If the run task generates bad profile data, the merge step in the
profile-use task will fail. However, retrying the profile-use task
doesn't fix the problem, and there isn't a straightforward way to retry
the run task in this situation. Instead we can add a clang toolchain to
all the run tasks, and perform the merge there.
This means the output from the run task will always be a successfully
merged file called 'merged.profdata', and we no longer need to perform
the merge as part of the profile-use build as a GENERATED_FILES step.
Depends on D45262
Differential Revision: https://phabricator.services.mozilla.com/D45263
--HG--
extra : moz-landing-system : lando
This code was already dead during the VS2017 era (since that compiler changed to the `Hostx64\x86` path scheme), let alone clang-cl.
Differential Revision: https://phabricator.services.mozilla.com/D33513
--HG--
extra : moz-landing-system : lando
We're moving to IR-level PGO instrumentation for clang-cl. We've also
moved to using static linker ordering files, which was the primary
application of the previous style of PGO instrumentation. We therefore
we no longer need this code.
Differential Revision: https://phabricator.services.mozilla.com/D31134
--HG--
extra : moz-landing-system : lando
Bug 1548941 restricted under which circumstances we allow the
browser.tabs.remote.autostart pref to turn off e10s. The PGO profileserver.py
script relied on the unittest-required user.js prefs collection to turn off
e10s (see also bug 1196094) via this pref. For PGO builds, we do not set the
MOZ_DISABLE_NONLOCAL_CONNECTIONS env var, which meant that we stopped
honouring the pref to turn off e10s. Unfortunately, this meant that
e10s was inadvertently now switched on for the pgo profiling, which
negatively impacted speedometer on PGO builds (and possibly other tests).
All this change does is re-disable e10s for PGO profiling. We should
investigate how to turn e10s on "properly" for PGO, but we can do that in
bug 1196094, without taking this temporary regression, especially as 68
branches.
Differential Revision: https://phabricator.services.mozilla.com/D31736
--HG--
extra : moz-landing-system : lando
In bug 1514288 we started to save the output from running Firefox during
PGO builds into log files, but they aren't correctly uploaded if the run
fails. This presents a problem for categorizing failed PGO builds if the
profileserver returns an error code (eg: bug 1517939), since the error
messages could be hidden away in log files that aren't uploaded.
Instead, we can generate them directly into the artifacts directory so
that they are always uploaded. Additionally, the log files are displayed
if the run fails so that the error messages are displayed in the log for
easier bug categorization.
Differential Revision: https://phabricator.services.mozilla.com/D15881
--HG--
extra : moz-landing-system : lando
The output from running the browser during PGO builds can have innocuous
error messages in them, but show up in treeherder as potential messages
to include when filing bugs. We can just save the output from these runs
as files and upload them as artifacts instead, so they don't show up in
the build log but are available for inspection if necessary.
MozReview-Commit-ID: 3VdVCKVkZNI
Differential Revision: https://phabricator.services.mozilla.com/D15154
--HG--
extra : moz-landing-system : lando
Currently, when Firefox crashes when running the PGO profile, the error
is ignored, and the result is a build that is not optimized as expected.
So instead of such things going unnoticed, we make profileserver.py
return an error code, further failing the build.
Unfortunately, this does not make the output useful to know what went
wrong exactly, but the most important thing is that the build fails
instead of going through and leading to bad results.
The incorrect LEGACY_BUILD_ID will be fixed in a subsequent changeset.
We must add https://www.mozilla.org/ to server-locations.txt and regenerate the mochitest certificates [1] because the new navigator.buildID test pretends to load content from https://www.mozilla.org/.
[1] https://searchfox.org/mozilla-central/source/build/pgo/certs/README
Differential Revision: https://phabricator.services.mozilla.com/D7982
--HG--
rename : dom/tests/mochitest/bugs/test_bug351601.html => dom/tests/mochitest/bugs/test_navigator_buildID.html
extra : rebase_source : 1deb142930f1a7a570cf719c4cb2bed8adfeabe2
extra : source : 408bff32f9623513a271cdf043d11ba6d1318e03
This change prevents javascript:-URLs from being passed at the command
line. This restriction was already applied to every URL but the first.
Even the first URL did not result in any visible effect when Firefox is
started. Yet somehow the PGO profiler script managed to rely on it.
This commit ensures that "javascript:" URLs are not activated regardless of
position in the command line, and switches to a data:-URL for the PGO script to
achieve the (previously) desired effect of quitting the browser on startup.
Depends on D4929
Differential Revision: https://phabricator.services.mozilla.com/D5958
--HG--
extra : moz-landing-system : lando
Add extra preferences for the profileserver script on-top of
the ones as inheritted from unittests.
--HG--
extra : rebase_source : 02d3297134e82c2425fc4c6c939cc5d0cc4a2c13
extra : histedit_source : 07f699b0ff8bec2a6eda7705a0fa13fb3b352f98
This serves two purposes:
1) It makes web-platform-tests pref downloading/handling a little more robust. When
run externally, it now downloads the entire testing/profiles directory. When loading
prefs it will look for both prefs_general.js (to support older versions of Firefox)
and profiles.json (for support moving forward).
This way we can add/remove/rename pref files under these directories without needing
to worry about breaking upstream wpt.
2) It provides developers an overview of which harnesses are using which base profiles.
Instead of hunting through test harness code to find this information, they can glance
at profiles.json.
MozReview-Commit-ID: AMzdnD8aGA2
--HG--
extra : rebase_source : 6fa0a802680417e49fcef99f3d03de7458a8fcba
This moves testing/profiles/prefs_general.js to
testing/profiles/common/user.js. It also adds an 'extensions' folder to the
common profile. Dropping extension files here will get them installed in all
test harnesses (useful for testing on try).
The idea is that all test harnesses will eventually use this 'common' profile.
We'll also create some new per harness profiles, e.g testing/profiles/mochitest
and testing/profiles/reftest. This way there will be a single location
developers can go to set preferences, both for a specific harness, and across
all harnesses.
MozReview-Commit-ID: 8sqBqLiypgU
--HG--
rename : testing/profiles/prefs_general.js => testing/profiles/common/user.js
extra : rebase_source : 72a4a4b691e93c77479c7e70647b0ca373862c51
This moves testing/profiles/prefs_general.js to
testing/profiles/common/user.js. It also adds an 'extensions' folder to the
common profile. Dropping extension files here will get them installed in all
test harnesses (useful for testing on try).
The idea is that all test harnesses will eventually use this 'common' profile.
We'll also create some new per harness profiles, e.g testing/profiles/mochitest
and testing/profiles/reftest. This way there will be a single location
developers can go to set preferences, both for a specific harness, and across
all harnesses.
MozReview-Commit-ID: 8sqBqLiypgU
--HG--
rename : testing/profiles/prefs_general.js => testing/profiles/common/user.js
extra : rebase_source : 7599913e547533f2f57b597ad0f238c6cd391c82
(This also fixes Bug 879740 and Bug 1204543.)
build/pgo/certs contains an NSS database set that has a bunch of hand-generated
certificates, and many of these hand-generated certificates are specifically
depended upon for a variety of unit tests. This patch changes all of these to
use the "pycert.py" and "pykey.py" utilities that produce deterministic keys
and certificates.
The naming convention here is new, and defined in the README. It is based on
the mochitest runtest.py naming convention that imports .ca and .client
PEM-encoded certificates.
Unfortunately, the updates to build/pgo/genpgocert.py to generate these files
depends on OpenSSL in order to produce PKCS12 archives for pk11tool to import
into NSS. This could be done with pure-NSS tooling, but it'd require some new
command line functionality, which is out-of-scope for this change.
Note that build/pgo/genpgocert.py no longer takes arguments when run. It's not
run automatically anywhere that I can see, but could (reasonably) be, now.
Differential Revision: https://phabricator.services.mozilla.com/D971
--HG--
extra : amend_source : bc389b9b0a807a4889feb14db439daa28635dfe9
dimi
Mike Shal
David Major
Gijs Kruitbosch
Mark Banner
Mihai Alexandru Michis
Nathan Froyd
Kevin Jacobs
Sebastian Streich
Noemi Erli
Ehsan Akhgari
Oana Pop Rus
Carolina Jimenez Gomez
Chris Manchester
Andreea Pavel
James Willcox
Johann Hofmann
Wes Kocher
Marco Bonardo
Mike Hommey
Cosmin Sabou
Gurzau Raul
Chris Peterson
Rob Wu
Henrik Skupin
Sylvestre Ledru
Andrew Halberstadt
Bogdan Tara
J.C. Jones