This is both for future proofing (fetches could move any time although
they likely won't), and to fix the path on the future Windows PGO
cross builds, where the fetches path is not under $WORKSPACE.
Differential Revision: https://phabricator.services.mozilla.com/D66358
--HG--
extra : moz-landing-system : lando
MOZ_PGO_PROFILE_USE is used to enable certain features in the automation
mozconfigs for profile-use builds, including the MOZ_PROFILE_USE
configure option. They aren't easily combined into one flag due to the
extra settings that are enabled only in automation mozconfigs.
Renaming the former variable to TASKCLUSTER_PGO_PROFILE_USE makes it
slightly more obvious where it comes from and avoids confusion.
Differential Revision: https://phabricator.services.mozilla.com/D56117
--HG--
extra : moz-landing-system : lando
If the run task generates bad profile data, the merge step in the
profile-use task will fail. However, retrying the profile-use task
doesn't fix the problem, and there isn't a straightforward way to retry
the run task in this situation. Instead we can add a clang toolchain to
all the run tasks, and perform the merge there.
This means the output from the run task will always be a successfully
merged file called 'merged.profdata', and we no longer need to perform
the merge as part of the profile-use build as a GENERATED_FILES step.
Depends on D45262
Differential Revision: https://phabricator.services.mozilla.com/D45263
--HG--
extra : moz-landing-system : lando
Some recent changes to how we set cross-language LTO for Windows
resulted in compilation-time decreases and small performance regressions
on a few benchmarks. The changes intended to remove explicit enablement
of cross-language LTO for all builds, but rely on shippable builds being
built with PGO and moz.configure's clever defaulting of cross-language
LTO for PGO'd builds on Windows, which would then enable cross-language
LTO for only shippable builds.
Obviously something went wrong with those changes.
The problem was our defaulting wasn't visible to moz.configure's logic
for how to pass command-line options to the JS subconfigure. We set the
value (`cross`) after the value for `--enable-lto` has been determined,
and the default value is off (that is, `--disable-lto`). Since
moz.configure is very thorough in passing configure options down into
JS, it dutifully looked at what the default value of `--enable-lto` was
supposed to be, and passed `--disable-lto` to JS's configure.
There's some evidence that we knew our defaulting was a little sketchy:
we'd only attempt cross-language LTO when we were performing the PGO use
phase, and only if the value of `--enable-lto` wasn't explicitly passed.
Which was a fine idea--you don't want to override what the user was
trying to do--but in the case of JS backfired on us: the value *was*
coming from the explicitly-passed command-line option of
`--disable-lto`. So JS didn't enable any kind of LTO, with attendant
consequences.
This problem *didn't* happen before the aforementioned change because we
were explicitly specifying that `--enable-lto=cross` should be passed in
the mozconfig, which ensured that the correct setting was passed into
JS. We were just setting `--enable-lto=cross` for *all* builds, which
was less than desirable.
The easiest way to fix all this is simply to put the
`--enable-lto=cross` setting in the Windows mozconfigs, conditional on
`MOZ_PGO_PROFILE_USE`. That placement captures the intent of the
previous attempt at defaulting, but without the troubles described
above: the option explicitly appears on the command line, and
moz.configure will correctly pass it through to the JS subconfigure.
This also makes our Windows configuration closer to our Linux
configuration (the Linux configuration enables cross-language LTO for
both PGO phases, which is arguably a bug).
Differential Revision: https://phabricator.services.mozilla.com/D41080
--HG--
extra : moz-landing-system : lando
MOZ_PGO_PROFILE_USE is set when the use-pgo attribute is defined in the
task. This environment variable is used to enable --enable-profile-use
and related configure flags.
Differential Revision: https://phabricator.services.mozilla.com/D34798
--HG--
extra : moz-landing-system : lando
We need to sign parts of the contents of the archives, so the mar's that we
ship get built as part of the repackage task. Thus, there is no reason to also
create and upload as part of the build, just to throw them away.
Differential Revision: https://phabricator.services.mozilla.com/D6213
--HG--
extra : moz-landing-system : lando
The previous commit moved creating installers to be side effect of creating
packages. This makes the installer step not actually do anything. So remove the
step from automation.
Differential Revision: https://phabricator.services.mozilla.com/D864
--HG--
extra : rebase_source : 75b67a6e57031ae189dafe1d9854e4105aa22621
extra : source : fcb756834abbdbe0ae6e59a8cfdf39024f50a226
The previous commit moved creating installers to be side effect of creating
packages. This makes the installer step not actually do anything. So remove the
step from automation.
Differential Revision: https://phabricator.services.mozilla.com/D864
--HG--
extra : rebase_source : 174a366890da295634ef8971d0608ea60979f110
extra : histedit_source : 06fdd0e2ccb93f061ba5a40c2a4137eed9898916
With all of our builds in Taskcluster now, we should never be uploading
symbols from build tasks. Unfortunately Windows builds were still doing so.
This patch removes MOZ_AUTOMATION_UPLOAD_SYMBOLS from all the in-tree
mozconfigs and a few other places so that it should always default off
(per moz-automation.mk). The rest of the uploadsymbols bits will be
removed once Thunderbird fixes their automation.
This patch was mostly autogenerated by running:
rg --files-with-matches UPLOAD_SYMBOLS browser/config/mozconfigs/ mobile/android/config/mozconfigs/ | xargs sed -ri '/.*UPLOAD_SYMBOLS.*/d'
sed -ri '/.*UPLOAD_SYMBOLS.*/d' build/unix/mozconfig.linux build/mozconfig.win-common build/macosx/local-mozconfig.common build/mozconfig.automation
Then mobile/android/config/mozconfigs/common and
taskcluster/scripts/builder/build-linux.sh were hand-edited.
MozReview-Commit-ID: Cy8kSEodSg4
--HG--
extra : rebase_source : 01caf1651b4eb428313e1f371aa585f8f34c4151
It turns out that running makecab to compress PDB files takes a significant
amount of time in the buildsymbols step. I wrote an implementation of
makecab in Rust that implements only the subset of features we use and
it's significantly faster:
https://github.com/luser/rust-makecab
This patch adds a makecab check to moz.configure, adds a release build of
the makecab binary to the Windows tooltool manifests, points the build at
it from mozconfig.win-common, and changes symbolstore.py to use MAKECAB
from substs instead of calling `makecab.exe` directly.
MozReview-Commit-ID: 76FHLIZFCXS
--HG--
extra : rebase_source : af4cf2e4db4607ec9329b2811cc0175d3e113b66
Mike Hommey
Razvan Maries
Gabriele Svelto
Csoregi Natalia
David Major
Mike Shal
Nathan Froyd
Tom Prince
Chris Peterson
Brindusan Cristian
Dorel Luca
Ted Mielczarek
Benjamin Smedberg
aleth
Ryan VanderMeulen