This includes scripts that involve `tup`, jobs that build `tup` in automation, `tup.configure`, and related infrastructure and documentation.
Differential Revision: https://phabricator.services.mozilla.com/D73921
This patch uses the new push-interval-10 to schedule the linux, windows plain and aarch64
builds on autoland every 10th push.
Tested locally with a local checkout whose pushlog_id was not divisible
by 10 using parameters.yml downloaded from the Gecko Decision Task using
./mach taskgraph optimized --verbose --parameters /tmp/parameters.yml
parameters.yml from autoland showed the following optimizations.
0:56.13 PushIntervalStrategy: Removing task build-linux64-aarch64/opt interval 10
0:56.13 PushIntervalStrategy: Removing task build-linux64-plain/debug interval 10
0:56.13 PushIntervalStrategy: Removing task build-signing-win64-aarch64/opt interval 10
0:56.13 PushIntervalStrategy: Removing task build-win64-aarch64/debug interval 10
0:56.13 PushIntervalStrategy: Removing task build-win64-plain/debug interval 10
0:56.18 PushIntervalStrategy: Removing task valgrind-linux64-valgrind/opt interval 10
while parameters.yml from mozilla-central did not show any PushIntervalStrategy
optimizations.
Differential Revision: https://phabricator.services.mozilla.com/D70182
This is similar to the same-named commit in bug 1619837.
For those few configurations that don't run xpcshell tests, we need a
`fix-stacks` dependency in the build job. This is because xpcshell self tests
run during the build job, but the existing xpcshell test dependency doesn't
cover that case.
Differential Revision: https://phabricator.services.mozilla.com/D66927
--HG--
extra : moz-landing-system : lando
This creates a new toolchain artifact that repacks a combination of the
linux64-clang compiler along with parts of the win64-clang-cl compiler.
This has multiple advantages:
- It removes some convoluted parts of build task definitions (limiting
that to only occur on the win-cross toolchain itself).
- It simplifies the build setup by not requiring to prepare for where
clang-cl.exe is.
- It speeds up getting compiler artifacts because the win64-clang-cl
artifact is very large (due to there not being a llvm shared library)
and bzipped, which is slow to decompress. Here, we only take what we
need for the cross builds.
- It adds the runtime files that e.g. PGO will require, and that
linux clang-cl insists lives in the clang directory, not the
win64-clang-cl one, and that would require some convoluted setup to make
it work with the two separate toolchains.
Differential Revision: https://phabricator.services.mozilla.com/D66543
--HG--
extra : moz-landing-system : lando
First, with `fix_stacks.py` now being used on most test suites, those test
suites need a dependency on `fix-stacks`.
Second, for those few configurations that don't run xpcshell tests, we need a
`fix-stacks` dependency in the build job. This is because xpcshell self tests
run during the build job, but the existing xpcshell test dependency doesn't
cover that case.
Differential Revision: https://phabricator.services.mozilla.com/D65943
--HG--
extra : moz-landing-system : lando
This duplicates the peculiarities of the win64-cross/opt task, while adding
a rust-size dependency it was lacking.
Differential Revision: https://phabricator.services.mozilla.com/D65483
--HG--
extra : moz-landing-system : lando
This moves the object directory and source directories around in all mozharness
jobs, to allow enabling caching on windows builders.
This makes a number of changes that all need to land at once:
- Move the source checkout for the workspace cache mount, to the checkouts
cache mount.
- Makes the object directoy from underneath the source directory, to directly
in the work directory (which is still under `workspace`).
- Sets the object directory to `obj-build` instead of `obj-firefox`.
- Stops caching the workspace directory (it is still a volume in docker workers,
so writes perform well; a followup revision add some checks around this).
- Removes one level of directory in the mozharness workdir (things were under
`workspace/build`, but are now just under `workspace/`.
- Adjust paths in environment variables and artifact specifications to match
the above changes.
Differential Revision: https://phabricator.services.mozilla.com/D62482
--HG--
extra : moz-landing-system : lando
Define and use fuzzing-builds optimization strategy that allows either
skip-unless-schedules or seta strategies.
Differential Revision: https://phabricator.services.mozilla.com/D64628
--HG--
extra : moz-landing-system : lando
Future work should hopefully find a way to not assume tests are packaged with beetmover, and to have a solution for w64's emefree design.
Differential Revision: https://phabricator.services.mozilla.com/D64429
--HG--
extra : moz-landing-system : lando
We don't need to package tests for builds that we don't actually run
tests from, but it is tricky to align this correctly by setting
MOZ_AUTOMATION_PACKAGE_TESTS=0 in relevant mozconfigs. Instead we can
set the environment variable in the task definition, and use a full
taskgraph verification check to ensure that the flag is only set on
builds that have tests.
The one tricky area is the win64-aarch64 builds, which have a workaround
by specifying the new skip-verify-test-packaging attribute.
In one case, win64-aarch64-shippable has tests that run against it, but
it copies those tests from a win64-aarch64-shippable-no-eme task, which
itself has no tests. Both of those tasks need to skip the verify check
as a result.
In another case, the win64-aarch64-eme task is an artifact build that
grabs test packages from the win64-aarch64 build. Since the win64-aarch64
build doesn't have tests, it needs to skip the verify check.
Differential Revision: https://phabricator.services.mozilla.com/D59426
--HG--
extra : moz-landing-system : lando
Ricky Stewart
Joel Maher
Bob Clary
Andrew Halberstadt
Cosmin Sabou
Nicholas Nethercote
Jesse Schwartzentruber
Mike Hommey
Christian Holler
Noemi Erli
Michael Froman
Arthur Iakab
Ciure Andrei
Sebastian Hengst
Tom Prince
Razvan Maries
Chris Manchester
Justin Wood
Bogdan Tara
Brindusan Cristian
Mike Shal