With dynamic-test-selection, we'll also need to query the bugbug service from
the transforms. Let's move the querying logic to a utility file to share it
more easily.
Differential Revision: https://phabricator.services.mozilla.com/D73088
The default way to split the 'arg' parameter for CompositeStrategies, is to
duplicate it across all substrategies. By setting 'split_arg=tuple', we instead
break the arg up so the first index goes to the first substrategy, the second
index goes to the second substrategy, etc.
This means that the length of the 'test' arg must be at least as long as the
number of substrategies.
Differential Revision: https://phabricator.services.mozilla.com/D72464
This ensures we don't run every build with every push via ./mach try auto. It
introduces a new 'optimization-overrides' try_config that can be used to
replace optimizations. For now, there is no user interface to pass this in via
the 'mach try' command line.
Differential Revision: https://phabricator.services.mozilla.com/D68207
Since we don't look at the changesets, there is no need for hgmo to generate
them for us.
Differential Revision: https://phabricator.services.mozilla.com/D70567
--HG--
extra : moz-landing-system : lando
Ensure a minimum of 1 chunk for "per-file" tests like test-verify when run on try, even when
no local file changes are detected. --full is still required.
Differential Revision: https://phabricator.services.mozilla.com/D67694
--HG--
extra : moz-landing-system : lando
Changes:
While stepping through the debugger for Bug 1608837, it was found that for `platform = windows` the conditional was always evaluating to `False` thereby loading the fallback `unix` runtimes.
Differential Revision: https://phabricator.services.mozilla.com/D67390
--HG--
extra : moz-landing-system : lando
Define and use fuzzing-builds optimization strategy that allows either
skip-unless-schedules or seta strategies.
Differential Revision: https://phabricator.services.mozilla.com/D64628
--HG--
extra : moz-landing-system : lando
There's no way for Thunderbird to define a new task kind at the moment because
the verify_docs function only checks one base path.
This adds a 'documentation_paths' registry that holds a list of paths to check
for documentation files. Calling documentation_paths() with a filename later
will read all the files it finds with that name under all of the base paths,
returning them in one chunk. This minimizes the necessary changes to verify_docs
itself.
Differential Revision: https://phabricator.services.mozilla.com/D60298
--HG--
extra : moz-landing-system : lando
Future work should hopefully find a way to not assume tests are packaged with beetmover, and to have a solution for w64's emefree design.
Differential Revision: https://phabricator.services.mozilla.com/D64429
--HG--
extra : moz-landing-system : lando
Also adjust artifacts based on dep_kind, and make build-notarization-part-1
a build-signing dependency.
Differential Revision: https://phabricator.services.mozilla.com/D57738
--HG--
extra : moz-landing-system : lando
We don't need to package tests for builds that we don't actually run
tests from, but it is tricky to align this correctly by setting
MOZ_AUTOMATION_PACKAGE_TESTS=0 in relevant mozconfigs. Instead we can
set the environment variable in the task definition, and use a full
taskgraph verification check to ensure that the flag is only set on
builds that have tests.
The one tricky area is the win64-aarch64 builds, which have a workaround
by specifying the new skip-verify-test-packaging attribute.
In one case, win64-aarch64-shippable has tests that run against it, but
it copies those tests from a win64-aarch64-shippable-no-eme task, which
itself has no tests. Both of those tasks need to skip the verify check
as a result.
In another case, the win64-aarch64-eme task is an artifact build that
grabs test packages from the win64-aarch64 build. Since the win64-aarch64
build doesn't have tests, it needs to skip the verify check.
Differential Revision: https://phabricator.services.mozilla.com/D59426
--HG--
extra : moz-landing-system : lando
We don't need to package tests for builds that we don't actually run
tests from, but it is tricky to align this correctly by setting
MOZ_AUTOMATION_PACKAGE_TESTS=0 in relevant mozconfigs. Instead we can
set the environment variable in the task definition, and use a full
taskgraph verification check to ensure that the flag is only set on
builds that have tests.
The one tricky area is the win64-aarch64 builds, which have a workaround
by specifying the new skip-verify-test-packaging attribute.
In one case, win64-aarch64-shippable has tests that run against it, but
it copies those tests from a win64-aarch64-shippable-no-eme task, which
itself has no tests. Both of those tasks need to skip the verify check
as a result.
In another case, the win64-aarch64-eme task is an artifact build that
grabs test packages from the win64-aarch64 build. Since the win64-aarch64
build doesn't have tests, it needs to skip the verify check.
Differential Revision: https://phabricator.services.mozilla.com/D59426
--HG--
extra : moz-landing-system : lando
Also adjust artifacts based on dep_kind, and make build-notarization-part-1
a build-signing dependency.
Differential Revision: https://phabricator.services.mozilla.com/D57738
--HG--
extra : moz-landing-system : lando
Provides denylisted attributes to copy over in the fusing of chunked portions of the taskgraph. Without
denylisting these we have changes on these related kinds in json, due simply to ordering of total tasks.
Differential Revision: https://phabricator.services.mozilla.com/D62637
--HG--
extra : moz-landing-system : lando
Changes:
Use `six.moves.urllib` instead of `urllib`.
Use `six.iteritems` instead of `iteritems`.
Differential Revision: https://phabricator.services.mozilla.com/D61044
--HG--
extra : moz-landing-system : lando
We don't need to package tests for builds that we don't actually run
tests from, but it is tricky to align this correctly by setting
MOZ_AUTOMATION_PACKAGE_TESTS=0 in relevant mozconfigs. Instead we can
set the environment variable in the task definition, and use a full
taskgraph verification check to ensure that the flag is only set on
builds that have tests.
The one tricky task is win64-aarch64-shippable/opt, which copies tests
from another build rather than building them itself. For this reason, it
explicitly sets MOZ_AUTOMATION_PACKAGE_TESTS: '0' in the environment
even though that is now the default. This is why the exception is only
raised if MOZ_AUTOMATION_PACKAGE_TESTS is not set at all, rather than
checking that it is set to 1.
Differential Revision: https://phabricator.services.mozilla.com/D59426
--HG--
extra : moz-landing-system : lando
Previously we simply ignored skipped tests when performing test chunking in the
taskgraph. This could sometimes result in a task that runs a single manifest
that happens to be skipped on that task's configuration. This results in an error
that no tests were run from the harness.
This patch attempts to guess a mozinfo based on a task's definition. Then it
uses those values to take skipped tests into account when running the chunking
algorithm. That way the chances of scheduling a manifest that would be skipped
anyway are much lower.
Depends on D58987
Differential Revision: https://phabricator.services.mozilla.com/D58988
--HG--
extra : moz-landing-system : lando
The new code is a superset of the old, so I've left it much the same to avoid conflicts until we come up with a more long term solution.
Differential Revision: https://phabricator.services.mozilla.com/D59181
--HG--
extra : moz-landing-system : lando
It will be relanded once these are complete. This prevents from those tasks
getting scheduled for every push until the initial ones have been completed.
When I was working on release-promotion for Thunderbird, I needed a way to
control the scriptworker worker-types used on try vs. on release branches.
At the time, there did not exist a way to vary worker types so I introduced
a scriptworker-specific way to vary the worker type.
Since then, we add support for worker aliases, which can currently vary by
level. This reuses that machinery for selecting signing workers.
Differential Revision: https://phabricator.services.mozilla.com/D58018
--HG--
extra : moz-landing-system : lando
Andrew Halberstadt
Bob Clary
Tom Prince
Geoff Brown
Edwin Takahashi
Rob Lemley
Justin Wood
Aki Sasaki
Mike Shal
Andreea Pavel
Dorel Luca
Ricky Stewart
Christian Holler
Noemi Erli
Razvan Maries
Ryan VanderMeulen
Daniel Varga
Simon Fraser
Sebastian Hengst
Gurzau Raul