This extracts an install-meson.sh helper script to install meson in both
the wrench-deps task for Firefox CI and the taskcluster.yml in WebRender CI.
Differential Revision: https://phabricator.services.mozilla.com/D90441
In order for Thunderbird to effectively cut down on excess builds, there needs
to be a way to define taskgraph optimization strategies beyond what is made
available in the mozilla repository.
taskgraph.optimize.register_strategy gets half of the job done. The other piece
is task schema validation which checks optimizations against OptimizationSchema.
OptimizationSchema gets moved to taskgraph.optimize.schema so that it is not set
too early by taskgraph.util.schema. Projects then call set_optimization_schema()
to replace the default schema object. set_optimization_schema() needs to be called
in the project's taskgraph:register function before any transform code is loaded.
Differential Revision: https://phabricator.services.mozilla.com/D88429
Without setting this variable, the call to `mach python` in `taskcluster/scripts/builder/build-linux.sh` will create a `virtualenv` in the default location -- on Linux in CI, that's `obj-x86_64-pc-linux-gnu`. Everything else about the build is going to end up in `obj-build`, so just set `MOZ_OBJDIR` appropriately to point to that directory.
Doing so enables us to land bug 1663755, as it causes the logic added to `symbols_archive.py` as part of bug 1654994 to behave as expected.
Differential Revision: https://phabricator.services.mozilla.com/D89975
Sets autoland to use the 'bugbug' test manifest loader. This is being enabled
as part of a temporary trial to see the impact it has on sheriffing.
Depends on D90159
Differential Revision: https://phabricator.services.mozilla.com/D90160
These tasks can only be selected on try by passing `--full` to |mach try fuzzy|
or |mach try chooser|. E.g:
$ ./mach try fuzzy --full -q "'windows7-32-qr"
Differential Revision: https://phabricator.services.mozilla.com/D89923
In the past, the 'backstop' optimization was applied to tasks by default across
all projects, even though it only really made sense on autoland. To choose what
would happen on non-autoland branches, we invented this 'remove_on_projects'
concept.
These days, we only apply the backstop optimization in the first place for
autoland. So 'remove_on_projects' is no longer necessary.
Depends on D88149
Differential Revision: https://phabricator.services.mozilla.com/D88150
This removes the last uses of the 'push-interval-10' and 'push-interval-20' strategies.
They are being removed because they are dangerous in that its easy to accidentally not run
tasks when they should.
Instead, task authors should decide whether they want their tasks to run on
"backstop" pushes (run everything) or "expanded" pushes (run more than usual,
but still not as much as a backstop). Note that using "expanded" means the task
will *also* run on backstop pushes. It'll just additionally run on "expanded"
pushes.
In practice 'backstop' pushes will be every 20th push and 'expanded' pushes
will be every 10th push. Though this may vary due to the time component in
backstops.
Differential Revision: https://phabricator.services.mozilla.com/D89503
This is a nomenclature change + refactoring. Now there is only a single
"backstop" push. Which is currently set to every 20th push on autoland (or
every push on non-autoland branches).
Now there is also a concept of an "expanded" push. These are pushes that run
more stuff than usual, but not as much as a backstop normally would. These are
currently set to run at half the interval of a backstop.
Concretely, here are the strategy changes:
* Renamed 'backstop' -> 'skip-unless-backstop'
* Introduced the 'skip-unless-expanded' strategy which has 'skip-unless-backstop' baked in
* Merged 'optimized-backstop' into the 'test' strategy (with expanded comments)
* Baked both 'skip-unless-expanded' and 'skip-unless-backstop' into the 'test' strategy
Differential Revision: https://phabricator.services.mozilla.com/D89501
It turns out that 'Not' is needed to negate "backstops". E.g, we normally
we want to use a pattern like so:
All("skip-unless-backstop", "test")
Since 'skip-unless-backstop' returns False on backstop pushes, it disables
the test strategy there.
However, suppose we wanted to run a special optimization, *only* on backstop
pushes. I.e, the opposite of the above example. Then we need to use:
All(Not("skip-unless-backstop"), "test-backstop")
Depends on D89500
Differential Revision: https://phabricator.services.mozilla.com/D89734
The intent of a "backstop" push, is to run everything so we can be absolutely sure that
the push in question does not cause any regressions.
Previously, backstops were thought to be only something that ran on autoland.
This was because the other branches *already* ran everything so the concept of
a "backstop" didn't make much sense.
But going by the above definition, it would make more sense to say that *every*
non autoland (or try) push is a backstop. Since the intent there is to run
everything to avoid regressions.
This change will allow us to simplify our optimization algorithms.
Differential Revision: https://phabricator.services.mozilla.com/D89499
The script uses the `cd $FOO && pwd` trick to fix the path style on Windows, but currently this happens before the `mkdir`, so we get an incorrect result:
```
[task 2020-09-11T03:38:59.656Z] ++ cd Z:/task_1599794667/public/build
[task 2020-09-11T03:38:59.656Z] ./src/taskcluster/scripts/builder/build-sm.sh: line 10: cd: Z:/task_1599794667/public/build: No such file or directory
[task 2020-09-11T03:38:59.656Z] ++ pwd
[task 2020-09-11T03:38:59.657Z] + export MOZ_UPLOAD_DIR=/z/task_1599794667
```
Differential Revision: https://phabricator.services.mozilla.com/D89925
This fixes a bug that was introduced in recent optimizations where
upload-symbols may not run for shipped release builds. We can add it as
a dependency for this, or some other per-platform task that runs for
releases. As long as it runs before we ship this should be fine.
Differential Revision: https://phabricator.services.mozilla.com/D89494
Corresponding debug and shippable tests already run as tier 1, but the plain
opt tests (which run on autoland and try) currently run as tier 2, apparently by
accident.
Differential Revision: https://phabricator.services.mozilla.com/D89255
Mihai Alexandru Michis
Daniel Holbert
Joel Maher
Kartikaya Gupta
Ricky Stewart
Tarek Ziadé
stransky
alexandru.ionescu
Rob Lemley
Aki Sasaki
Dzmitry Malyshau
Doug Thayer
june wilde
Gregory Mierzwinski
Csoregi Natalia
Andrew Halberstadt
Michael Pobega
Sylvestre Ledru
Andrew Halberstadt
David Major
Dorel Luca
Ben Hearsum
Narcis Beleuzu
Tom Ritter
Cristina Coroiu
Andreea Pavel
Geoff Brown