The 'webgl-ext' suites explicitly disable the 'webgl-ipc' variant on
ccov builds. It looks like the fact that 'webtl-core' doesn't was an
omission.
The added tests fail without the change in this patch.
Differential Revision: https://phabricator.services.mozilla.com/D91937
Sets autoland to use the 'bugbug' test manifest loader. This is being enabled
as part of a temporary trial to see the impact it has on sheriffing.
Differential Revision: https://phabricator.services.mozilla.com/D90160
Until we remove IonBuilder/TI (October or November if all goes well) we need to
make sure we don't break it when we enable Warp by default in the JS shell too.
This patch adds a temporary no-warp job based on the plaindebug variant that runs
shell tests with --no-warp.
The next step is enabling Warp by default in the JS shell + removing the SM(warp) job.
Differential Revision: https://phabricator.services.mozilla.com/D91389
When enabling manifest scheduling, several interdependencies between tests were
revealed resulting in too many new intermittents. Make sure we disable
manifest-scheduling there for now.
Differential Revision: https://phabricator.services.mozilla.com/D91589
When enabling manifest scheduling, several interdependencies between tests were
revealed resulting in too many new intermittents. Make sure we disable
manifest-scheduling there for now.
Differential Revision: https://phabricator.services.mozilla.com/D91589
We started using the "backstop" index added by bug 1660506 to determine whether
a push should be a backstop based on a time interval. The problem is that this
index gets added by an index-task that runs after the decision task has
completed. Therefore, if two pushes land at roughly the same time (i.e, the
second decision task starts before the first completes), then they can both
determine themselves as backstops.
This patch gets around the problem by inserting the "backstop" index as early
as possible (immediately after resolving parameters), so the chances of this
happening become very low. It's still theoretically possible that it could
happen again, but we don't need this to be 100% perfect. As long as it is rare,
it's good enough.
Depends on D91191
Differential Revision: https://phabricator.services.mozilla.com/D91192
We did some simple statistical analysis and found that increasing the iteration count on the Fenix view tests from 14 to 25 would give us 95% confidence margin of error that is comparable to the local FNPRMS system.
Differential Revision: https://phabricator.services.mozilla.com/D91356
This causes them to get filtered out at the target tasks phase instead. We need
to do this since backfilling depends on the task existing in the full task
graph.
Differential Revision: https://phabricator.services.mozilla.com/D90579
Remove the use of conditioned profiles for Fenix, GVE, since they create a user scenario that we are not seeing in local testing.
Differential Revision: https://phabricator.services.mozilla.com/D90450
This extracts an install-meson.sh helper script to install meson in both
the wrench-deps task for Firefox CI and the taskcluster.yml in WebRender CI.
Differential Revision: https://phabricator.services.mozilla.com/D90441
In order for Thunderbird to effectively cut down on excess builds, there needs
to be a way to define taskgraph optimization strategies beyond what is made
available in the mozilla repository.
taskgraph.optimize.register_strategy gets half of the job done. The other piece
is task schema validation which checks optimizations against OptimizationSchema.
OptimizationSchema gets moved to taskgraph.optimize.schema so that it is not set
too early by taskgraph.util.schema. Projects then call set_optimization_schema()
to replace the default schema object. set_optimization_schema() needs to be called
in the project's taskgraph:register function before any transform code is loaded.
Differential Revision: https://phabricator.services.mozilla.com/D88429
Without setting this variable, the call to `mach python` in `taskcluster/scripts/builder/build-linux.sh` will create a `virtualenv` in the default location -- on Linux in CI, that's `obj-x86_64-pc-linux-gnu`. Everything else about the build is going to end up in `obj-build`, so just set `MOZ_OBJDIR` appropriately to point to that directory.
Doing so enables us to land bug 1663755, as it causes the logic added to `symbols_archive.py` as part of bug 1654994 to behave as expected.
Differential Revision: https://phabricator.services.mozilla.com/D89975
Sets autoland to use the 'bugbug' test manifest loader. This is being enabled
as part of a temporary trial to see the impact it has on sheriffing.
Depends on D90159
Differential Revision: https://phabricator.services.mozilla.com/D90160
These tasks can only be selected on try by passing `--full` to |mach try fuzzy|
or |mach try chooser|. E.g:
$ ./mach try fuzzy --full -q "'windows7-32-qr"
Differential Revision: https://phabricator.services.mozilla.com/D89923
In the past, the 'backstop' optimization was applied to tasks by default across
all projects, even though it only really made sense on autoland. To choose what
would happen on non-autoland branches, we invented this 'remove_on_projects'
concept.
These days, we only apply the backstop optimization in the first place for
autoland. So 'remove_on_projects' is no longer necessary.
Depends on D88149
Differential Revision: https://phabricator.services.mozilla.com/D88150
This removes the last uses of the 'push-interval-10' and 'push-interval-20' strategies.
They are being removed because they are dangerous in that its easy to accidentally not run
tasks when they should.
Instead, task authors should decide whether they want their tasks to run on
"backstop" pushes (run everything) or "expanded" pushes (run more than usual,
but still not as much as a backstop). Note that using "expanded" means the task
will *also* run on backstop pushes. It'll just additionally run on "expanded"
pushes.
In practice 'backstop' pushes will be every 20th push and 'expanded' pushes
will be every 10th push. Though this may vary due to the time component in
backstops.
Differential Revision: https://phabricator.services.mozilla.com/D89503
This is a nomenclature change + refactoring. Now there is only a single
"backstop" push. Which is currently set to every 20th push on autoland (or
every push on non-autoland branches).
Now there is also a concept of an "expanded" push. These are pushes that run
more stuff than usual, but not as much as a backstop normally would. These are
currently set to run at half the interval of a backstop.
Concretely, here are the strategy changes:
* Renamed 'backstop' -> 'skip-unless-backstop'
* Introduced the 'skip-unless-expanded' strategy which has 'skip-unless-backstop' baked in
* Merged 'optimized-backstop' into the 'test' strategy (with expanded comments)
* Baked both 'skip-unless-expanded' and 'skip-unless-backstop' into the 'test' strategy
Differential Revision: https://phabricator.services.mozilla.com/D89501
It turns out that 'Not' is needed to negate "backstops". E.g, we normally
we want to use a pattern like so:
All("skip-unless-backstop", "test")
Since 'skip-unless-backstop' returns False on backstop pushes, it disables
the test strategy there.
However, suppose we wanted to run a special optimization, *only* on backstop
pushes. I.e, the opposite of the above example. Then we need to use:
All(Not("skip-unless-backstop"), "test-backstop")
Depends on D89500
Differential Revision: https://phabricator.services.mozilla.com/D89734
The intent of a "backstop" push, is to run everything so we can be absolutely sure that
the push in question does not cause any regressions.
Previously, backstops were thought to be only something that ran on autoland.
This was because the other branches *already* ran everything so the concept of
a "backstop" didn't make much sense.
But going by the above definition, it would make more sense to say that *every*
non autoland (or try) push is a backstop. Since the intent there is to run
everything to avoid regressions.
This change will allow us to simplify our optimization algorithms.
Differential Revision: https://phabricator.services.mozilla.com/D89499
The script uses the `cd $FOO && pwd` trick to fix the path style on Windows, but currently this happens before the `mkdir`, so we get an incorrect result:
```
[task 2020-09-11T03:38:59.656Z] ++ cd Z:/task_1599794667/public/build
[task 2020-09-11T03:38:59.656Z] ./src/taskcluster/scripts/builder/build-sm.sh: line 10: cd: Z:/task_1599794667/public/build: No such file or directory
[task 2020-09-11T03:38:59.656Z] ++ pwd
[task 2020-09-11T03:38:59.657Z] + export MOZ_UPLOAD_DIR=/z/task_1599794667
```
Differential Revision: https://phabricator.services.mozilla.com/D89925
This fixes a bug that was introduced in recent optimizations where
upload-symbols may not run for shipped release builds. We can add it as
a dependency for this, or some other per-platform task that runs for
releases. As long as it runs before we ship this should be fine.
Differential Revision: https://phabricator.services.mozilla.com/D89494
Corresponding debug and shippable tests already run as tier 1, but the plain
opt tests (which run on autoland and try) currently run as tier 2, apparently by
accident.
Differential Revision: https://phabricator.services.mozilla.com/D89255
Pipenv is heavy weight and overkill for the purposes it is being used. We'd like to remove it from the tree and |mach python-test| was one of the last remanining use cases.
Remove the `--python` command-line argument as a result. Users who wish to run unit tests with Python 2 can do `MACH_PY2=1 ./mach python-test ...` or `python2 ./mach python-test ...`.
Also update a few unit tests that would break otherwise in the presence of this change.
There were a couple lines in the `setup.py` for `mozlog` that were problematic for tests and was resulting in errors due to the `mozlog` plugin being loaded by `pytest` more than once. We just delete those lines and bump up the major version number of the package to fix it.
Differential Revision: https://phabricator.services.mozilla.com/D88296
We want to try to align 'push-interval' tasks to the 'backstop'. This way
we have greater confidence in our backstop pushes, and it will allow us to
simplify a lot of our backstop logic.
Depends on D89055
Differential Revision: https://phabricator.services.mozilla.com/D89056
Each file represents a new taskgraph generation in the tests. But now that these exist, we
can add new assertions to the existing files without worry.
Depends on D89054
Differential Revision: https://phabricator.services.mozilla.com/D89055
Under this model, each test file should correspond to a static set of parameters. Each
set of parameters will only ever generate the taskgraph a single time (so adding a new
file will have a perf hit, but adding new test functions to an existing file will not).
Differential Revision: https://phabricator.services.mozilla.com/D89054
The new Cranelift compiler for WebAssembly makes jittest run a bit slower, so
raise the timeout value for those tests, until performance enhancements make it
run faster again.
Differential Revision: https://phabricator.services.mozilla.com/D88973
Backstop pushes have a time component so that we can guarantee they run at least every N
hours. Prior, they measured the time since the last push. This means that as long as a new
push comes in at least once every 4 hours, the backstop would keep being pushed out.
This means we could theoretically go 80 hours without a backstop push, which makes finding
merge candidates tricky.
This change ensures we measure the 4 hours as time since last backstop, as opposed to time
since last push.
Differential Revision: https://phabricator.services.mozilla.com/D88913
Sometimes we want to conditionally add index routes to the decision task
based on a parameter. E.g, a route that tracks all the backstop pushes.
This adds a new index task specifically for the decision task, which allows
to defined new index routes within the decision task itself.
Differential Revision: https://phabricator.services.mozilla.com/D88911
I found myself needing to test a push with 'backstop=True' on try. Though, I
couldn't set it because the call to 'is_backstop' happens after the
'try_task_config' parameters are added in. This adds a simple check to handle
that case.
Differential Revision: https://phabricator.services.mozilla.com/D88910
We had thought ReferenceBrowser tests would be useful in isolating Fenix startup regressions.
However the startup of the Reference Browser is not directly comparable to Fenix and so we have not used these tests.
Differential Revision: https://phabricator.services.mozilla.com/D88666
This patch removes all raptor power-usage-only tests from CI. It also removes all resource usage testing done on raptor from Fenix.
Differential Revision: https://phabricator.services.mozilla.com/D88516
Older versions of kaniko didn't handle $PATH correctly in `RUN`
commands, and we worked around this by using full paths for the
executables. Now that the base image builder is upgraded to kaniko 1,
we can remove those workarounds.
Differential Revision: https://phabricator.services.mozilla.com/D88932
This changes most of our automation builds to clang 11.0.0 rc2.
Not included:
* code coverage builds, per bug 1660341
* mingw builds, which have traditionally been on their own update cadence, and in this case are blocked anyway by bug 1658632
This will leave some unused clang-9 task definitions. I intend to clean them up, but at a later date. For now I want to focus on making sure this update sticks, since patches like this have a tendency to bounce.
Differential Revision: https://phabricator.services.mozilla.com/D88313
`flake8`, `black`, and `pylint` are all installed in `virtualenv`s when running `mach lint`, so installing these packages globally on image setup isn't necessary.
Differential Revision: https://phabricator.services.mozilla.com/D87904
We want to be able to retroactively tell whether a push was a backstop or not.
This patch stores whether or not a push was a "backstop" directly in the
parameters. The optimization strategy now simply returns 'not
params["backstop"]'.
For simplicity, I'm not counting the 'optimized-backstop' as a backstop. It's
unclear if we'll want to be able to detect these types of the pushes in the
future or not, but we can cross that bridge when we get there.
Differential Revision: https://phabricator.services.mozilla.com/D88151
In the past, the 'backstop' optimization was applied to tasks by default across
all projects, even though it only really made sense on autoland. To choose what
would happen on non-autoland branches, we invented this 'remove_on_projects'
concept.
These days, we only apply the backstop optimization in the first place for
autoland. So 'remove_on_projects' is no longer necessary.
Depends on D88149
Differential Revision: https://phabricator.services.mozilla.com/D88150
This patch cleans up some of the backstop strategy names. Specifically:
1. Rename 'full-backstop' -> 'backstop'. The old 'backstop' algorithm was
unused anyway, so there is no conflict. It is also just defined directly in
the decorator rather than using 'Alias'.
So now rather than 'full-backstop' and 'optimized-backstop', it's just
'backstop' and 'optimized-backstop'.
2. Remove 'backstop-X-hours-Y-minutes' strategies, and replace them with
the corresponding 'push-interval-X' strategy.
This means we lose the time component in the 'optimized-backstop'. But it isn't
a problem, because we shouldn't be using a time component there at all anyway
(we should just use it with the 'backstop').
Differential Revision: https://phabricator.services.mozilla.com/D88149
Using nested Any generates deeply nested non-trivial schemas because we
also end up nesting optionally_keyed_by's, and the addition of one field
in one optionally_keyed_by in bug 1657769 made the whole schema
compilation step an order of magnitude slower.
Even before bug 1657769, the schema compilation step was slow for tests,
and with this change, it's virtually instantaneous.
Differential Revision: https://phabricator.services.mozilla.com/D88543
Certain tasks are meant to perform follow-up work to another task. For example,
a 'signing' task signs an artifact from a dependency task. An 'upload-symbols'
tasks uploads an artifact from a dependency task.
In general, we only want to run these types of tasks when the dependency task
is running. But in practice, these tasks often cause the dependency to get
pulled in.
We've tried to fix this in the past by making the 'run-on-projects' and
'optimization' keys match their primary dependency. But it's very easy to
mess this up, and some optimizations (like bugbug) take the content of the
task definition into account. So even if both tasks use the same optimization,
they could have different results.
This revision adds a new 'if-dependencies' key that denotes a task that should
only run if one of the dependencies specified in this list is also run. It
will allow us to stop playing whack-a-mole when trying to make sure tasks
aren't being pulled in solely due to these types of dependencies.
This feature is implemented independently of the 'optimization', so tasks that
specify this can still be optimized even if their primary dependency does run.
Differential Revision: https://phabricator.services.mozilla.com/D80508
Edges are defined as a (left, right, name) tuple, which means task left depends on task right using an edge called name. Previously the optimization code never took dependencies into account, so the mock graph generator never bothered to create them. But in the child revision, we start using these values, so the generation should set it up appropriately.
I had to make it conditional due to the way the assertions in the test_get_subgraph test are set up. I figured it was easier to do that than to re-write the assertions.
Depends on D88483
Differential Revision: https://phabricator.services.mozilla.com/D88484
This changes most of our automation builds to clang 11.0.0 rc2.
Not included:
* code coverage builds, per bug 1660341
* mingw builds, which have traditionally been on their own update cadence, and in this case are blocked anyway by bug 1658632
This will leave some unused clang-9 task definitions. I intend to clean them up, but at a later date. For now I want to focus on making sure this update sticks, since patches like this have a tendency to bounce.
Differential Revision: https://phabricator.services.mozilla.com/D88313
This is a fork of the release-partner-repack-beetmover kind and transform. It's modified to cope with having one upstream task with many partner builds, rather than a many beetmover tasks dealing with a single config-platform-locale combination.
Differential Revision: https://phabricator.services.mozilla.com/D87730
A single task is created to do all partner attributions. The partner_attribution transform processes the configuration into an environment variable for the tools/attribution/attribute.py script to use. This is quite verbose so a large number of configurations may cause problems.
Applies the same priority modification to attribution tasks as to partner repacks, to not impede the main part of the graph.
Differential Revision: https://phabricator.services.mozilla.com/D87729
The partner attribution config is stored in the same repository as the repo manifest for partner repacks, but all in attribution_config.yml instead of default.xml. This extends the existing support for using the Github API to read files to retrieve and process the attribution config.
Differential Revision: https://phabricator.services.mozilla.com/D87728
Renames the release_enable_partners parameter to release_enable_partner_repack, and adds release_enable_partner_attribution for attribution. This it to provide support for disabling them independently in main releases, and in respins.
Adds docs for attribution, update docs for repacks.
Hardwire values for the enable params for the respin flavors, other wise read from the input (defaulting to on in promotion, off otherwise).
Fixes up the rebuild-kinds for partner repacks so that they reflect the current set, although the top level may be all that is needed.
Differential Revision: https://phabricator.services.mozilla.com/D87727
This improves the integrity of downloads of upstream artifacts when using fetch-content. If `verify-hash: True` is set on the fetch config, then the chain-of-trust.json of the upstream is used to retieve the expected sha256 of the artifact, and this is checked.
Differential Revision: https://phabricator.services.mozilla.com/D87725
Prior to this patch the task graph would always include a release-partner-repack-<platform> task for all 6 platforms, regardless of what was specified in release_partner_config. This was particularly obvious in the off-cycle respin scenario when a single partner is repacked. By moving and reusing get_repack_ids_by_platform() it's easy to skip unneeded platforms.
Differential Revision: https://phabricator.services.mozilla.com/D87724
This is a fork of the release-partner-repack-beetmover kind and transform. It's modified to cope with having one upstream task with many partner builds, rather than a many beetmover tasks dealing with a single config-platform-locale combination.
Differential Revision: https://phabricator.services.mozilla.com/D87730
A single task is created to do all partner attributions. The partner_attribution transform processes the configuration into an environment variable for the tools/attribution/attribute.py script to use. This is quite verbose so a large number of configurations may cause problems.
Applies the same priority modification to attribution tasks as to partner repacks, to not impede the main part of the graph.
Differential Revision: https://phabricator.services.mozilla.com/D87729
The partner attribution config is stored in the same repository as the repo manifest for partner repacks, but all in attribution_config.yml instead of default.xml. This extends the existing support for using the Github API to read files to retrieve and process the attribution config.
Differential Revision: https://phabricator.services.mozilla.com/D87728
Renames the release_enable_partners parameter to release_enable_partner_repack, and adds release_enable_partner_attribution for attribution. This it to provide support for disabling them independently in main releases, and in respins.
Adds docs for attribution, update docs for repacks.
Hardwire values for the enable params for the respin flavors, other wise read from the input (defaulting to on in promotion, off otherwise).
Fixes up the rebuild-kinds for partner repacks so that they reflect the current set, although the top level may be all that is needed.
Differential Revision: https://phabricator.services.mozilla.com/D87727
This improves the integrity of downloads of upstream artifacts when using fetch-content. If `verify-hash: True` is set on the fetch config, then the chain-of-trust.json of the upstream is used to retieve the expected sha256 of the artifact, and this is checked.
Differential Revision: https://phabricator.services.mozilla.com/D87725
Prior to this patch the task graph would always include a release-partner-repack-<platform> task for all 6 platforms, regardless of what was specified in release_partner_config. This was particularly obvious in the off-cycle respin scenario when a single partner is repacked. By moving and reusing get_repack_ids_by_platform() it's easy to skip unneeded platforms.
Differential Revision: https://phabricator.services.mozilla.com/D87724
This changes most of our automation builds to clang 11.0.0 rc2.
Not included:
* code coverage builds, per bug 1660341
* mingw builds, which have traditionally been on their own update cadence, and in this case are blocked anyway by bug 1658632
This will leave some unused clang-9 task definitions. I intend to clean them up, but at a later date. For now I want to focus on making sure this update sticks, since patches like this have a tendency to bounce.
Differential Revision: https://phabricator.services.mozilla.com/D88313
This patch fixes an issue in the custom-v8 toolchain where there is tooling missing that is needed to build v8 (the libtinfo5 package).
Differential Revision: https://phabricator.services.mozilla.com/D88310
From mwoodrow:
bcv is 'basic compositor video', which explicitly disables acceleration using a
pref and then runs some video tests. For normal -qr platforms this isn't
useful, since the pref would disable WebRender, and it'd be an identical
configuation to the non-qr variant.
When software webrender is enabled, disabling acceleration won't disable WR,
and test remains useful (testing video performance of the software webrender
configuration).
The baseline I want to compare against here is the non-qr bcv, to see if
webrender-software is worse than layers-software.
Differential Revision: https://phabricator.services.mozilla.com/D88192
It's often useful to nest 'by-*' statements in the CI configs. However, calling
'resolve_keyed_by' will keep drilling down and resolving these statements all
at once. Normally this is fine, but sometimes we may want to use, e.g 'by-foo'
when 'foo' hasn't been resolved yet (i.e, it is being set by a later transform
or something like that).
Currently, this means that we'll hit the 'default' case as 'foo' doesn't exist.
This revision adds the ability to 'defer' evaluation of 'foo' until a later
time.
While running into this scenario might be a code smell, re-organizing
transforms to untangle these kinds of ordering dependencies will be a major
undertaking. Plus, there are legitimate reasons we might want one kind of
ordering with certain tasks, then another ordering for other tasks.
Differential Revision: https://phabricator.services.mozilla.com/D88191
This adds toolchain definitions for clang 11.0.0 rc2, so that developers can get a sneak peek, but nothing in automation uses these tasks yet. We'll make the switch in a later patch.
NB: most of `clang.yml` is rote copy-paste, except for `macosx64-clang-11` which makes a deliberate departure, described in a comment.
Differential Revision: https://phabricator.services.mozilla.com/D88189
Also reflects the move from CraneStation to the WebAssembly account. And we need a small tweak to the build script to accommodate one of the changes that got picked up along the way.
Differential Revision: https://phabricator.services.mozilla.com/D88187
This avoids a set of intermittent issues related to `zstd` decompression failures, which in the absence of these changes break the entire build.
This also requires [updating an environment variable](https://github.com/mozilla/sccache/pull/822), which we do in `client.mk` as well as documentation.
Differential Revision: https://phabricator.services.mozilla.com/D88184
In Bug 1466660, we started deleting the fetches after a task had run, to avoid
interference between tasks. It turns out the only tasks this was for were the
`source-test-jsshell` tasks, which were changed to use an absolute directory in
Bug 1465181. However, since Bug 1568460 we've always used a per-task directory
for fetches, so can remove the work-around of removing fethes.
Differential Revision: https://phabricator.services.mozilla.com/D86670
We define a value `run.workdir` to point to the defaut directory to use for
all sorts of paths, including checkouts, in the job transform. However, that
directory only makes sense for docker-worker tasks, and relative paths should
be used elsewhere. Adjust the paths on non-docker-worker linux tasks to match
macOS paths.
Differential Revision: https://phabricator.services.mozilla.com/D86668
Bogdan Tara
Gregory Mierzwinski
Aki Sasaki
Andrew Halberstadt
Raphael Pierzina
Andrew Halberstadt
Jan de Mooij
Sylvestre Ledru
Mitchell Hentges
Dorel Luca
Marco Castelluccio
Tom Ritter
Dzmitry Malyshau
James Graham
Narcis Beleuzu
Joel Maher
Andrew Creskey
Razvan Maries
Michael Froman
David Major
Justin Wood
Tarek Ziadé
Csoregi Natalia
Mihai Tabara
Mihai Alexandru Michis
Kimberly Sereduck
Kartikaya Gupta
Daniel Holbert
Ricky Stewart
stransky
alexandru.ionescu
Rob Lemley
Doug Thayer
june wilde
Michael Pobega
Ben Hearsum
Cristina Coroiu
Andreea Pavel
Geoff Brown
Mike Hommey
Benjamin Bouvier
Jeff Gilbert
Emil Farisan
Nick Thomas
Butkovits Atila
Florin Strugariu
James Willcox
Tom Prince