Remove duplicate OSX 10.14/10.15 jobs and run on 10.15. Also setup for proper taskgrpah configs for easier migration of unittest suites via test-sets.yml
Differential Revision: https://phabricator.services.mozilla.com/D107467
Switching linux64-ccov builds to clang from gcc in bug 1499663 uncovered some performance problems that lead to test timeouts. These are addressed in clang 12. In order to ease the starring burden on sheriffs, let's give those builds a sneak preview of clang 12 even though general builds are not yet switching over.
Differential Revision: https://phabricator.services.mozilla.com/D107104
This check was likely added to try and limit the types of tasks that can be
created with fission. However it doesn't make sense to be filtering tasks based
on the project during the transforms stage. Tasks filtered out here don't exist
at all, so it's not possible to even schedule them on try with --full. This
type of filtering should be left to the target tasks stage of generation.
As a side effect, this patch enables the following tasks on autoland:
> test-linux1804-64-qr/debug-mochitest-webgpu-fis-e10s
> test-linux1804-64-qr/opt-web-platform-tests-print-reftest-fis-e10s
> test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-1
> test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-2
> test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-3
> test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-4
> test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-5
> test-linux1804-64-qr/opt-web-platform-tests-reftest-fis-e10s-6
> test-linux1804-64/opt-marionette-fis-e10s
> test-linux1804-64/opt-marionette-headless-fis-e10s
> test-windows10-64-qr/opt-web-platform-tests-print-reftest-fis-e10s
> test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-1
> test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-2
> test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-3
> test-windows10-64-qr/opt-web-platform-tests-reftest-fis-e10s-4
> test-windows10-64/opt-marionette-fis-e10s
And the following tasks on central:
> test-linux1804-64-qr/debug-mochitest-webgpu-fis-e10s
> test-linux1804-64/debug-mochitest-webgpu-fis-e10s
While this change would ideally happen in a separate commit, fission team
indicated it was desirable to enable these tasks anyway, so I decided not
to spend effort disabling them here, only to enable them again later.
Depends on D107113
Differential Revision: https://phabricator.services.mozilla.com/D107114
This task gets enabled as a side effect of the last patch in this stack. This
patch preserves the status quo.
Depends on D107112
Differential Revision: https://phabricator.services.mozilla.com/D107113
These -profiling tasks are not currently running on fission. But the last patch in this stack
enables them as a side effect. This patch preserves the status quo.
Depends on D107107
Differential Revision: https://phabricator.services.mozilla.com/D107112
All the other browsertime tasks ignore non-shippable platforms except for this
one. It was causing problems for a later patch in this stack.
Differential Revision: https://phabricator.services.mozilla.com/D107107
Just adding the toolchain tasks for now. They are not yet used by anything, but bug 1693288 might be interested eventually.
As with earlier releases, clang-tidy and mingw builds are not included here, they'll be separate patches.
Differential Revision: https://phabricator.services.mozilla.com/D106554
Bug 1694775 had to get a fixup to install the rename utility, that is
not in the base Debian image for buster, while it was there for jessie.
However, we only use rename for a hack when cross-building geckodriver
for Windows, and we don't need to.
Differential Revision: https://phabricator.services.mozilla.com/D106875
As of that bug, the default docker image for toolchain builds is
deb10-toolchain-build, so setting the docker image in the task is
redundant.
Differential Revision: https://phabricator.services.mozilla.com/D106874
This adds a `mach wpt-fission-regressions` command that uses the wpt
expectation data to look for tests which have a worse result in
fission. With the `--all-json=<path>` argument it will output a JSON
file containing details of all the regressions. With the
`--untriaged=<path>` argument it will output a file containing a list
of failures that have not yet been triaged.
It also adds a try job to produce those files as artifacts whenever
wpt metadata is changed.
The actual implementation is based on reading the wpt expectation data
with sample run_info values corresponding to the configurations in
which we have fission enabled, but with the "fission" property set to
False (to get a baseline result) and True (to get a with-fission
result) and then comparing the resulting expectations.
The implemenation is pretty suboptimal performance wise since we end
up reading the metadata once per configuration i.e. 6 times, and this
is slow. It could be optimised by using the conditional metadata
backend, reading it once, and then evaluating per
configuration. However that would require a little more work and the
presumption is that this will be shortlived until fission becomes the
default configuration.
Differential Revision: https://phabricator.services.mozilla.com/D106954
This patch switches us from using the Firefox window reorder to the ffmpeg recorder because of some issues that give bad visual metrics values.
Differential Revision: https://phabricator.services.mozilla.com/D106785
Because we use sysroots, this makes no difference to the shipped builds.
This would normally allow to remove the debian8-amd64-build docker image,
but we need to keep it for the valgrind jobs for now.
Differential Revision: https://phabricator.services.mozilla.com/D106404
The mozjs_sys crate is actually responsible for all the problems
encountered with the js crate we just removed, but the sm-mozjs-crate
task that builds and runs tests for it doesn't fail because... there is
actually no test to build and run, so all it builds is a static library,
which doesn't expose all the problems that exist.
The content of this crate in mozilla-central is also outdated compared
to servo upstream.
Differential Revision: https://phabricator.services.mozilla.com/D106390
The sm-rust-bindings task that builds and runs tests for it actually only
doesn't fail by three(!) distinct happenstances of chance:
- It somehow doesn't hit a race condition that can happen, but that
attempting to change the build process for jsglue.cpp did,
- The second call to `cargo test` without `--feature debugmozjs` only
works because nothing was cleaned up from the first and the second
build uses artifacts from the first that are otherwise not there if
only running the second.
- The addition of rust code to spidermonkey (the jsrust crate in
js/src/rust) somehow didsn't cause problems in the current
configuration, but in other configurations (which fixing the second
thing above unveils) the build fails because of the multiple
definition of rust_eh_personality between the jsrust staticlib and the
rust stdlib that rustc links to test programs.
On top of all the above, the content of the crate in mozilla-central
has not received updates from upstream servo in years, and doesn't
reflect what servo actually uses.
Differential Revision: https://phabricator.services.mozilla.com/D106389
Now that all builds use sysroots, we:
- don't need to install -dev packages,
- don't need multi-arch packages,
- don't need workarounds for partial multi-arch awareness,
- however need a few packages that were installed as indirect
dependencies.
While here, we haven't really needed autoconf2.13 since bug 1663863
(except for one job, which switched in bug 1694784)
Differential Revision: https://phabricator.services.mozilla.com/D106357
Initially, they were using a debian7-amd64-build image, when linux64 were
the only 3-step PGO builds. There wasn't really a profound reason not to
use another image. Then later bug 632954 and bug 1561147 added android
and linux32, using desktop1604-test images, the test image back then,
with no rationale given as to why using different image than builds, but
there's also no reason not to use the test images.
As we're going to remove things from the build images that will probably
prevent the profiling step to run, we might as well use the test image
(now ubuntu1804-test) everywhere.
Differential Revision: https://phabricator.services.mozilla.com/D106356
We don't need the valgrind package for the valgrind.h header anymore,
because it's in the sysroot, and we only needed the package in the build
docker image because of the header. We still do need it in the valgrind
build image, because we run valgrind in the builds using that image.
The valgrind build image Dockerfile doesn't need an update because
valgrind will be pulled through the install of valgrind-dbg that already
happens there.
Differential Revision: https://phabricator.services.mozilla.com/D106355
Now that all builds are using a sysroot, they don't need a specifically
different docker image to cross-build from x86-64.
This also means we don't need to build 32-bits specific packages
anymore, so we don't need the docker image to build them either
(although in theory, we might need it if we even need to build packages
for the sysroot later on, but we'll be able to dig this out if we
actually come to that).
Subsequently, this means we don't need the debian8-i386-raw image either.
Differential Revision: https://phabricator.services.mozilla.com/D106354
While here, remove the -dev packages we have in the toolchain sysroot
and thus don't need in the docker image anymore.
Two exceptions: gcc and binutils need to stay on the older docker image,
at least for now.
Differential Revision: https://phabricator.services.mozilla.com/D106325
While not strictly necessary for CI on the longer term (because builds
will also run on a newer docker image soon), for transition purposes, we
need the toolchain to keep running on old systems.
However, eventually, we'll want libdmg-hfsplus to be a local-toolchain,
so it would need to use a sysroot in the long run anyways.
Differential Revision: https://phabricator.services.mozilla.com/D106323
This will make the diffs both easier to review, and smaller; the latter
should avoid bumping into arc's hardcoded 4MB size limit for diffs.
Differential Revision: https://phabricator.services.mozilla.com/D106774
This patch decreases the test timeout for some Talos tests which are hitting task timeouts. It also increases the task timeout for one of the tests.
Differential Revision: https://phabricator.services.mozilla.com/D106638
Use python3 in more places so we don't have to explicitly install
the python 2.x package.
The newer version of curl in 20.04 will hopefully fix the intermittent
http2 framing errors we've been seeing recently.
Differential Revision: https://phabricator.services.mozilla.com/D106259
ubuntu focal comes with mercurial 5.3.1 which is new enough for us right
now, so install it directly from the distro instead of fetching the
older 4.8.1 from tooltool.
Differential Revision: https://phabricator.services.mozilla.com/D106258
Joel Maher
David Major
Gregory Mierzwinski
Dave Hunt
Andrew Creskey
Mike Hommey
Andrew Halberstadt
James Graham
Jens Stutte
Cosmin Sabou
Andrew Osmond
Ryan VanderMeulen
Andi-Bogdan Postelnicu
Csoregi Natalia
Noemi Erli
Julien Cristau