This makes --enable-thread-sanitizer turn on Rust tsan (-Zsanitizer=thread).
This requires changing SpiderMonkey tsan to use the tsan rust nightly.
In future changes, more Rust tsan integration will key off of MOZ_TSAN.
Differential Revision: https://phabricator.services.mozilla.com/D96453
Since e10s-multi is going to be enabled by default, this gives us the ability
to run junit in a secondary, e10s-single configuration so that we can see them
both side-by-side for a little while, at least until we're settled in with the
e10s-multi configuration.
Differential Revision: https://phabricator.services.mozilla.com/D94883
When the virtualenv is activated, subprocesses of mozharness inherit the
PATH that it sets, making plain python resolve to the virtualenv python,
which is actually not desirable for builds.
Ideally, we'd avoid creating a virtualenv for mozharness entirely,
especially considering it creates its own, and then the build creates
yet another one, but that doesn't work because of PYTHONPATH interfering
with the build virtualenv.
Differential Revision: https://phabricator.services.mozilla.com/D96168
This patch changes a few different things:
- The Docker image used for both scrapers are now based off of our standard
Debian 10 image instead of an old Ubuntu one
- Both images were changed to make it easy to work with them when an
interactive task is used
- The python packages used by the scripts had their versions pinned and
cryptographic hashes have been recorded so that they can be verified before
running the task
- The dump_syms version used on Windows was bumped to include fixes that allow
us to scrape public symbols from DLLs and EXEs that have no PDB file available
- The maximum open file limit of the Window scraper was bumped up to cope with
runs where we gather plenty of symbols in one go
- Similarly the maximum number of symultaneous connections to a single symbol
server has been limited to 4 to avoid being throttled
- The macOS Python tools were modified to work with Python 3
Differential Revision: https://phabricator.services.mozilla.com/D95853
This patch will make sure that we select the right
wheels for numpy and scipy when `--visualmetrics` is used,
so we don't need to compile them.
It also make sure we don't install all those packages
unless the environment wants to use visual metrics.
Differential Revision: https://phabricator.services.mozilla.com/D91740
This patch makes webrender-enabled testing the main method for testing and also adds a subset of tests that will without it enabled until the switch is fully complete.
Differential Revision: https://phabricator.services.mozilla.com/D95744
This patch upgrade browsertime to v10, enables webdriver-based navigation in browsertime tests, and also disables the Firefox WindowRecorder for a fairer comparison between Chrome and Firefox.
Differential Revision: https://phabricator.services.mozilla.com/D95425
This patch flips things around so that the non-QR macosx test platforms are
running a "skeleton crew" set of tests, and the QR macosx test platforms are
running all the tests that were previously running with non-QR. As of Fx83
macOS has WebRender enabled by default for 90%+ of the population so it makes
sense to test that configuration more heavily, but leave some tests running
for the non-QR configuration as well.
Differential Revision: https://phabricator.services.mozilla.com/D95430
This patch flips things around so that the non-QR macosx test platforms are
running a "skeleton crew" set of tests, and the QR macosx test platforms are
running all the tests that were previously running with non-QR. As of Fx83
macOS has WebRender enabled by default for 90%+ of the population so it makes
sense to test that configuration more heavily, but leave some tests running
for the non-QR configuration as well.
Differential Revision: https://phabricator.services.mozilla.com/D95430
The migration of the task images from Ubuntu 16.04 to Ubuntu 18.04 in
bug 1602863 added the Ubuntu version to the platform. The platform of Linux x64
DevEdition didn't get added and used the default tier 2. We don't run tests
for Linux 32-bit anymore.
Differential Revision: https://phabricator.services.mozilla.com/D95754
This decreases the scheduling algorithm from a CT_MEDIUM (0.8) threshold to
CT_LOW (0.7). It also reduces the threshold used in the manifest resolving
algorithm down to 0.5. This means we'll have more "ride-along" manifests that
happen to be in the same chunk as the more important ones.
Differential Revision: https://phabricator.services.mozilla.com/D94757
The first versions with 1.48 had a regression that was fixed later
during the 1.48.0 development phase. Use the last of those to have
something closer to a release, but still versioned 1.48.
Differential Revision: https://phabricator.services.mozilla.com/D94573
Rather than putting a top-level "mozharness" directory in "mozharness.zip",
this puts the root of mozharness at the root of the archive. Then we extract
the archive into a "mozharness" directory. This way the path on disk ends up
being the same, but generic-worker won't attempt to modify the permissions of
files in the cwd.
Differential Revision: https://phabricator.services.mozilla.com/D95138
Rather than putting a top-level "mozharness" directory in "mozharness.zip",
this puts the root of mozharness at the root of the archive. Then we extract
the archive into a "mozharness" directory. This way the path on disk ends up
being the same, but generic-worker won't attempt to modify the permissions of
files in the cwd.
Differential Revision: https://phabricator.services.mozilla.com/D95138
This moves most of the windows ccov tests to run with WebRender enabled by
putting them in the windows10-64-ccov-qr test platform. (The -qr suffix causes
the taskgraph code to enable WebRender for these jobs).
The only one skipped is the marionette GPU test because it contains a test that
specifically requires WebRender disabled.
The existing non-regex config keys that referred to `windows10-64-ccov` also
get updated, and in a couple of places we need to add a new non-regex key to
disambiguate between two matching regex keys.
Differential Revision: https://phabricator.services.mozilla.com/D94769
This moves most of the windows ccov tests to run with WebRender enabled by
putting them in the windows10-64-ccov-qr test platform. (The -qr suffix causes
the taskgraph code to enable WebRender for these jobs).
The only one skipped is the marionette GPU test because it contains a test that
specifically requires WebRender disabled.
The existing non-regex config keys that referred to `windows10-64-ccov` also
get updated, and in a couple of places we need to add a new non-regex key to
disambiguate between two matching regex keys.
Differential Revision: https://phabricator.services.mozilla.com/D94769
This allows additional suffixes (e.g. -qr) to be added to ccov test platforms,
and in most cases those labels will be treated the same as the other ccov jobs.
In a few cases exact (non-regex) keys were left unmodified, because changing
a non-regex key to a regex key can result in taskgraph generation failure, if
more than one regex key matches.
I verified that this patch does not modify the output generated by `./mach
taskgraph full --json`.
Differential Revision: https://phabricator.services.mozilla.com/D94725
As with the previous patch, the negative regex makes changes hard. This replaces
the negative regex with a positive list of matching labels, similar to the one
a few lines down. I verified that this patch does not modify the output for
`./mach taskgraph full --json`.
Differential Revision: https://phabricator.services.mozilla.com/D94724
I need to modify the ccov regexes to allow additional suffixes. The negative-
search regex makes this hard, and seems quite complex/error-prone. Instead of the
negative-search regex, we can accomplish the same thing by splitting out a
new test-set.
Note that with this patch, there are a handful of jobs that get removed from
the list generated by `./mach taskgraph full` (listed below). I believe this
is a desirable change since per the comment in the code it seems like we don't
want these jobs to run. In fact, I don't see these jobs on TreeHerder, so maybe
some other step is pruning them out anyway.
The jobs being removed are below. The '*' at the end represents the chunk number,
each of the labels with a '*' has 6 chunks which I've collapsed into a single
label for brevity.
test-linux1804-64-asan/opt-jittest-1proc-*
test-linux1804-64/debug-jittest-1proc-*
test-linux1804-64-devedition/opt-jittest-1proc-*
test-linux1804-64/opt-jittest-1proc-*
test-linux1804-64-qr/debug-jittest-1proc-*
test-linux1804-64-qr/opt-jittest-1proc-*
test-linux1804-64-shippable/opt-jittest-1proc-*
test-linux1804-64-shippable-qr/opt-jittest-1proc-*
test-windows10-64-asan/opt-jittest-1proc
test-windows10-64/debug-jittest-1proc
test-windows10-64-devedition/opt-jittest-1proc
test-windows10-64/opt-jittest-1proc
test-windows10-64-shippable/opt-jittest-1proc
test-windows7-32/debug-jittest-1proc
test-windows7-32/opt-jittest-1proc
test-windows7-32-shippable/opt-jittest-1proc
Differential Revision: https://phabricator.services.mozilla.com/D94723
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
Depends on D94045
Dave House
Alexis Beingessner
Aaron Klotz
Gregory Mierzwinski
Julian Descottes
Mike Hommey
Sylvestre Ledru
Gabriele Svelto
Tom Ritter
Tarek Ziadé
Kartikaya Gupta
Steve Fink
Matt Woodrow
Sebastian Hengst
Razvan Maries
Andrew Halberstadt
Agi Sferro
Narcis Beleuzu
Jeff Gilbert
Kershaw Chang
Ricky Stewart