This commit adds a config flag '--wasm-no-experimental' which disables all
experimental features that have a conditional compile flag. This flag has
a higher precedence than the manual enable flags such as '--enable-wasm-gc'.
This commit then also adds a try task to run with this configuration on
linux64debug. It also runs jit-tests to cover all the wasm tests, but the
important thing is testing that the build succeeds.
Differential Revision: https://phabricator.services.mozilla.com/D166375
These are pretty redundant right now, and not even consistent with what we do with other chunked things (such as tests).
Differential Revision: https://phabricator.services.mozilla.com/D170114
Minimize diff noise when comparing taskgraphs written as JSON files with -J.
json.dumps() has sort_keys=True, but that does not affect lists.
Differential Revision: https://phabricator.services.mozilla.com/D170020
This is hardcoded to point at tip of mozilla-central instead of the revision
corresponding to the actual build, for simplicity's sake.
Thanks to barthalion for the tip.
Differential Revision: https://phabricator.services.mozilla.com/D169838
We already support this when it's a string, we ought to when it's a list as well. This is particularly useful when you have a job or task template in a kind, and need to be able to append to the command in concrete jobs, while still using command-context.
Differential Revision: https://phabricator.services.mozilla.com/D169847
Two main issues I've found:
1) release_promotion_flavor is required, but has no default, so it doesn't show up in the default list of fields
2) The decision task ended up failing because partner repacks were enabled, but I didn't provide partner repack config. We should default these to disabled (any production use case explicitly sets them, so this should be harmless and no-opt to those).
Differential Revision: https://phabricator.services.mozilla.com/D169639
Locales in the flatpak desktop files have the language and the country
separated by a hyphen, but it should be separated by underscore.
"-mac" in Japanese locale is also removed according to bug 418485.
Spanish translation is added for the action to open the profile manager.
Differential Revision: https://phabricator.services.mozilla.com/D169220
This patch disables most of the android perftests on the toolchain branch. After this patch, only 1 pageload test, and 1 benchmark test will run on android on that branch.
Differential Revision: https://phabricator.services.mozilla.com/D168684
In the time since we last bumped MinGW many of our patches
no longer apply.
The enum patch had one of the fixes incorporated, but not the
other.
The winrt patch added several files that were added to MinGW -
but not all the classes we need were added, so I've copied them
into a new patch in a subsequent commit.
While we're here, also add the mingw patches as resources so
they trigger rebuilds when edited.
Differential Revision: https://phabricator.services.mozilla.com/D168831
Instead of excluding android builds based on the `project` parameter, use
`release_type`, which is properly set to e.g. `esr102` by `mach try release`.
Differential Revision: https://phabricator.services.mozilla.com/D168563
The hardcoded use of libtool was removed upstream in 2019, replaced with
the use of AR, so we can drop the related hack.
Differential Revision: https://phabricator.services.mozilla.com/D168036
These flags used to be necessary when we first were cross-compiling
clang, but more recent (although now old) changes have made them
actually unnecessary.
Differential Revision: https://phabricator.services.mozilla.com/D168033
We have not removed the no-eft mode yet, so we want to keep the variant alive.
We will try to pay attention to potential breakage with this job during the next 6 months,
and if we see that it didn't detect any implementation issue, we might remove it when we
reach the next deadline.
Differential Revision: https://phabricator.services.mozilla.com/D168529
The hardcoded use of libtool was removed upstream in 2019, replaced with
the use of AR, so we can drop the related hack.
Differential Revision: https://phabricator.services.mozilla.com/D168036
These flags used to be necessary when we first were cross-compiling
clang, but more recent (although now old) changes have made them
actually unnecessary.
Differential Revision: https://phabricator.services.mozilla.com/D168033
These tasks run basically once a month, so having them expire after 3
months means we're left with very little history to try and investigate
trends/issues.
Differential Revision: https://phabricator.services.mozilla.com/D168280
This patch adds a new test for our regression detection system. It produces a single constant value that we can easily change in the test manifest to trigger an alert. This is being done to in preparation for work on the entire alerting system, and to test if some unimplemented fields in the harness work (min_back_window, max_back_window, and fore_window). They all exist in the performance artifact schemas in mozilla-central, and the treeherder repo. I've also disabled the push/backstop optimizations for the regression tests so that they run on each push. It's limited to running on linux machines.
Differential Revision: https://phabricator.services.mozilla.com/D168116
At the moment, they all run for every type of change under
python/mozbood/mozboot. But a change to e.g. debian.py doesn't require a
test on a non-Debian-based distro.
Differential Revision: https://phabricator.services.mozilla.com/D167892
Bump the number of retries from 5 (5+10+20+40=75 seconds worst case) to 7 (5+10+20+40+80+160=315 seconds worst case).
Also log the url being used for the first fetch, in case that helps diagnose future failures.
Differential Revision: https://phabricator.services.mozilla.com/D167704
Updates several tests results for accelerated canvas. Affected tests are set as skip to handle both enabling/disabling accelerated canvas. They could be updated after accelerated canvas is enabled until release on Android.
Some skipped reftest failures could be addressed by updating Android emulator to more than or equal to 25.
On Android Emulator, accelerated canvas is set as disabled with mochitest-media "Mochitest media run" and web-platform-tests-wdspec "Web platform webdriver-spec run". Android Emulator does not use hardware GPU. Accelerated canvas handling becomes slow. Then these tests were failed with TIMEOUT.
Differential Revision: https://phabricator.services.mozilla.com/D166515
* Use enumerate(...) instead of manual indexing
* It's safe to modify a list we're iterating on as long as we break out
of the iteration right after the modification.
Differential Revision: https://phabricator.services.mozilla.com/D165053
Basic loop invariant code motion. The impact on execution time is not
totally negligible due to the number of tests involved.
Differential Revision: https://phabricator.services.mozilla.com/D165052
This module defines a single, efficient function to deepcopy a task. It
is faster than deepcopy because it doesn't need to track cycles and
duplicate references that don't make sense for tree (and not graph)
structures.
I measure a speedup > 10% on mach taskgraph tasks --fast >/dev/null.
Differential Revision: https://phabricator.services.mozilla.com/D164789
Firstly, python hasn't pointed to python2.6 in a long while. Secondly,
the gn build script has switched to python3 a year and a half ago.
Differential Revision: https://phabricator.services.mozilla.com/D166511
None of the few remaining things that are still using pytoml are
expected to be running on this docker image, let alone outside mach.
Differential Revision: https://phabricator.services.mozilla.com/D166509
A side effect of switching to `cargo install` in bug 1806960 is that
Cargo.lock is ignored, which has undesirable consequences, such as
pulling new dependencies that might not compile because e.g. they
require a newer version of rust.
Differential Revision: https://phabricator.services.mozilla.com/D166400
This is _mostly_ boring, standard stuff. The most notable part is the refactor of the test transforms to process `variant` as early as possible. This is needed to key `suite` by variant (and is likely going to be helpful in other places as well, eg: we can now key all `mozharness` keys on variant). See the previous patch for why we need a separate xpcshell suite for this.
I don't know if I'm adding the runtime manifest correctly, or how those times are supposed to be generated/updated (the commit log seems to suggest that it's broken right now?) - let me know if I should do something differently here.
Note: I'm still working through a few more tests that need to be disabled, so this isn't ready to be landed yet, but I don't anticipate any changes to this patch.
Differential Revision: https://phabricator.services.mozilla.com/D164230
This patch updates the S3 commit used, fixes an issue with local Windows benchmark runs, and enables the windows S3 tests in CI.
Differential Revision: https://phabricator.services.mozilla.com/D166323
* Use enumerate(...) instead of manual indexing
* It's safe to modify a list we're iterating on as long as we break out
of the iteration right after the modification.
Differential Revision: https://phabricator.services.mozilla.com/D165053
Basic loop invariant code motion. The impact on execution time is not
totally negligible due to the number of tests involved.
Differential Revision: https://phabricator.services.mozilla.com/D165052
This module defines a single, efficient function to deepcopy a task. It
is faster than deepcopy because it doesn't need to track cycles and
duplicate references that don't make sense for tree (and not graph)
structures.
I measure a speedup > 10% on mach taskgraph tasks --fast >/dev/null.
Differential Revision: https://phabricator.services.mozilla.com/D164789
This patch adds the speedometer 3 benchmark test to raptor enables it for CI. In CI, it'll start by running on Linux, and MacOSX on Firefox, Chrome, and Chromium. The patch also adds a new method for parsing benchmark tests that is more generic and works in conjunction with the `custom_data` option.
Differential Revision: https://phabricator.services.mozilla.com/D165988
These jobs often run very close to the 60min max and are prone to random
timeouts as a result. We can't chunk this suite at the moment, so increasing
the max runtime is the only real option we have.
Differential Revision: https://phabricator.services.mozilla.com/D166109
This patch moves the benchmark setup logic out of the mach_commands.py file and into the Benchmark class in Raptor that is built to handle our benchmark tests. At the same time, the code is completely reworked to be simpler to understand, and to use pathlib instead of os for path manipulations. The existing benchmark tests that use code from our perf-automation repository are then modified to make use of this new method (fetch methods are also removed). The tests which exist in-tree don't need modifications.
Differential Revision: https://phabricator.services.mozilla.com/D164365
In mozharness, setting --download-symbols=True or --download-symbols=ondemand
triggers an attempt to access the symbols url (either to download them or just
verify that they exist). asan and tsan builds only have the special "full"
symbols, which are never used by mozharness. Removing the --download-symbols
option from the command line avoids the url check entirely, which avoids
retries, warnings, and errors in the log.
Differential Revision: https://phabricator.services.mozilla.com/D165921
As of bug 1806960, all the rust toolchains are built the same way, and
they all use lld, so they don't need binutils anymore.
Differential Revision: https://phabricator.services.mozilla.com/D165659
Because this involves affecting the MSVC + Windows SDK packs, we need to
do this for toolchain tasks as well as Firefox builds in a single pass.
The MSVC + Windows SDK packs are altered to keep the original case
instead of lowercasing everything (except .lib files), and contain
an overlay file that we now automatically use from the WINSYSROOT.
This requires adjusting some paths to match what the original case
is, as well as removing everything related to the use of liblowercase,
which conflicts with the use of the overlay file. People using
liblowercase locally will still have a working setup as long as they
don't set WINSYSROOT (it's new, so they won't have it), and don't have
an overlay file in there (which they only would if they ran the new
pack_vs.py).
Differential Revision: https://phabricator.services.mozilla.com/D165596
We leave out liblowercase and geckodriver because
- their source is in-tree, which is different from the others
- liblowercase will eventually be removed
- geckodriver has different needs wrt its artifacts
We switch to using cargo install instead of cargo build, because it
gives us strictly what we want, as opposed to pulling things off
target/release, which currently ends up copying .d files.
Differential Revision: https://phabricator.services.mozilla.com/D165381
Convert all other imports of `pytoml` within mozilla-central to imports
of `toml`.
As both libraries patterned their API on that of the Python standard-
library module `json`, this is mostly a straight replace, with two
caveats:
* pytoml's exception messages when loading a file provide the file
name; toml's do not. Some additional machinery has been added or
repurposed in a few places to ensure that the relevant filename is
printed.
* In `python_lockfile.py`, the order of arguments to `dump` needed to
be reversed. (`toml` follows `json` in this; `pytoml` differs.)
This patchset does not remove pytoml from `requirements.in`, as `pytoml`
is still used by `compare-locales`.
Differential Revision: https://phabricator.services.mozilla.com/D164155
The `pytoml` library is abandoned, and does not support TOML v0.5.0,
which Cargo uses. This prevents mach from properly parsing many
Cargo.toml files.
Include the `toml` library as a slightly more modern and currently-
maintained alternative for use in mach.
Supply-chain notes:
- toml v0.10.2 is MIT-licensed, which is an approved MPL-compatible
license.
- I have personally audited this version of `toml` and consider it
"safe-to-run", as much as any Python code can be: other than opening
and reading/writing a file when specifically requested via `load`/
`dump` respectively, it performs only string manipulation and
ordinary object creation.
Differential Revision: https://phabricator.services.mozilla.com/D164150
We sign autoland builds with dep certificates. MSIX packages require some of their metadata set in a way that matches the signing cert, which is currently not done correctly for autoland.
Differential Revision: https://phabricator.services.mozilla.com/D165037
This patch adds a test that triggers a benchmark test on the Grandprix website (live). The patch also adds the CI tasks for this test to firefox, chrome, and chromium. The firefox tasks run on autoland, and mozilla-central. Some new code is added to flatten a custom data type if requested. This allows us to easily comb large datasets and organize them into our replicates as long as the key paths match. We remove the "iterations", and "tests" strings as they are unnecessary for unique pathing and we do this with a new test field called `subtest_metric_filters`. The `test_summary` field lets you turn the flattening on and off since some tests don't need this.
Differential Revision: https://phabricator.services.mozilla.com/D164946
This pool runs the 'index' and 'eager-index' tasks. It is not actually a part
of any graph and doesn't show up on Treeherder.
Differential Revision: https://phabricator.services.mozilla.com/D164941
While debugging https://bugzilla.mozilla.org/show_bug.cgi?id=1805989 I noticed this seemingly unnecessary fallback branch in one of the scriptworker helpers. It appears to me that we can drop the second and third branches there, and let everything not in the first branch get handled by get_signing_cert_scope instead.
That function ultimately has its own fallback for things without explicit matches, which applies the same default as the current third branch.
I ran this through tgdiff on try, which found no changes to any tasks for any set of parameters
Differential Revision: https://phabricator.services.mozilla.com/D165022
None of the others will work either because the platform is not windows, or the build type on Windows doesn't support signing (eg: asan, ccov).
Differential Revision: https://phabricator.services.mozilla.com/D164938
Ryan Hunt
Mike Hommey
Ben Hearsum
Rob Lemley
Joel Maher
Greg Mierzwinski
Tom Ritter
Julien Cristau
Mike Conley
Barret Rennie
Marco Castelluccio
Andrew Halberstadt
William Durand
Ryan VanderMeulen
Jaime Soriano Pastor
Bobby Holley
Olek Czubiak
Alex Ionescu
Jamie Nicol
Kelsey Gilbert
Gabriele Svelto
andrej
Ed Guloien
Stanca Serban
Julian Descottes
Alexandra Borovova
ogiorgis
Geoff Brown
Ron Waldon-Howe
Nicolas Chevobbe
alwu
serge-sans-paille
sotaro
Ben Hearsum
Sebastian Hengst
Kash Shampur
Marian-Vasile Laza
Michelle Goossens
Sylvestre Ledru
Narcis Beleuzu
Sandor Molnar
Ray Kraesig
Mark Banner
Andi-Bogdan Postelnicu
Csoregi Natalia
Noemi Erli
Cristian Tuns