Because this involves affecting the MSVC + Windows SDK packs, we need to
do this for toolchain tasks as well as Firefox builds in a single pass.
The MSVC + Windows SDK packs are altered to keep the original case
instead of lowercasing everything (except .lib files), and contain
an overlay file that we now automatically use from the WINSYSROOT.
This requires adjusting some paths to match what the original case
is, as well as removing everything related to the use of liblowercase,
which conflicts with the use of the overlay file. People using
liblowercase locally will still have a working setup as long as they
don't set WINSYSROOT (it's new, so they won't have it), and don't have
an overlay file in there (which they only would if they ran the new
pack_vs.py).
Differential Revision: https://phabricator.services.mozilla.com/D165596
We leave out liblowercase and geckodriver because
- their source is in-tree, which is different from the others
- liblowercase will eventually be removed
- geckodriver has different needs wrt its artifacts
We switch to using cargo install instead of cargo build, because it
gives us strictly what we want, as opposed to pulling things off
target/release, which currently ends up copying .d files.
Differential Revision: https://phabricator.services.mozilla.com/D165381
Convert all other imports of `pytoml` within mozilla-central to imports
of `toml`.
As both libraries patterned their API on that of the Python standard-
library module `json`, this is mostly a straight replace, with two
caveats:
* pytoml's exception messages when loading a file provide the file
name; toml's do not. Some additional machinery has been added or
repurposed in a few places to ensure that the relevant filename is
printed.
* In `python_lockfile.py`, the order of arguments to `dump` needed to
be reversed. (`toml` follows `json` in this; `pytoml` differs.)
This patchset does not remove pytoml from `requirements.in`, as `pytoml`
is still used by `compare-locales`.
Differential Revision: https://phabricator.services.mozilla.com/D164155
The `pytoml` library is abandoned, and does not support TOML v0.5.0,
which Cargo uses. This prevents mach from properly parsing many
Cargo.toml files.
Include the `toml` library as a slightly more modern and currently-
maintained alternative for use in mach.
Supply-chain notes:
- toml v0.10.2 is MIT-licensed, which is an approved MPL-compatible
license.
- I have personally audited this version of `toml` and consider it
"safe-to-run", as much as any Python code can be: other than opening
and reading/writing a file when specifically requested via `load`/
`dump` respectively, it performs only string manipulation and
ordinary object creation.
Differential Revision: https://phabricator.services.mozilla.com/D164150
We sign autoland builds with dep certificates. MSIX packages require some of their metadata set in a way that matches the signing cert, which is currently not done correctly for autoland.
Differential Revision: https://phabricator.services.mozilla.com/D165037
This patch adds a test that triggers a benchmark test on the Grandprix website (live). The patch also adds the CI tasks for this test to firefox, chrome, and chromium. The firefox tasks run on autoland, and mozilla-central. Some new code is added to flatten a custom data type if requested. This allows us to easily comb large datasets and organize them into our replicates as long as the key paths match. We remove the "iterations", and "tests" strings as they are unnecessary for unique pathing and we do this with a new test field called `subtest_metric_filters`. The `test_summary` field lets you turn the flattening on and off since some tests don't need this.
Differential Revision: https://phabricator.services.mozilla.com/D164946
This pool runs the 'index' and 'eager-index' tasks. It is not actually a part
of any graph and doesn't show up on Treeherder.
Differential Revision: https://phabricator.services.mozilla.com/D164941
While debugging https://bugzilla.mozilla.org/show_bug.cgi?id=1805989 I noticed this seemingly unnecessary fallback branch in one of the scriptworker helpers. It appears to me that we can drop the second and third branches there, and let everything not in the first branch get handled by get_signing_cert_scope instead.
That function ultimately has its own fallback for things without explicit matches, which applies the same default as the current third branch.
I ran this through tgdiff on try, which found no changes to any tasks for any set of parameters
Differential Revision: https://phabricator.services.mozilla.com/D165022
None of the others will work either because the platform is not windows, or the build type on Windows doesn't support signing (eg: asan, ccov).
Differential Revision: https://phabricator.services.mozilla.com/D164938
To help test parallel marking this patch enables it in some browser TSAN tests.
This change will be removed when it is enabled by default.
Differential Revision: https://phabricator.services.mozilla.com/D164579
I ended up making what I think is an overall improvement to build-signing: moving it out to test-platforms.yml. This was more or less necessary due to the need to key it on variant. (It would've been possible to keep in the loader, but it would've been much uglier...)
As far as I can tell, all current jobs are keeping their current dependencies correctly from a run of `./mach taskgraph target-graph -p project=mozilla-central -J --diff`
Differential Revision: https://phabricator.services.mozilla.com/D164082
clang is built using the sysroot, and not using the sysroot to build the
afl-llvm-pass plugin can lead to ABI differences that lead to crashes.
Differential Revision: https://phabricator.services.mozilla.com/D163905
As of bug 1791454 m-c and autoland both also imply the toolchains
branch, which doesn't work for every kind of task, some of which are
really meant to only run on those branches.
Differential Revision: https://phabricator.services.mozilla.com/D164149
We remove the python2 packages that are of no use, and bzr, which causes
problems because it pulls a dulwich package that conflicts with the
dulwich that pip wants to install when installing poetry. AFAIK, we have
no use for bzr.
Differential Revision: https://phabricator.services.mozilla.com/D164042
Now that we can run performance tests on Safari, this patch adds these tests to the general_perf_testing cron.
This let's us run the existing Safari live and benchmark tests at the same frequency as Chrome tests.
Differential Revision: https://phabricator.services.mozilla.com/D163987
Switches from `buildconfig.substs` to `bootstrap_toolchain` for the repackage job
Refactors mozpack/dmg to not depend on the environment; Moves path resolution of tools up to `mozbuild.action.{make_dmg|unpack_dmg}`
Update syntax to python 3.7+
Differential Revision: https://phabricator.services.mozilla.com/D163129
This patch adds the ability to run raptor performance tests (specifically benchmark and live pageload tests) with Safari on our MacOS hardware.
This will let us compare Safari to Firefox for a subset of our tests.
Bug 1802922 is a follow up to look into some of the failing tests in Safari that were disabled for this patch.
Differential Revision: https://phabricator.services.mozilla.com/D160890
This patch fixes several issues and needs to work together with matching
changes in reposado:
* The package dumper handles nested packages correctly
* Disk image files (.dmg) are handled as well as package ones (.pkg) and ZIP archives (.zip)
* Fixed some code that still used python 2 semantics
* Deleting temporary directories was made more reliable
* Use the libdmg build provided in the toolchain
Differential Revision: https://phabricator.services.mozilla.com/D137207
Switches from `buildconfig.substs` to `bootstrap_toolchain` for the repackage job
Refactors mozpack/dmg to not depend on the environment; Moves path resolution of tools up to `mozbuild.action.{make_dmg|unpack_dmg}`
Update syntax to python 3.7+
Differential Revision: https://phabricator.services.mozilla.com/D163129
We want these tests to run by default on Android Fission to get some statistics on the failures, and to let the wpt-sync bot adjust the expectations. Once the work on fixing wpt tests on Android Fission is complete, we plan to revisit this and make the tier of these tests to be default
Differential Revision: https://phabricator.services.mozilla.com/D163262
It's unclear exactly what is happening, but ensuring that gecko_taskgraph is loaded immediately after being unloaded will ensure that it is always before taskcluster_taskgraph in sys.path, which avoids the issue.
Differential Revision: https://phabricator.services.mozilla.com/D163377
# ignore-this-changeset
Output from `mach lint`:
error: problem with lint setup, skipping android-api-lint, android-checkstyle, android-format, android-javadoc, android-lint, android-test
Unable to match yaml regex against output: Site not up-to-date reason: The pthfile at "/repos/mozilla-central/obj-x86_64-pc-linux-gnu/_virtualenvs/common/lib/python3.9/site-packages/mach.pth" does not match the expected value.
Unable to match yaml regex against output: # --- on-disk pthfile: ---
Unable to match yaml regex against output: /repos/mozilla-central/build
Unable to match yaml regex against output: /repos/mozilla-central/config
Unable to match yaml regex against output: /repos/mozilla-central/config/mozunit
Unable to match yaml regex against output: /repos/mozilla-central/dom/bindings
Unable to match yaml regex against output: /repos/mozilla-central/dom/bindings/parser
Unable to match yaml regex against output: /repos/mozilla-central/layout/tools/reftest
Unable to match yaml regex against output: /repos/mozilla-central/python/l10n
Unable to match yaml regex against output: /repos/mozilla-central/python/mach
Unable to match yaml regex against output: /repos/mozilla-central/python/mozboot
Unable to match yaml regex against output: /repos/mozilla-central/python/mozbuild
Unable to match yaml regex against output: /repos/mozilla-central/python/mozlint
Unable to match yaml regex against output: /repos/mozilla-central/python/mozperftest
Unable to match yaml regex against output: /repos/mozilla-central/python/mozrelease
Unable to match yaml regex against output: /repos/mozilla-central/python/mozterm
Unable to match yaml regex against output: /repos/mozilla-central/python/mozversioncontrol
Unable to match yaml regex against output: /repos/mozilla-central/security/manager/tools
Unable to match yaml regex against output: /repos/mozilla-central/taskcluster
Unable to match yaml regex against output: /repos/mozilla-central/testing
Unable to match yaml regex against output: /repos/mozilla-central/testing/awsy
Unable to match yaml regex against output: /repos/mozilla-central/testing/condprofile
Unable to match yaml regex against output: /repos/mozilla-central/testing/firefox-ui/harness
Unable to match yaml regex against output: /repos/mozilla-central/testing/marionette/client
Unable to match yaml regex against output: /repos/mozilla-central/testing/marionette/harness
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/manifestparser
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozcrash
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozdebug
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozdevice
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozfile
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozhttpd
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozgeckoprofiler
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozinfo
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozinstall
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozleak
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozlog
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/moznetwork
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozpower
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozprocess
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozprofile
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozproxy
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozrunner
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozsystemmonitor
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozscreenshot
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/moztest
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozversion
Unable to match yaml regex against output: /repos/mozilla-central/testing/raptor
Unable to match yaml regex against output: /repos/mozilla-central/testing/talos
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform
Unable to match yaml regex against output: /repos/mozilla-central/testing/xpcshell
Unable to match yaml regex against output: /repos/mozilla-central/toolkit/components/telemetry/tests/marionette/harness
Unable to match yaml regex against output: /repos/mozilla-central/tools
Unable to match yaml regex against output: /repos/mozilla-central/tools/moztreedocs
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/ds/tools
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/geckoprocesstypes_generator
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/idl-parser
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/funcsigs
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/h2
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/hpack
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/html5lib
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/hyperframe
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/pywebsocket3
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/webencodings
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/wptserve
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/wptrunner
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/aiohttp
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ansicon
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/appdirs
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/async_timeout
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/attrs
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/blessed
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cbor2
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/certifi
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/chardet
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/click
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/colorama
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/compare_locales
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cookies
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cram
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/diskcache
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/distro
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/dlmanager
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ecdsa
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/esprima
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/fluent.migrate
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/fluent.syntax
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/giturlparse
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/gyp/pylib
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/idna
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/idna-ssl
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/importlib_metadata
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/iso8601
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/Jinja2
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jinxed
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jsmin
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/json-e
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jsonschema
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/looseversion
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/MarkupSafe/src
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mohawk
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mozilla_repo_urls
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mozilla_version
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/multidict
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/packaging
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pathspec
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pip
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pip_tools
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ply
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyasn1
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyasn1_modules
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pylru
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyparsing
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyrsistent
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/python-hglib
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pytoml
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/PyYAML/lib3
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/redo
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/requests
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/requests_unixsocket
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/responses
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/rsa
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/setuptools
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/sentry_sdk
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/six
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/slugid
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster_taskgraph
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster_urls
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/tqdm
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/typing_extensions
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/urllib3
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/voluptuous
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/wcwidth
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/wheel
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/yamllint
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/yarl
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/zipp
Unable to match yaml regex against output: /app/.mozbuild/srcdirs/mozilla-central-b9868e2b6607/_virtualenvs/mach/lib/python3.9/site-packages
Unable to match yaml regex against output: import sys; sys.path = [p for p in sys.path if p.lower() != '/repos/mozilla-central/obj-x86_64-pc-linux-gnu/_virtualenvs/common/lib/python3.9/site-packages'.lower()]
Unable to match yaml regex against output: import sys; sys.path.append('/repos/mozilla-central/obj-x86_64-pc-linux-gnu/_virtualenvs/common/lib/python3.9/site-packages')
Unable to match yaml regex against output: # --- expected pthfile contents ---
Unable to match yaml regex against output: /repos/mozilla-central/build
Unable to match yaml regex against output: /repos/mozilla-central/config
Unable to match yaml regex against output: /repos/mozilla-central/config/mozunit
Unable to match yaml regex against output: /repos/mozilla-central/dom/bindings
Unable to match yaml regex against output: /repos/mozilla-central/dom/bindings/parser
Unable to match yaml regex against output: /repos/mozilla-central/layout/tools/reftest
Unable to match yaml regex against output: /repos/mozilla-central/python/l10n
Unable to match yaml regex against output: /repos/mozilla-central/python/mach
Unable to match yaml regex against output: /repos/mozilla-central/python/mozboot
Unable to match yaml regex against output: /repos/mozilla-central/python/mozbuild
Unable to match yaml regex against output: /repos/mozilla-central/python/mozlint
Unable to match yaml regex against output: /repos/mozilla-central/python/mozperftest
Unable to match yaml regex against output: /repos/mozilla-central/python/mozrelease
Unable to match yaml regex against output: /repos/mozilla-central/python/mozterm
Unable to match yaml regex against output: /repos/mozilla-central/python/mozversioncontrol
Unable to match yaml regex against output: /repos/mozilla-central/security/manager/tools
Unable to match yaml regex against output: /repos/mozilla-central/taskcluster
Unable to match yaml regex against output: /repos/mozilla-central/testing
Unable to match yaml regex against output: /repos/mozilla-central/testing/awsy
Unable to match yaml regex against output: /repos/mozilla-central/testing/condprofile
Unable to match yaml regex against output: /repos/mozilla-central/testing/firefox-ui/harness
Unable to match yaml regex against output: /repos/mozilla-central/testing/marionette/client
Unable to match yaml regex against output: /repos/mozilla-central/testing/marionette/harness
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/manifestparser
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozcrash
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozdebug
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozdevice
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozfile
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozhttpd
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozgeckoprofiler
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozinfo
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozinstall
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozleak
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozlog
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/moznetwork
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozpower
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozprocess
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozprofile
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozproxy
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozrunner
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozsystemmonitor
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozscreenshot
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/moztest
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozversion
Unable to match yaml regex against output: /repos/mozilla-central/testing/raptor
Unable to match yaml regex against output: /repos/mozilla-central/testing/talos
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform
Unable to match yaml regex against output: /repos/mozilla-central/testing/xpcshell
Unable to match yaml regex against output: /repos/mozilla-central/toolkit/components/telemetry/tests/marionette/harness
Unable to match yaml regex against output: /repos/mozilla-central/tools
Unable to match yaml regex against output: /repos/mozilla-central/tools/moztreedocs
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/ds/tools
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/geckoprocesstypes_generator
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/idl-parser
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/h2
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/hpack
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/html5lib
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/hyperframe
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/pywebsocket3
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/webencodings
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/wptserve
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/wptrunner
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/aiohttp
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ansicon
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/appdirs
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/async_timeout
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/attrs
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/blessed
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cbor2
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/certifi
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/chardet
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/click
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/colorama
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/compare_locales
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cookies
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cram
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/diskcache
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/distro
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/dlmanager
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ecdsa
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/esprima
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/fluent.migrate
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/fluent.syntax
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/giturlparse
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/gyp/pylib
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/idna
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/idna-ssl
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/importlib_metadata
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/iso8601
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/Jinja2
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jinxed
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jsmin
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/json-e
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jsonschema
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/looseversion
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/MarkupSafe/src
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mohawk
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mozilla_repo_urls
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mozilla_version
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/multidict
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/packaging
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pathspec
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pip
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pip_tools
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ply
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyasn1
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyasn1_modules
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pylru
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyparsing
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyrsistent
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/python-hglib
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pytoml
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/PyYAML/lib3
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/redo
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/requests
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/requests_unixsocket
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/responses
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/rsa
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/setuptools
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/sentry_sdk
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/six
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/slugid
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster_taskgraph
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster_urls
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/tqdm
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/typing_extensions
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/urllib3
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/voluptuous
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/wcwidth
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/wheel
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/yamllint
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/yarl
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/zipp
Unable to match yaml regex against output: /app/.mozbuild/srcdirs/mozilla-central-b9868e2b6607/_virtualenvs/mach/lib/python3.9/site-packages
Unable to match yaml regex against output: import sys; sys.path = [p for p in sys.path if p.lower() != '/repos/mozilla-central/obj-x86_64-pc-linux-gnu/_virtualenvs/common/lib/python3.9/site-packages'.lower()]
Unable to match yaml regex against output: import sys; sys.path.append('/repos/mozilla-central/obj-x86_64-pc-linux-gnu/_virtualenvs/common/lib/python3.9/site-packages')
Unable to match yaml regex against output: # ---
Fixing /repos/mozilla-central/taskcluster/gecko_taskgraph/transforms/bootstrap.py
✖ 0 problems (0 errors, 0 warnings, 7 fixed)
--enable-application is the historic flag, and --enable-project was
added to handle the non-application things we build, while encompassing
the meaning of --enable-application. --enable-project has been preferred
for a while and we should reflect that more consistently in mozconfigs,
documentation, etc.
Differential Revision: https://phabricator.services.mozilla.com/D162625
Some consumers, in particular some lint tasks, may have their builds
substituted with artifact builds. Therefore, in an artifact build, we
re-package the fetched XPT artifacts.
Using the `zip` Python action makes the resulting archive reproducible.
Depends on D162071
Differential Revision: https://phabricator.services.mozilla.com/D161640
When running `./mach doc` locally it is a poor experience to have the build error out
due to fatal warnings, as it's possible the missing refs simply aren't implemented yet.
Or worse, it's possible that the developer ran `./mach doc <subtree>` and the missing
refs don't exist simply because they are outside of <subtree>.
This patch ensures we only enable fatal warnings in CI, or if the user opts into it with
the `--fatal-warnings` flag.
Differential Revision: https://phabricator.services.mozilla.com/D161633
In clang tasks, cctools-port is used to merge macos compiler-rts, or to
build clang itself for macos hosts. It's not needed for tasks for
non-macos hosts that don't depend on compiler-rt.
Differential Revision: https://phabricator.services.mozilla.com/D161223
`ThreadAllows.txt` has a sibling file `ThreadFileAllows.txt` which
should also be counted among the clang-plugin's dependencies.
Followup to D159671.
Differential Revision: https://phabricator.services.mozilla.com/D160283
This is the last place that in `gecko_taskgraph/generator.py` that calls into
`gecko_taskgraph` code. By removing it, the two generator.py files will be
identical and the `gecko_taskgraph` copy can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D161080
This is the last place that in `gecko_taskgraph/generator.py` that calls into
`gecko_taskgraph` code. By removing it, the two generator.py files will be
identical and the `gecko_taskgraph` copy can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D161080
Make all expiration date comparison from the same time reference, making
the comparison pedantically valid, and also avoiding repeated implicit
calls to `datetime.datetime.utcnow()`
Use `dict.setdefault` and `dict.get` to decrease the number of control path
in the code, hopefully making it easier to read.
As a bonus, this yields a modest 4% runtime improvement on my setup when
comparing the runtime of
./mach taskgraph tasks --fast --no-optimize -q -o /dev/null
Differential Revision: https://phabricator.services.mozilla.com/D160404
Avoid long lines by using appropriate temporary variables.
Use dict.pop instead of dict.get + del.
Move loop invariant outside of loops.
Differential Revision: https://phabricator.services.mozilla.com/D160467
Currently, they are kept a meager 2 days, which is not enough to even
look at logs from the preceding week on a Monday. Those logs are useful
because as per bug 1744706, they show what specific git revision was
fetched (we use uncached fetch tasks for "get tip of git branch" type
of fetches, it can be useful to know what the tip actually was then).
We don't necessarily need the artifact that long.
Differential Revision: https://phabricator.services.mozilla.com/D160870
Brew's Python 3.10 causes `virtualenv==20.7.2` to produce a wonky folder
structure (`$venv/opt/homebrew/lib/python3.10/site-packages`?).
This is likely fixed with newer `virtualenv`, but the simpler workaround
here is to use `venv` instead now that Python 3 is always used.
Adds `python3-venv` to docker image so that tests and debian-based tasks
can leverage it.
Differential Revision: https://phabricator.services.mozilla.com/D144872
This is a bit more general purpose and flexible than hardcoding `hg-push` here.
It is also something that can be upstreamed to standalone taskgraph.
Differential Revision: https://phabricator.services.mozilla.com/D160536
This mostly avoids the repacking step which is CPU-heavy, but is only
valid for git repo that don't make use of submodule.
On large repo like htpps://github.com/llvm/llvm-project, this speeds up
the artifact creation from 6min to 1min.
Differential Revision: https://phabricator.services.mozilla.com/D159680
Before bug 1289641, we didn't have pdbstr in the VS artifact, but we now
do as a side effect of using vsdownload. So instead of exfiltrating the
file from a Windows worker, we can just use the one from VS.
Differential Revision: https://phabricator.services.mozilla.com/D159659
It looks like it takes a long time to run unity-webgl tests. Because of this
long run time, it's perma failing with "max run time exceeded" error when there
is an additional profiler-enabled run. This revision disables this additional
run for this test so it doesn't perma fail.
Differential Revision: https://phabricator.services.mozilla.com/D159257
The `always_target` attribute is a feature which I believe I previously
mis-implemented. It doesn't belong in `generator.py` and should instead be
implemented somewhere in `target_tasks.py`. I chose to wrap the registration
decorator to guarantee backwards compatibility within Gecko. Though we may wish
to move it out to a filter in the future.
I'm making this change now to facilitate merging standalone and Gecko
taskgraphs. The `always_target` feature will be removed from standalone (as it
isn't being used and should consumers need something like it, they can roll
their own implementations).
With the feature removed from both Gecko and standalone in generator.py, this
file will now be identical across both Taskgraphs and can therefore be removed
from Gecko.
Differential Revision: https://phabricator.services.mozilla.com/D159181
Very few tasks actually need the attribute these days. The few that
still do can use the corresponding environment variable directly.
Differential Revision: https://phabricator.services.mozilla.com/D159350
Brew's Python 3.10 causes `virtualenv==20.7.2` to produce a wonky folder
structure (`$venv/opt/homebrew/lib/python3.10/site-packages`?).
This is likely fixed with newer `virtualenv`, but the simpler workaround
here is to use `venv` instead now that Python 3 is always used.
Adds `python3-venv` to docker image so that tests and debian-based tasks
can leverage it.
Differential Revision: https://phabricator.services.mozilla.com/D144872
What we are doing:
- Upgrading FFMPEG version from 4.1.4 to 4.4.1
- Moving the binaries out of a personal github to the perf-automation mozilla repository
Differential Revision: https://phabricator.services.mozilla.com/D155411
The last time we re-recorded the google-docs site, it had been updated to the canvas version. There is no reason to keep the separate google-docs-canvas page load test around.
Differential Revision: https://phabricator.services.mozilla.com/D157507
Most importantly for a newer Mesa version.
The docker image apparently does not need to get updated to 22.08, leave
it on 21.08 as there appear to be issues creating the new one.
Differential Revision: https://phabricator.services.mozilla.com/D157305
Joel Maher
Mike Hommey
Sandor Molnar
Marco Castelluccio
Ryan VanderMeulen
Ray Kraesig
Tom Ritter
Mark Banner
Andi-Bogdan Postelnicu
Csoregi Natalia
Greg Mierzwinski
Ben Hearsum
Noemi Erli
Andrew Halberstadt
Michelle Goossens
Cristian Tuns
andrej
Jon Coppeard
Iulian Moraru
Julien Cristau
Gabriele Svelto
Marian-Vasile Laza
Alexandre Lissy
Kash Shampur
serge-sans-paille
Heitor Neiva
Geoff Brown
Stanca Serban
Gabriel Bustamante
Brenden Hyde
Alex Finder
Dave Hunt
Sylvestre Ledru
Cosmin Sabou
Henrik Skupin
Otto Länd
Butkovits Atila
Norisz Fay
owlishDeveloper
Bobby Holley
Jesse Schwartzentruber
Alex Ionescu
Emilio Cobos Álvarez
Nick Alexander
Eitan Isaacson
Makoto Kato
Barret Rennie
june wilde
Jason Kratzer
trickypr
ahochheiden
steven100695
Dana Keeler
stransky
Nazım Can Altınova
alwu
Sebastian Hengst
Robert Mader
Tyson Smith
Andreea Pavel
Johan Lorenzo
criss