This patch adds the ability to run raptor performance tests (specifically benchmark and live pageload tests) with Safari on our MacOS hardware.
This will let us compare Safari to Firefox for a subset of our tests.
Bug 1802922 is a follow up to look into some of the failing tests in Safari that were disabled for this patch.
Differential Revision: https://phabricator.services.mozilla.com/D160890
This patch fixes several issues and needs to work together with matching
changes in reposado:
* The package dumper handles nested packages correctly
* Disk image files (.dmg) are handled as well as package ones (.pkg) and ZIP archives (.zip)
* Fixed some code that still used python 2 semantics
* Deleting temporary directories was made more reliable
* Use the libdmg build provided in the toolchain
Differential Revision: https://phabricator.services.mozilla.com/D137207
Switches from `buildconfig.substs` to `bootstrap_toolchain` for the repackage job
Refactors mozpack/dmg to not depend on the environment; Moves path resolution of tools up to `mozbuild.action.{make_dmg|unpack_dmg}`
Update syntax to python 3.7+
Differential Revision: https://phabricator.services.mozilla.com/D163129
We want these tests to run by default on Android Fission to get some statistics on the failures, and to let the wpt-sync bot adjust the expectations. Once the work on fixing wpt tests on Android Fission is complete, we plan to revisit this and make the tier of these tests to be default
Differential Revision: https://phabricator.services.mozilla.com/D163262
It's unclear exactly what is happening, but ensuring that gecko_taskgraph is loaded immediately after being unloaded will ensure that it is always before taskcluster_taskgraph in sys.path, which avoids the issue.
Differential Revision: https://phabricator.services.mozilla.com/D163377
# ignore-this-changeset
Output from `mach lint`:
error: problem with lint setup, skipping android-api-lint, android-checkstyle, android-format, android-javadoc, android-lint, android-test
Unable to match yaml regex against output: Site not up-to-date reason: The pthfile at "/repos/mozilla-central/obj-x86_64-pc-linux-gnu/_virtualenvs/common/lib/python3.9/site-packages/mach.pth" does not match the expected value.
Unable to match yaml regex against output: # --- on-disk pthfile: ---
Unable to match yaml regex against output: /repos/mozilla-central/build
Unable to match yaml regex against output: /repos/mozilla-central/config
Unable to match yaml regex against output: /repos/mozilla-central/config/mozunit
Unable to match yaml regex against output: /repos/mozilla-central/dom/bindings
Unable to match yaml regex against output: /repos/mozilla-central/dom/bindings/parser
Unable to match yaml regex against output: /repos/mozilla-central/layout/tools/reftest
Unable to match yaml regex against output: /repos/mozilla-central/python/l10n
Unable to match yaml regex against output: /repos/mozilla-central/python/mach
Unable to match yaml regex against output: /repos/mozilla-central/python/mozboot
Unable to match yaml regex against output: /repos/mozilla-central/python/mozbuild
Unable to match yaml regex against output: /repos/mozilla-central/python/mozlint
Unable to match yaml regex against output: /repos/mozilla-central/python/mozperftest
Unable to match yaml regex against output: /repos/mozilla-central/python/mozrelease
Unable to match yaml regex against output: /repos/mozilla-central/python/mozterm
Unable to match yaml regex against output: /repos/mozilla-central/python/mozversioncontrol
Unable to match yaml regex against output: /repos/mozilla-central/security/manager/tools
Unable to match yaml regex against output: /repos/mozilla-central/taskcluster
Unable to match yaml regex against output: /repos/mozilla-central/testing
Unable to match yaml regex against output: /repos/mozilla-central/testing/awsy
Unable to match yaml regex against output: /repos/mozilla-central/testing/condprofile
Unable to match yaml regex against output: /repos/mozilla-central/testing/firefox-ui/harness
Unable to match yaml regex against output: /repos/mozilla-central/testing/marionette/client
Unable to match yaml regex against output: /repos/mozilla-central/testing/marionette/harness
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/manifestparser
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozcrash
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozdebug
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozdevice
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozfile
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozhttpd
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozgeckoprofiler
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozinfo
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozinstall
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozleak
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozlog
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/moznetwork
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozpower
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozprocess
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozprofile
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozproxy
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozrunner
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozsystemmonitor
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozscreenshot
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/moztest
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozversion
Unable to match yaml regex against output: /repos/mozilla-central/testing/raptor
Unable to match yaml regex against output: /repos/mozilla-central/testing/talos
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform
Unable to match yaml regex against output: /repos/mozilla-central/testing/xpcshell
Unable to match yaml regex against output: /repos/mozilla-central/toolkit/components/telemetry/tests/marionette/harness
Unable to match yaml regex against output: /repos/mozilla-central/tools
Unable to match yaml regex against output: /repos/mozilla-central/tools/moztreedocs
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/ds/tools
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/geckoprocesstypes_generator
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/idl-parser
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/funcsigs
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/h2
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/hpack
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/html5lib
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/hyperframe
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/pywebsocket3
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/webencodings
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/wptserve
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/wptrunner
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/aiohttp
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ansicon
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/appdirs
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/async_timeout
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/attrs
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/blessed
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cbor2
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/certifi
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/chardet
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/click
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/colorama
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/compare_locales
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cookies
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cram
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/diskcache
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/distro
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/dlmanager
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ecdsa
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/esprima
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/fluent.migrate
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/fluent.syntax
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/giturlparse
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/gyp/pylib
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/idna
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/idna-ssl
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/importlib_metadata
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/iso8601
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/Jinja2
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jinxed
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jsmin
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/json-e
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jsonschema
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/looseversion
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/MarkupSafe/src
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mohawk
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mozilla_repo_urls
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mozilla_version
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/multidict
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/packaging
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pathspec
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pip
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pip_tools
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ply
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyasn1
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyasn1_modules
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pylru
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyparsing
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyrsistent
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/python-hglib
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pytoml
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/PyYAML/lib3
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/redo
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/requests
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/requests_unixsocket
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/responses
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/rsa
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/setuptools
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/sentry_sdk
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/six
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/slugid
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster_taskgraph
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster_urls
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/tqdm
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/typing_extensions
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/urllib3
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/voluptuous
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/wcwidth
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/wheel
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/yamllint
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/yarl
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/zipp
Unable to match yaml regex against output: /app/.mozbuild/srcdirs/mozilla-central-b9868e2b6607/_virtualenvs/mach/lib/python3.9/site-packages
Unable to match yaml regex against output: import sys; sys.path = [p for p in sys.path if p.lower() != '/repos/mozilla-central/obj-x86_64-pc-linux-gnu/_virtualenvs/common/lib/python3.9/site-packages'.lower()]
Unable to match yaml regex against output: import sys; sys.path.append('/repos/mozilla-central/obj-x86_64-pc-linux-gnu/_virtualenvs/common/lib/python3.9/site-packages')
Unable to match yaml regex against output: # --- expected pthfile contents ---
Unable to match yaml regex against output: /repos/mozilla-central/build
Unable to match yaml regex against output: /repos/mozilla-central/config
Unable to match yaml regex against output: /repos/mozilla-central/config/mozunit
Unable to match yaml regex against output: /repos/mozilla-central/dom/bindings
Unable to match yaml regex against output: /repos/mozilla-central/dom/bindings/parser
Unable to match yaml regex against output: /repos/mozilla-central/layout/tools/reftest
Unable to match yaml regex against output: /repos/mozilla-central/python/l10n
Unable to match yaml regex against output: /repos/mozilla-central/python/mach
Unable to match yaml regex against output: /repos/mozilla-central/python/mozboot
Unable to match yaml regex against output: /repos/mozilla-central/python/mozbuild
Unable to match yaml regex against output: /repos/mozilla-central/python/mozlint
Unable to match yaml regex against output: /repos/mozilla-central/python/mozperftest
Unable to match yaml regex against output: /repos/mozilla-central/python/mozrelease
Unable to match yaml regex against output: /repos/mozilla-central/python/mozterm
Unable to match yaml regex against output: /repos/mozilla-central/python/mozversioncontrol
Unable to match yaml regex against output: /repos/mozilla-central/security/manager/tools
Unable to match yaml regex against output: /repos/mozilla-central/taskcluster
Unable to match yaml regex against output: /repos/mozilla-central/testing
Unable to match yaml regex against output: /repos/mozilla-central/testing/awsy
Unable to match yaml regex against output: /repos/mozilla-central/testing/condprofile
Unable to match yaml regex against output: /repos/mozilla-central/testing/firefox-ui/harness
Unable to match yaml regex against output: /repos/mozilla-central/testing/marionette/client
Unable to match yaml regex against output: /repos/mozilla-central/testing/marionette/harness
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/manifestparser
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozcrash
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozdebug
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozdevice
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozfile
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozhttpd
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozgeckoprofiler
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozinfo
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozinstall
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozleak
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozlog
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/moznetwork
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozpower
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozprocess
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozprofile
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozproxy
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozrunner
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozsystemmonitor
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozscreenshot
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/moztest
Unable to match yaml regex against output: /repos/mozilla-central/testing/mozbase/mozversion
Unable to match yaml regex against output: /repos/mozilla-central/testing/raptor
Unable to match yaml regex against output: /repos/mozilla-central/testing/talos
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform
Unable to match yaml regex against output: /repos/mozilla-central/testing/xpcshell
Unable to match yaml regex against output: /repos/mozilla-central/toolkit/components/telemetry/tests/marionette/harness
Unable to match yaml regex against output: /repos/mozilla-central/tools
Unable to match yaml regex against output: /repos/mozilla-central/tools/moztreedocs
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/ds/tools
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/geckoprocesstypes_generator
Unable to match yaml regex against output: /repos/mozilla-central/xpcom/idl-parser
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/h2
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/hpack
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/html5lib
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/hyperframe
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/pywebsocket3
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/third_party/webencodings
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/wptserve
Unable to match yaml regex against output: /repos/mozilla-central/testing/web-platform/tests/tools/wptrunner
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/aiohttp
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ansicon
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/appdirs
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/async_timeout
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/attrs
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/blessed
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cbor2
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/certifi
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/chardet
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/click
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/colorama
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/compare_locales
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cookies
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/cram
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/diskcache
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/distro
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/dlmanager
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ecdsa
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/esprima
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/fluent.migrate
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/fluent.syntax
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/giturlparse
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/gyp/pylib
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/idna
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/idna-ssl
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/importlib_metadata
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/iso8601
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/Jinja2
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jinxed
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jsmin
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/json-e
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/jsonschema
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/looseversion
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/MarkupSafe/src
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mohawk
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mozilla_repo_urls
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/mozilla_version
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/multidict
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/packaging
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pathspec
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pip
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pip_tools
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/ply
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyasn1
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyasn1_modules
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pylru
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyparsing
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pyrsistent
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/python-hglib
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/pytoml
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/PyYAML/lib3
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/redo
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/requests
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/requests_unixsocket
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/responses
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/rsa
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/setuptools
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/sentry_sdk
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/six
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/slugid
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster_taskgraph
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/taskcluster_urls
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/tqdm
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/typing_extensions
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/urllib3
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/voluptuous
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/wcwidth
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/wheel
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/yamllint
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/yarl
Unable to match yaml regex against output: /repos/mozilla-central/third_party/python/zipp
Unable to match yaml regex against output: /app/.mozbuild/srcdirs/mozilla-central-b9868e2b6607/_virtualenvs/mach/lib/python3.9/site-packages
Unable to match yaml regex against output: import sys; sys.path = [p for p in sys.path if p.lower() != '/repos/mozilla-central/obj-x86_64-pc-linux-gnu/_virtualenvs/common/lib/python3.9/site-packages'.lower()]
Unable to match yaml regex against output: import sys; sys.path.append('/repos/mozilla-central/obj-x86_64-pc-linux-gnu/_virtualenvs/common/lib/python3.9/site-packages')
Unable to match yaml regex against output: # ---
Fixing /repos/mozilla-central/taskcluster/gecko_taskgraph/transforms/bootstrap.py
✖ 0 problems (0 errors, 0 warnings, 7 fixed)
--enable-application is the historic flag, and --enable-project was
added to handle the non-application things we build, while encompassing
the meaning of --enable-application. --enable-project has been preferred
for a while and we should reflect that more consistently in mozconfigs,
documentation, etc.
Differential Revision: https://phabricator.services.mozilla.com/D162625
Some consumers, in particular some lint tasks, may have their builds
substituted with artifact builds. Therefore, in an artifact build, we
re-package the fetched XPT artifacts.
Using the `zip` Python action makes the resulting archive reproducible.
Depends on D162071
Differential Revision: https://phabricator.services.mozilla.com/D161640
When running `./mach doc` locally it is a poor experience to have the build error out
due to fatal warnings, as it's possible the missing refs simply aren't implemented yet.
Or worse, it's possible that the developer ran `./mach doc <subtree>` and the missing
refs don't exist simply because they are outside of <subtree>.
This patch ensures we only enable fatal warnings in CI, or if the user opts into it with
the `--fatal-warnings` flag.
Differential Revision: https://phabricator.services.mozilla.com/D161633
In clang tasks, cctools-port is used to merge macos compiler-rts, or to
build clang itself for macos hosts. It's not needed for tasks for
non-macos hosts that don't depend on compiler-rt.
Differential Revision: https://phabricator.services.mozilla.com/D161223
`ThreadAllows.txt` has a sibling file `ThreadFileAllows.txt` which
should also be counted among the clang-plugin's dependencies.
Followup to D159671.
Differential Revision: https://phabricator.services.mozilla.com/D160283
This is the last place that in `gecko_taskgraph/generator.py` that calls into
`gecko_taskgraph` code. By removing it, the two generator.py files will be
identical and the `gecko_taskgraph` copy can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D161080
This is the last place that in `gecko_taskgraph/generator.py` that calls into
`gecko_taskgraph` code. By removing it, the two generator.py files will be
identical and the `gecko_taskgraph` copy can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D161080
Make all expiration date comparison from the same time reference, making
the comparison pedantically valid, and also avoiding repeated implicit
calls to `datetime.datetime.utcnow()`
Use `dict.setdefault` and `dict.get` to decrease the number of control path
in the code, hopefully making it easier to read.
As a bonus, this yields a modest 4% runtime improvement on my setup when
comparing the runtime of
./mach taskgraph tasks --fast --no-optimize -q -o /dev/null
Differential Revision: https://phabricator.services.mozilla.com/D160404
Avoid long lines by using appropriate temporary variables.
Use dict.pop instead of dict.get + del.
Move loop invariant outside of loops.
Differential Revision: https://phabricator.services.mozilla.com/D160467
Currently, they are kept a meager 2 days, which is not enough to even
look at logs from the preceding week on a Monday. Those logs are useful
because as per bug 1744706, they show what specific git revision was
fetched (we use uncached fetch tasks for "get tip of git branch" type
of fetches, it can be useful to know what the tip actually was then).
We don't necessarily need the artifact that long.
Differential Revision: https://phabricator.services.mozilla.com/D160870
Brew's Python 3.10 causes `virtualenv==20.7.2` to produce a wonky folder
structure (`$venv/opt/homebrew/lib/python3.10/site-packages`?).
This is likely fixed with newer `virtualenv`, but the simpler workaround
here is to use `venv` instead now that Python 3 is always used.
Adds `python3-venv` to docker image so that tests and debian-based tasks
can leverage it.
Differential Revision: https://phabricator.services.mozilla.com/D144872
This is a bit more general purpose and flexible than hardcoding `hg-push` here.
It is also something that can be upstreamed to standalone taskgraph.
Differential Revision: https://phabricator.services.mozilla.com/D160536
This mostly avoids the repacking step which is CPU-heavy, but is only
valid for git repo that don't make use of submodule.
On large repo like htpps://github.com/llvm/llvm-project, this speeds up
the artifact creation from 6min to 1min.
Differential Revision: https://phabricator.services.mozilla.com/D159680
Before bug 1289641, we didn't have pdbstr in the VS artifact, but we now
do as a side effect of using vsdownload. So instead of exfiltrating the
file from a Windows worker, we can just use the one from VS.
Differential Revision: https://phabricator.services.mozilla.com/D159659
It looks like it takes a long time to run unity-webgl tests. Because of this
long run time, it's perma failing with "max run time exceeded" error when there
is an additional profiler-enabled run. This revision disables this additional
run for this test so it doesn't perma fail.
Differential Revision: https://phabricator.services.mozilla.com/D159257
The `always_target` attribute is a feature which I believe I previously
mis-implemented. It doesn't belong in `generator.py` and should instead be
implemented somewhere in `target_tasks.py`. I chose to wrap the registration
decorator to guarantee backwards compatibility within Gecko. Though we may wish
to move it out to a filter in the future.
I'm making this change now to facilitate merging standalone and Gecko
taskgraphs. The `always_target` feature will be removed from standalone (as it
isn't being used and should consumers need something like it, they can roll
their own implementations).
With the feature removed from both Gecko and standalone in generator.py, this
file will now be identical across both Taskgraphs and can therefore be removed
from Gecko.
Differential Revision: https://phabricator.services.mozilla.com/D159181
Very few tasks actually need the attribute these days. The few that
still do can use the corresponding environment variable directly.
Differential Revision: https://phabricator.services.mozilla.com/D159350
Brew's Python 3.10 causes `virtualenv==20.7.2` to produce a wonky folder
structure (`$venv/opt/homebrew/lib/python3.10/site-packages`?).
This is likely fixed with newer `virtualenv`, but the simpler workaround
here is to use `venv` instead now that Python 3 is always used.
Adds `python3-venv` to docker image so that tests and debian-based tasks
can leverage it.
Differential Revision: https://phabricator.services.mozilla.com/D144872
What we are doing:
- Upgrading FFMPEG version from 4.1.4 to 4.4.1
- Moving the binaries out of a personal github to the perf-automation mozilla repository
Differential Revision: https://phabricator.services.mozilla.com/D155411
The last time we re-recorded the google-docs site, it had been updated to the canvas version. There is no reason to keep the separate google-docs-canvas page load test around.
Differential Revision: https://phabricator.services.mozilla.com/D157507
Most importantly for a newer Mesa version.
The docker image apparently does not need to get updated to 22.08, leave
it on 21.08 as there appear to be issues creating the new one.
Differential Revision: https://phabricator.services.mozilla.com/D157305
This patch is to help address scenarios where duplicate graphs are created for the daily browsertime tests on mozilla-beta, off of the same revision. This occurs when the 2nd cron job finishes before the 1st one finishes. Here, the logic prevents a 2nd graph from being created.
Differential Revision: https://phabricator.services.mozilla.com/D157366
We want these tests to run by default on Android Fission to get some statistics on the failures, and to let the wpt-sync bot adjust the expectations. Once the work on fixing wpt tests on Android Fission is complete, we plan to revisit this and make the tier of these tests to be default (bug 1790520).
Differential Revision: https://phabricator.services.mozilla.com/D157177
This patch limits the frequency of performance tests running on the mozilla-release branch. In particular, we need to reduce the load on our android devices for the time being, similar to how we did this for the beta branch in bug 1788026.
Differential Revision: https://phabricator.services.mozilla.com/D157159
Because the relevant SDK is not installed on the mac workers, we pull it
via fetches and adjust the plain build mozconfig as well as mozconfigs
for rusttest, grouping most things in build/macosx/mozconfig.common.
And because the SDK itself now has all the relevant headers, we don't
need the old check for system C++ headers (which also happens to have
outdated instructions)
Differential Revision: https://phabricator.services.mozilla.com/D156280
We keep the older 11.0 and 10.11 SDKs for openh264.
Ideally, we'd rename the SDK directory not to contain its version, but
ld64 actually relies on the directory name to figure out the platform
version when it's not given to it, which it's not. We can make clang do
that, but it's rather involved (because it also requires setting up the
host linker properly, which we don't do), so we just go the easy route
and keep the SDK version in its directory.
Differential Revision: https://phabricator.services.mozilla.com/D156279
Renaming `nightly` to `daily` is intended for avoiding confusion with Nightly builds and better reflect what we are doing with these performance tests.
Differential Revision: https://phabricator.services.mozilla.com/D156324
There will be changes in mozilla-beta that will increase the frequency of pushes. This will increase the load on our android devices so this patch will limit the amount of tests to be run nightly rather than on every push.
Differential Revision: https://phabricator.services.mozilla.com/D156198
We would like to enable this run for all the raptor browsertime tests on
desktop because would like to get gecko profile data for all our browsertime
performance results. This will help our users to get profiles easily and will
help our performance sheriffs to file performance regressions with profile data
easily.
Differential Revision: https://phabricator.services.mozilla.com/D155284
This patch makes available the new chromedriver version to our tests in CI to keep up with the monthly chrome webdriver updates.
Differential Revision: https://phabricator.services.mozilla.com/D156019
Moving the comments outside of the `command` entry as they were preventing the
command to be executed since the command lines are then turned into a single line,
and the leading `#` character was commenting out the whole command.
Differential Revision: https://phabricator.services.mozilla.com/D155712
They were added at some point in the past to possibly upgrade from
Ubuntu 18.04, but they're unused and the upgrade will happen to
something else later.
The system-symbols-* images use some files from it for the interactive
task shell, but those seem to have been cargo-culted rather than having
been actually necessary in those images.
Differential Revision: https://phabricator.services.mozilla.com/D155531
- Amended commit to match revisions requested in https://phabricator.services.mozilla.com/D155404
- Amended commit to match cached task and cached task artifact expirations per Phab. D155404
- Add note about ASCII date string comparison
- Add comment about potential for mismatch artifact / task expirations.
- Change the way artifact and task expiry are set based on cached_task attribute.
Differential Revision: https://phabricator.services.mozilla.com/D155404
This bump gets a lot of changes - the significant job processing refactoring
where we will now check on all past jobs and see if any of them need to be
processed and tracking multiple phabricator revisions for when a library
has patches.
Differential Revision: https://phabricator.services.mozilla.com/D155398
What we are doing:
- Disabling chrome-m tests to run on Pixel2 devices
Why:
- Recent bitbar cleanup script update is causing permafails on chrome
Differential Revision: https://phabricator.services.mozilla.com/D154274
This adds support for breakpad sym files with inline information.
Without this patch, things would break if we started emitting
the new format.
Differential Revision: https://phabricator.services.mozilla.com/D153656
This adds support for breakpad sym files with inline information.
Without this patch, things would break if we started emitting
the new format.
Differential Revision: https://phabricator.services.mozilla.com/D153655
This patch adds the ability to skip tests to Talos. This lets us prevent tests from running depending on which branch we are on. It also adds some code that will skip tests on the pine branch.
Differential Revision: https://phabricator.services.mozilla.com/D152513
The baldrdash integration of Cranelift is agreed between SM and CL
to be the wrong shape. Our import of the code base is also old and
causes difficulties for us when upgrading some crates (see bug
1774829). We should remove it for now to unblock bug 1774829.
Differential Revision: https://phabricator.services.mozilla.com/D152806
The "initial" verifications import gecko_taskgraph.transforms.job.registry.
Doing so brings in gecko.transforms.task as a depdendency, which imports
gecko_taskgraph.optimize.schema.OptimizationSchema.
Thunderbird's call to gecko_taskgraph.optimize.schema.set_optimization_schema()
needs to happen before gecko.transforms.task is loaded so that Thunderbird-
specific optimizations are registered. That happens when graph_config.register()
is called, so to fix the situation, run the "initial" verifications after
calling graph_config.register().
Differential Revision: https://phabricator.services.mozilla.com/D152446
This hopefully makes things less confusing, e.g. we now check that the
*new* file is non-empty, not the old one, and HSTS_DIFF_ARTIFACT is no
longer reversed.
Differential Revision: https://phabricator.services.mozilla.com/D152501
bzip2 has "important" priority in ubuntu 18.04 but "standard" in debian 11, so
it went missing when we switched the base image.
Differential Revision: https://phabricator.services.mozilla.com/D152420
The fxms schema job has been updated with a script that writes out the test
corpus for the test. This means we no longer need to keep these JSON files in
tree, since the test will automatically generate them.
Differential Revision: https://phabricator.services.mozilla.com/D151985
While the test lives in taskgraph, it's actually testing code in
mozbuild.mozpack. That code relies on python's tarfile to create tar
archives, and python 3.9 changed how it fills tar headers:
674935b8ca
Differential Revision: https://phabricator.services.mozilla.com/D152245
This patch adds a new set of tests (essential only) to run in CI at tier 2. A command line argument is added to turn bytecode cache testing on in warm pageloads. This also disables the testing in our primary pageload tests to fix the variance and increases in runtime.
Differential Revision: https://phabricator.services.mozilla.com/D151757
What we are doing:
Switching mac power data to run on osx-1015 from osx-1014
Why:
Generic workers were failing and attempting to pull from osx-1014, so we are setting them to pull from the osx-1015 worker pool
Differential Revision: https://phabricator.services.mozilla.com/D151268
To ensure that we don't ship a schema that breaks Experimenter, we now have
in-tree tests that validate FxMS messages against our schema using the same
JSON Schema evaluator (python-jsonchema).
Our test corpus is the same as test_PanelTestProvider.js. We cannot have
PanelTestProvider.jsm fetch the messages from the JSON due to fetch() not being
supported in the newtab npm tests.
Differential Revision: https://phabricator.services.mozilla.com/D151169
Sanitizer builds require unstable features from the rust compiler, and
the common way to achieve this is to use a nightly rust compiler,
which we had been doing for a while.
Later on, we also needed to use unstable features that actually require
some patches to both rustc and cargo to work properly for our needs, so
we switched to taking the source of a nightly rust compiler, patch and
build it, and use that.
But finding an appropriate nightly rust compiler is not always easy. We
want something close enough from the stable rust compiler we're using
otherwise, which we've been using the first nightly that has the
following version number for. But it also can happen that those versions
have bugs that have been fixed later and then uplifted during the beta
period (and thus fixed in the stable rust compiler).
On the other hand, taking the source of the stable rust compiler and
compiling it with a `channel` set to `dev` (actually the default) or
`nightly` yields a compiler that is very close to the actual stable rust
compiler, but acts like a nightly compiler and allows to enable unstable
features.
So instead of trying to pick a good version from the nightly channel,
just use the same version as the stable rust version we use.
Differential Revision: https://phabricator.services.mozilla.com/D151341
The longest of these tasks take around 1 hour and a half, by having a max
runtime of 10 hours we risk wasting a lot of time in case of bugs.
Differential Revision: https://phabricator.services.mozilla.com/D150855
The Nimbus Features corresponding to FxMS messaging surfaces are actually
intended to map to FxMS message groups, which can accept *any* FxMS message.
The features have been updated with schemas that accept any FxMS message.
As part of this, all FxMS schemas have been updated with an `$id` so that they
can be bundled into feature schemas and have their internal `$ref`s work.
(Otherwise, a `$ref` would be relative to the top-level schema instead of the
sub-schema).
Schemas for individual message types are no longer exposed as resource:// URIs,
except in tests, as indivual schemas are no longer required at runtime.
Additionally, each FxMS schema has had its `template` field become required and
requires a constant value for that schema (e.g., Spotlight requires a template
value of "spotlight").
A test has been added to ensure that if any of the messaging surfaces schemas
change that the feature schemas are also updated. The feature schemas can be
regenerated via:
```
cd ./browser/components/newtab/content-src/asrouter/schemas
../../../../../../mach make-schemas.py
```
Differential Revision: https://phabricator.services.mozilla.com/D147332
Add non-android debug builds/tests back to mozilla-beta, to catch
post-central leaks and debug-only test failures before reaching esr.
Differential Revision: https://phabricator.services.mozilla.com/D150621
This patch makes available the new chromedriver version to our tests in CI to keep up with the monthly chrome webdriver updates.
Differential Revision: https://phabricator.services.mozilla.com/D150424
Stanca Serban
Alexandre Lissy
Andrew Halberstadt
Gabriel Bustamante
Brenden Hyde
Alex Finder
Dave Hunt
Sylvestre Ledru
Sandor Molnar
Cosmin Sabou
Csoregi Natalia
Gabriele Svelto
Henrik Skupin
Marco Castelluccio
Cristian Tuns
Otto Länd
Mark Banner
Butkovits Atila
serge-sans-paille
Norisz Fay
Heitor Neiva
owlishDeveloper
Bobby Holley
Ben Hearsum
andrej
Jesse Schwartzentruber
Iulian Moraru
Mike Hommey
Alex Ionescu
Tom Ritter
Emilio Cobos Álvarez
Noemi Erli
Nick Alexander
Michelle Goossens
Eitan Isaacson
Makoto Kato
Greg Mierzwinski
Barret Rennie
june wilde
Jason Kratzer
trickypr
Ray Kraesig
ahochheiden
Ryan VanderMeulen
steven100695
Dana Keeler
stransky
Andi-Bogdan Postelnicu
Nazım Can Altınova
Julien Cristau
alwu
Sebastian Hengst
Kash Shampur
Marian-Vasile Laza
Robert Mader
Tyson Smith
Andreea Pavel
Johan Lorenzo
criss
Markus Stange
Joel Maher
Suhaib Mujahid
Nicolas Chevobbe
Geoff Brown
Narcis Beleuzu
Aki Sasaki
Jeff Muizelaar
Ryan Hunt
Rob Lemley
Tooru Fujisawa
Kelsey Gilbert
Myeongjun Go
Benjamin VanderSloot
Andi-Bogdan Postelnicu
Yulia
Alexandre Lissy