Removes a bunch of zstandard installations that were ran from
within the mach virtualenv, which should already have zstandard
installed.
Differential Revision: https://phabricator.services.mozilla.com/D98387
The zstandard package is always installed in the mach virtualenv.
The patch assumes that zstandard is only used from the mach virtualenv,
and never the build virtualenv.
Differential Revision: https://phabricator.services.mozilla.com/D98387
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Today we don't require that `mach` `CommandProvider`s subclass from any particular parent class and we're very lax about the requirements they must meet. While that's convenient in certain circumstances, it has some unfortunate implications for feature development.
Today the only requirements that we have for `CommandProvider`s are that they have an `__init__()` method that takes either 1 or 2 arguments, the second of which must be called `context` and is populated with the `mach` `CommandContext`. Again, while this flexibility is occasionally convenient, it is limiting. As we add features to `mach`, having a better idea what the shape of our `CommandProvider`s are and how we can instantiate them and use them is increasingly important, and this gives us additional control when having `mach` configure `CommandProvider`s based on data that is only available at the `mach` level. In particular, we plan to leverage this in bugs 985141 and 1654074.
Here we add validation to the `CommandProvider` decorator to ensure all classes inherit from `MachCommandBase`, update all `CommandProvider`s in-tree to inherit from `MachCommandBase`, and update source and test code accordingly.
Follow-up work: we now require (de facto) that the `context` be populated with a `topdir` attribute by the `populate_context_handler` function, since instantiating the `MachCommandBase` requires a `topdir` be provided. This is fine for now in the interest of keeping this patch reasonably sized, but some additional refactoring could make this cleaner.
Differential Revision: https://phabricator.services.mozilla.com/D86255
If we are generating only a part of the graph, to given kind, don't fail if a
build is packaging tests and there is no corresponding test task, as the tests
may not have been generated.
Differential Revision: https://phabricator.services.mozilla.com/D82097
This causes actions to fail on graphs with large parameters, such as
release graphs and try pushes.
Differential Revision: https://phabricator.services.mozilla.com/D37499
--HG--
extra : moz-landing-system : lando
Previously, this code looked parameters under `gecko.v2`, but that doesn't work
for projects using the out-of-tree taskgraph code, or Thunderbird. This moves
the parameter loading slightly later to vary the index used based on trust domain.
Differential Revision: https://phabricator.services.mozilla.com/D19028
--HG--
extra : moz-landing-system : lando
This allows the target tasks to include cached tasks, which by default, get
optimized out of the graph on on-`try` branches.
Differential Revision: https://phabricator.services.mozilla.com/D15278
--HG--
extra : moz-landing-system : lando
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015). But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere, and TASKCLUSTER_PROXY_URL
is set wherever the proxy is active.
The setup for the mach commands defaults to https://taskcluster.net for user
convenience. When the production instance's URL changes, we can simply change
that default.
This changes the docker build process propagate TASKCLUSTER_ROOT_URL into the
docker images where necessary (using %ARG), specifically to create URLs for
debian repo paths.
--HG--
extra : rebase_source : 0626ebdb66a9f4078cb75ab71be256c334297363
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015). But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere, and TASKCLUSTER_PROXY_URL
is set wherever the proxy is active.
The setup for the mach commands defaults to https://taskcluster.net for user
convenience. When the production instance's URL changes, we can simply change
that default.
This changes the docker build process to propagate TASKCLUSTER_ROOT_URL into
the docker images, and for good measure includes some code to use that value to
generate debian repo paths.
Differential Revision: https://phabricator.services.mozilla.com/D14196
--HG--
extra : moz-landing-system : lando
`mach artifact toolchain` gets task definitions from taskgraph, to get the
index path to find the artifacts at. Now that these index paths depend on the
digests of fetch tasks, those kinds need to be loaded as well. This adds a
supported API to get task definitions for a given kind, which loads all the
kind dependencies.
Differential Revision: https://phabricator.services.mozilla.com/D12161
--HG--
extra : moz-landing-system : lando
For kind=hook, the spec doesn't include this value as it's untrustworthy.
For kind=task, it's still untrustworthy, but there is no privilege escalation
so that's not important. Still, it dramatically expands the size of the task
definition.
MozReview-Commit-ID: 6scQ2ZwxP10
--HG--
extra : rebase_source : 4dc34390a510091ddc26023755992995fe358e47
The latest python-zstandard uses a newer zstandard that is faster.
It also has wheels available, which means installation doesn't require
Python development headers, etc.
MozReview-Commit-ID: 5gRq81KYmX4
--HG--
extra : rebase_source : 96ccc64e9707c6b4815c1bfa5c1a98b9a428b387
Mitchell Hentges
Cosmin Sabou
Ricky Stewart
Bogdan Tara
Dorel Luca
Mike Hommey
Tom Prince
Narcis Beleuzu
Chris AtLee
Razvan Maries
Andrew Halberstadt
Sebastian Hengst
Dustin J. Mitchell
Margareta Eliza Balazs
Kartikaya Gupta
Nick Thomas
Brian Stack
Ciure Andrei
Gregory Szorc