Граф коммитов

155 Коммитов

Автор SHA1 Сообщение Дата
Andrew Halberstadt 5c93c1892b Bug 1746414 - Use standalone taskgraph's parameters module in Gecko, r=taskgraph-reviewers,jmaher
Rather than defining a distinct `Parameter` class for Gecko, this now uses the
`extend_parameters_schema` utility function (which mobile repos are already
using).

As a consequence, shared parameters are now defined in standalone taskgraph.
And only Gecko-specific parameters are listed in
`gecko_taskgraph/parameters.py`

The only exception is `project` which gets redefined so we can override the
standalone taskgraph default (since it derives `project` from the repo name,
which doesn't work for Gecko).

Differential Revision: https://phabricator.services.mozilla.com/D134515
2022-01-31 17:03:19 +00:00
Andreea Pavel 77276416aa Backed out changeset 976680f20082 (bug 1746414) for causing local bootstrap bustage a=backout 2022-01-28 02:03:53 +02:00
Andrew Halberstadt c2b9d1b50f Bug 1746414 - Use standalone taskgraph's parameters module in Gecko, r=taskgraph-reviewers,jmaher
Rather than defining a distinct `Parameter` class for Gecko, this now uses the
`extend_parameters_schema` utility function (which mobile repos are already
using).

As a consequence, shared parameters are now defined in standalone taskgraph.
And only Gecko-specific parameters are listed in
`gecko_taskgraph/parameters.py`

The only exception is `project` which gets redefined so we can override the
standalone taskgraph default (since it derives `project` from the repo name,
which doesn't work for Gecko).

Differential Revision: https://phabricator.services.mozilla.com/D134515
2022-01-27 14:45:19 +00:00
Alexandru Michis e0e0fe9d03 Backed out changeset 34f8cfa0813b (bug 1746414) for breaking local builds.
CLOSED TREE
2022-01-06 00:01:54 +02:00
Andrew Halberstadt d3f1ddd6d3 Bug 1746414 - Use standalone taskgraph's parameters module in Gecko, r=taskgraph-reviewers,jmaher
Rather than defining a distinct `Parameter` class for Gecko, this now uses the
`extend_parameters_schema` utility function (which mobile repos are already
using).

As a consequence, shared parameters are now defined in standalone taskgraph.
And only Gecko-specific parameters are listed in
`gecko_taskgraph/parameters.py`

The only exception is `project` which gets redefined so we can override the
standalone taskgraph default (since it derives `project` from the repo name,
which doesn't work for Gecko).

Differential Revision: https://phabricator.services.mozilla.com/D134515
2022-01-05 16:37:43 +00:00
Andrew Halberstadt 95449daa6d Bug 1732723 - Rename "taskgraph" Python module to "gecko_taskgraph". r=jmaher
For a long time two copies of the 'taskgraph' module have existed in parallel.
We've attempted to keep them in sync, but over time they have diverged and the
maintenance burden has increased.

In order to reduce this burden, we'd like to re-join the two code bases. The
canonical repo will be the one that lives outside of mozilla-central, and this
module will depend on it. Since they both have the same module name (taskgraph)
we need to rename the version in mozilla-central to avoid collisions.

Other consumers of 'taskgraph' (like mobile repos) have standardized on
'<project>_taskgraph' as their module names. So replicating that here as well.

Differential Revision: https://phabricator.services.mozilla.com/D127118
2021-09-30 09:50:08 -04:00
Alex Lopez 63022efc7a Bug 1696251: Allow mach commands as stand-alone functions and adapt existing commands. r=mhentges,webdriver-reviewers,perftest-reviewers,sparky,whimboo
This removes the `@CommandProvider` decorator and the need to implement
mach commands inside subclasses of `MachCommandBase`, and moves all
existing commands out from classes to module level functions.

Differential Revision: https://phabricator.services.mozilla.com/D121512
2021-09-27 18:12:51 +00:00
Butkovits Atila eb735ac57e Backed out changeset 53b1fa0faa6d (bug 1696251) for breaking the static-analysis integration. a=backout 2021-09-23 13:06:40 +03:00
Alex Lopez a8e7083c84 Bug 1696251: Allow mach commands as stand-alone functions and adapt existing commands. r=mhentges,webdriver-reviewers,perftest-reviewers,sparky,whimboo
This removes the `@CommandProvider` decorator and the need to implement
mach commands inside subclasses of `MachCommandBase`, and moves all
existing commands out from classes to module level functions.

Differential Revision: https://phabricator.services.mozilla.com/D121512
2021-09-21 20:38:16 +00:00
Butkovits Atila c06a3dd9fa Backed out changeset 510dd46a9de7 (bug 1696251) for causing Android build bustages. 2021-09-21 05:16:50 +03:00
Alex Lopez fe61e94ec8 Bug 1696251: Allow mach commands as stand-alone functions and adapt existing commands. r=mhentges,webdriver-reviewers,perftest-reviewers,sparky,whimboo
This removes the `@CommandProvider` decorator and the need to implement
mach commands inside subclasses of `MachCommandBase`, and moves all
existing commands out from classes to module level functions.

Differential Revision: https://phabricator.services.mozilla.com/D121512
2021-09-20 20:21:07 +00:00
Noemi Erli 91aa2e2b9c Backed out changeset 5f5b612878f3 (bug 1696251) for causing multiple bustages 2021-09-18 02:22:39 +03:00
Alex Lopez bf860fe7dc Bug 1696251: Allow mach commands as stand-alone functions and adapt existing commands. r=mhentges,webdriver-reviewers,perftest-reviewers,sparky,whimboo
This removes the `@CommandProvider` decorator and the need to implement
mach commands inside subclasses of `MachCommandBase`, and moves all
existing commands out from classes to module level functions.

Differential Revision: https://phabricator.services.mozilla.com/D121512
2021-09-17 19:00:39 +00:00
Andrew Halberstadt 07b4805381 Bug 1729060 - [taskgraph] Use --diff implementation from 'main.py', r=taskgraph-reviewers,bhearsum
This replaces Gecko's old '--diff' implementation with the one included with
standalone taskgraph.

Note this revision is a straight sync from standalone taskgraph and should not
contain any additional changes.

Depends on D124835

Differential Revision: https://phabricator.services.mozilla.com/D124836
2021-09-08 20:31:46 +00:00
Kershaw Chang a27769ef1f Bug 1716566 - Enable mochitest for socket process, r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D123109
2021-09-07 14:42:44 +00:00
Andrew Halberstadt c832a0b42a Bug 1725404 - Only print warning about -J if '--diff' was specified, r=bhearsum
A drive-by fix I noticed.

Depends on D122778

Differential Revision: https://phabricator.services.mozilla.com/D122779
2021-08-17 13:35:13 +00:00
Andrew Halberstadt b3348551e1 Bug 1725404 - Refactor '--target-kind' / '--output-file' logic to use standalone taskgraph's implementation, r=bhearsum
This syncs in the latest 'main.py' from standalone taskgraph and refactors
mach_commands.py to use it.

Differential Revision: https://phabricator.services.mozilla.com/D122778
2021-08-17 13:35:12 +00:00
Andrew Halberstadt 4ca883c33a Bug 1725404 - [taskgraph] Implement 'mach taskcluster-image-digest' command, r=bhearsum
I'm not sure if this is needed in Gecko, but it matches the similar command in
standalone taskgraph so makes the two more in sync. I don't think having it
will hurt.

Differential Revision: https://phabricator.services.mozilla.com/D122527
2021-08-16 14:56:27 +00:00
Andrew Halberstadt 5df4495f68 Bug 1725404 - [taskgraph] Re-use 'action-callback' subcommand args from main.py, r=bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D122526
2021-08-16 14:56:26 +00:00
Andrew Halberstadt 7b9414647d Bug 1725404 - [taskgraph] Re-use 'decision' args from main.py, r=bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D122524
2021-08-16 14:56:26 +00:00
Andrew Halberstadt 30bd3e4513 Bug 1725404 - [taskgraph] Use copy of 'get_filtered_taskgraph' from main.py, r=bhearsum
These two functions were identical.

Differential Revision: https://phabricator.services.mozilla.com/D122523
2021-08-16 14:56:26 +00:00
Andrew Halberstadt b3999e374e Bug 1725404 - [taskgraph] Re-use 'docker' subcommand args from main.py, r=bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D122522
2021-08-16 14:56:25 +00:00
Andrew Halberstadt b9135a7111 Bug 1725404 - [taskgraph] Re-use 'show' subcommand args from main.py, r=bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D122521
2021-08-16 14:56:25 +00:00
Alex Lopez 77ce415604 Bug 1696251 - Replace self with command_context where possible in existing mach commands. r=mhentges,webdriver-reviewers,perftest-reviewers,whimboo
This step removes all the dependencies of mach commands to
having a MachCommandBase as the `self` by using the `command_context`
argument instead. This also removes any remaining statefulness from those
classes that implement mach commands, ultimately making it easier to move
existing commands out of classes in a follow-up.

Differential Revision: https://phabricator.services.mozilla.com/D118058
2021-07-19 16:04:25 +00:00
Butkovits Atila a07f790e42 Backed out changeset e1921c5112d8 (bug 1696251) for causing bustages complaining about 'CommandContext'. CLOSED TREE 2021-07-16 20:35:55 +03:00
Alex Lopez 190e03aaab Bug 1696251 - Replace self with command_context where possible in existing mach commands. r=mhentges,webdriver-reviewers,perftest-reviewers,whimboo
This step removes all the dependencies of mach commands to
having a MachCommandBase as the `self` by using the `command_context`
argument instead. This also removes any remaining statefulness from those
classes that implement mach commands, ultimately making it easier to move
existing commands out of classes in a follow-up.

Differential Revision: https://phabricator.services.mozilla.com/D118058
2021-07-16 15:51:29 +00:00
Ben Hearsum 6290622edf No bug: Add helpful suggestion to ./mach taskgraph --diff for some circumstances r=ahal
This helps hint that in some cases, a user may have forgotten to diff
with `-J` if they were changing task bodies/payloads instead of adding
or removing a task.

Differential Revision: https://phabricator.services.mozilla.com/D116384
2021-06-01 14:10:07 +00:00
Rob Lemley c16a2ae79e Bug 1696251 - Fix "mach taskgraph full" command. r=mhentges
Differential Revision: https://phabricator.services.mozilla.com/D115543
2021-05-19 20:50:05 +00:00
Alex Lopez 455d9a088b Bug 1696251 - Pass MachCommandBase object as first argument for Mach Commands. r=mhentges,remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).

Differential Revision: https://phabricator.services.mozilla.com/D109650
2021-05-17 16:15:58 +00:00
Andrew Halberstadt 8bb5b785c1 No Bug - [taskgraph] Add 'check=True' to subprocess when running with --diff, r=taskgraph-reviewers,jmaher
Differential Revision: https://phabricator.services.mozilla.com/D113860
2021-05-03 20:55:19 +00:00
Andrew Halberstadt 2976b622e7 No Bug - [taskgraph] Generate current taskgraph before base when running with --diff, r=taskgraph-reviewers,aki
This ensures that when your patch has an error, taskgraph generation will fail
faster rather than needing to wait for the base generation to complete.

Depends on D113854

Differential Revision: https://phabricator.services.mozilla.com/D113859
2021-05-03 20:55:18 +00:00
Andrew Halberstadt 89f7a5f21c Bug 1702819 - [taskgraph] Support Python 3.6 with ./mach taskgraph <sub> --diff, r=taskgraph-reviewers,jmaher
Differential Revision: https://phabricator.services.mozilla.com/D113854
2021-05-03 20:55:18 +00:00
Butkovits Atila 2e34e363b9 Backed out 2 changesets (bug 1696251) for causing js-bench-sm failures. CLOSED TREE
Backed out changeset 1c84c9a34575 (bug 1696251)
Backed out changeset e169193b7423 (bug 1696251)
2021-04-23 02:53:36 +03:00
Alex Lopez d1a82b8092 Bug 1696251 - Pass MachCommandBase object as first argument for Mach Commands. r=mhentges,remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).

Differential Revision: https://phabricator.services.mozilla.com/D109650
2021-04-22 18:56:15 +00:00
Cosmin Sabou 785f9b8a87 Backed out changeset d4a5d8567977 (bug 1696251) for non-unified build bustages. CLOSED TREE 2021-04-19 19:43:31 +03:00
Alex Lopez 75dfe35468 Bug 1696251 - Pass MachCommandBase object as first argument for Mach Commands. r=mhentges,remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).

Differential Revision: https://phabricator.services.mozilla.com/D109650
2021-04-19 16:15:11 +00:00
Ben Hearsum 93969911bd No bug: ensure ./mach taskgraph --diff returns you to your starting branch when using git. r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D111848
2021-04-14 13:49:01 +00:00
Andrew Halberstadt 24e8551d71 Bug 1409733 - [taskcluster] Add a mach setting to override the diff command in |mach taskgraph --diff|, r=taskgraph-reviewers,bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D107542
2021-03-09 16:18:25 +00:00
Andrew Halberstadt 8ce220cc24 Bug 1409733 - [taskcluster] Implement a --diff flag on |mach taskgraph|, r=taskgraph-reviewers,bhearsum
This implements a --diff flag on the suite of |mach taskgraph| commands. E.g:

    ./mach taskgraph target -p project=autoland -J --diff

The above will update to the base revision of your stack (e.g the public
revision), generate the taskgraph, update to the current revision, generate the
taskgraph again, then display a diff of the two.

You can also specify a specifier to arg, e.g:

    ./mach taskgraph target -J --diff .~1

This will diff against the parent revision in mercurial (use 'HEAD~1' for git).

Differential Revision: https://phabricator.services.mozilla.com/D107274
2021-03-09 16:18:24 +00:00
Andrew Halberstadt 16b495e6df Bug 1409733 - [taskcluster] Refactor |mach taskgraph| generation into separate format function, r=taskgraph-reviewers,bhearsum
This will allow us to more easily generate and format the taskgraph multiple
times from the same session (e.g, needed for diffing multiple graphs).

Differential Revision: https://phabricator.services.mozilla.com/D107273
2021-03-09 16:18:24 +00:00
Andrew Halberstadt d66ad7cf65 Bug 1409733 - [taskcluster] Sort imports in |mach_commands.py|, r=taskgraph-reviewers,bhearsum
This was generated using isort and pyright.

Differential Revision: https://phabricator.services.mozilla.com/D107272
2021-03-09 16:18:24 +00:00
Mitchell Hentges 984958944e Bug 1661153: Removes _ensure_zstd() and usages r=firefox-build-system-reviewers,glandium
Removes a bunch of zstandard installations that were ran from
within the mach virtualenv, which should already have zstandard
installed.

Differential Revision: https://phabricator.services.mozilla.com/D98387
2020-12-03 22:58:52 +00:00
Cosmin Sabou fc64a95f3a Backed out changeset f1d785de7ab3 (bug 1661153) for causing symbols bustages. a=backout. 2020-12-03 00:49:17 +02:00
Mitchell Hentges 4184bca3d4 Bug 1661153: Removes _ensure_zstd() and usages r=firefox-build-system-reviewers,glandium
The zstandard package is always installed in the mach virtualenv.
The patch assumes that zstandard is only used from the mach virtualenv,
and never the build virtualenv.

Differential Revision: https://phabricator.services.mozilla.com/D98387
2020-12-02 15:16:14 +00:00
Ricky Stewart 02a7b4ebdf Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.

5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-26 18:34:53 +00:00
Bogdan Tara da1098d4aa Backed out 10 changesets (bug 1654103, bug 1672023, bug 1518999) for PanZoomControllerTest.touchEventForResult gv-junit failures CLOSED TREE
Backed out changeset ff3fb0b4a512 (bug 1672023)
Backed out changeset e7834b600201 (bug 1654103)
Backed out changeset 807893ca8069 (bug 1518999)
Backed out changeset 13e6b92440e9 (bug 1518999)
Backed out changeset 8b2ac5a6c98a (bug 1518999)
Backed out changeset 575748295752 (bug 1518999)
Backed out changeset 65f07ce7b39b (bug 1518999)
Backed out changeset 4bb80556158d (bug 1518999)
Backed out changeset 8ac8461d7bd7 (bug 1518999)
Backed out changeset e8ba13ee17f5 (bug 1518999)
2020-10-24 03:36:18 +03:00
Ricky Stewart c0cea3b0fa Bug 1654103: Standardize on Black for Python code in `mozilla-central`. r=remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers,devtools-backward-compat-reviewers,jgilbert,preferences-reviewers,sylvestre,maja_zf,webcompat-reviewers,denschub,ntim,whimboo,sparky
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.

5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-23 20:40:42 +00:00
Dorel Luca 1ff59cb7a3 Backed out changeset 7558c8821a07 (bug 1654103) for multiple failures. CLOSED TREE 2020-10-22 03:51:06 +03:00
Ricky Stewart 50762dacab Bug 1654103: Standardize on Black for Python code in `mozilla-central`. r=remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers,devtools-backward-compat-reviewers,jgilbert,preferences-reviewers,sylvestre,maja_zf,webcompat-reviewers,denschub,ntim,whimboo,sparky
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-21 21:27:27 +00:00
Mike Hommey 2df59e529f Bug 1661589 - Track decision task duration on perfherder. r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D88495
2020-08-28 10:45:23 +00:00