Граф коммитов

174 Коммитов

Автор SHA1 Сообщение Дата
Mitchell Hentges b8108d6064 Bug 1632429: Migrate |./mach try| to python 3; r=rstewart,ahal
|./mach try| subcommands are now compatible with both python 2 and 3.

Hand-tested with many combinations of subcommand and subcommand flags.

Updates tryselect unit tests to use Python 3.

Differential Revision: https://phabricator.services.mozilla.com/D73398
2020-05-06 22:17:14 +00:00
Andrew Halberstadt 3220639fe4 Bug 1634484 - [tryselect] Change default ./mach try auto strategy to 'bugbug_reduced', r=marco
Depends on D73349

Differential Revision: https://phabricator.services.mozilla.com/D73350
2020-04-30 21:50:40 +00:00
Mitchell Hentges 6e1e037cd3 Bug 1569115 - Don't throw TypeError exception when using |./mach try chooser| r=rstewart
This was an issue around "six" and "click", this thread sheds some light:
https://github.com/pallets/click/issues/564.

This issue was reproducible by having a local python 2 flask app with Flask==1.0.2, Werkzeug==0.14.1 and Click==7.0. Upon self-shutdown, the TypeError was raised.

Upgrading Click to 7.1.2 resolves this issue both in that separate "reproduce app" and here in-tree.

Differential Revision: https://phabricator.services.mozilla.com/D73351
2020-05-01 14:45:40 +00:00
Mitchell Hentges 26afa282b0 Bug 1634116 - Removes "mach try again" history migration r=rstewart
The history migration has been around since February 2019 and is ready for cleanup.

Differential Revision: https://phabricator.services.mozilla.com/D73099
2020-04-29 19:40:51 +00:00
Andrew Halberstadt 5454fdd8bb Bug 1624859 - [taskgraph] Always optimize build tasks with ./mach try auto, r=tomprince
This ensures we don't run every build with every push via ./mach try auto. It
introduces a new 'optimization-overrides' try_config that can be used to
replace optimizations. For now, there is no user interface to pass this in via
the 'mach try' command line.

Differential Revision: https://phabricator.services.mozilla.com/D68207
2020-04-22 15:51:35 +00:00
Andrew Halberstadt a0bcf43891 Bug 1624859 - [taskgraph] Remove 'bugbug' internal class in experimental optimization strategies, r=tomprince
It was making things more complicated.

Differential Revision: https://phabricator.services.mozilla.com/D71821
2020-04-22 15:50:13 +00:00
Edwin Takahashi e2f3473016 Bug 1630350 - apply mach try filter to try syntax selector r=jmaher,gbrown
Changes:

Applies the `filter_tasks_by_blacklist` method to try syntax pushes as well.

  - moved `TARGET_TASK_BLACKLIST`and `filter_tasks_by_blacklist` method to live in `taskcluster/taskgraph/target_tasks.py`.
  - removed existing filters against `ccov, windows10-aarch64` and `android-hw` filters against try syntax pushes.
  - update imports for `fuzzy` and `chooser` selectors to refer to the new location of `filter_tasks_by_blacklist` method.

The reason for moving the logic (again) from `tools/tryselect` to `taskcluster/` is due to the placement of `try_option_syntax` and `target_tasks` files and both of those files handle the processing of `mach try syntax` pushes.

Differential Revision: https://phabricator.services.mozilla.com/D71698
2020-04-21 19:51:25 +00:00
Edwin Takahashi 66ea42cb18 Bug 1627340 - separate filter for try selectors and apply them to try chooser r=bc,ahal,jmaher
Changes:

  - implement filtering of taskgraph items against a list of known task filters in `tasks.py`.
  - apply these filters to both `fuzzy` and `chooser` selectors, unless `--full` is specified.

Differential Revision: https://phabricator.services.mozilla.com/D70097

--HG--
extra : moz-landing-system : lando
2020-04-14 13:43:17 +00:00
Mihai Alexandru Michis 9a16882603 Backed out changeset c5907d661bb7 (bug 1627340) for causing lint failures in tryselect/selectors/chooser/__init__.py
CLOSED TREE
2020-04-10 04:48:08 +03:00
Edwin Takahashi ae9fc8a3c6 Bug 1627340 - separate filter for try selectors and apply them to try chooser r=ahal,jmaher
Changes:

  - implement filtering of taskgraph items against a list of known task filters in `tasks.py`.
  - apply these filters to both `fuzzy` and `chooser` selectors, unless `--full` is specified.

Differential Revision: https://phabricator.services.mozilla.com/D70097

--HG--
extra : moz-landing-system : lando
2020-04-09 22:01:41 +00:00
Andrew Halberstadt 7349bc85c9 Bug 1628810 - [tryselect] Use bugbug.debug optimization strategy by default in ./mach try auto, DONTBUILD, r=marco
Differential Revision: https://phabricator.services.mozilla.com/D70419

--HG--
extra : moz-landing-system : lando
2020-04-09 18:31:24 +00:00
Bob Clary 770a924bf7 Bug 1627344 - Remove leading and trailing .* from mach try fuzzy.py's TARGET_TASK_FILTERS, r=ahal.
Differential Revision: https://phabricator.services.mozilla.com/D69622

--HG--
extra : moz-landing-system : lando
2020-04-06 14:48:46 +00:00
Andrew Halberstadt 367ad2118d Bug 1627074 - [tryselect] Make it more clear that 'mach try auto' is experimental, DONTBUILD, r=marco,sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D69445

--HG--
extra : moz-landing-system : lando
2020-04-02 20:43:13 +00:00
Andrew Halberstadt d45b3cf7a4 Bug 1624902 - [taskgraph] Implement substrategies in the bugbug scheduler for dealing with platforms, r=marco
The bugbug scheduler currently chooses which manifests are important, and we
then run *every* task that contains those manifests. This is likely overkill
and we can reduce the number of configurations we run these manifests on.

Differential Revision: https://phabricator.services.mozilla.com/D68466

--HG--
extra : moz-landing-system : lando
2020-04-01 13:43:48 +00:00
Andrew Halberstadt 9359f4cba8 Bug 1624902 - [taskgraph] Move experimental strategies into a common namespace, r=marco
Small cleanup to help compartmentalize these strategies.

Differential Revision: https://phabricator.services.mozilla.com/D68463

--HG--
extra : moz-landing-system : lando
2020-04-01 13:43:17 +00:00
James Graham 5eba193cb7 Bug 1624280 - Allow specifying routes in mach try fuzzy, r=ahal
These routes are applied to all tasks except the decision task. There
is currently no validity checking in the frontend, so if the provided
routes aren't valid the decision task will fail.

Differential Revision: https://phabricator.services.mozilla.com/D67828

--HG--
extra : moz-landing-system : lando
2020-03-27 19:02:07 +00:00
Csoregi Natalia e1af17b6c1 Backed out 4 changesets (bug 1624902) for failures on test_optimize_strategies.py. CLOSED TREE
Backed out changeset b500403daa0c (bug 1624902)
Backed out changeset bf817363344e (bug 1624902)
Backed out changeset c7a62f2d9034 (bug 1624902)
Backed out changeset 08356d478d16 (bug 1624902)
2020-04-01 00:09:34 +03:00
Andrew Halberstadt 8787bbc73a Bug 1624902 - [taskgraph] Implement substrategies in the bugbug scheduler for dealing with platforms, r=marco
The bugbug scheduler currently chooses which manifests are important, and we
then run *every* task that contains those manifests. This is likely overkill
and we can reduce the number of configurations we run these manifests on.

Differential Revision: https://phabricator.services.mozilla.com/D68466

--HG--
extra : moz-landing-system : lando
2020-03-31 17:57:50 +00:00
Andrew Halberstadt 9a971df100 Bug 1624902 - [taskgraph] Move experimental strategies into a common namespace, r=marco
Small cleanup to help compartmentalize these strategies.

Differential Revision: https://phabricator.services.mozilla.com/D68463

--HG--
extra : moz-landing-system : lando
2020-03-31 18:06:58 +00:00
Simon Fraser 5736cab8ce Bug 1625933 Only complain about unmanaged fzf if it was found r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D68792

--HG--
extra : moz-landing-system : lando
2020-03-30 13:49:57 +00:00
Simon Fraser ba6f2afbaa Bug 1623197 Change fzf management check to avoid side effects r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D67830

--HG--
extra : moz-landing-system : lando
2020-03-25 10:07:26 +00:00
Sylvestre Ledru be18fd4dba Bug 1622328 - add license info to all eslintrc files r=Standard8,webcompat-reviewers,miketaylr
Differential Revision: https://phabricator.services.mozilla.com/D67319

--HG--
extra : moz-landing-system : lando
2020-03-19 13:47:51 +00:00
Andrew Halberstadt 2cf0c4fbc5 Bug 1603463 - [tryselect] Add the ability to run 'mach try auto' with alternative scheduling algorithms, r=tomprince
This allows us to change the default optimization strategy used in try pushes.
While probably not super useful to developers, it can help us easily test
changes to new and experimental optimizations on try.

This also changes the default to the 'bugbug_push_schedules' strategy, since
SETA is more or less random and shouldn't be used by 'mach try auto'. In the
future, we'll switch this back to simply using the default optimization as the
default will ideally be the best one that we have.

Differential Revision: https://phabricator.services.mozilla.com/D65746

--HG--
extra : moz-landing-system : lando
2020-03-13 20:32:14 +00:00
Andrew Halberstadt 56ba8a9ddf Bug 1603463 - Implement |mach try auto| which selects tasks on try automatically, r=tomprince
The 'auto' in 'mach try auto' stands for two things:

1. It automatically picks tasks for you.
2. It runs the same scheduling algorithms as autoland.

It accomplishes this by creating a new target_tasks method that spoofs the
'project' parameter to autoland.

Differential Revision: https://phabricator.services.mozilla.com/D60184

--HG--
extra : moz-landing-system : lando
2020-03-13 20:35:43 +00:00
Simon Fraser dfdd97e607 Bug 1621658 Update |mach try fuzzy| fzf version check r=ahal
Add filename support for plain display

Update fzf version in tests

Update tooltool url for fzf

Differential Revision: https://phabricator.services.mozilla.com/D66571

--HG--
extra : moz-landing-system : lando
2020-03-12 12:56:35 +00:00
Simon Fraser 54f21ea31e Bug 1619554 Improve performance of |mach try fuzzy| preview r=ahal
Remove use of requests module in preview pane

Reformat task duration data to avoid reprocessing in preview pane

Avoid loading task durations json more than once.

Increase required fzf version, use temporary file instead of arglist

Differential Revision: https://phabricator.services.mozilla.com/D65094

--HG--
extra : moz-landing-system : lando
2020-03-05 19:58:43 +00:00
Mark Banner 15cfe23b88 Bug 1620542 - Automatically fix ESLint errors in .eslintrc.js files. r=mossop
# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D65703

--HG--
extra : moz-landing-system : lando
2020-03-07 10:09:44 +00:00
Tom Prince 5a52cec97c No bug: Fix missing space in `mach try syntax` help text.
Differential Revision: https://phabricator.services.mozilla.com/D65840

--HG--
extra : moz-landing-system : lando
2020-03-07 01:27:13 +00:00
Edwin Takahashi 6e6db2f667 Bug 1620044 - remove unnecessary debian and ubuntu-bionic codepaths r=jmaher
Changes:

Remove `ubuntu-bionic` flag that was used during development to enable use of ubuntu1804-test docker image.

Remove unnecessary conditional and in the process rewrite how `runtests.py` checks the environment for `pactl` prior to running mochitests.

Differential Revision: https://phabricator.services.mozilla.com/D65389

--HG--
extra : moz-landing-system : lando
2020-03-04 20:47:43 +00:00
Andrew Halberstadt 8649491808 Bug 1618633 - Add --disable-target-task-filter to |mach try fuzzy| to bypass task filtering, r=jgraham
In |mach try fuzzy| there's a TARGET_TASK_FILTERS variable that we use to make
it more difficult to run certain kinds of expensive / resource constrained
tasks. The problem is that the only way to run these is to use '--full' at
which point there's no way to distinguish which tasks are "valid" (i.e run on
mozilla-central) and which tasks don't.

This adds a flag that will ensure the default set truly matches what we run
on central.

Differential Revision: https://phabricator.services.mozilla.com/D64648

--HG--
extra : moz-landing-system : lando
2020-02-28 20:56:10 +00:00
Tom Prince e797b75c9a Bug 1618079: [tryselect] Use public ship-it api from scriptworker selector; r=rail
Differential Revision: https://phabricator.services.mozilla.com/D64663

--HG--
extra : moz-landing-system : lando
2020-02-27 21:30:34 +00:00
octavian.negru 5348ca01b5 Bug 1617877 - TypeError: environment can only contain strings with mach try fuzzy on Windows r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D64062

--HG--
extra : moz-landing-system : lando
2020-02-26 09:07:31 +00:00
Bob Clary a3aec07b06 Bug 1610879 - restrict Tier 2 fission web-platform-tests on Linux to try with --full, r=jgraham
Differential Revision: https://phabricator.services.mozilla.com/D64050

--HG--
extra : moz-landing-system : lando
2020-02-25 11:43:33 +00:00
Andrew Halberstadt 903826d9cd Bug 1616902 - [try] Move tools/tryselect/preview.py to the 'selectors' directory, r=sfraser
Differential Revision: https://phabricator.services.mozilla.com/D63676

--HG--
rename : tools/tryselect/preview.py => tools/tryselect/selectors/preview.py
extra : moz-landing-system : lando
2020-02-21 16:45:33 +00:00
Simon Fraser dfe6a6ca65 Bug 1615929 Pass PYTHONPATH into fzf preview script r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D63217

--HG--
extra : moz-landing-system : lando
2020-02-18 19:08:15 +00:00
Simon Fraser bb0152beb3 Bug 1611989 - Refactor try estimates for code re-use. r=ahal
Add duration estimates to push summary. Refactored preview script - needed to be moved in order to import module.

Differential Revision: https://phabricator.services.mozilla.com/D61195

--HG--
rename : tools/tryselect/formatters/preview.py => tools/tryselect/preview.py
extra : moz-landing-system : lando
2020-02-12 21:06:56 +00:00
Barret Rennie 90d19ac0d4 Bug 1613743 - Remove `./mach try --visual-metrics-jobs` r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D62358

--HG--
extra : moz-landing-system : lando
2020-02-11 20:01:01 +00:00
Tom Prince 0ddf6e6ce0 No bug: [tryselect] Use up-to-date URL for code-coverage information; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D61645

--HG--
extra : moz-landing-system : lando
2020-02-10 18:52:06 +00:00
Florin Strugariu f28920ebc4 Bug 1610791 - Do not run try tests on Windows 10 x64 2017 Ref HW by default r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D60661

--HG--
extra : moz-landing-system : lando
2020-01-22 15:02:03 +00:00
Ricky Stewart 0fa8a98bdb Bug 1524639: [taskgraph] Enforce unicode strings in schemas r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D18376

--HG--
extra : moz-landing-system : lando
2020-01-21 17:12:08 +00:00
Razvan Maries ffc90b2ec8 Backed out changeset a57aed49dc58 (bug 1524639) for beetmover bustages. a=backout 2020-01-16 14:04:31 +02:00
Ricky Stewart fecc988cc6 Bug 1524639: [taskgraph] Enforce unicode strings in schemas r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D18376

--HG--
extra : moz-landing-system : lando
2020-01-15 20:33:20 +00:00
Tom Prince fb985314d1 Bug 1608205: [tryselect] Allow scriptworker selector to use nightly graphs; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D59388

--HG--
extra : moz-landing-system : lando
2020-01-15 17:19:02 +00:00
Edwin Takahashi b2e141b8ed Bug 1599197 - do not run linux32 on autoland or try by default r=jmaher,gbrown
Changes:

Do not run `linux32` builds on `try` or `autoland` by default, but ensure it is able to run on all other repos (eg. `mozilla-beta`).

Filter out all tasks with `linux-shippable` in the name, including builds, auxiliary tasks (balrog, etc).

Differential Revision: https://phabricator.services.mozilla.com/D58982

--HG--
extra : moz-landing-system : lando
2020-01-09 01:09:07 +00:00
Tom Prince 569e26cdb0 Bug 1605275: [tryselect] Add more task sets to scriptworker selector; r=Callek
This adds support most of the remaining desktop promote phase task types.

Differential Revision: https://phabricator.services.mozilla.com/D58020

--HG--
extra : moz-landing-system : lando
2020-01-07 18:08:44 +00:00
Tom Prince 83d95abcbe Bug 1605275: [taskgraph] Provide a function to get all tasks from a previous set of graphs; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D58450

--HG--
extra : moz-landing-system : lando
2020-01-07 18:08:42 +00:00
Tom Prince 29144620e2 Bug 1605275: [tryselect] Get l10n files from release commit; r=Callek
This should lead to the chunking of L10n tasks on try to match the base
release.

Differential Revision: https://phabricator.services.mozilla.com/D58449

--HG--
extra : moz-landing-system : lando
2020-01-07 18:08:31 +00:00
Tom Prince f1826cfaf7 Bug 1605277: [tryselect] Handle v2 try_task_config in `mach try again --list`; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D58420

--HG--
extra : moz-landing-system : lando
2020-01-06 19:44:12 +00:00
Tom Prince e053400510 Bug 1605277: [tryselect] Add option to allow overriding worker types on try; r=ahal
This allows easy testing against staging worker types, without needing to
change anything in-tree.

Differential Revision: https://phabricator.services.mozilla.com/D58017

--HG--
extra : moz-landing-system : lando
2020-01-07 17:23:18 +00:00
Edwin Takahashi d1af0a1c8e Bug 1599197 - demote linux32-shippable/opt tasks to tier 2 and limit builds to try and mozilla-central r=jmaher,gbrown
Changes:

Demote linux32-shippable/opt tasks to tier 2 on all repositories. Other linux32 jobs are defined but do not appear to run.

Limit the build of linux32-shippable/opt to `try`/`mozilla-central`.

Differential Revision: https://phabricator.services.mozilla.com/D58207

--HG--
extra : moz-landing-system : lando
2020-01-06 18:42:02 +00:00