With this in place, all `-j`obs will not run by default on try. This will omit
such jobs in most try pushes even if files-changed matches. This is
unfortunate, but better than running them unconditionally. Fuzzy selections,
and later `just try it` pushes, are the ultimate solution here.
With this change, a push with no try syntax or try_task_config.json will schedule
no tasks at all.
MozReview-Commit-ID: FGjqlDW1FT6
--HG--
extra : rebase_source : 727ceafb1b6d24f83c0c7382b6a877ecb65863ab
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
--HG--
extra : rebase_source : 25e9966696d78d899783d9f38533d5ae66f9ccb9
extra : source : b53ff084c2d7968a1d9864d1343f2d9381fb652b
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
--HG--
extra : rebase_source : 03a10610aa3337269fe76a1196bb9b1665e1ab20
extra : source : b53ff084c2d7968a1d9864d1343f2d9381fb652b
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
--HG--
extra : rebase_source : fdd8c3cfb9abf759a3c43c5713e62e4772c5bd06
This fixes the issue where "-u web-platform-tests-wdspec" scheduled
nothing. Now it will schedule a e10s-enabled run of wdspec tests.
MozReview-Commit-ID: AdHZqtk1hLy
--HG--
extra : rebase_source : 8d5926277b37952616c4dffdb20133868913bab2
The upload now uses MOZ_SCM_LEVEL to determine which secret and bucket to
upload to, so it can potentially run at any level.
This also modifies task descriptions to allow {level} in scopes, and updates
try syntax to allow `-j doc-upload` even though run-on-tasks says it doesn't
run on try by default.
MozReview-Commit-ID: Dm27TGPa7IM
--HG--
extra : rebase_source : f1131abc8cd639251e085c8ebf776827a6b831ed
extra : amend_source : b2b0cb253c7f6e90fdd710c2c788877411bd9e1d
The first version of the patch didn't account for the fact that if we did -p all we still couldn't do -u foo[Windows 8]
specifically: -p all means self.platforms is None
https://hg.mozilla.org/mozilla-central/file/16ffc1d05422/taskcluster/taskgraph/try_option_syntax.py#l563
Then we check if the *test* has a run_on_projects that has either `all` or `try`, it doesn't so we abort.
In new code, we merely set a flag to say don't-run-me-by-default and use that in a few places.
MozReview-Commit-ID: 6rjNVZvpYGA
--HG--
extra : rebase_source : 4c791c9e566a5eddc2bb2e820bab0090f9db1237
Land date changes to support windows nightlies onto central
MozReview-Commit-ID: 8wlzBReWe5r
--HG--
extra : rebase_source : 72def80209aefb656b236d5858d8bb1ef5a68aab
It's a bit hacky to single out 'build' dependencies, but most tasks have a dependency on a docker
image, so we can't blanket skip all 'job' tasks with any dependencies at all. This is far from ideal
but is an improvement on the current behaviour of running build dependencies all the time, even if
the 'job' task gets optimized away.
There is likely a cleverer solution, but that can be follow-up work.
MozReview-Commit-ID: 6T68LT5VSrg
--HG--
extra : rebase_source : 1a7c43891c6e5da9a13c588c1daeba6121a47750
This copies the behavior of mozilla-taskcluster when it
submits try pushes. This will allow us to eventually stop
using mozilla-taskcluster.
MozReview-Commit-ID: J9zC92AE7HZ
--HG--
extra : rebase_source : 36575ef1fa5c43b6b6d5c0243c323af69e4ebd23
The logic here is a bit tricky to grok, but essentially there are two kinds of "job" tasks, those that
should always be considered (and possibly be optimized away later due to "skip-unless-changed"), and
those that should only be considered if their associated build-platform is also going to be scheduled.
If -j is specified, that should supercede both cases.
This patch uses the prescence of the 'build_platform' attribute to draw the distinction.
MozReview-Commit-ID: H9SjeYuZ8F0
--HG--
extra : rebase_source : 4c33061e882533df74159f3299278619a06d5945
We have a `unittest_try_name` for all unit test tasks, and similarly a
`talos_try_name`. The `-j` flag controls tasks known as "jobs" (although the
word has dozens of other meanings, too). Some of those set `job_try_name`, but
others do not and have special-case support in `try_option_syntax.py`. With
this change, all "jobs" set `job_try_name` and the special-case is removed.
MozReview-Commit-ID: 9hvW7wBIl2B
--HG--
extra : rebase_source : 120b5e9e7aa8f81fe49e72f4dadafdbd145ac357
In order to migrate from buildbot to taskcluster gradually, we read an
external json file to tells the percentage of jobs that must run in
Taskcluster.
MozReview-Commit-ID: JQhvsXKpcLz
--HG--
extra : rebase_source : 0c1a83b31831b788fb374adcb9fd181f97121239
In order to migrate from buildbot to taskcluster gradually, we read an
external json file to tells the percentage of jobs that must run in
Taskcluster.
MozReview-Commit-ID: K9KZMhe36Cu
--HG--
extra : rebase_source : 9619e5cdb010527538f03b3effe4756fc9574c4e
This uses the email provided by mozilla-taskcluster to find who to
email about failed decision tasks. It also adds some validation of
the try syntax that we've previously ignored.
Any platforms or build types specified in try sytax that don't exist
in the full task graph will throw an error.
MozReview-Commit-ID: JOKkLle7hEe
--HG--
extra : rebase_source : 65299c95c60bcb3c6a295b1f8f2f18477c2852db
this patch:
- adds linux{32,64}-nightly/opt test platforms that mirror the non-nightly test platforms.
- adds an `include_nightly` per-project parameter; this is refered to in the default `target_task_method`. It's still possible to launch custom `target_task_method`s to trigger nightlies against, say, try.
- adds a `filter_for_project` method in `target_tasks.py` that allows for `include_nightly` and `run_on_projects` filtering in the various `target_task_method`s.
- adds nightly filtering into the `TryOptionSyntax` object. By default, this will be off. To trigger nightly tests on try, either submit a new decision task with a different `target_task_method` (e.g. `nightly_fennec`) or flip the `include_nightly` flag to True.
- adds the `nightly` attribute to tests if their builds have that attribute.
MozReview-Commit-ID: DttIZH0BHS2
--HG--
extra : rebase_source : d8acbe4c741f570b2e8d33a8e6a7f5c791b24ff6
This fixes the ability to run mozbase via `-j mozbase`, with the
added advantage that it will obey `-p` too.
MozReview-Commit-ID: 1zkitUephXk
--HG--
extra : rebase_source : 0ebb65363d5f5813bc7ccb379768df54310d39c1
This formats the marionette-harness python tests to be a regular |mach python-test| suite. Though
we add subsuite=marionette, this is just for automation purposes. The new preferred way to run the
marionette harness tests locally is:
./mach python-test testing/marionette
They will also run if running the full suite.
The mozbase packages.txt file modifies mozlog to use 'setup.py' instead of 'pth'. The reason for
this is that the marionette-harness tests use the pytest_mozlog pytest plugin for formatting
their results (converts pytest format into something resembling the standard tbpl logging format).
In order for this plugin to get picked up however, mozlog's setup.py file needs to be processed.
MozReview-Commit-ID: Ata99evHxbd
--HG--
extra : rebase_source : 22382e3d65ce8454a1682cfced0d03477762e8fe
This formats the marionette-harness python tests to be a regular |mach python-test| suite. Though
we add subsuite=marionette, this is just for automation purposes. The new preferred way to run the
marionette harness tests locally is:
./mach python-test testing/marionette
They will also run if running the full suite.
The mozbase packages.txt file modifies mozlog to use 'setup.py' instead of 'pth'. The reason for
this is that the marionette-harness tests use the pytest_mozlog pytest plugin for formatting
their results (converts pytest format into something resembling the standard tbpl logging format).
In order for this plugin to get picked up however, mozlog's setup.py file needs to be processed.
MozReview-Commit-ID: Ata99evHxbd
--HG--
extra : rebase_source : 16ed70edd38a53c3279d8632d7cba3df4d5216c3
The name 'source-check' is a bit of a misnomer, because it already includes a bunch
of tasks that are proper unittests, not lints. Some of these unittests will soon
depend on a build task, which makes 'source-check' feel even more wrong.
They still have a lot in common with the lint tasks though, so it's probably not
worth splitting this into two separate kinds. Instead, let's just rename this kind
to 'source-test', which means, any task that tests stuff and is run from the source
directory (instead of a tests.zip). I think both lints and python-tests fall squarely
under this definition.
MozReview-Commit-ID: K0gZ5rVLyeD
--HG--
rename : taskcluster/ci/source-check/doc.yml => taskcluster/ci/source-test/doc.yml
rename : taskcluster/ci/source-check/kind.yml => taskcluster/ci/source-test/kind.yml
rename : taskcluster/ci/source-check/mozlint.yml => taskcluster/ci/source-test/mozlint.yml
rename : taskcluster/ci/source-check/python-tests.yml => taskcluster/ci/source-test/python-tests.yml
rename : taskcluster/ci/source-check/webidl.yml => taskcluster/ci/source-test/webidl.yml
extra : rebase_source : a683b5b6e243849de57f2681993046c776d6a8f2
-w, --taskcluster-worker: schedule taskcluster-worker jobs.
This is necessary to relieve the load of the small pool of data center
machines.
MozReview-Commit-ID: IiOLHUs2ALi
--HG--
extra : rebase_source : 96acbbd40aef856f68c7ab94ae23bcee4f2f078d
We parse_message needs to return an args object even if try commit
message is empty.
MozReview-Commit-ID: BPWvzjAyjHX
--HG--
extra : rebase_source : 57a699e95e55eabaad3e6a7596086c7f06248e91
We add the following command line options to Taskcluster try syntax:
--spsProfile - enable profile mode.
--rebuild-talos <N> - retrigger talos tests N times.
--setenv <VAR>=<val> - add extra environments variables.
--tag <TAG> - run tests only the tag TAG.
--no-retry - doesn't retry failed jobs.
We have a chicken-egg problem, as we first generate the full task graph
and then parse the try message. But the graph generation step needs to
know the try message to process the aforementioned options. The
solution is to parse the message before graph generation and then
pass the command line options to the transforms. Then, each transform
can look at the option that interests it and process it accordingly.
The message parse function is configured in kind.yml, which gives some
flexibility for future implementations of alternative syntaxes.
MozReview-Commit-ID: GPFdi0FD6Vn
--HG--
extra : rebase_source : b992786158851f1099aedfce8669a163228edc51
Tom Prince
Dustin J. Mitchell
Mike Hommey
Phil Ringnalda
Kris Maglione
J. Ryan Stinnett
Nick Fitzgerald
Ryan VanderMeulen
Johan Lorenzo
Maja Frydrychowicz
Wes Kocher
Justin Wood
Nathan Froyd
Bryce Van Dyk
Andrew Halberstadt
Brian Stack
Wander Lairson Costa
Sebastian Hengst
Mike Conley
Peter Moore
Steve Fink
Aki Sasaki
Chris Manchester