We only use `include_nigthly` where we are also using
`filter_beta_release_tasks`, so just change the later to include nightly.
Differential Revision: https://phabricator.services.mozilla.com/D9687
--HG--
extra : moz-landing-system : lando
Currently, `mach try fuzzy` generates a taskgraph that is configured exactly
like the most recent push to mozilla-central. This isn't always desirabe, so
pass some configuration down, to allow the taskgraph to behave differently.
Differential Revision: https://phabricator.services.mozilla.com/D2729
--HG--
extra : rebase_source : 99d6958b33211697227e65df17edc1eb337f63a4
extra : histedit_source : 69b5ff6805bc8409340eb71323a1f6fc637259d7
This used to only be relevant to Devedition and Firefox releases.
In bug 1433536 we're going to add RC Fennec releases. Let's rename
the parameter now, for less parameters churn.
MozReview-Commit-ID: 28e1Y5FG4On
--HG--
extra : rebase_source : 59d41887d856481ab85bb8d2221dfcebca4430b0
extra : source : 4c96e312d88a3f7037c1eb39a3031d4baf997015
also clean up and move more config to the promotion config.
MozReview-Commit-ID: FmTWNNPcEaZ
--HG--
extra : rebase_source : 40431217fafb6796dbd65c7dfeab0e891ac1bbd4
extra : source : 0f5418a83477c1b6b221e4d28515792410e504d0
Add support for the three firefox and devedition relpro flavors (we
could probably reduce devedition to 2).
Also, instead of defining which kinds to use from the previous graph
in `previous_graph_kinds`, specify which kinds to rebuild (ignore)
from the previous graph in `rebuild_kinds`. This list will be much
smaller (currently empty).
MozReview-Commit-ID: 5rH1TW7GbAD
--HG--
extra : rebase_source : b4294a0d17a99b2ffd48f5d62821c724324b242c
extra : histedit_source : b3fe87101e4595f5fc5c7daaa2d4a0bd88418667
Instead of relying on environment variables, pass these in as parameters.
MozReview-Commit-ID: An58Bu2kd1g
--HG--
extra : rebase_source : 9c7b6beb13b676a3376a897f2c8143cc042b8276
Add a release promotion custom action for releng's TC release promotion migration work.
This action generates a graph dependent on previously built tasks. To track these, we add the `do_not_optimize` and `existing_tasks` parameters. The `do_not_optimize` parameter specifies tasks that we want to explicitly exclude from taskgraph optimization. The `existing_tasks` parameter specifies a label-to-taskid map for tasks from previous graphs.
MozReview-Commit-ID: 1vKrNUavM4V
--HG--
extra : rebase_source : b8ba95d270aafe1464c2b3bfc318b9568500a7a1
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
--HG--
extra : rebase_source : 25e9966696d78d899783d9f38533d5ae66f9ccb9
extra : source : b53ff084c2d7968a1d9864d1343f2d9381fb652b
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
--HG--
extra : rebase_source : 03a10610aa3337269fe76a1196bb9b1665e1ab20
extra : source : b53ff084c2d7968a1d9864d1343f2d9381fb652b
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
--HG--
extra : rebase_source : bb1d194b49caa4b69ce8faf9f324c80850d15d3f
extra : histedit_source : a933e0a6f8819b8d0e9220ecc10c34d096a0d477
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
--HG--
extra : rebase_source : fdd8c3cfb9abf759a3c43c5713e62e4772c5bd06
This provides a mechanism to modify the behaviour of tasks from a try push. The try_task_config.json
looks something like:
{
"tasks": ["build-linux64/opt", "test-linux64/opt-mochitest-e10s-1"],
"templates": {
"artifact": {"enabled": 1}
}
}
This tells taskgraph to apply the 'artifact' template to all tasks. Templates are JSONe based
.yml files that live under taskcluster/taskgraph/templates. Taskgraph will render every template
against every task definition. The templates themselves can then use JSONe condition statements to
filter out which tasks they should or shouldn't apply to.
MozReview-Commit-ID: J8HVZzOt4mX
--HG--
extra : rebase_source : 95a78bc56d3f90ff1b34aabd84ed92aff1e3d954
The purpose of this parameter has been superseded by the `include_nightly`
property.
MozReview-Commit-ID: 4iXQsv9Drqg
--HG--
extra : rebase_source : c94142282909a88c29fe6809d87721bef1f198c2
This patch enables `run-on-projects` to work appropriately for
nightly builds and tests. Initially, we were setting an empty
`run-on-projects` for nightly `build_platform`s, then explicitly
targeting the platforms in nightly-specific `target_task_method`s.
Instead, this patch enables nightlies to `run-on-projects` everywhere,
but governs the use of nightlies by either the `include_nightly`
parameter, or the `--include-nightly` try option. This lets us filter
nightly-related `target_task_method`s against `run-on-projects` without
losing all nightly tasks.
Then, enable spidermonkey tests by removing optimization from beta and
release. This patch also enables everything then disables specific
tasks, rather than disabling everything and enabling specific tasks.
Since we're beginning with a `filter_for_project` call, we should be
able to reduce these if blocks to zero over time, if desired.
MozReview-Commit-ID: A9tolynaChF
--HG--
extra : rebase_source : 3465ee2c714de3e0359f14109096fc94de27aadf
this patch:
- adds linux{32,64}-nightly/opt test platforms that mirror the non-nightly test platforms.
- adds an `include_nightly` per-project parameter; this is refered to in the default `target_task_method`. It's still possible to launch custom `target_task_method`s to trigger nightlies against, say, try.
- adds a `filter_for_project` method in `target_tasks.py` that allows for `include_nightly` and `run_on_projects` filtering in the various `target_task_method`s.
- adds nightly filtering into the `TryOptionSyntax` object. By default, this will be off. To trigger nightly tests on try, either submit a new decision task with a different `target_task_method` (e.g. `nightly_fennec`) or flip the `include_nightly` flag to True.
- adds the `nightly` attribute to tests if their builds have that attribute.
MozReview-Commit-ID: DttIZH0BHS2
--HG--
extra : rebase_source : d8acbe4c741f570b2e8d33a8e6a7f5c791b24ff6
Previously, we ran a single "target task" function to mutate the full
task graph into a subset based on input parameters (try syntax,
repository being built for, etc). This concept is useful. But
the implementation was limiting because we could only have a single
"target tasks" function.
This commit introduces the concept of "filters." They conceptually
do the same thing as "target tasks methods" but you can run more than
1 of them.
Filters are simply functions that examine an input graph+parameters
and emit nodes that should be retained. Filters, like target tasks
methods, are defined via decorated functions in a module.
TaskGraphGenerator has been converted to use filters. The list of
defined filters can be defined in the parameters dict passed into
TaskGraphGenerator. A default filter list is provided in decision.py.
The intent is to eventually convert target tasks to filters. Until
that happens, we always run the registered target tasks method via
a filter proxy function.
No new tests have been added because we don't yet have any
functionality relying explicitly on filters. Tests will be added in
a subsequent commit once we add a new filter.
While I was here, I also snuck in some logging on the size of the
graphs.
MozReview-Commit-ID: ERn2hIYbMRp
--HG--
extra : rebase_source : 36b8e86aa64b2f52b03b31b5497759b0009fb921
The `from_parameters` method was never used, and let do confusion over the role
of these parameters. Now there are only two, and they are always required.
MozReview-Commit-ID: AbPqijXucu5
--HG--
extra : rebase_source : 85affd063a543c549afaaa36ce7ee31ed1f943d5
Jobs reporting to treeherder should rely on the task route for project,
revision, and pushlog ID rather than things stuffed into task.extra.treeherder.
This also removes the need for a revision_hash that was calculated by mozilla-taskcluster.
MozReview-Commit-ID: EcQM9QRZzgG
--HG--
extra : rebase_source : f04f6724feef2dd51b4b98c67c9a261b093f452b
extra : amend_source : 0590605834d93359206f49edd94396c43b57f6dd
* Implement & document optimization (although legacy kind doesn't do much of it)
* Introduce `optimize_target_tasks` parameter to control whether tasks in the
target set can be optimized (no for try, yes for most other branches)
* Refactor to include resolved taskIds in the optimized task graph
* Include a `label-to-taskid.json` artifact.
* Introduce {'task-reference': '... <dependency-name> ...'} for referring to
parent tasks' taskId.
MozReview-Commit-ID: LWvlWNz49U5
--HG--
extra : rebase_source : 780e0e23d24b268ade33ecdcbccb5081f32aac48
Aki Sasaki
Andreea Pavel
Johan Lorenzo
Narcis Beleuzu
Tom Prince
Aki Sasaki
Sylvestre Ledru
Dustin J. Mitchell
Phil Ringnalda
Simon Fraser
Sebastian Hengst
Wes Kocher
Ryan VanderMeulen
Andrew Halberstadt
Paul Bignier
Gregory Szorc
Jordan Lund
Gregory Arndt