* Implement & document optimization (although legacy kind doesn't do much of it)
* Introduce `optimize_target_tasks` parameter to control whether tasks in the
target set can be optimized (no for try, yes for most other branches)
* Refactor to include resolved taskIds in the optimized task graph
* Include a `label-to-taskid.json` artifact.
* Introduce {'task-reference': '... <dependency-name> ...'} for referring to
parent tasks' taskId.
MozReview-Commit-ID: LWvlWNz49U5
--HG--
extra : rebase_source : 780e0e23d24b268ade33ecdcbccb5081f32aac48
MozReview-Commit-ID: KFlaQoeGPRc
* I guess this has to be uplifted to aurora and included in the aurora->beta
merge
--HG--
extra : rebase_source : eb97b061f193e0f6d976ea374a3ed2677435520b
extra : amend_source : 843c4791f09dc39e21499dd1782614b2c5442b7e
This test was removed as part of commit 2a7dbc0411d1, but we still try to run
it.
MozReview-Commit-ID: HfFkugGz2vI
--HG--
extra : rebase_source : 6073292e18b39cd54c976322c89e2371a27d3d24
extra : amend_source : 3b918c333ef0e07a07aa0bc3d957ea3ac65d79ad
this patch:
* forces always passing '-r' when pushing after a migration run
* removes '--new-branch' from beta_to_release push. we don't generate new branches on releases anymore because of release promotion!
* has migration runs use a unique share dir for each repo.
this can be used as a stop gap. maybe even a more permanent solution so we don't need to add complexity to robustcheckout for an edgecase like this.
MozReview-Commit-ID: HXY5vDI1pIt
--HG--
extra : rebase_source : 717e48dbf79115817e48adba8ed2f082d832fca0
Enables flake8 linting! To start, only these directories are actually linted:
- python/mozlint
- tools/lint
To enable new directories, add them to the 'include' directive at the bottom of:
tools/lint/flake8.lint
Edit topsrcdir/.flake8 to modify global configuration. Add a new .flake8 to a
subdirectory to override the global. The current configuration is more or less
just the default and we should tweak it to our needs.
MozReview-Commit-ID: iXbToRhm3b
--HG--
extra : rebase_source : 78c4491261ba36421110b0d9fca73c1f6d1f85c3
This adds flake8 dependencies. Note that ubuntu 16.04 repos include pip > 8.0
which has peep merged into it, so there's no need to install peep separately.
I also ran into a locale issue which was causing a UnicodeDecodeError anytime
python tried to print a unicode character. The "locale-gen/dpkg-reconfigure locales"
fix the problem.
MozReview-Commit-ID: DQhYV6Emklw
--HG--
extra : rebase_source : ffefb07a17a43423142b3417628b6dd475d5c54a
MozReview-Commit-ID: oDGtQ2Vnq4
Marionette harness has undergone lots of changes, so I decided to take current sources from M-C and do the copy + changes patches again with my own changes to get a Session harness.
--HG--
extra : transplant_source : D5%AA%A1E%96%8E%FC%06%BC%AF%C1T%DA%83%AF%1C%AB9%B5
MozReview-Commit-ID: 9e6HNAaF0Yo
In case of in-process restarts it can happen that the new process gets forked into a new process group.
When that happens we loose the capability to kill the process. To prevent a hang when joining the output
reader threads in wait(), we simply skip that call by passing-through the IO error.
--HG--
extra : rebase_source : 702dfec407ed13114f59fa6ccb0d82c5b0790550
robustcheckout barfs on symbolic revisions when using "revision."
MozReview-Commit-ID: B7YXqbWG0G1
--HG--
extra : rebase_source : d852930ac24be79004bce978c8ed6542ab58600f
This commit teaches the resource monitor in mozharness to emit
Perfherder data for system metrics and step times. This will
allow us to see when the timing or resource characteristics
of jobs in automation changes.
The recorded data includes overall CPU percent usage and I/O.
Each step has its time and CPU percent recorded. There is
certainly more data we could record. However, the immediate
goal of this change is to see if the data provides any benefit.
I'd rather start small and expand reporting once value from
this data is proved.
The wonkiest part of this patch is likely the mechanism to
define the Perfherder "test" names. We don't appear to have
an identifier in mozharness suitable for distinguishing
between job types. e.g. the "desktop_unittest.py" script is
responsible for running a few dozen jobs. So we invent code
for creating an identifier from the script config options.
I /think/ Treeherder will automatically assign the
project/branch, platform, and build type, which is why these
aren't included in the identifier.
MozReview-Commit-ID: HjhtXfxOvzJ
--HG--
extra : rebase_source : a3f0f2de4a091cde10c5a6815f1b4646bb5dc2f2
Having the latest schema available seems like a good thing. This is a
direct copy of schemas/performance-artifact.json from
https://github.com/mozilla/treeherder.git at commit
7bed1b22ceb01e3e71536fa1c4ecd14ddc87e803.
MozReview-Commit-ID: JQC4CeW6szM
--HG--
extra : rebase_source : a57d4e93b9334b5c571b05e0ef52f637a45432dd
Currently, only Talos accesses this file. An uncoming commit will add
a non-Talos consumer. Enable all mozharness consumers to access the
file by including it in the mozharness directory (previously it was
part of the Talos test archive).
MozReview-Commit-ID: ADlCj9E5BwC
--HG--
rename : testing/talos/treeherder-schemas/performance-artifact.json => testing/mozharness/external_tools/performance-artifact-schema.json
extra : rebase_source : ce5fcaf700941ce260c97c6daeefa07b4ef5e617
Phil Ringnalda
Justin D'Arcangelo
Dustin J. Mitchell
Wes Kocher
Dan Minor
Jordan Lund
Mike Conley
Carsten "Tomcat" Book
bechen
Jan Varga
Dave Hunt
fkang
Valentin Gosu
Henrik Skupin
Andrew Halberstadt
Kim Moir
Andrea Marchesini
André Reinald
Kris Maglione
David Burns
Gregory Szorc
Rail Aliiev
Masatoshi Kimura