d8029a3fe7
Bug 1643661 Fix shell quoting with hg commands r=mtabara
...
shellcheck will moan about this, as it's potentially unsafe
Differential Revision: https://phabricator.services.mozilla.com/D78486
2020-06-05 12:06:21 +00:00
9e95f1a2e5
Bug 1637544 - Add a 'project' to the fake parameters used during taskgraph tests. r=tomprince DONTBUILD
...
Differential Revision: https://phabricator.services.mozilla.com/D78399
2020-06-04 21:52:34 +00:00
525a47b947
Bug 1637544 - Run no-sccache verification only on release projects. r=tomprince
...
Differential Revision: https://phabricator.services.mozilla.com/D77248
2020-06-04 19:40:54 +00:00
b2e275bf31
Bug 1637544 - Support project-specific verifications in taskgraph. r=tomprince
...
Differential Revision: https://phabricator.services.mozilla.com/D77247
2020-06-04 19:40:46 +00:00
0f1bc1a064
Bug 1633866 - [taskgraph] Improve 'guess_mozinfo_from_task' for more balanced chunks, r=gbrown
...
Adds a few properties to the mozinfo guess that were previously missing.
Differential Revision: https://phabricator.services.mozilla.com/D78156
2020-06-04 18:35:07 +00:00
ddaf15c72f
Bug 1633866 - [taskgraph] Implement a multiplier to tweak the duration for certain platforms / suites, r=gbrown
...
This is needed because xpcshell runs in parallel. Therefore we need to know the number
of threads are used (which varies by platform) in order to accurately determine how
many chunks are needed.
This allows us to specify a platform / suite specific multiplier to compensate via trial
and error.
Differential Revision: https://phabricator.services.mozilla.com/D78153
2020-06-04 15:34:51 +00:00
d4001d553e
Bug 1633866 - [taskgraph] Truncate runtimes that exceed DYNAMIC_CHUNK_DURATION so they don't artificially increase chunks, r=gbrown
...
For example, assume we have three manifests:
foo -> 98 minutes
bar -> 1 minute
baz -> 1 minute
And let's say we want average chunk lengths of ~20 minutes. The calculation
would currently say we need 5 chunks ((98 + 1 + 1) / 20). Even if set the max
chunks to the # of manifests, it would still return 3. Clearly this only needs
at most 2 chunks to run.
By truncating the runtime, we'd have:
foo -> 20 minutes (truncated)
bar -> 1 minute
baz -> 1 minute
So we'd actually only assign one chunk (round((20 + 1 + 1) / 20)). Despite
having a single 100 minute chunk, this outcome is desirable as we now avoid the
overhead of a second chunk that only runs 2 minutes worth of tests.
Depends on D77871
Differential Revision: https://phabricator.services.mozilla.com/D77873
2020-06-03 21:00:43 +00:00
b3fbd0cf70
Bug 1633866 - [taskgraph] Always use chunk number as symbol if it would otherwise be empty, r=jmaher
...
Depends on D76523
Differential Revision: https://phabricator.services.mozilla.com/D77871
2020-06-02 18:56:54 +00:00
27d78d7059
Bug 1633866 - [taskgraph] Create a 'bugbug' based test manifest loader, r=marco
...
Loads manifests using bugbug's push/schedules endpoint. For now we use as low
confidence threshold to select manifests. This is to try and both:
1) Improve regression detection rate
2) Save resources
at the same time. This way we theoretically shouldn't regress either dimension.
Note that we still optimize with CT_MEDIUM. My thinking is that at least all
of the "extra" stuff is still guaranteed to be relevant. Though we may want to
consider making these two thresholds match.
Differential Revision: https://phabricator.services.mozilla.com/D76523
2020-06-04 18:45:09 +00:00
be4e9089ec
Bug 1633866 - [taskgraph] Simplify how 'suites' are handled in manifest resolving and chunking, r=egao
...
This change is a straight refactor and results in identical full taskgraphs.
Differential Revision: https://phabricator.services.mozilla.com/D77543
2020-06-02 20:00:46 +00:00
e46cf863d7
Backed out changeset 447cbc28fda3 (bug 1641948) for Android 7 perma failures. CLOSED TREE
2020-06-04 18:55:43 +03:00
179572857b
Backed out changeset d51c3c722d65 (bug 1643335) for breaking Gecko Decision Task. CLOSED TREE
2020-06-04 18:29:19 +03:00
0f57f34c18
Bug 1643335 - Run hazard-linux64-haz/debug less often r=jmaher,sfink
...
Differential Revision: https://phabricator.services.mozilla.com/D78275
2020-06-04 15:19:38 +00:00
dd5a8a4ba7
Bug 1642713: allow diff-artifact tasks to be optimized by push interval r=tomprince
...
Differential Revision: https://phabricator.services.mozilla.com/D77859
2020-06-04 14:27:42 +00:00
3dca5e3f67
Bug 1641948: remove android opt builds from non-mozilla-central branches; reduce frequency of fat-aar builds r=snorp,Callek
...
snorp - I'm not sure if you want/need to review this yourself so I'm flagging you as optional.
Differential Revision: https://phabricator.services.mozilla.com/D77834
2020-06-03 20:52:38 +00:00
96ed60b4bd
Bug 1641863 - Upgrade Node 10.x (in preparation for ESR 78) r=Standard8. CLOSED TREE
...
Node v10.21.0 (LTS) just released https://nodejs.org/en/blog/release/v10.21.0/
Differential Revision: https://phabricator.services.mozilla.com/D77904
2020-06-03 18:00:52 +00:00
9eac413f54
Bug 1642446: Add an attribute to annotate toolchain tasks used by `mach bootstrap`; r=firefox-build-system-reviewers,rstewart
...
Toolchains that are used for local development need to be built on a level-3
branch to installable via `mach bootstrap`. Add an attribute to track the fact
that a toolchain is used that way, and:
- ensure that everything installed via `mach boostrap` has that attribute set
- ensure that everything with that attribute set is built on trunk projects
We could additionally verify that attribute is only set on things used by
bootstrap, but bootstrap doesn't currently have an exhaustive list of things
that it might install, making that difficult.
Differential Revision: https://phabricator.services.mozilla.com/D77706
2020-06-03 15:55:26 +00:00
40348063b3
Bug 1643029 - remove build-.*-aarch64 from being hidden behind --full; r=ahal
...
It's not good to hide Android AArch64 builds, a tier 1 platform, behind `--full`.
Differential Revision: https://phabricator.services.mozilla.com/D78070
2020-06-03 16:15:43 +00:00
3428cd2244
Bug 1601782 - [ci] Enable crashtest-fis on same platforms where reftest-fis is running, r=jmaher
...
This adds the following task:
test-linux1804-64-qr/debug-crashtest-fis-e10s
Differential Revision: https://phabricator.services.mozilla.com/D78030
2020-06-03 14:09:42 +00:00
b086844bd9
Bug 1642720 - Disable reftests on win7/mingw builds. r=tjr
...
These reftests seem to hang quite frequently, caused by the content process
hanging on file create/open operations. See dependencies of bug 1642719 for
some examples.
Differential Revision: https://phabricator.services.mozilla.com/D77875
2020-06-03 14:17:08 +00:00
6b213d3c19
Bug 1637810 - Define a new strategy that optimizes builds away if they are not selected by bugbug. r=ahal DONTBUILD
...
Differential Revision: https://phabricator.services.mozilla.com/D75621
2020-06-03 09:53:17 +00:00
8073961f7e
Bug 1641971: [taskgraph] Use task-reference to refer to the decision task in task; r=Callek
...
Differential Revision: https://phabricator.services.mozilla.com/D77552
2020-06-03 02:35:00 +00:00
ca6c9020de
Bug 1641971: [taskgraph] Always pass an explicit decision task id to create_tasks; r=Callek
...
Differential Revision: https://phabricator.services.mozilla.com/D77551
2020-06-03 02:35:03 +00:00
9f3450be95
Bug 1641971: [taskgraph] Pass explicit decision task id to action.json generation; r=Callek
...
Differential Revision: https://phabricator.services.mozilla.com/D77550
2020-06-03 02:35:05 +00:00
86711ecee8
Bug 1641971: Use `task-reference` to pass the decision task id to shadow schedulers; r=ahal
...
Differential Revision: https://phabricator.services.mozilla.com/D77549
2020-06-03 02:35:07 +00:00
581eba974b
Bug 1641971: Pass `mach perftest` options explicitly, rather than inpsect parameters; r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-03 02:35:09 +00:00
6617f03f05
Bug 1641971: Allow references to decision task via task-reference/artifact-reference; r=Callek
...
Differential Revision: https://phabricator.services.mozilla.com/D77547
2020-06-03 02:43:34 +00:00
01e244f62c
Bug 1641971: Allow making references to a task itself in the task definition; r=Callek
...
This is to support linking to the a specific task in taskcluster.
Differential Revision: https://phabricator.services.mozilla.com/D77529
2020-06-03 02:35:14 +00:00
12f6324f02
Bug 1623128 - [remote] Run Puppeteer unit tests on ubuntu18.04. r=egao
...
Differential Revision: https://phabricator.services.mozilla.com/D77765
2020-06-03 04:07:43 +00:00
98b889cf83
Bug 1622963 - Install python 3.6 in build docker images. CLOSED TREE
...
Differential Revision: https://phabricator.services.mozilla.com/D77753
2020-06-02 03:51:14 +00:00
479c45bbea
Backed out 7 changesets (bug 1641971) for breaking Gecko Decision Task opt . CLOSED TREE
...
Backed out changeset a039e60a26d2 (bug 1641971164197116419711641971164197116419711641971
2020-06-03 05:30:51 +03:00
658f5f0eeb
Bug 1642411: Removes unused test-macosx.sh script r=ahal
...
A grep through the repository and a brief discussion with #releng seems to point towards this script not being used.
Differential Revision: https://phabricator.services.mozilla.com/D77689
2020-06-02 18:55:35 +00:00
6f0a7bbba6
Bug 1641971: [taskgraph] Use task-reference to refer to the decision task in task; r=Callek
...
Differential Revision: https://phabricator.services.mozilla.com/D77552
2020-06-02 18:23:23 +00:00
f32e4a4f4d
Bug 1641971: [taskgraph] Always pass an explicit decision task id to create_tasks; r=Callek
...
Differential Revision: https://phabricator.services.mozilla.com/D77551
2020-06-02 18:23:19 +00:00
7c6a413ca9
Bug 1641971: [taskgraph] Pass explicit decision task id to action.json generation; r=Callek
...
Differential Revision: https://phabricator.services.mozilla.com/D77550
2020-06-02 18:23:20 +00:00
73b2e6f745
Bug 1641971: Use `task-reference` to pass the decision task id to shadow schedulers; r=ahal
...
Differential Revision: https://phabricator.services.mozilla.com/D77549
2020-06-02 18:23:11 +00:00
11aedca9b2
Bug 1641971: Pass `mach perftest` options explicitly, rather than inpsect parameters; r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-02 18:23:07 +00:00
602da9cdcd
Bug 1641971: Allow references to decision task via task-reference/artifact-reference; r=Callek
...
Differential Revision: https://phabricator.services.mozilla.com/D77547
2020-06-02 18:50:10 +00:00
de6126516d
Bug 1641971: Allow making references to a task itself in the task definition; r=Callek
...
This is to support linking to the a specific task in taskcluster.
Differential Revision: https://phabricator.services.mozilla.com/D77529
2020-06-02 18:22:55 +00:00
4925d4d08e
Bug 1638990: Use requests for fetching parameters r=tomprince
...
Differential Revision: https://phabricator.services.mozilla.com/D76133
2020-05-28 19:01:42 +00:00
b6edad89bf
Bug 1638990: Sort values in taskgraph in order to make comparing output easier r=rstewart
...
Differential Revision: https://phabricator.services.mozilla.com/D77320
2020-05-29 15:01:50 +00:00
273a7905e6
Bug 1635826 - platform docs: Document WebRender tasks r=jbonisteel
...
Differential Revision: https://phabricator.services.mozilla.com/D77677
2020-06-01 15:28:13 +00:00
c5994e6bba
Bug 1628289 - platform docs: the Documentation jobs moved away from the linting section
...
Depends on D76185
Differential Revision: https://phabricator.services.mozilla.com/D77657
2020-06-01 15:28:13 +00:00
58f742cd8b
Bug 1635826 - platform docs: Remove the nightly platforms r=Callek
...
Remove in treeherder in this patch:
1568eaf479https://phabricator.services.mozilla.com/D76185
2020-06-01 11:11:22 +00:00
3efe2edb5c
Bug 1641395 - install amd64 variants of existing i386 packages and insert additional startup parameters for pulseaudio r=jmaher
...
Changes:
- install the `amd64` variants of various packages alongside `i386` packages
- install `amd64` version of `fontconfig` instead
- additional command line arguments for pulseaudio initialization
- do not add configuration files to `~/.pulse` in the docker image
Differential Revision: https://phabricator.services.mozilla.com/D77512
2020-06-01 20:44:47 +00:00
4e005a2cbd
Bug 1641711 - Update artifacts for mozlint-android. r=nalexander
...
Differential Revision: https://phabricator.services.mozilla.com/D77544
2020-06-01 15:57:58 +00:00
98dd864200
No bug: Don't use proxy for accessing taskcluster index; r=Callek
...
The index is not protected by scopes, so don't use the proxy to access it
Differential Revision: https://phabricator.services.mozilla.com/D77537
2020-05-29 21:37:46 +00:00
7e13ebcb0a
Bug 1628289 - move documentation jobs from lint/opt to doc/opt. r=sylvestre
...
Differential Revision: https://phabricator.services.mozilla.com/D77481
2020-05-29 21:12:13 +00:00
2a66b39bc4
Bug 1641962: Defer imports of distutils.util to ensure bootstrap can run r=rstewart,perftest-reviewers,sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D77516
2020-05-29 20:17:27 +00:00
f0e5f6ed7b
Bug 1633866 - [taskgraph] Create a system to choose how test manifests are loaded via a parameter, r=tomprince
...
Currently test manifests are loaded by instantiating a TestResolver and
traversing moz.build files to find manifests.
With 'manifest-scheduling', we'll want to grab the manifests directly from the
bugbug service instead. Initially we'll want to enable manifest-scheduling with
|mach try auto|, but not on autoland yet.
This patch will allow |mach try auto| to set the parameter that causes bugbug
to be used (see future commits in this bug).
Differential Revision: https://phabricator.services.mozilla.com/D76522
2020-05-29 18:51:14 +00:00
Simon Fraser
Marco Castelluccio
Andrew Halberstadt
Razvan Maries
Sylvestre Ledru
Ben Hearsum
Ed Lee
Tom Prince
Nathan Froyd
Kartikaya Gupta
Henrik Skupin
Mike Hommey
Narcis Beleuzu
Mitchell Hentges
Chris AtLee
Edwin Takahashi
Agi Sferro
emilfs