Граф коммитов

8736 Коммитов

Автор SHA1 Сообщение Дата
Tom Prince e837225a92 Bug 1638990: [taskgraph] Make test_generator pass on python 3; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D84075
2020-07-20 19:26:59 +00:00
Tom Prince 65572b1ec0 Bug 1638990: [taskgraph] Make test_decision pass on python 3; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D84074
2020-07-20 19:26:57 +00:00
Tom Prince 066d97917f Bug 1638990: [taskgraph] Make test_try_option_syntax pass on python 3; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D84073
2020-07-20 19:26:47 +00:00
Tom Prince 7c05ff5c43 Bug 1653717: [taskgraph] Handle composite description nicely; r=Callek
When links to specific tasks in treeherder was added to the description in
Bug 1630710, This caused the descriptions for several derived tasks to include
the `{'task-reference': '...'}` wrapper in the derived description. This is
in addition to the already duplicate links to treeherder those tasks had.

To ameliorate this issue, keep track of the original description without the
treeherder link and use that in derived tasks.

Differential Revision: https://phabricator.services.mozilla.com/D84047
2020-07-18 15:59:27 +00:00
Kartikaya Gupta c1424e1a0c Bug 1643847 - Allow running talos-profiling jobs on QR platforms. r=perftest-reviewers,sparky
This adds the talos-profiling jobs to the relevant test-platforms so that they
can be run for Linux and Windows shippable builds with WebRender enabled.
However they are also disabled from running on m-c by default; they must be
run explicitly using `./mach try fuzzy --full` or using the TreeHerder add jobs
feature.

Differential Revision: https://phabricator.services.mozilla.com/D84011
2020-07-20 18:14:26 +00:00
Kartikaya Gupta 3424be1f06 Bug 1643847 - Remove redundant run-on-projects block. r=ahal
This block is identical to the run-on-projects applied to all the talos
profiling jobs in the defaults section at the top of the file. Therefore
it is redundant.

Differential Revision: https://phabricator.services.mozilla.com/D84010
2020-07-17 21:14:25 +00:00
Kartikaya Gupta 38770c344a Bug 1486004 - Part 18 - Update wrench to use mozdevice 4.0.0 on bitbar. r=gbrown
Since the system mozdevice version on bitbar is old, and mozharness now requires
mozdevice 4.0.0, we simply have the build job make a tarball of mozdevice 4.0.0
from the gecko source tree and export it as an artifact. The wrench job that
runs on bitbar can unpack the tarball and add it to the python path so it gets
picked up.

Differential Revision: https://phabricator.services.mozilla.com/D84070
2020-07-20 15:51:45 +00:00
Andrew Halberstadt 85e7c622aa No bug: [taskgraph] Simplify bugbug strategies; r=ahal
This change the strategy aliases that have bugbug, so that they handle the extra
arguments that are needed by bugbug.

Differential Revision: https://phabricator.services.mozilla.com/D83438
2020-07-20 15:10:41 +00:00
Johan Lorenzo b651a3366d Bug 1631839 - Update mobile routes to use new namespace r=tarek,sparky,acreskey
Differential Revision: https://phabricator.services.mozilla.com/D81038
2020-07-17 16:00:57 +00:00
Gregory Mierzwinski b0fde5b680 Bug 1652508 - Add chimera pageload testing mode. r=tarek
This patch adds a new pageload testing mode called chimera mode. This mode runs the cold and warm pageload tests together within a single browser session (cold first, warm second). Some changes had to be made in the visual metrics task to be able to do this (per-test specification of extraOptions).

Differential Revision: https://phabricator.services.mozilla.com/D83356
2020-07-17 22:26:07 +00:00
Butkovits Atila 96f3218871 Backed out 3 changesets (bug 1648851) for webgl issues. CLOSED TREE
Backed out changeset 1f0d6e32a0f4 (bug 1648851)
Backed out changeset df5ec3dbab05 (bug 1648851)
Backed out changeset fbda45ebde5c (bug 1648851)
2020-07-20 12:44:43 +03:00
Mihai Alexandru Michis d37fd5ad51 Backed out changeset 5e3a55fa20f8 for causing Gecko Decision Task bustages on try. a=backout
DONTBUILD
2020-07-19 21:49:30 +03:00
Andrew Halberstadt 53afe3b2f2 No bug: [taskgraph] Simplify bugbug strategies; r=ahal
This change the strategy aliases that have bugbug, so that they handle the extra
arguments that are needed by bugbug.

Differential Revision: https://phabricator.services.mozilla.com/D83438
2020-07-18 04:51:06 +00:00
Tom Prince 41b5c511be Bug 1652086: [taskgraph] Don't compile schemas if we aren't going to check them; r=Callek
I'm planning on replacing voluptuous with something faster in Bug 1652123, but
in the meantime, compile the schema is slow, so there is no reason to do so if
we aren't going to check them anyway.

Differential Revision: https://phabricator.services.mozilla.com/D83540
2020-07-18 05:02:45 +00:00
Tom Prince a72836e1a2 No bug: Cleanup language. DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D84071
2020-07-18 04:56:52 +00:00
Edwin Takahashi 857b749921 Bug 1648851 - Part 2: create android-specific bootstrapping script r=jmaher,gbrown
Changes:
  - create a `test-android.sh` script that is based on, but removes non-android related paths from the `test-linux.sh` script.

Differential Revision: https://phabricator.services.mozilla.com/D82252
2020-07-17 22:02:41 +00:00
Edwin Takahashi 81fbf89023 Bug 1648851 - Part 0: create android-test docker image based on Ubuntu 20.04 r=jmaher,gbrown
Changes:
  - set up a new minimal docker image for `android-em` tests.
  - reorganize commonly referenced files into a specific folder `taskcluster/docker/files` to reduce instances of same file being copy-pasted into directories.
  - add android-test docker build task to taskcluster.

Differential Revision: https://phabricator.services.mozilla.com/D82204
2020-07-17 20:19:41 +00:00
Butkovits Atila 1d11c9f1df Backed out changeset a3af9b09a6ea (bug 1648851) for causing failure at test_2_conformance2__glsl3__matrix-row-major-dynamic-indexing.html. CLOSED TREE 2020-07-17 21:49:40 +03:00
Edwin Takahashi 1e84a04fb4 Bug 1648851 - Part 0: create android-test docker image based on Ubuntu 20.04 r=jmaher,gbrown
Changes:
  - set up a new minimal docker image for `android-em` tests.
  - reorganize commonly referenced files into a specific folder `taskcluster/docker/files` to reduce instances of same file being copy-pasted into directories.
  - add android-test docker build task to taskcluster.

Differential Revision: https://phabricator.services.mozilla.com/D82204
2020-07-16 23:36:01 +00:00
Razvan Maries 2af0f9a132 Backed out 4 changesets (bug 1648851) for perma failues on test_2_conformance2__glsl3__matrix-row-major-dynamic-indexing.html and test_2_conformance__state__gl-get-calls.html. CLOSED TREE
Backed out changeset 2d2686a389dc (bug 1648851)
Backed out changeset e05b2f2fdbb2 (bug 1648851)
Backed out changeset b576e680a9fa (bug 1648851)
Backed out changeset 901f16e24a60 (bug 1648851)
2020-07-17 02:19:47 +03:00
Edwin Takahashi 53eace3ac3 Bug 1648851 - Part 3: add ability to specify bootstrap script r=jmaher
Changes:
  - in `mozharness_test.py`, add new optional key `job-script` similar to existing provision for toolchain tasks.
  - set the default bootstrapping script depending on platform.
  - substitute the correct script in the final mozharness call.

Differential Revision: https://phabricator.services.mozilla.com/D83559
2020-07-16 14:00:27 +00:00
Edwin Takahashi cdce692d0a Bug 1648851 - Part 2: create android-specific bootstrapping script r=jmaher,gbrown
Changes:
  - create a `test-android.sh` script that is based on, but removes non-android related paths from the `test-linux.sh` script.

Differential Revision: https://phabricator.services.mozilla.com/D82252
2020-07-14 22:51:40 +00:00
Edwin Takahashi 92d7799961 Bug 1648851 - Part 1: use android-test docker image for android-em test tasks r=jmaher
Changes:
  - when the taskgraph is being generated for android-em tasks, use the new `android-test` docker image.

Differential Revision: https://phabricator.services.mozilla.com/D82478
2020-07-14 22:49:10 +00:00
Edwin Takahashi afd4560cb8 Bug 1648851 - Part 0: create android-test docker image based on Ubuntu 20.04 r=jmaher,gbrown
Changes:
  - set up a new minimal docker image for `android-em` tests.
  - reorganize commonly referenced files into a specific folder `taskcluster/docker/files` to reduce instances of same file being copy-pasted into directories.
  - add android-test docker build task to taskcluster.

Differential Revision: https://phabricator.services.mozilla.com/D82204
2020-07-08 17:46:37 +00:00
Kartikaya Gupta bb263023fa Bug 1653279 - Expand set of file changes that trigger android wrench jobs. r=ahal
This should help catch failures due to mozharness refactorings/changes faster.

Differential Revision: https://phabricator.services.mozilla.com/D83793
2020-07-16 20:30:07 +00:00
Marco Castelluccio b69737572a Bug 1653030 - Use Python's not operator
Differential Revision: https://phabricator.services.mozilla.com/D83849
2020-07-16 20:38:14 +00:00
Marco Castelluccio ab73b2650b Bug 1653030 - Make 'All' optimization strategy return early as soon as one of its substrategies returns False. r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D83715
2020-07-16 20:24:39 +00:00
Andrew Creskey d1317020f2 Bug 1653293 - Disable Moto G5 performance tuning for perftests until it is tuned r=sparky
Disabled because we've observed that this is introducing noise on G5.
Also added perftuning to the other Pixel 2 tests.

Differential Revision: https://phabricator.services.mozilla.com/D83808
2020-07-16 17:00:20 +00:00
Cosmin Sabou 1129658f1a Backed out 2 changesets (bug 1652870) for bustages on Android GeckoView multi-arch fat AAR.
Backed out changeset c279e71779dc (bug 1652870)
Backed out changeset 0d8e1da6bb40 (bug 1652870)
2020-07-15 21:50:08 +03:00
Nathan Froyd 6c7c60a3ca Bug 1652870 - use new dump_syms for Android builds; r=geckoview-reviewers,snorp
Faster and more maintainable is good.

Differential Revision: https://phabricator.services.mozilla.com/D83561
2020-07-15 12:59:00 +00:00
Dorel Luca 775a192261 Backed out 2 changesets (bug 1652870) for Android bustage. CLOSED TREE
Backed out changeset 48a10a9249b0 (bug 1652870)
Backed out changeset 94ffbec77e89 (bug 1652870)
2020-07-15 15:35:45 +03:00
Nathan Froyd c7df38111d Bug 1652870 - use new dump_syms for Android builds; r=geckoview-reviewers,snorp
Faster and more maintainable is good.

Depends on D83560

Differential Revision: https://phabricator.services.mozilla.com/D83561
2020-07-14 21:33:12 +00:00
Tom Prince 198865bdd3 Bug 1652678: [taskgraph] Fix `test_transforms_job`.
Differential Revision: https://phabricator.services.mozilla.com/D83470
2020-07-14 21:12:01 +00:00
David Major 2e6b6721b9 Bug 1652600 - Rename build-clang-linux-{win,macosx}-cross.sh to 'repack' r=froydnj
These scripts don't call `build-clang.py`, they just repackage artifacts from other tasks that do.

I went with `repack` over `repackage` since that seems to be the established pattern in `taskcluster/scripts/misc/`.

Differential Revision: https://phabricator.services.mozilla.com/D83532
2020-07-14 15:55:20 +00:00
Tom Prince e0998f9228 Bug 1652786: [taskgraph] Fix parameter handling `load_tasks_for_kind; r=Callek a=Callek
DONTBUILD

Differential Revision: https://phabricator.services.mozilla.com/D83566
2020-07-14 21:09:34 +00:00
Nathan Froyd 94d53dfdef Bug 1652054 - add a job name for the codeql build job; r=nalexander,tomprince
This change doesn't seem to affect anything now, but when I was pulling out
some bits into `job-defaults`, the lack of a `job-name` resulted in some
extremely cryptic errors from inside taskcluster code.

Differential Revision: https://phabricator.services.mozilla.com/D83127
2020-07-14 16:07:48 +00:00
Nathan Froyd 0724eb65cb Bug 1652759 - mark all dump_syms toolchains as available for local development; r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D83512
2020-07-14 16:11:22 +00:00
Nathan Froyd aa851e0e30 Bug 1652098 - move linux base toolchain builds to their own file; r=dmajor
The explanatory comment at the top of the new file provides some context.
The larger motivating context here is wanting to use `job-defaults` to clean
up the toolchain fetches for the Linux build jobs.  But if we did that, we
run into the problem that such fetches are merged with whatever the
individual job(s) specify for their toolchain fetches, not replaced.  So
we'd wind up with multiple clang toolchains being downloaded, or similar,
and things would go downhill from there, as workers attempt to unpack those
things to the same location (which is bad) or perhaps error out (which is
also bad).

This change therefore paves the way for providing `job-defaults` in linux.yml.

Differential Revision: https://phabricator.services.mozilla.com/D83158
2020-07-14 13:55:31 +00:00
David Major 82ce441b57 Bug 1616694 - Turn the macosx-cross toolchain task into a mere repackage r=nalexander
Currently the macosx-cross toolchain build pulls in a linux64-clang toolchain, uses it to build a Mac native toolchain, and then clobbers the result with pieces of the Linux toolchain. This means that the same version of LLVM is used to build the Mac pieces and be part of the final artifact. This will become a problem with upcoming LLVM 11 where we'll want to build the Mac pieces with LLVM 9 but otherwise repackage an LLVM 11 Linux toolchain.

So this commit makes the macosx-cross workflow look more like the win-cross one: take two compilers built elsewhere and just merge them.

Differential Revision: https://phabricator.services.mozilla.com/D83378
2020-07-14 11:33:50 +00:00
Tom Prince e92bbf5dd8 No bug: [taskgraph] Make `verify_test_packaging` handle `--target-kind` gracefully; r=Callek
If we are generating only a part of the graph, to given kind, don't fail if a
build is packaging tests and there is no corresponding test task, as the tests
may not have been generated.

Differential Revision: https://phabricator.services.mozilla.com/D82097
2020-07-14 07:17:26 +00:00
Tom Prince 51f106a3a3 Bug 1652086: Make `kind_dependencies_tasks` a dictionary based on the label; r=Callek
I was looking at py-spy for generating the taskgraph, and found that a bunch
of time was spent in taskgraph.transforms.job.use_fetches[1]. Use a dictionary
there instead saves about 20-30s on my machine.

[1] https://searchfox.org/mozilla-central/rev/622dbd3409610ad3f71b56c9a6a92da905dab0aa/taskcluster/taskgraph/transforms/job/__init__.py#243-247

Differential Revision: https://phabricator.services.mozilla.com/D83166
2020-07-14 07:39:20 +00:00
David Major 63aa7238ca Bug 1616694 - Allow build-clang to work with different Mac SDKs r=froydnj
LLVM 11 introduces a hard requirement for SDK 10.12 in order to build for Mac. We want to keep building older LLVMs with 10.11 though, so this patch adds some flexibility so that build-clang can make use of whatever SDK package a particular task pulls from tooltool (but still requesting a deployment target of 10.11).

Differential Revision: https://phabricator.services.mozilla.com/D82621
2020-07-13 22:44:54 +00:00
Tom Prince 1691c9892d Bug 1652184: Report to perherder stats about bugbug timings; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D83212
2020-07-13 19:53:00 +00:00
Gregory Mierzwinski a009f016fa Bug 1652451 - Disable 2020 youtube-playback tests temporarily. r=perftest-reviewers,davehunt
This patch disables all failing youtube-playback tests.

Differential Revision: https://phabricator.services.mozilla.com/D83368
2020-07-13 19:29:12 +00:00
Tarek Ziadé 7dfca9db55 Bug 1644778 - add a mozperftest to measure browser.xhtml DOM size, r=tarek,dhouse,sparky
Differential Revision: https://phabricator.services.mozilla.com/D79152
2020-07-13 15:44:38 +00:00
Tom Prince e460c39b9a Bug 1649168: Add title to sciptworker-canary action.
Differential Revision: https://phabricator.services.mozilla.com/D83362
2020-07-13 17:35:36 +00:00
Nathan Froyd 9bdf21a5da Bug 1652089 - update dump_syms version; r=firefox-build-system-reviewers,rstewart
This change pulls in the ELF and MachO support and few other miscellaneous
things.

Differential Revision: https://phabricator.services.mozilla.com/D83151
2020-07-13 13:07:27 +00:00
Edwin Takahashi 8e3cdb9917 Bug 1652128 - only schedule web-platform-tests-backlog suites on opt variant of each platform r=jmaher
Changes:
  - removes the ability to schedule web-platform-tests-reftest-backlog and web-platform-tests-backlog from non-opt variants of each platform.
  - in addition, removes the ability to schedule the above from windows7-32, regardless of the variant.
  - clean up references to now-deprecated platforms in backlog task definitions.

Differential Revision: https://phabricator.services.mozilla.com/D83172
2020-07-10 21:08:31 +00:00
Ricky Stewart c60f8ad9aa Bug 1652149 - Run `mozwebidlcodegen` unit test as part of `mozbuild` suite r=froydnj
We could make a new task for this, but `mozwebidlcodegen` depends on code in `mozbuild`, and vice-versa, so there doesn't really seem to be any meaningful advantage to that.

Differential Revision: https://phabricator.services.mozilla.com/D83187
2020-07-13 13:02:01 +00:00
Nathan Froyd d9e3a915cf Bug 1652100 - use job-defaults for macosx builds; r=nalexander
This change reduces clutter.

Differential Revision: https://phabricator.services.mozilla.com/D83159
2020-07-10 20:16:05 +00:00
Nathan Froyd bbdcb7d9d4 Bug 1652095 - don't repeatedly look up values when merging dicts; r=tomprince
We already have `value = source[key]` thanks to iteration; we should be
using `value` directly rather than re-doing `source[key]`.

Differential Revision: https://phabricator.services.mozilla.com/D83157
2020-07-10 20:43:41 +00:00
Nathan Froyd 89cf8970bb Bug 1652142 - add binutils toolchain to {fuzzing-,}ccov builds; r=firefox-build-system-reviewers,rstewart
This change brings the aforementioned toolchains to parity with the
asan-fuzzing-ccov toolchain.

Differential Revision: https://phabricator.services.mozilla.com/D83185
2020-07-10 21:45:25 +00:00
Sylvestre Ledru f7a0736a34 Bug 1617369 - Reformat recent rust changes with rustfmt r=emilio
# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D83259
2020-07-12 17:01:13 +00:00
David Major 6c935d09d1 Bug 1630092 - Don't pull in wasm sandboxing in plain or base-toolchain builds r=froydnj
The `plain` builds are intended to make sure that building with an empty mozconfig doesn't break. Since an empty mozconfig build wouldn't have wasm sandboxing, neither should the `plain` builds.

While here, also remove wasi/lucet dependencies from the base-toolchain tasks, since they don't enable sandboxing either.

Differential Revision: https://phabricator.services.mozilla.com/D83011
2020-07-11 12:48:06 +00:00
Kris Maglione 9d78661f88 Bug 1649221: Update ChromeUtils.generateQI callers to pass strings. r=mccr8,remote-protocol-reviewers,marionette-reviewers,perftest-reviewers,webcompat-reviewers,geckoview-reviewers,preferences-reviewers,agi,whimboo,Bebe,twisniewski
Differential Revision: https://phabricator.services.mozilla.com/D81594
2020-07-10 23:58:28 +00:00
Mitchell Hentges e1aa2a866a Bug 1629337: Implements incremental dafsa algorithm r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D76893
2020-07-10 21:08:10 +00:00
Cosmin Sabou 72bb6a7f58 Backed out 2 changesets (bug 1652089, bug 1652087) for toolchains bustage.
Backed out changeset 617233706940 (bug 1652087)
Backed out changeset ebcebafde8e0 (bug 1652089)
2020-07-11 00:48:41 +03:00
Ricky Stewart e95ebbb152 Bug 1652097 - Run `featuregates` Python unit tests in separate task and not in `make check` r=mythmon
The unit test is broken under Python 3 on macOS, so I haven't included any macOS version of this task; one should probably be added if/when that's fixed.

Differential Revision: https://phabricator.services.mozilla.com/D83169
2020-07-10 20:54:12 +00:00
Mike Hommey ed90d5c5e6 Bug 1651938 - Update sccache to pick up basic `/clang:` support. r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D83050
2020-07-10 20:53:19 +00:00
Nathan Froyd 60d15a3571 Bug 1652089 - update dump_syms version; r=firefox-build-system-reviewers,rstewart
This change pulls in the ELF and MachO support and few other miscellaneous
things.

Differential Revision: https://phabricator.services.mozilla.com/D83151
2020-07-10 20:32:57 +00:00
Narcis Beleuzu d32b539f51 Bug 1649168 - Fix lint failure a=lint-fix. CLOSED TREE 2020-07-10 19:51:24 +03:00
Narcis Beleuzu 3445ca16f5 Bug 1649168 - Fix gecko decision task failure. r=fix CLOSED TREE 2020-07-10 19:31:46 +03:00
Tom Prince 388e735d63 Bug 1649168: Add scriptworker-canary action and corresponding task; r=rail
Add an action that will trigger a task that runs
`mach release push-scriptworker-canary`
to test a new scriptworker deployment.

Differential Revision: https://phabricator.services.mozilla.com/D82821
2020-07-10 16:12:05 +00:00
Tom Prince ee007d06fc Bug 1649168: Add a new docker-image with the tools for pushing to try; r=rail
Differential Revision: https://phabricator.services.mozilla.com/D82820
2020-07-10 16:11:03 +00:00
Nathan Froyd 726b4f3b99 Bug 1649757 - don't upload artifacts for various SM builds; r=sfink
The artifacts (`js`, `jsapi-tests`) generated by these builds are rather
large, and we don't need them for anything.

Differential Revision: https://phabricator.services.mozilla.com/D81880
2020-07-10 15:53:38 +00:00
Nathan Froyd df458c1e95 Bug 1651410 - include updated binutils for clang 5 toolchain; r=firefox-build-system-reviewers,glandium
This change brings this toolchain into parity with other clang toolchains.

Differential Revision: https://phabricator.services.mozilla.com/D82741
2020-07-10 12:51:36 +00:00
Tom Prince 204d796929 Bug 1651731: [lint] Python and shell files without `#!` should not be executable; r=linter-reviewers,perftest-reviewers,geckoview-reviewers,agi,sylvestre,sparky
Differential Revision: https://phabricator.services.mozilla.com/D82954
2020-07-09 20:29:18 +00:00
Tom Prince 2c52622b85 Bug 1651731: [lint] Add a bunch more types of files that should never be executable; r=linter-reviewers,perftest-reviewers,geckoview-reviewers,preferences-reviewers,agi,sylvestre,sparky
Differential Revision: https://phabricator.services.mozilla.com/D82953
2020-07-09 21:28:49 +00:00
Andrew Halberstadt d5e8716e90 Bug 1640902 - [taskcluster.test] Add some additional checks to test_mach_try_auto.py, r=marco
Adds some additional assertions against what would be ran with |mach try auto|.
This also fixes a legit bug that the 'test_important_manifests_only' test
caught.

Differential Revision: https://phabricator.services.mozilla.com/D81404
2020-07-09 12:46:49 +00:00
Andrew Halberstadt 8dc68edc8e Bug 1640902 - [ci] Add integration tests for the 'taskcluster' directory, r=tomprince
Initially this suite will only include a test for |mach try auto| pushes.

Differential Revision: https://phabricator.services.mozilla.com/D81403
2020-07-08 18:48:26 +00:00
Andrew Halberstadt 0d3d8d4744 Bug 1640902 - [python-test] Add ability to mark tests as "slow" and an arugment to run them, r=raphael
This creates a general pytest plugin under "config/mozunit" which implements
the ability to mark tests as "slow". Slow tests will be skipped by default
unless "--run-slow" is passed in.

Differential Revision: https://phabricator.services.mozilla.com/D81402
2020-07-09 17:46:07 +00:00
Andrew Halberstadt eadca174d1 Bug 1651677 - [taskgraph] Remove 'xfail' from tests that were fixed by bug 1650067, r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D82952
2020-07-09 17:46:07 +00:00
Armen Zambrano G. 5e2be1d274 Bug 1650224 - Support backfilling of manifest scheduled tasks r=tomprince
This adds the support for backfilling of manifest scheduled tasks while
preserving the existing features (e.g. schedule a specific test path).

Remove temporary new-backfill action.

Differential Revision: https://phabricator.services.mozilla.com/D82599
2020-07-09 15:20:58 +00:00
Armen Zambrano G. 28728bfafb Bug 1650224 - Drop support of testPath from backfill action r=tomprince
This optional mode to specify a test path when backfilling has been broken for a while
and is not being used to make it worth fixing. It is a mode
originally designed to be used with the test-verify tasks.

Differential Revision: https://phabricator.services.mozilla.com/D82722
2020-07-09 06:52:08 +00:00
Andrew Halberstadt 2e0601326b Bug 1650067 - [taskgraph] Fix some Python 3 compatibility issues in the 'optimize' directory, r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D81577
2020-07-08 18:19:24 +00:00
Andi-Bogdan Postelnicu 222c13b10b Bug 1648710 - reduce `max-run-time` for coverity full tree analysis. r=marco
Differential Revision: https://phabricator.services.mozilla.com/D82871
2020-07-09 12:40:43 +00:00
Florin Strugariu f0e6aa38e1 Bug 1531358 - [meta] ensure windows 10 ux machines running performance are only running on mozilla-central/try as tier-2 r=perftest-reviewers,sparky
Differential Revision: https://phabricator.services.mozilla.com/D81669
2020-07-09 09:04:26 +00:00
Chris AtLee 676e2f71c7 Bug 1555560: Run black format check in automation; r=ahal,sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D33126
2020-07-08 22:08:06 +00:00
Mike Hommey 675ed50acd Bug 1651549 - Remove now unnecessary order_image_tasks. r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D82810
2020-07-09 03:54:46 +00:00
Mike Hommey 459cfbe348 Bug 1651227 - Use the in-tree kaniko image builder image to build docker images. r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D82622
2020-07-09 03:54:46 +00:00
Mihai Alexandru Michis 5f385df474 Backed out 2 changesets (bug 1555560) for causing multiple lint failures.
CLOSED TREE

Backed out changeset bd9460ac6e48 (bug 1555560)
Backed out changeset f2896032ced6 (bug 1555560)
2020-07-09 01:04:18 +03:00
Chris AtLee 92897f1567 Bug 1555560: Run black format check in automation; r=ahal,sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D33126
2020-07-08 20:44:41 +00:00
Tom Prince f27be8d7e7 No bug: Be strict about treeherder symbol splitting; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D82624
2020-07-08 20:02:26 +00:00
Dorel Luca 7ff1c0fb61 Backed out changeset b17f8ba4211d for Gecko Decision task bustage. CLOSED TREE 2020-07-08 22:53:59 +03:00
Tom Prince 440eed985f Bug 1643817: Use a fresh taskgroup for create-interactive tasks; r=Callek
In Bug 1641971 (e7d4c490c31a) we changed `taskgraph.create.create_tasks` to
always get the decision task id to use. However, the call in create-interactive
passed `None`, expecting to have on generated. This changes that to use the
`create-interactive`s action task as the task id.

Differential Revision: https://phabricator.services.mozilla.com/D82652
2020-07-08 18:41:17 +00:00
Tom Prince fc3ae614bc No bug: Update links to hg.mozilla.org to use `default` instead of `tip` in `taskcluster/`; r=Callek
The latter sometimes bites us if we have a legacy support branch that gets
pushed more recently than default.

Differential Revision: https://phabricator.services.mozilla.com/D82628
2020-07-08 18:41:17 +00:00
Tom Prince 927f176328 No bug: Be strict about treeherder symbol splitting; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D82624
2020-07-08 18:18:57 +00:00
Edwin Takahashi 02a12126c0 Bug 1648851 - reduce max-run-time value of android-api-16-shippable/opt task to 1800 r=gbrown,jmaher
Changes:
  - max-run-time reduced to 1800 which should suffice.

Differential Revision: https://phabricator.services.mozilla.com/D82253
2020-07-08 17:46:37 +00:00
emilfs dafec58d06 Bug 1643293 - added new job in doc to render documentation at review phase r=sylvestre,ahal
Differential Revision: https://phabricator.services.mozilla.com/D80428
2020-07-08 13:53:45 +00:00
Christian Holler fcbc0dd643 Bug 1651243 - Remove leftover crashsymbols attribute from fuzzing debug job. r=truber
Differential Revision: https://phabricator.services.mozilla.com/D82683
2020-07-08 12:14:15 +00:00
alexandru.ionescu 1feda07409 Bug 1651353 Re-enable youtube widevine tests r=sparky,perftest-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D82692
2020-07-08 12:38:10 +00:00
Gregory Mierzwinski 8d10f86507 Bug 1650871 - Add all browsertime desktop tests. r=perftest-reviewers,AlexandruIonescu
This patch adds all desktop pageload tests to browsertime. It adds taskcluster tasks for these but doesn't enable them in production. They can only be run when --full is used.

Differential Revision: https://phabricator.services.mozilla.com/D82432
2020-07-08 07:48:16 +00:00
Gregory Mierzwinski 6e1d467ad0 Bug 1650871 - Add option to disable firefox window recorder. r=perftest-reviewers,AlexandruIonescu
This patch adds an option to disable the firefox window recorder in browsertime, it also adds it to the browsertime-mobile taskcluster config since it is disabled there (but not on desktop).

Differential Revision: https://phabricator.services.mozilla.com/D82431
2020-07-08 07:50:11 +00:00
Tom Prince a22772e2a2 Bug 1626058: Build docker images with kaniko, instead of dind; r=glandium,hwine
Differential Revision: https://phabricator.services.mozilla.com/D77864
2020-07-07 19:53:32 +00:00
Tom Prince 033c399c15 Bug 1626058: Don't add prefix when building a docker-context at graph generation time; r=glandium
We were building a context with a top-level directory matching the image name
during graph generation. This is not accepted by docker build. This didn't
cause issues since we only used it for digest generation. Since we are
switching to use the decision generated context, change this to match the
context being generated when building an image.

Differential Revision: https://phabricator.services.mozilla.com/D77841
2020-07-07 19:53:17 +00:00
Tom Prince d64ad0967b Bug 1626058: Add support for writting out artifacts from transforms in the decision task; r=Callek,ahal
To support using kaniko[1] for building images, we need to generate the docker
contexts in a seprate task from the docker-image task. Since we use the hash of
the context as the cached-task digest, we generate the context in the decision
task already, so this adds a way to write that out to be used by downstream
tasks.

[1] https://github.com/GoogleContainerTools/kaniko

Differential Revision: https://phabricator.services.mozilla.com/D77839
2020-07-07 19:56:11 +00:00
Tom Prince 50513f07b4 Bug 1637544: Use sccache on instrumented builds on non-release branches; r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D82485
2020-07-07 16:39:28 +00:00
Tom Prince cfe40e25ca Bug 1637544: Set `use-sccache` explicitly, rather than based on an sccache toolchain being provided; r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D82484
2020-07-07 16:39:31 +00:00
Tom Prince 7ed26bf1c6 Bug 1637544: Rename `needs-sccache` to `use-sccache`; r=glandium
I'm going to change this from being autodected, to specified explicitly, and so
`use-sccache` better captures the meaning.

Differential Revision: https://phabricator.services.mozilla.com/D82483
2020-07-07 16:39:33 +00:00
Tom Prince aa8615681b No bug: Remove unused `desktop1604-test` image; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D82543
2020-07-07 16:17:16 +00:00
Justin Wood bff89afddf Bug 1649383 - don't do geckoview builds on esr. r=jcristau
Differential Revision: https://phabricator.services.mozilla.com/D81742
2020-07-07 16:09:35 +00:00
Mihai Alexandru Michis a8ccb677a1 Bug 1648591 - Fix failures regarding test_cron_util.py a=fix
CLOSED TREE
2020-07-07 19:35:38 +03:00
Mihai Alexandru Michis 5acb93239f Backed out 3 changesets (bug 1637544) for causing gecko decision task bustages.
CLOSED TREE

Backed out changeset 93b955e5c048 (bug 1637544)
Backed out changeset be0717d76643 (bug 1637544)
Backed out changeset 447fea64b68d (bug 1637544)
2020-07-07 19:31:41 +03:00
Tom Prince 9c3f3b3519 Bug 1637544: Use sccache on instrumented builds on non-release branches; r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D82485
2020-07-07 08:39:25 +00:00
Tom Prince 2d287e7d96 Bug 1637544: Set `use-sccache` explicitly, rather than based on an sccache toolchain being provided; r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D82484
2020-07-07 08:39:20 +00:00
Tom Prince 8a8dbbb549 Bug 1637544: Rename `needs-sccache` to `use-sccache`; r=glandium
I'm going to change this from being autodected, to specified explicitly, and so
`use-sccache` better captures the meaning.

Differential Revision: https://phabricator.services.mozilla.com/D82483
2020-07-07 08:39:19 +00:00
Tom Prince b89d332bae No bug: Don't include push-id in treeherder routes; r=aki
Treeherder ignores this.

Differential Revision: https://phabricator.services.mozilla.com/D82251
2020-07-03 23:37:49 +00:00
Tom Prince 909e7275e8 Bug 1648591: Remove taskgraph based cron implementation; r=aki
Differential Revision: https://phabricator.services.mozilla.com/D81270
2020-07-07 16:04:00 +00:00
David Major 06d9c7e99f Bug 1650239 - Build llvm-dsymutil out of the clang-10 source tree r=firefox-build-system-reviewers,rstewart
The current version hits OOM errors when dsymutil-ing files created by clang 10 and 11.

The choice of clang-10 here is somewhat arbitrary in that it's a fetch job that we conveniently already had in the tree. It doesn't have to be exactly version 10 specifically.

Differential Revision: https://phabricator.services.mozilla.com/D82453
2020-07-06 21:42:42 +00:00
James Graham 6fdb192e27 Bug 1649866 - Promote wpt print-reftests to tier-1 on desktop, r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D81979
2020-07-02 12:32:44 +00:00
David Major d38d9b8bdb Bug 1650239 - Update cctools-port to make it buildable with clang 11 r=froydnj
Fixes the bustage described in 4e1faecce8

In theory I could leave this task alone and fork a new toolchain for use by clang 11, but this seems internal-enough that existing builds shouldn't be negatively affected by this.

Differential Revision: https://phabricator.services.mozilla.com/D82444
2020-07-06 20:43:42 +00:00
Gregory Mierzwinski 99a1058fe5 Bug 1649819 - Temporarily disable youtube widevine tests on linux and windows7. r=tarek
This patch disables the linux and windows7 platforms for the youtube widevine tests.

Differential Revision: https://phabricator.services.mozilla.com/D82370
2020-07-07 11:50:28 +00:00
David Major 466f74b7ac Bug 1650239 - Name clang-9 toolchains as such explicitly r=firefox-build-system-reviewers,rstewart
We already do this for e.g. `linux64-clang-9`; this patch extends that pattern everywhere.

This will make it easy to do try runs with other clangs: just move the `toolchain-alias` lines from the `9` tasks to the `trunk` tasks.

Also, this makes dependencies more explicit: for example the gn task specifically requests a clang-9 compiler, so it will also need a clang-9-based cctools-port, rather than whatever moving value the alias represents.

Differential Revision: https://phabricator.services.mozilla.com/D82441
2020-07-06 21:42:22 +00:00
David Major c65f6e0aff Bug 1650239 - Rename cctools-clang-10-port to cctools-port-clang-10 r=firefox-build-system-reviewers,rstewart
Some cleanup before I add more copies of this task.

Since this is based on a repo called `cctools-port` it seems like it would be better to keep that substring intact.

Differential Revision: https://phabricator.services.mozilla.com/D82439
2020-07-06 21:42:35 +00:00
Armen Zambrano G. bcdae74a2f Bug 1639873 - Fix new backfill issue r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D82420
2020-07-06 20:41:11 +00:00
Mihai Tabara 2db7ce0b92 Bug 1627834 - update the text and screenshots on Flathub.r=rail
Differential Revision: https://phabricator.services.mozilla.com/D82360
2020-07-06 18:00:10 +00:00
Simon Fraser 28ddc204af Bug 1649930 Diagnostics for AWS request issues r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D82358
2020-07-06 13:07:45 +00:00
Ben Hearsum dc6fa9e083 Bug 1641058: fix unit test failures from remove_tasks interface change. a=fix. CLOSED TREE
Differential Revision: https://phabricator.services.mozilla.com/D82213
2020-07-03 17:49:05 +00:00
Marco Castelluccio 9e5cdda0dd Bug 1648414 - Define some additional bugbug-based strategies and remove poorly performing ones. r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D81050
2020-07-03 16:30:37 +00:00
Marco Castelluccio a6c9cd86e2 Bug 1648414 - Add a new argument to the BugBugPushSchedules optimization strategy to only consider tasks. r=ahal
So we can define alternative strategies that only use tasks and compare them with the reduced ones.

Differential Revision: https://phabricator.services.mozilla.com/D81049
2020-07-01 15:32:48 +00:00
Marco Castelluccio c0efb461f5 Bug 1648414 - Support defining custom DisperseGroup strategies with a different target_counts dict. r=ahal
If we don't copy the dict, all strategies will share the same.

Differential Revision: https://phabricator.services.mozilla.com/D81048
2020-07-01 15:32:25 +00:00
Ben Hearsum 8289a7e7cb Bug 1641058: tasks that only exist because of another that is later optimized away should not be scheduled r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D81745
2020-07-02 21:28:57 +00:00
Geoff Brown e17ca5f1b5 Bug 1585726 - Run mozlint on macosx64 and win64 too; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D82200
2020-07-03 16:07:33 +00:00
Armen Zambrano G. 17affe362d Bug 1639873 - Add support for test manifest backfilling r=tomprince
We use the symbol of the original task we backfill and suffix the revision of the push
from which we backfill. This makes it a bit more clear as to from which push we requested
the backfill.

We also modify the group symbol (if available) to group backfilled tasks.

***

***

Differential Revision: https://phabricator.services.mozilla.com/D81379
2020-07-03 14:18:28 +00:00
Armen Zambrano G. 9425bdc44a Bug 1639873 - New backfill action r=tomprince
This backfill action schedules a secondary action per push. This secondary action
schedules the intended task on behalf of the primary action.

This introduces logic on how to trigger a hook by an action.

Differential Revision: https://phabricator.services.mozilla.com/D79351
2020-07-03 13:45:21 +00:00
Edwin Takahashi 3ae2d81379 Bug 1648674 - fix unary operator expected error in test-liux.sh r=glandium
Changes:
  - use parameter expansion to provide a default value of 'false' if `$NEED_COMPIZ` is not set.

Differential Revision: https://phabricator.services.mozilla.com/D81388
2020-07-02 23:03:49 +00:00
Gregory Mierzwinski f671ac2725 Bug 1649539 - Disable non-shippable G5 raptor tests in production. r=perftest-reviewers,Bebe
A patch landed recently which added this build. This patch disables it in production.

Differential Revision: https://phabricator.services.mozilla.com/D81733
2020-07-02 19:46:36 +00:00
Narcis Beleuzu bb1da0fa97 Backed out changeset 6a42da21836e (bug 1641058) for breaking action tasks like retriggers . CLOSED TREE 2020-07-02 21:45:53 +03:00
Tom Prince 02ebadf1a2 Bug 1649356: [taskgraph] Enforce schema identifier restrictions for source_test schema; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D81658
2020-07-02 12:45:53 +00:00
Tom Prince 4df2e5b458 Bug 1649356: [taskgraph] Don't use `Schema(..., required=True)`; r=ahal
This is the only place where we override the default of `required=False`.
Change it to match the rest of the schemas, so as not to confuse people who
don't happen to notice the final line of the schema.

Differential Revision: https://phabricator.services.mozilla.com/D81657
2020-07-02 18:01:45 +00:00
Tom Prince dcbbfca3e4 Bug 1649356: [taskgraph] `optionally_keyed_by` takes multiple fields to key by; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D81656
2020-06-30 14:11:30 +00:00
Tom Prince 78f88bf76b Bug 1649356: [taskgraph] We do not honour voluptuous `default` values, so don't specify them; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D81655
2020-06-30 14:10:36 +00:00
Tom Prince da997b746b Bug 1649356: [taskgraph] Don't have unused `depname` key in release transforms; r=Callek
We don't ever set `depname` in any of the kinds that use these transforms.
Since we don't apply default values in schemas, `Required(..., default=...)`
behaves equivalently to `Optional(...)`, so we can just remove these options.

Differential Revision: https://phabricator.services.mozilla.com/D81654
2020-06-30 14:08:27 +00:00
Ben Hearsum 2890d89ae8 Bug 1641058: tasks that only exist because of another that is later optimized away should not be scheduled r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D81745
2020-07-02 17:00:03 +00:00
Ben Hearsum 9626260f04 Bug 1646511: disable webgpu tests for mingw builds on autoland r=marco
Differential Revision: https://phabricator.services.mozilla.com/D81139
2020-07-01 19:54:04 +00:00
Florin Strugariu 6999f35c94 Bug 1621344 - Add Android Browsertime youtube-playback tests r=perftest-reviewers,sparky,AlexandruIonescu
Differential Revision: https://phabricator.services.mozilla.com/D67437
2020-07-02 09:33:41 +00:00
Nathan Froyd fe5aabec65 Bug 1647481 - bump sccache version to pick up zstd changes; r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D80550
2020-07-01 21:18:27 +00:00
Martin Stransky dc523400eb Bug 1631400 [Flatpak] Add 'Open the Profile Manager' to the right click menu on desktop actions, r=mtabara
Differential Revision: https://phabricator.services.mozilla.com/D81863
2020-07-01 19:30:08 +00:00
Tarek Ziadé 410c14bebe Bug 1644320 - Fix how we setup the virtualenv r=sparky
Fix how we setup the virtualenv

Differential Revision: https://phabricator.services.mozilla.com/D81597
2020-07-01 18:12:06 +00:00
Justin Wood f050825aa3 Bug 1640278 - Remove 'try' from checking run-on-projects for try-syntax. r=bhearsum
This could go further and test for m-c or the trunk/integration aliases, etc. but I opted for not letting this increase try-syntax jobs.

Differential Revision: https://phabricator.services.mozilla.com/D81756
2020-07-01 17:17:28 +00:00
Justin Wood 908076db9f Bug 1640278 - Adjust docs. r=sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D81755
2020-07-01 14:11:45 +00:00
Justin Wood a9b4324248 Bug 1640278 - Fixup run-on-projects verification failures. r=sylvestre
This is in-part a followup to 1638260 which fixed some cases of this problem.

Depends on D76540

Differential Revision: https://phabricator.services.mozilla.com/D76746
2020-07-01 14:04:19 +00:00
Justin Wood 725d121ff5 Bug 1640278 - Do validation of run-on-projects. r=catlee
Differential Revision: https://phabricator.services.mozilla.com/D76540
2020-07-01 14:03:38 +00:00
Gregory Mierzwinski 23288e112e Bug 1649641 - Re-enable live site Fenix tests. r=perftest-reviewers,Bebe
Differential Revision: https://phabricator.services.mozilla.com/D81786
2020-07-01 09:24:15 +00:00
Simon Fraser f02cac8eef Bug 1648664 Reduce download concurrency in partials r=aki
Differential Revision: https://phabricator.services.mozilla.com/D81692
2020-06-30 16:34:06 +00:00
Henrik Skupin 6f921302c7 Bug 1643099 - [geckodriver] Build geckodriver as static executable for Linux. r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D81605
2020-06-30 13:19:41 +00:00
alexandru.ionescu 2a81df9ece Bug 1648902 Remove webrender variant from fennec r=perftest-reviewers,Bebe
Differential Revision: https://phabricator.services.mozilla.com/D81660
2020-06-30 10:37:22 +00:00
Florin Strugariu 1749affc05 Bug 1635711 - Enable cold page load tests with Fission enabled r=perftest-reviewers,AlexandruIonescu
Differential Revision: https://phabricator.services.mozilla.com/D81308
2020-06-30 07:27:41 +00:00
Tarek Ziadé 9294a5bfc1 Bug 1649176 - add batch attribute to add --test-date r=sparky
Right now setup_perftest_test_date adds --test-date yesterday to all perftest
runs. we want that only for the ones doing batches

Differential Revision: https://phabricator.services.mozilla.com/D81562
2020-06-29 20:12:42 +00:00
Edwin Takahashi 63d17a46c6 Bug 1648846 - more robust detection in conditional for xwit to be called in test-linux.sh r=jmaher
Changes:
  - instead of relying only on `$DISPLAY` env var, also use `$NEED_WINDOW_MANAGER` in the check.

Differential Revision: https://phabricator.services.mozilla.com/D81398
2020-06-27 09:19:17 +00:00
Andi-Bogdan Postelnicu 68409df6ba Bug 1648709 - run `Coverity Static Analysis` on `mozilla-central` once per day. r=sylvestre
Right now we are running `Coverity Static Analysis` for each merge to `mozilla-central`,
this is suboptimal since the resources required to do so are very expensive. It's
enough to run it once a day.

Differential Revision: https://phabricator.services.mozilla.com/D81510
2020-06-29 12:44:51 +00:00
Mike Hommey 48d63aca96 Bug 1524723 - Upgrade apt in the Ubuntu 16.04 docker image. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D81282
2020-06-26 16:29:30 +00:00
Edwin Takahashi 4e16b8cde4 Bug 1646885 - add backlog flag to be passed to mozharness and set timeout multiplier appropriately r=jmaher
Changes:
  - add new flag in `web_platform_tests.py`, with name `backlog` that is a boolean value.
  - simplify the timeout multiplier conditional to check for presence of the `backlog` flag.
  - add the new flag to the `mozharness_extra_options` flag in `web-platform.yml`

Differential Revision: https://phabricator.services.mozilla.com/D81184
2020-06-26 18:28:45 +00:00
Edwin Takahashi 5e8e090932 Bug 1646813 - Part 2. write tests for taskgraph/utils/chunking.py r=ahal
Changes:
  - added tests that exercise manifest loading, mozinfo guessing and the overall process of chunking.
  - tests added for both web-platform and traditional mochitest/xpcshell suites.

Differential Revision: https://phabricator.services.mozilla.com/D80985
2020-06-26 21:52:20 +00:00
Andrew Creskey 76d2dabaea Bug 1648183 - Disable live site testing for 1 day, or otherwise disable telemetry pings r=davehunt
Disabling these unsheriffed tests to determine if they are causing the spikes in new users that telemetry reveals.

Differential Revision: https://phabricator.services.mozilla.com/D81326
2020-06-26 16:59:38 +00:00
Emilio Cobos Álvarez 7c995807da Bug 1646936 - Generate a single metadata file in the objdir, and feed it to cbindgen. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D80360
2020-06-26 10:41:26 +00:00
Coroiu Cristina 302c2fa31a Backed out changeset 8f948dd74aba (bug 1646936) for SM and Toolchain failures on a CLOSED TREE 2020-06-26 13:08:09 +03:00
Emilio Cobos Álvarez 9c7c03bf30 Bug 1646936 - Generate a single metadata file in the objdir, and feed it to cbindgen. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D80360
2020-06-26 09:37:16 +00:00
Mike Hommey 90a47ff4ef Bug 1648343 - Require rustc 1.43 to build. r=dmajor
Differential Revision: https://phabricator.services.mozilla.com/D81246
2020-06-25 23:41:16 +00:00
Mike Hommey 81697e521f Bug 1632735 - Upgrade valgrind builds to the same rust version as other builds. r=erahm
And add suppressions for the new errors that this adds to the valgrind
run. It's not clear if it's a legitimate thing that LLVM does that
valgrind doesn't cope with, like many others, but the fact is running
valgrind on a PGO/LTO build doesn't show these errors, so we're not
actually shipping them anyways (but does show _different_ errors that we
don't see on the build the valgrind jobs do)

Differential Revision: https://phabricator.services.mozilla.com/D81016
2020-06-25 22:29:59 +00:00
Mike Hommey 65b1898329 Bug 1648359 - Use Debian 10 based docker images for macOS builds. r=froydnj
That brings them on par with all the other cross-compiled builds.

Differential Revision: https://phabricator.services.mozilla.com/D81020
2020-06-25 20:32:23 +00:00
Edwin Takahashi 2e63f99098 Bug 1607713 - install xwit in ubuntu1804 docker image and set cursor position to 0,0 r=karlt
Changes:
  - install `xwit` package in the ubuntu1804 docker image setup phase.
  - in `test-linux.sh` call xwit to reposition the cursor to a coodinate of 0,0

Differential Revision: https://phabricator.services.mozilla.com/D80698
2020-06-25 18:02:11 +00:00
Andrew Halberstadt 3371baa8d6 Bug 1648455 - [taskgraph] Adjust manifest scheduling confidence threshold from CT_LOW to CT_MEDIUM, r=marco
Using a higher confidence threshold means manifest scheduling (i.e just |mach
try auto| for now), will schedule fewer manifests. We have data suggests that
this bump should reduce the number of manifests scheduled by more than half,
while only very slightly reducing the regression detection rate (still above
95%).

Differential Revision: https://phabricator.services.mozilla.com/D81085
2020-06-25 13:28:38 +00:00
Ben Hearsum d003f46a59 Bug 1646767: Android opt tests completely missing from taskgraph on try r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D80920
2020-06-24 18:33:10 +00:00
Mike Hommey bc8366ddca Bug 1648256 - Upgrade valgrind to latest release. r=firefox-build-system-reviewers,rstewart
There isn't a Debian package for that verson yet, so keep using the code
that pulls from the git repository, but use the revision that was
released as 3.16.1.

Differential Revision: https://phabricator.services.mozilla.com/D80957
2020-06-24 20:55:05 +00:00
Mike Hommey 3437f37ac4 Bug 1647880 - Upgrade cmake to 3.16.3 for toolchain builds. r=firefox-build-system-reviewers,rstewart
This gets us to the same version as the one used on the Debian 10
toolchain docker image.

Differential Revision: https://phabricator.services.mozilla.com/D80818
2020-06-24 15:17:03 +00:00
Tom Prince 39034e6b47 Bug 1639873: Be explicit in the permission used for a given action; r=aki
We currently base this on the cb_name, but it would make be better to make this
explicit, so that different actions can share a permisssion. In particular, we are
adding a pair of new backfill actions that needs a singlle non-default permission.

Differential Revision: https://phabricator.services.mozilla.com/D80940
2020-06-24 20:40:29 +00:00
Edwin Takahashi 507fa3db90 Bug 1647383 - chunk web-platform-tests-crashtest and web-platform-tests-wdspec in decision task r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D80529
2020-06-24 18:23:39 +00:00
Andrew Creskey 1670be2396 Bug 1646987 - Increase browsertime-iterations for the app-link tests to 14 r=tarek
We are seeing a pretty broad distribution of results for the applink tests.
This bug increases the number of browsertime iterations from 10 to 14.

Differential Revision: https://phabricator.services.mozilla.com/D80363
2020-06-24 15:30:41 +00:00
Mihai Alexandru Michis bebac5dcfa Merge mozilla-central to autoland. a=merge CLOSED TREE 2020-06-24 16:09:44 +03:00
Mihai Alexandru Michis 5e50c08b10 Backed out changeset 47a06752bda8 (bug 1647383) as requested by jmaher. a=backout
DONTBUILD
2020-06-24 16:06:16 +03:00
VandanRogheliya 1979f2279c Bug 1645197 - Type pageload added to browsertime visualmetrics data. r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D80647
2020-06-24 11:12:15 +00:00
Rob Lemley f518507ad0 Bug 1646843 - Add comm-esr78 support to taskgraph. r=Callek
Follow-up to bug 1646588 to support comm-esr78 repository.

Differential Revision: https://phabricator.services.mozilla.com/D80271
2020-06-18 20:42:03 +00:00
Edwin Takahashi 0fc6588f5d Bug 1647383 - chunk web-platform-tests-crashtest and web-platform-tests-wdspec in decision task r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D80529
2020-06-23 18:31:42 +00:00
Edwin Takahashi cb1543b5f6 Bug 1646813 - write tests for chunking.get_runtimes and chunking.chunk_manifests for web-platform-tests r=ahal
Changes:
  - change default return value for `get_runtimes` method to an empty dictionary if data is not found.
  - add in fixtures and tests for `get_runtimes` and `chunk_manifests` method for web-platform-tests and subsuites.

Differential Revision: https://phabricator.services.mozilla.com/D80250
2020-06-22 22:00:02 +00:00
Edwin Takahashi ebdff4d00a Bug 1647498 - set default docker-image attribute to ubuntu1804-test r=jmaher
This change affects some android-em tests that continued to run on the deprecated `desktop1604-test` image.

Differential Revision: https://phabricator.services.mozilla.com/D80561
2020-06-22 22:24:07 +00:00
Justin Wood 0493942024 Bug 1647749 - Don't test updates from esr52. r=mtabara a=release
Differential Revision: https://phabricator.services.mozilla.com/D80670
2020-06-23 15:13:11 +00:00
Justin Wood 31ae9b52fb Bug 1647634 - Update esr rules too for esr78. r=mtabara
Differential Revision: https://phabricator.services.mozilla.com/D80658
2020-06-23 14:24:04 +00:00
Andrew Creskey e23be904f3 Bug 1647644 - perftest(applink-fenix) tasks fail and logs too big to parse r=tarek
Removed --verbose and increased test duration to prevent failures.

Differential Revision: https://phabricator.services.mozilla.com/D80645
2020-06-23 12:55:22 +00:00
James Graham 91d4b40822 Bug 1644395 - Don't try to run wpt print-reftests on Android,
Differential Revision: https://phabricator.services.mozilla.com/D80641
2020-06-23 12:41:17 +00:00
James Graham 2ade9fa1fd Bug 1644395 - Add wpt print reftests to TC config, r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D79082
2020-06-23 10:00:28 +00:00
Marian Raiciof c2bbc30a7b Bug 1623421 - Add WebRender visual metrics pageload tests Android r=perftest-reviewers,sparky
This patch is allowing visual metrics android pageload tests to run with webrender.

Differential Revision: https://phabricator.services.mozilla.com/D70668
2020-06-23 10:36:33 +00:00
Gregory Mierzwinski 7d24231bc3 Bug 1646967 - Disable conditioned profiles for fennec. r=perftest-reviewers,Bebe
Differential Revision: https://phabricator.services.mozilla.com/D80439
2020-06-23 09:23:19 +00:00
Gregory Mierzwinski 59c1208baf Bug 1502032 - Increase raptor ytp task timeout. r=perftest-reviewers,Bebe
Differential Revision: https://phabricator.services.mozilla.com/D80458
2020-06-23 09:20:20 +00:00
Rail Aliiev d58a49504e Bug 1647558 - Add esr78 balrog scopes, r=RyanVM
Differential Revision: https://phabricator.services.mozilla.com/D80580
2020-06-23 02:12:19 +00:00
Edwin Takahashi 7e28d8d3d2 Bug 1643177 - separate web-platform-tests-backlog tests into its own test-set r=jmaher
Changes:
  - remove web-platform-tests from the test-set `macosx1014-tests`
  - create new group `web-platform-tests-backlog` and move all backlog types into that group
  - add the new group into test-platforms where applicable

Differential Revision: https://phabricator.services.mozilla.com/D80555
2020-06-22 21:30:31 +00:00
Ben Hearsum 29b214847d Bug 1645728: All android-em opt tests are now tier 2 r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D79716
2020-06-22 19:24:58 +00:00
Narcis Beleuzu a0c8ce672c Backed out 4 changesets (bug 1644395) for bc failures on browser_all_files_referenced.js . CLOSED TREE
Backed out changeset 582c1dd945ca (bug 1644395)
Backed out changeset 9dbba532a700 (bug 1644395)
Backed out changeset 59e998eb5e59 (bug 1644395)
Backed out changeset 035ddba4a4d9 (bug 1644395)
2020-06-23 00:56:49 +03:00
Edwin Takahashi a4ca63453e Bug 1643177 - apply timeout multiplier reduction to web-platform-tests-reftest-backlog and reduce chunks r=jmaher
Changes:

Previous patch to apply the reduced timeout multiplier was only applicable to vanilla `web-platform-tests`. This patch reduces the chunk count for `web-platform-tests-reftest` and applies the reduction multiplier.

Differential Revision: https://phabricator.services.mozilla.com/D80520
2020-06-22 19:50:34 +00:00
Julien Cristau 58f17114d5 Bug 1647347 - add missing release-to-esr merge configuration. r=Callek
We need to update the display version, and flip the MOZ_REQUIRE_SIGNING
default, not just the explanatory comment.

Differential Revision: https://phabricator.services.mozilla.com/D80493
2020-06-22 16:51:17 +00:00
James Graham 5706e82bce Bug 1644395 - Add wpt print reftests to TC config, r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D79082
2020-06-22 13:44:13 +00:00
Gregory Mierzwinski 2868ced012 Bug 1645874 - Simplify the perftest crons and remove some tests in production. r=perftest-reviewers,davehunt
This patch does a few small things to clean up our CI tests:
(1) The GPT cron is simplified so that it can be more manageable. In this change I also include a new helper method to make sure we run our android tests on the build variants that we should be running them on (and no others).
(2) The geckoview browsertime warm tests were running on 1 platform too many - this is no longer the case.
(3) Fennec vismet tests were previously excluded but are now included.
(4) All fennec tests that were in GPT cron have been moved to the Fennec_v68 cron so they only run once a week.

Differential Revision: https://phabricator.services.mozilla.com/D79891
2020-06-22 10:26:39 +00:00
Sylvestre Ledru 46af48052b Bug 1646421 - flake8 Fix a bunch of actual errors r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D80078
2020-06-20 09:46:45 +00:00
Justin Wood f66f5a235e Bug 1646588 - Add esr78 support. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D79731
2020-06-18 19:57:20 +00:00
Justin Wood c202d47df3 Bug 1646588 - Remove old esr52 code. r=aki
Differential Revision: https://phabricator.services.mozilla.com/D79730
2020-06-18 19:56:22 +00:00
Mike Hommey 70a05665ae Bug 1645393 - Install python3 development files in build docker images. r=firefox-build-system-reviewers,rstewart
It's required to properly setup psutil in the python3 virtualenv, and
psutil is required for build resources tracking by mach.

It's not clear to me why this hasn't been a problem since `mach build`
switched to python3, though. ¯\_(ツ)_/¯

Differential Revision: https://phabricator.services.mozilla.com/D80343
2020-06-19 13:53:28 +00:00
Gregory Mierzwinski 91291fe122 Bug 1646697 - Modify how the multi-commit tasks work. r=mozperftest-reviewers,tarek
This patch adds a `--test-date` option to mozperftest that lets us specify the date that we are testing on. This lets us trigger multi-commit tests on past dates, and also ensures that retriggers of the multi-commit tests use the same day that they were pushed on (giving us reproducibility).

Differential Revision: https://phabricator.services.mozilla.com/D80173
2020-06-18 22:37:35 +00:00
Gregory Mierzwinski b1676248ea Bug 1646368 - Add performance tuning feature to android layer. r=mozperftest-reviewers,tarek
This patch ports the performance tuning functionality from Raptor to mozperftest and adds a small test for it.

Differential Revision: https://phabricator.services.mozilla.com/D80168
2020-06-18 22:37:27 +00:00
Andrew Creskey 959282ad93 Bug 1644773 - Add a nightly cron job for the Fenix HomeActivity (MAIN) perf test r=sparky,tarek
Also fixes a logcat encoding read error in the transformer.

Differential Revision: https://phabricator.services.mozilla.com/D79909
2020-06-18 18:05:13 +00:00
Sylvestre Ledru 0fa8b022f0 Bug 1623024 - mozlint: run pylint at review phase r=linter-reviewers,ahal
Depends on D79099

Differential Revision: https://phabricator.services.mozilla.com/D79430
2020-06-18 20:05:20 +00:00
Sylvestre Ledru 25fd29ad52 Bug 1623024 - mozlint: Add pylint to the CI r=linter-reviewers,ahal
Depends on D79076

Differential Revision: https://phabricator.services.mozilla.com/D79099
2020-06-18 20:05:02 +00:00
Edwin Takahashi 46dc1bb99f Bug 1637011 - enable chunking in decision task for web-platform-tests-reftest r=ahal
Changes:
  - enable chunking in the decision task for web-platform-tests-reftest.

Differential Revision: https://phabricator.services.mozilla.com/D76763
2020-06-17 16:37:28 +00:00
Julien Cristau d9c88a0948 Bug 1646763 - sanity check that the generated HPKPStaticPins header isn't truncated. r=sfraser
It looks like genHPKPStaticPins.js can fail and still exit 0, so try and
make sure its output is plausible.  The last thing it writes is the
expiration time, so check that it's there, which should mean everything
else got generated as well.

Differential Revision: https://phabricator.services.mozilla.com/D80206
2020-06-18 16:28:44 +00:00
Gijs Kruitbosch 272fe15123 Bug 1644863 - include CSS, HTML and localization (ftl, properties, dtd) files in the whitespace linter, r=sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D79204
2020-06-17 22:46:42 +00:00
Tom Prince 91916ebd33 Bug 1646514: Use a single manifest loader for all tests; r=Callek
Since `@memoize` on a method is per-instance, use a single instance for all
tests, rather than a new one for each tasks. This gives a ~4x speedup in test
task generation.

Differential Revision: https://phabricator.services.mozilla.com/D79764
2020-06-17 21:21:35 +00:00
Ted Campbell c72c2a34d1 Bug 1642708 - Stop running BinAST tests in automation r=perftest-reviewers,Bebe,sfink
Remove the BinAST tests from following jobs:
    - SM(p)
    - SM(cgc)

Remove the BinAST performance tests:
    - binast-instagram

Mark the WPT BinAST tests as backlog.

Differential Revision: https://phabricator.services.mozilla.com/D77919
2020-06-17 13:57:04 +00:00
David Major af8ed110e8 Bug 1646394 - Use the generic compiler alias for ccov builds r=froydnj
AFAIK there's no reason to pin to clang-9 specifically. In fact I already landed this change in the bouncy attempts at bug 1616692. I'm just separating it out in order to have fewer things in the air since I'm currently juggling clangs 9, 10, and 11 locally.

Differential Revision: https://phabricator.services.mozilla.com/D80018
2020-06-17 15:43:47 +00:00
Csoregi Natalia e722c196bc Backed out 5 changesets (bug 1642708) for build bustages. CLOSED TREE
Backed out changeset d307b00c7e1b (bug 1642708)
Backed out changeset d210a60ad435 (bug 1642708)
Backed out changeset bcc2728b5ca5 (bug 1642708)
Backed out changeset 1c245d4e8244 (bug 1642708)
Backed out changeset ad613fa94a83 (bug 1642708)
2020-06-17 16:51:16 +03:00
Ted Campbell c2620d4c2f Bug 1642708 - Stop running BinAST tests in automation r=perftest-reviewers,Bebe,sfink
Remove the BinAST tests from following jobs:
    - SM(p)
    - SM(cgc)

Remove the BinAST performance tests:
    - binast-instagram

Mark the WPT BinAST tests as backlog.

Differential Revision: https://phabricator.services.mozilla.com/D77919
2020-06-17 08:06:08 +00:00
Mihai Tabara e6bfa572e4 Bug 1644973 - remove Nazgul jobs as they become the new vanilla bouncer jobs. r=rail
Differential Revision: https://phabricator.services.mozilla.com/D79829
2020-06-16 15:17:05 +00:00
Edwin Takahashi 94fa9788e6 Bug 1643177 - reduce web-platform-tests-backlog chunk counts r=jmaher
Changes:
  - reduce the number of web-platform-tests-backlog chunks across the board to save on machine setup costs.

Differential Revision: https://phabricator.services.mozilla.com/D79414
2020-06-16 22:58:20 +00:00
Razvan Maries 08aad2dacd Backed out 2 changesets (bug 1643177) for multiple webdriver perma failures. CLOSED TREE
Backed out changeset 745ca9baa9c8 (bug 1643177)
Backed out changeset 3acd697a9131 (bug 1643177)
2020-06-17 01:52:28 +03:00
Razvan Maries e29397a370 Backed out changeset 722854464579 (bug 1623520) for multiple failures on Win10 x64 asan. CLOSED TREE 2020-06-17 01:49:36 +03:00
Edwin Takahashi 4ba892e64c Bug 1643177 - reduce web-platform-tests-backlog chunk counts r=jmaher
Changes:
  - reduce the number of web-platform-tests-backlog chunks across the board to save on machine setup costs.

Differential Revision: https://phabricator.services.mozilla.com/D79414
2020-06-16 20:50:36 +00:00
Mike Hommey a0075e4138 Bug 1623520 - Switch Windows asan and fuzzing builds to cross-builds. r=decoder
Differential Revision: https://phabricator.services.mozilla.com/D79417
2020-06-12 13:50:26 +00:00
Edwin Takahashi 04a1969fa4 Bug 1645974 - update task suite check when setting TESTS_BY_MANIFEST_URL r=jmaher
Changes:

This addresses the fallout of the change in https://phabricator.services.mozilla.com/D78318, to use the `re` module to check for presence of `test-verify-wpt` or `test-coverage-wpt`.

Differential Revision: https://phabricator.services.mozilla.com/D79851
2020-06-16 20:05:59 +00:00
Edwin Takahashi dc3f2aa657 Bug 1643177 - discontinue web-platform-tests-backlog on ccov/devedition platforms r=jmaher
Changes:
  - discontinue ability to run web-platform-tests-backlog on ccov/devedition platforms, including `web-platform-tests-reftest`.

Differential Revision: https://phabricator.services.mozilla.com/D79867
2020-06-16 19:54:21 +00:00
Sylvestre Ledru e9674f79d4 Bug 1645829 - mozlint: Show the version of rustc r=linter-reviewers,ahal
Also, make a change to force the installation of a more recent version
of clippy & rustfmt until we have a proper fix

Differential Revision: https://phabricator.services.mozilla.com/D79735
2020-06-16 18:52:50 +00:00
Andrew Halberstadt 09f67bf165 Bug 1641131 - [ci] Don't run test-verify tasks on projects where the associated build is skipped, r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D79856
2020-06-16 18:39:53 +00:00
Andrew Halberstadt ba66e735dd Bug 1641131 - [ci] Don't use shippable builds as dependencies for source-test tasks on try, r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D79854
2020-06-16 18:11:43 +00:00
Sylvestre Ledru cb7d556407 Bug 1642825 - mozlint: Run the rejected-words linter in the CI r=Gijs,linter-reviewers,ahal
Depends on D78158

Differential Revision: https://phabricator.services.mozilla.com/D78230
2020-06-16 17:03:46 +00:00
Edwin Takahashi 4db89d4e49 Bug 1634554 - Part 3. simplify chunking of web-platform-tests suites r=ahal
Changes:
  - stop using runtime information for the chunking process of web-platform-tests
  - remove the wpt_group_translation workaround as it is no longer necessary
  - simplify chunking process to iterate and assign test paths to chunks in order

Simply assigning test paths (stand-in for manifests in other suites) is now possible because other patches in the stack now prevent the web-platform-tests harness (when an external process handles the grouping) from loading all tests in a given path.

Differential Revision: https://phabricator.services.mozilla.com/D78321
2020-06-15 19:38:39 +00:00
Edwin Takahashi a779c60ef5 Bug 1634554 - Part 1: download decision task artifact and strip out non-WPT tests in mozharness/web_platform_tests.py r=ahal,jgraham
Changes:
  - add new environment variable `TESTS_BY_MANIFEST_URL` referencing the tests-by-manfiest.json.gz artifact generated by decision task.
  - add new method `download_manifest` in mozharness/web_platform_tests.py to download the artifact, then remove non-web-platform-test manifests.
  - the processed artifact is then saved as JSON file containing only web-platform-tests for downstream harnesses to use.

Differential Revision: https://phabricator.services.mozilla.com/D78318
2020-06-15 20:27:21 +00:00
Tom Prince 900dbd7e06 Bug 1643842: Don't try generate artifact builds of builds without tests; r=nalexander
There are various binaries that are included in test archives that are
necessary for artifact builds to be repackaged. Don't generate artifact build
variants of tasks without them.

Differential Revision: https://phabricator.services.mozilla.com/D79575
2020-06-15 20:26:58 +00:00
Tom Prince 9ad2d142a7 Bug 1643842: Don't depend on profile-generation if we are an artifact build; r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D79550
2020-06-15 19:49:30 +00:00
Tom Prince 8a031b09b0 Bug 1643842: Add an attribute to indicate which builds are explicitly artifact builds; r=nalexander
This deliberately excludes builds that are implemented using the artifact build machinery,
but are not primarly intended to short-circuit build time. In particular the Windows aarch64
builds are not marked this way.

Differential Revision: https://phabricator.services.mozilla.com/D79549
2020-06-15 19:49:30 +00:00
Nick Alexander 7e3c305443 Bug 1644046 - Post: Bump Sonatype nexus to latest 2.x version. r=froydnj
Differential Revision: https://phabricator.services.mozilla.com/D78946
2020-06-15 16:29:38 +00:00
Nick Alexander 571c7fd42e Bug 1644046 - Get Sonatype nexus with fetch task rather than when building `android-build` image. r=froydnj
This brings the `android-build` image very close to other build
images, paving the way for it to be folded in completely.  It also
makes us more resilient in the face of upstream service interruptions.

Differential Revision: https://phabricator.services.mozilla.com/D78945
2020-06-15 16:29:30 +00:00
Bogdan Tara 30c59d36db Backed out 5 changesets (bug 1634554) for lint failure CLOSED TREE
Backed out changeset e8db70067dbf (bug 1634554)
Backed out changeset ae24c9902708 (bug 1634554)
Backed out changeset 177cc940d57f (bug 1634554)
Backed out changeset 3240acbd85b6 (bug 1634554)
Backed out changeset 71aa576ada8b (bug 1634554)
2020-06-15 22:25:12 +03:00
Edwin Takahashi ec7c82ed3a Bug 1634554 - Part 3. simplify chunking of web-platform-tests suites r=ahal
Changes:
  - stop using runtime information for the chunking process of web-platform-tests
  - remove the wpt_group_translation workaround as it is no longer necessary
  - simplify chunking process to iterate and assign test paths to chunks in order

Simply assigning test paths (stand-in for manifests in other suites) is now possible because other patches in the stack now prevent the web-platform-tests harness (when an external process handles the grouping) from loading all tests in a given path.

Differential Revision: https://phabricator.services.mozilla.com/D78321
2020-06-15 17:56:16 +00:00
Edwin Takahashi 0c1148f15a Bug 1634554 - Part 1: download decision task artifact and strip out non-WPT tests in mozharness/web_platform_tests.py r=ahal,jgraham
Changes:
  - add new environment variable `TESTS_BY_MANIFEST_URL` referencing the tests-by-manfiest.json.gz artifact generated by decision task.
  - add new method `download_manifest` in mozharness/web_platform_tests.py to download the artifact, then remove non-web-platform-test manifests.
  - the processed artifact is then saved as JSON file containing only web-platform-tests for downstream harnesses to use.

Differential Revision: https://phabricator.services.mozilla.com/D78318
2020-06-15 18:41:08 +00:00
Ben Hearsum 0b49e01a94 Bug 1645265: -b o -p macosx64 chooses opt and shippable builds instead of one or the other r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D79400
2020-06-13 13:17:29 +00:00
Ben Hearsum 0462efd54d Bug 1642004: `-p all` try syntax selects shippable builds instead of opt r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D79169
2020-06-12 19:08:54 +00:00
Gregory Mierzwinski 83d7628dc7 Bug 1641878 - Migrate fennec68 raptor tests to browsertime. r=perftest-reviewers,davehunt
This patch disables and removes the raptor-webext fennec68 tests and replaces them with the browsertime variants. The raptor tests which ran through a cron are also swapped for the browsertime tests.

Differential Revision: https://phabricator.services.mozilla.com/D77674
2020-06-12 11:38:26 +00:00
Adam Gashlin aec8dd9651 Bug 1645162 - Use nightly key for oak MARs. r=sfraser
Differential Revision: https://phabricator.services.mozilla.com/D79401
2020-06-12 08:06:51 +00:00
Simon Fraser 65cbd9e05b Bug 1643625 Clear up references to MAR_OLD_FORMAT in packaging tools r=nthomas
I think we don't need these checks any more.

Differential Revision: https://phabricator.services.mozilla.com/D78480
2020-06-11 22:23:10 +00:00
Ben Hearsum 3f2018eaf1 Bug 1641948: remove android opt builds from non-mozilla-central branches; reduce frequency of fat-aar builds. r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D78547
2020-06-11 19:25:07 +00:00
Tom Prince 9d6432258c Bug 1630710: Add link to specfic as specific task in treeherder; r=Callek
Since treeherder added support for linking to a specific task, lets make use of
it in taskgraph.

Differential Revision: https://phabricator.services.mozilla.com/D76049
2020-06-11 23:45:58 +00:00
Gregory Mierzwinski 06555779e5 Bug 1645135 - Disable conditioned-profiles for raptor-webext geckoview P2 tests. r=perftest-reviewers,davehunt
This patch disables conditioned-profiles on geckoview P2 raptor-webext tests temporarily.

Differential Revision: https://phabricator.services.mozilla.com/D79331
2020-06-11 21:04:03 +00:00
Mihai Tabara 97baffd8ad Bug 1645001 - remove vanilla bouncer as deps in release automation. r=jcristau
Differential Revision: https://phabricator.services.mozilla.com/D79268
2020-06-11 12:13:21 +00:00
Joel Maher f16377e4f4 Bug 1591466 - do not adjust tier and optimization for mobile. r=sparky
do not adjust tier and optimization for mobile.

Differential Revision: https://phabricator.services.mozilla.com/D79279
2020-06-11 14:38:01 +00:00
Gregory Mierzwinski 9a6913c3d3 Bug 1644706 - Make a multi-commit test for app-link in CI. r=mozperftest-reviewers,tarek
This patch adds a new method to test multiple Fenix builds within one test run.

Depends on D78926

Differential Revision: https://phabricator.services.mozilla.com/D79107
2020-06-10 22:10:35 +00:00
Bogdan Tara 633c636ab1 Backed out 3 changesets (bug 1644706, bug 1644437, bug 1644368) for python related bustage CLOSED TREE
Backed out changeset 2a6645398c5a (bug 1644706)
Backed out changeset e695f5748e04 (bug 1644437)
Backed out changeset d615591507b9 (bug 1644368)
2020-06-11 01:04:33 +03:00
Gregory Mierzwinski 8d81e0549f Bug 1644706 - Make a multi-commit test for app-link in CI. r=mozperftest-reviewers,tarek
This patch adds a new method to test multiple Fenix builds within one test run.

Depends on D78926

Differential Revision: https://phabricator.services.mozilla.com/D79107
2020-06-10 21:18:57 +00:00
Tooru Fujisawa a2591607d9 Bug 1641369 - Part 5: Make sure the workspace is clean in SM package build. r=sfink
Differential Revision: https://phabricator.services.mozilla.com/D77643
2020-06-03 05:34:20 +00:00
Tooru Fujisawa 34aa7e8d55 Bug 1641369 - Part 3: Move objdir of SpiderMonkey jobs outside of checkouts directory. r=tomprince
Depends on D77642

Differential Revision: https://phabricator.services.mozilla.com/D77991
2020-06-10 18:27:18 +00:00
Tooru Fujisawa ad7ff99b9d Bug 1641369 - Part 2: Stop caching workspace directory in SpiderMonkey jobs. r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D77642
2020-06-10 18:21:38 +00:00
Tooru Fujisawa d2368696d6 Bug 1641369 - Part 1: Use GECKO_PATH instead of SRCDIR in SpiderMonkey jobs. r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D77990
2020-06-10 18:21:20 +00:00
Gregory Mierzwinski b4e1c09e30 Bug 1644787 - Disable all raptor-webext geckoview pageload tests. r=perftest-reviewers,davehunt
This patch is the final patch for the raptor to browsertime geckoview migration. It disables all raptor-webext geckoview pageload tests running in production.

Differential Revision: https://phabricator.services.mozilla.com/D79157
2020-06-10 18:44:00 +00:00
Mitchell Hentges 0bc6909185 Bug 1621960: Change |mach python| default from Python 2 to Python 3 r=rstewart
Depends on D77967

Differential Revision: https://phabricator.services.mozilla.com/D78181
2020-06-10 16:10:08 +00:00
Gregory Mierzwinski 5222ae9898 Bug 1644491 - Run fennec68 youtube tests once a week. r=perftest-reviewers,davehunt
This patch disables fennec68 youtube playback tests on mozilla-central and runs them once a week through the fennec_v68 cron job.

Differential Revision: https://phabricator.services.mozilla.com/D78937
2020-06-10 17:02:17 +00:00
Joel Maher bd184eb8bb Bug 1591466 - split raptor tests into tier-1 (high value) and tier-2 (lower value). r=sparky
split raptor tests into tier-1 (high value) and tier-2 (lower value)

Differential Revision: https://phabricator.services.mozilla.com/D78247
2020-06-10 15:00:58 +00:00
Simon Fraser 4fbe0ba259 Bug 1644712 Remove datadog metrics submission from partials r=mtabara
Differential Revision: https://phabricator.services.mozilla.com/D79121
2020-06-10 13:04:29 +00:00
Mihai Tabara 7bd3b1d36c Bug 1635525 - fix broken nazgul links and deps. r=rail
Differential Revision: https://phabricator.services.mozilla.com/D79084
2020-06-10 14:17:02 +00:00
Mike Hommey 01f30cd88f Bug 1643504 - Fix WR bustage. r=fix CLOSED TREE 2020-06-10 00:41:49 +03:00
Mike Hommey ef2aafa5a5 Bug 1643504 - Remove not unused Debian stretch-based docker images. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D78433
2020-06-09 01:01:33 +00:00
Mike Hommey 57fa3660c9 Bug 1643504 - Upgrade webrender and wgpu docker images to Debian buster. r=kats
Differential Revision: https://phabricator.services.mozilla.com/D78419
2020-06-09 13:35:40 +00:00
Florin Strugariu cfbc528f9c Bug 1641820 - Remove chrome variant of binast test. r=sparky,perftest-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D78869
2020-06-09 18:35:24 +00:00
Andrew Halberstadt 851c45354a Bug 1643811 - [ci] Add a SCHEDULES rule to exclusively run browser-chrome tests when '**/test/browser_*' files are modified, r=gbrown,Gijs
Also makes sure that 'mochitest-devtools-chrome' tasks are triggered by the
'mochitest-browser-chrome' component since they use the same manifest / test
formats.

Depends on D78906

Differential Revision: https://phabricator.services.mozilla.com/D78927
2020-06-09 17:14:09 +00:00
Noemi Erli 9017f973e3 Backed out 4 changesets (bug 1621960) for causing Android bustages CLOSED TREE
Backed out changeset f36a95234a5f (bug 1621960)
Backed out changeset 0e4661c9061d (bug 1621960)
Backed out changeset 3e2cbe4dd777 (bug 1621960)
Backed out changeset d7eb101e5df7 (bug 1621960)
2020-06-09 20:59:26 +03:00
Mitchell Hentges 4006a0890c Bug 1621960: Change |mach python| default from Python 2 to Python 3 r=rstewart
Depends on D77967

Differential Revision: https://phabricator.services.mozilla.com/D78181
2020-06-09 16:44:51 +00:00
Razvan Maries ef73a88351 Backed out changeset 37a5223eda5f (bug 1643625) for Rpk bustages. CLOSED TREE 2020-06-09 18:30:01 +03:00
Simon Fraser 1422db5e20 Bug 1643625 Clear up references to MAR_OLD_FORMAT in packaging tools r=nthomas
I think we don't need these checks any more.

Differential Revision: https://phabricator.services.mozilla.com/D78480
2020-06-09 11:53:26 +00:00
Kunchanapalli Shriram 75c48c9453 Bug 1518313 - Removes --geckoProfile from talos-test mach command and replaces it with --gecko-profile; r=AlexandruIonescu,perftest-reviewers
Also removes import argparse as per https://bugzilla.mozilla.org/show_bug.cgi?id=1518313#c15
Addresses defect found in code analysis as per https://code-review.moz.tools/#/diff/242580
Changes parameters value as per https://phabricator.services.mozilla.com/D66380#2132590

Differential Revision: https://phabricator.services.mozilla.com/D66380
2020-06-09 13:22:15 +00:00
Simon Fraser d5c5c1164e Bug 1598055 Fix up output redirection in ubuntu1804-test creation r=egao
I think this is meant to be a 'stderr into stdout' redirection

Differential Revision: https://phabricator.services.mozilla.com/D78482
2020-06-08 19:21:23 +00:00
Mike Hommey 7142d5cabc Bug 1643504 - Upgrade visual metrics docker image to Debian buster. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D78421
2020-06-09 01:02:04 +00:00
Gregory Mierzwinski 141cc114ba Bug 1644256 - Make raptor-youtube-playback test use 2019 branch. r=perftest-reviewers,davehunt
The old raptor-youtube-playback test was converted to a 2020 branch H264 test and this patch reverts that change. It also adds a new test (and task) for the H264 standard tests.

Differential Revision: https://phabricator.services.mozilla.com/D78803
2020-06-09 00:18:27 +00:00
Mike Hommey a5ea1d83c5 Bug 1643504 - Upgrade code-review task to Debian buster. r=sylvestre,andi
Differential Revision: https://phabricator.services.mozilla.com/D78432
2020-06-08 21:46:07 +00:00
Mike Hommey a0dec00425 Bug 1643504 - Upgrade coverity tasks to Debian buster. r=andi
Differential Revision: https://phabricator.services.mozilla.com/D78429
2020-06-08 21:42:02 +00:00
Mike Hommey d5ce330d0b Bug 1643504 - Upgrade static analysis docker image to Debian buster. r=andi
An explicit install of the libstdc++ dev package is now necessary.
See https://bugzilla.mozilla.org/show_bug.cgi?id=1578535#c6

Differential Revision: https://phabricator.services.mozilla.com/D78422
2020-06-08 21:45:02 +00:00
Mike Hommey afac0aa5fb Bug 1643504 - Upgrade android build docker images to Debian buster. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D78418
2020-06-08 10:53:29 +00:00
Simon Fraser 61d97f3f00 Bug 1642351 Set tasks_for in merge automation action to prevent source-tests r=rail
Differential Revision: https://phabricator.services.mozilla.com/D78672
2020-06-08 16:46:25 +00:00
Joel Maher eeadf81538 Bug 1643284 - require --full to schedule talos/raptor profiling jobs on try. r=davehunt,ahal
require --full to schedule talos/raptor profiling jobs on try.

Differential Revision: https://phabricator.services.mozilla.com/D78250
2020-06-08 15:19:55 +00:00
Tarek Ziadé 7489c5233a Bug 1642779 - implement the 'androidlog' test r=acreskey,sparky
Add a layer that scans the android logcat

Differential Revision: https://phabricator.services.mozilla.com/D78268
2020-06-08 10:47:52 +00:00
Narcis Beleuzu c4c6cd3a30 Backed out changeset c324ff9bf17e (bug 1643504) for breaking Docker Images. CLOSED TREE 2020-06-08 02:01:41 +03:00
Mike Hommey 75e08f6365 Bug 1643504 - Upgrade release-notify-started tasks to Debian buster. r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D78430
2020-06-07 22:29:17 +00:00
Mike Hommey 266e2a5a8c Bug 1643504 - Build wine on Debian buster. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D78428
2020-06-07 22:29:15 +00:00
Mike Hommey 2acf07e0d0 Bug 1643504 - Upgrade Windows searchfox task to Debian buster. r=kats
Differential Revision: https://phabricator.services.mozilla.com/D78431
2020-06-07 22:29:07 +00:00
Mike Hommey 65b8d3beb1 Bug 1643504 - Upgrade the Linux64 generate-profile task to Debian buster. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D78427
2020-06-07 22:29:05 +00:00
Mike Hommey 33f75a13db Bug 1643504 - Upgrade Windows cross builds to Debian buster. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D78426
2020-06-07 22:29:02 +00:00
Mike Hommey 80c76c5881 Bug 1643504 - Upgrade Linux plain builds to Debian buster. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D78425
2020-06-07 22:29:00 +00:00
Mike Hommey a79f981981 Bug 1643504 - Upgrade mingw build docker image to Debian buster. r=firefox-build-system-reviewers,rstewart
Differential Revision: https://phabricator.services.mozilla.com/D78424
2020-06-07 22:28:58 +00:00
Mike Hommey bcf92befab Bug 1643504 - Upgrade partner repack docker image to Debian buster. r=bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D78423
2020-06-07 22:28:55 +00:00
Mike Hommey 4e9162942f Bug 1643504 - Upgrade custom-v8 docker image to Debian buster. r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D78420
2020-06-07 22:28:58 +00:00
Mike Hommey 563675005b Bug 1643504 - Upgrade android build docker images to Debian buster. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D78418
2020-06-07 22:28:57 +00:00
Bogdan Tara 72a33db2ac Backed out changeset 2ed99d13012d (bug 1591466) on ahal's request 2020-06-06 06:49:13 +03:00
Andrew Halberstadt c40751d405 Bug 1641131 - [taskgraph] Filter out devedition builds and tests with |mach try auto|, r=marco
Differential Revision: https://phabricator.services.mozilla.com/D78569
2020-06-05 20:28:44 +00:00
Joel Maher 7ea3ea9209 Bug 1591466 - split raptor tests into tier-1 (high value) and tier-2 (lower value). r=sparky
split raptor tests into tier-1 (high value) and tier-2 (lower value)

Differential Revision: https://phabricator.services.mozilla.com/D78247
2020-06-05 21:07:21 +00:00
Bogdan Tara 7e299a95cf Backed out 3 changesets (bug 1640278) for verify lint failure CLOSED TREE
Backed out changeset e63b60fe21fe (bug 1640278)
Backed out changeset dbf1e5788c22 (bug 1640278)
Backed out changeset 27ae032f17be (bug 1640278)
2020-06-05 22:59:20 +03:00
Justin Wood 7fa31e9752 Bug 1640278 - Fix bustage because empty {} is a dict not a set CLOSED TREE
Differential Revision: https://phabricator.services.mozilla.com/D78564
2020-06-05 19:40:48 +00:00
Justin Wood fe8c054562 Bug 1640278 - Fixup run-on-projects verification failures. r=sylvestre
This is in-part a followup to 1638260 which fixed some cases of this problem.

Differential Revision: https://phabricator.services.mozilla.com/D76746
2020-06-05 14:52:03 +00:00
Justin Wood b1ec9e9331 Bug 1640278 - Do validation of run-on-projects. r=catlee
Differential Revision: https://phabricator.services.mozilla.com/D76540
2020-06-05 16:41:21 +00:00
Sylvestre Ledru c5a6924cbf Bug 1643335 - Run hazard-linux64-haz/debug less often r=jmaher,sfink
Differential Revision: https://phabricator.services.mozilla.com/D78275
2020-06-05 16:39:58 +00:00
Narcis Beleuzu 3fe80978c1 Backed out changeset c5e87f7e82af (bug 1643335) for breaking the Gecko Decision Task. CLOSED TREE 2020-06-05 19:37:04 +03:00
Sylvestre Ledru 07064e1400 Bug 1643335 - Run hazard-linux64-haz/debug less often r=jmaher,sfink
Differential Revision: https://phabricator.services.mozilla.com/D78275
2020-06-05 16:01:43 +00:00
Andrew Halberstadt 191a0b1695 Bug 1641131 - [taskgraph] Fix the 'push-interval-*' schema to not require an argument, r=jmaher
The Backstop optimization doesn't take any arguments, yet the schema for 'push-optimization-*'
requires a 'schedules-component' that goes ignored. Fix this.

Differential Revision: https://phabricator.services.mozilla.com/D77058
2020-06-05 15:38:24 +00:00
Gregory Mierzwinski acf95bb010 Bug 1641877 - Migrate reference browser raptor pageload tests to browsertime. r=perftest-reviewers,Bebe
This patch moves all existing reference browser pageload tests from raptor-webext to browsertime. We also delete all raptor-webext reference browser tests.

Differential Revision: https://phabricator.services.mozilla.com/D77474
2020-06-05 11:00:49 +00:00
Simon Fraser d8029a3fe7 Bug 1643661 Fix shell quoting with hg commands r=mtabara
shellcheck will moan about this, as it's potentially unsafe

Differential Revision: https://phabricator.services.mozilla.com/D78486
2020-06-05 12:06:21 +00:00
Marco Castelluccio 9e95f1a2e5 Bug 1637544 - Add a 'project' to the fake parameters used during taskgraph tests. r=tomprince DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D78399
2020-06-04 21:52:34 +00:00
Marco Castelluccio 525a47b947 Bug 1637544 - Run no-sccache verification only on release projects. r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D77248
2020-06-04 19:40:54 +00:00
Marco Castelluccio b2e275bf31 Bug 1637544 - Support project-specific verifications in taskgraph. r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D77247
2020-06-04 19:40:46 +00:00
Andrew Halberstadt 0f1bc1a064 Bug 1633866 - [taskgraph] Improve 'guess_mozinfo_from_task' for more balanced chunks, r=gbrown
Adds a few properties to the mozinfo guess that were previously missing.

Differential Revision: https://phabricator.services.mozilla.com/D78156
2020-06-04 18:35:07 +00:00
Andrew Halberstadt ddaf15c72f Bug 1633866 - [taskgraph] Implement a multiplier to tweak the duration for certain platforms / suites, r=gbrown
This is needed because xpcshell runs in parallel. Therefore we need to know the number
of threads are used (which varies by platform) in order to accurately determine how
many chunks are needed.

This allows us to specify a platform / suite specific multiplier to compensate via trial
and error.

Differential Revision: https://phabricator.services.mozilla.com/D78153
2020-06-04 15:34:51 +00:00
Andrew Halberstadt d4001d553e Bug 1633866 - [taskgraph] Truncate runtimes that exceed DYNAMIC_CHUNK_DURATION so they don't artificially increase chunks, r=gbrown
For example, assume we have three manifests:

foo -> 98 minutes
bar -> 1 minute
baz -> 1 minute

And let's say we want average chunk lengths of ~20 minutes. The calculation
would currently say we need 5 chunks ((98 + 1 + 1) / 20). Even if set the max
chunks to the # of manifests, it would still return 3. Clearly this only needs
at most 2 chunks to run.

By truncating the runtime, we'd have:

foo -> 20 minutes (truncated)
bar -> 1 minute
baz -> 1 minute

So we'd actually only assign one chunk (round((20 + 1 + 1) / 20)). Despite
having a single 100 minute chunk, this outcome is desirable as we now avoid the
overhead of a second chunk that only runs 2 minutes worth of tests.

Depends on D77871

Differential Revision: https://phabricator.services.mozilla.com/D77873
2020-06-03 21:00:43 +00:00
Andrew Halberstadt b3fbd0cf70 Bug 1633866 - [taskgraph] Always use chunk number as symbol if it would otherwise be empty, r=jmaher
Depends on D76523

Differential Revision: https://phabricator.services.mozilla.com/D77871
2020-06-02 18:56:54 +00:00
Andrew Halberstadt 27d78d7059 Bug 1633866 - [taskgraph] Create a 'bugbug' based test manifest loader, r=marco
Loads manifests using bugbug's push/schedules endpoint. For now we use as low
confidence threshold to select manifests. This is to try and both:

1) Improve regression detection rate
2) Save resources

at the same time. This way we theoretically shouldn't regress either dimension.

Note that we still optimize with CT_MEDIUM. My thinking is that at least all
of the "extra" stuff is still guaranteed to be relevant. Though we may want to
consider making these two thresholds match.

Differential Revision: https://phabricator.services.mozilla.com/D76523
2020-06-04 18:45:09 +00:00
Andrew Halberstadt be4e9089ec Bug 1633866 - [taskgraph] Simplify how 'suites' are handled in manifest resolving and chunking, r=egao
This change is a straight refactor and results in identical full taskgraphs.

Differential Revision: https://phabricator.services.mozilla.com/D77543
2020-06-02 20:00:46 +00:00
Razvan Maries e46cf863d7 Backed out changeset 447cbc28fda3 (bug 1641948) for Android 7 perma failures. CLOSED TREE 2020-06-04 18:55:43 +03:00
Razvan Maries 179572857b Backed out changeset d51c3c722d65 (bug 1643335) for breaking Gecko Decision Task. CLOSED TREE 2020-06-04 18:29:19 +03:00
Sylvestre Ledru 0f57f34c18 Bug 1643335 - Run hazard-linux64-haz/debug less often r=jmaher,sfink
Differential Revision: https://phabricator.services.mozilla.com/D78275
2020-06-04 15:19:38 +00:00
Ben Hearsum dd5a8a4ba7 Bug 1642713: allow diff-artifact tasks to be optimized by push interval r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D77859
2020-06-04 14:27:42 +00:00
Ben Hearsum 3dca5e3f67 Bug 1641948: remove android opt builds from non-mozilla-central branches; reduce frequency of fat-aar builds r=snorp,Callek
snorp - I'm not sure if you want/need to review this yourself so I'm flagging you as optional.

Differential Revision: https://phabricator.services.mozilla.com/D77834
2020-06-03 20:52:38 +00:00
Ed Lee 96ed60b4bd Bug 1641863 - Upgrade Node 10.x (in preparation for ESR 78) r=Standard8. CLOSED TREE
Node v10.21.0 (LTS) just released https://nodejs.org/en/blog/release/v10.21.0/

Differential Revision: https://phabricator.services.mozilla.com/D77904
2020-06-03 18:00:52 +00:00
Tom Prince 9eac413f54 Bug 1642446: Add an attribute to annotate toolchain tasks used by `mach bootstrap`; r=firefox-build-system-reviewers,rstewart
Toolchains that are used for local development need to be built on a level-3
branch to installable via `mach bootstrap`. Add an attribute to track the fact
that a toolchain is used that way, and:
- ensure that everything installed via `mach boostrap` has that attribute set
- ensure that everything with that attribute set is built on trunk projects

We could additionally verify that attribute is only set on things used by
bootstrap, but bootstrap doesn't currently have an exhaustive list of things
that it might install, making that difficult.

Differential Revision: https://phabricator.services.mozilla.com/D77706
2020-06-03 15:55:26 +00:00
Nathan Froyd 40348063b3 Bug 1643029 - remove build-.*-aarch64 from being hidden behind --full; r=ahal
It's not good to hide Android AArch64 builds, a tier 1 platform, behind `--full`.

Differential Revision: https://phabricator.services.mozilla.com/D78070
2020-06-03 16:15:43 +00:00
Andrew Halberstadt 3428cd2244 Bug 1601782 - [ci] Enable crashtest-fis on same platforms where reftest-fis is running, r=jmaher
This adds the following task:
test-linux1804-64-qr/debug-crashtest-fis-e10s

Differential Revision: https://phabricator.services.mozilla.com/D78030
2020-06-03 14:09:42 +00:00
Kartikaya Gupta b086844bd9 Bug 1642720 - Disable reftests on win7/mingw builds. r=tjr
These reftests seem to hang quite frequently, caused by the content process
hanging on file create/open operations. See dependencies of bug 1642719 for
some examples.

Differential Revision: https://phabricator.services.mozilla.com/D77875
2020-06-03 14:17:08 +00:00
Marco Castelluccio 6b213d3c19 Bug 1637810 - Define a new strategy that optimizes builds away if they are not selected by bugbug. r=ahal DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D75621
2020-06-03 09:53:17 +00:00
Tom Prince 8073961f7e Bug 1641971: [taskgraph] Use task-reference to refer to the decision task in task; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77552
2020-06-03 02:35:00 +00:00
Tom Prince ca6c9020de Bug 1641971: [taskgraph] Always pass an explicit decision task id to create_tasks; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77551
2020-06-03 02:35:03 +00:00
Tom Prince 9f3450be95 Bug 1641971: [taskgraph] Pass explicit decision task id to action.json generation; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77550
2020-06-03 02:35:05 +00:00
Tom Prince 86711ecee8 Bug 1641971: Use `task-reference` to pass the decision task id to shadow schedulers; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D77549
2020-06-03 02:35:07 +00:00
Tom Prince 581eba974b Bug 1641971: Pass `mach perftest` options explicitly, rather than inpsect parameters; r=tarek
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-03 02:35:09 +00:00
Tom Prince 6617f03f05 Bug 1641971: Allow references to decision task via task-reference/artifact-reference; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77547
2020-06-03 02:43:34 +00:00
Tom Prince 01e244f62c Bug 1641971: Allow making references to a task itself in the task definition; r=Callek
This is to support linking to the a specific task in taskcluster.

Differential Revision: https://phabricator.services.mozilla.com/D77529
2020-06-03 02:35:14 +00:00
Henrik Skupin 12f6324f02 Bug 1623128 - [remote] Run Puppeteer unit tests on ubuntu18.04. r=egao
Differential Revision: https://phabricator.services.mozilla.com/D77765
2020-06-03 04:07:43 +00:00
Mike Hommey 98b889cf83 Bug 1622963 - Install python 3.6 in build docker images. CLOSED TREE
Differential Revision: https://phabricator.services.mozilla.com/D77753
2020-06-02 03:51:14 +00:00
Narcis Beleuzu 479c45bbea Backed out 7 changesets (bug 1641971) for breaking Gecko Decision Task opt . CLOSED TREE
Backed out changeset a039e60a26d2 (bug 1641971)
Backed out changeset e7d4c490c31a (bug 1641971)
Backed out changeset f2322981b294 (bug 1641971)
Backed out changeset d123b08d2c20 (bug 1641971)
Backed out changeset 57772c0b9b3e (bug 1641971)
Backed out changeset 11a887f73dbb (bug 1641971)
Backed out changeset e9c7a59d4bc5 (bug 1641971)
2020-06-03 05:30:51 +03:00
Mitchell Hentges 658f5f0eeb Bug 1642411: Removes unused test-macosx.sh script r=ahal
A grep through the repository and a brief discussion with #releng seems to point towards this script not being used.

Differential Revision: https://phabricator.services.mozilla.com/D77689
2020-06-02 18:55:35 +00:00
Tom Prince 6f0a7bbba6 Bug 1641971: [taskgraph] Use task-reference to refer to the decision task in task; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77552
2020-06-02 18:23:23 +00:00
Tom Prince f32e4a4f4d Bug 1641971: [taskgraph] Always pass an explicit decision task id to create_tasks; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77551
2020-06-02 18:23:19 +00:00
Tom Prince 7c6a413ca9 Bug 1641971: [taskgraph] Pass explicit decision task id to action.json generation; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77550
2020-06-02 18:23:20 +00:00
Tom Prince 73b2e6f745 Bug 1641971: Use `task-reference` to pass the decision task id to shadow schedulers; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D77549
2020-06-02 18:23:11 +00:00
Tom Prince 11aedca9b2 Bug 1641971: Pass `mach perftest` options explicitly, rather than inpsect parameters; r=tarek
Differential Revision: https://phabricator.services.mozilla.com/D77548
2020-06-02 18:23:07 +00:00
Tom Prince 602da9cdcd Bug 1641971: Allow references to decision task via task-reference/artifact-reference; r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77547
2020-06-02 18:50:10 +00:00
Tom Prince de6126516d Bug 1641971: Allow making references to a task itself in the task definition; r=Callek
This is to support linking to the a specific task in taskcluster.

Differential Revision: https://phabricator.services.mozilla.com/D77529
2020-06-02 18:22:55 +00:00
Chris AtLee 4925d4d08e Bug 1638990: Use requests for fetching parameters r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D76133
2020-05-28 19:01:42 +00:00
Chris AtLee b6edad89bf Bug 1638990: Sort values in taskgraph in order to make comparing output easier r=rstewart
Differential Revision: https://phabricator.services.mozilla.com/D77320
2020-05-29 15:01:50 +00:00
Sylvestre Ledru 273a7905e6 Bug 1635826 - platform docs: Document WebRender tasks r=jbonisteel
Differential Revision: https://phabricator.services.mozilla.com/D77677
2020-06-01 15:28:13 +00:00
Sylvestre Ledru c5994e6bba Bug 1628289 - platform docs: the Documentation jobs moved away from the linting section
Depends on D76185

Differential Revision: https://phabricator.services.mozilla.com/D77657
2020-06-01 15:28:13 +00:00
Sylvestre Ledru 58f742cd8b Bug 1635826 - platform docs: Remove the nightly platforms r=Callek
Remove in treeherder in this patch:
1568eaf479

Differential Revision: https://phabricator.services.mozilla.com/D76185
2020-06-01 11:11:22 +00:00
Edwin Takahashi 3efe2edb5c Bug 1641395 - install amd64 variants of existing i386 packages and insert additional startup parameters for pulseaudio r=jmaher
Changes:
  - install the `amd64` variants of various packages alongside `i386` packages
  - install `amd64` version of `fontconfig` instead
  - additional command line arguments for pulseaudio initialization
  - do not add configuration files to `~/.pulse` in the docker image

Differential Revision: https://phabricator.services.mozilla.com/D77512
2020-06-01 20:44:47 +00:00
Agi Sferro 4e005a2cbd Bug 1641711 - Update artifacts for mozlint-android. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D77544
2020-06-01 15:57:58 +00:00
Tom Prince 98dd864200 No bug: Don't use proxy for accessing taskcluster index; r=Callek
The index is not protected by scopes, so don't use the proxy to access it

Differential Revision: https://phabricator.services.mozilla.com/D77537
2020-05-29 21:37:46 +00:00
emilfs 7e13ebcb0a Bug 1628289 - move documentation jobs from lint/opt to doc/opt. r=sylvestre
Differential Revision: https://phabricator.services.mozilla.com/D77481
2020-05-29 21:12:13 +00:00
Mitchell Hentges 2a66b39bc4 Bug 1641962: Defer imports of distutils.util to ensure bootstrap can run r=rstewart,perftest-reviewers,sparky
Differential Revision: https://phabricator.services.mozilla.com/D77516
2020-05-29 20:17:27 +00:00
Andrew Halberstadt f0e5f6ed7b Bug 1633866 - [taskgraph] Create a system to choose how test manifests are loaded via a parameter, r=tomprince
Currently test manifests are loaded by instantiating a TestResolver and
traversing moz.build files to find manifests.

With 'manifest-scheduling', we'll want to grab the manifests directly from the
bugbug service instead. Initially we'll want to enable manifest-scheduling with
|mach try auto|, but not on autoland yet.

This patch will allow |mach try auto| to set the parameter that causes bugbug
to be used (see future commits in this bug).

Differential Revision: https://phabricator.services.mozilla.com/D76522
2020-05-29 18:51:14 +00:00
Bogdan Tara c72f5589b1 Backed out changeset 2c18e64584af (bug 1641131) for gecko decision task bustage CLOSED TREE 2020-05-29 18:33:23 +03:00
Andrew Halberstadt ec4749227c Bug 1641131 - [taskgraph] Make 'repackage' and 'repackage-signing' tasks copy their dependency's optimization, r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77346
2020-05-29 14:43:29 +00:00
Andrew Halberstadt 12b0e45ee9 Bug 1641131 - [taskgraph] Make 'upload-symbols' tasks copy their dependency's optimization r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D77048
2020-05-29 14:42:51 +00:00