It could go into its own test suite, but it 1) depends on `mozbuild` code, so the `mozbuild` suite as well as this new suite would be running on any push that touches `mozbuild` code anyway, and 2) this is code that runs during the build, so it's not out of place.
Differential Revision: https://phabricator.services.mozilla.com/D84547
It could go into its own test suite, but it 1) depends on `mozbuild` code, so the `mozbuild` suite as well as this new suite would be running on any push that touches `mozbuild` code anyway, and 2) this is code that runs during the build, so it's not out of place.
Differential Revision: https://phabricator.services.mozilla.com/D84547
The Reference Browser represents our tech stack up to and including the Android-Components layer.
It would be useful to have an applink startup test for this browser to detect improvements and regressions.
Differential Revision: https://phabricator.services.mozilla.com/D81407
This creates a new set of optimization strategies
(taskgraph.optimize:project.autoland) to use with autoland. Among other things,
it also means there's no need for the 'test-try' optimization as the autoland
strategies are no longer the default behaviour.
Differential Revision: https://phabricator.services.mozilla.com/D79705
Creates a new utility function in 'taskgraph.util.taskcluster' to return task
definitions from a task group. Also refactors the existing
'list_task_group_incomplete_tasks' function to use it.
Differential Revision: https://phabricator.services.mozilla.com/D74407
It could go into its own test suite, but it 1) depends on `mozbuild` code, so the `mozbuild` suite as well as this new suite would be running on any push that touches `mozbuild` code anyway, and 2) this is code that runs during the build, so it's not out of place.
Differential Revision: https://phabricator.services.mozilla.com/D84547
I suspect these were an artifact of building the checked-out repository
inside the Firefox source directory, but that is not a problem anymore.
Differential Revision: https://phabricator.services.mozilla.com/D84363
We don't need this for our (current) builds, which are cross-builds, but we
would need this at some future date if we ditched the breakpad `dump_syms`.
Depends on D83528
Differential Revision: https://phabricator.services.mozilla.com/D83529
I suspect these were an artifact of building the checked-out repository
inside the Firefox source directory, but that is not a problem anymore.
Differential Revision: https://phabricator.services.mozilla.com/D83528
When links to specific tasks in treeherder was added to the description in
Bug 1630710, This caused the descriptions for several derived tasks to include
the `{'task-reference': '...'}` wrapper in the derived description. This is
in addition to the already duplicate links to treeherder those tasks had.
To ameliorate this issue, keep track of the original description without the
treeherder link and use that in derived tasks.
Differential Revision: https://phabricator.services.mozilla.com/D84047
This adds the talos-profiling jobs to the relevant test-platforms so that they
can be run for Linux and Windows shippable builds with WebRender enabled.
However they are also disabled from running on m-c by default; they must be
run explicitly using `./mach try fuzzy --full` or using the TreeHerder add jobs
feature.
Differential Revision: https://phabricator.services.mozilla.com/D84011
This block is identical to the run-on-projects applied to all the talos
profiling jobs in the defaults section at the top of the file. Therefore
it is redundant.
Differential Revision: https://phabricator.services.mozilla.com/D84010
Since the system mozdevice version on bitbar is old, and mozharness now requires
mozdevice 4.0.0, we simply have the build job make a tarball of mozdevice 4.0.0
from the gecko source tree and export it as an artifact. The wrench job that
runs on bitbar can unpack the tarball and add it to the python path so it gets
picked up.
Differential Revision: https://phabricator.services.mozilla.com/D84070
This change the strategy aliases that have bugbug, so that they handle the extra
arguments that are needed by bugbug.
Differential Revision: https://phabricator.services.mozilla.com/D83438
This patch adds a new pageload testing mode called chimera mode. This mode runs the cold and warm pageload tests together within a single browser session (cold first, warm second). Some changes had to be made in the visual metrics task to be able to do this (per-test specification of extraOptions).
Differential Revision: https://phabricator.services.mozilla.com/D83356
This change the strategy aliases that have bugbug, so that they handle the extra
arguments that are needed by bugbug.
Differential Revision: https://phabricator.services.mozilla.com/D83438
I'm planning on replacing voluptuous with something faster in Bug 1652123, but
in the meantime, compile the schema is slow, so there is no reason to do so if
we aren't going to check them anyway.
Differential Revision: https://phabricator.services.mozilla.com/D83540
Changes:
- create a `test-android.sh` script that is based on, but removes non-android related paths from the `test-linux.sh` script.
Differential Revision: https://phabricator.services.mozilla.com/D82252
Changes:
- set up a new minimal docker image for `android-em` tests.
- reorganize commonly referenced files into a specific folder `taskcluster/docker/files` to reduce instances of same file being copy-pasted into directories.
- add android-test docker build task to taskcluster.
Differential Revision: https://phabricator.services.mozilla.com/D82204
Changes:
- set up a new minimal docker image for `android-em` tests.
- reorganize commonly referenced files into a specific folder `taskcluster/docker/files` to reduce instances of same file being copy-pasted into directories.
- add android-test docker build task to taskcluster.
Differential Revision: https://phabricator.services.mozilla.com/D82204
Changes:
- in `mozharness_test.py`, add new optional key `job-script` similar to existing provision for toolchain tasks.
- set the default bootstrapping script depending on platform.
- substitute the correct script in the final mozharness call.
Differential Revision: https://phabricator.services.mozilla.com/D83559
Changes:
- create a `test-android.sh` script that is based on, but removes non-android related paths from the `test-linux.sh` script.
Differential Revision: https://phabricator.services.mozilla.com/D82252
Changes:
- set up a new minimal docker image for `android-em` tests.
- reorganize commonly referenced files into a specific folder `taskcluster/docker/files` to reduce instances of same file being copy-pasted into directories.
- add android-test docker build task to taskcluster.
Differential Revision: https://phabricator.services.mozilla.com/D82204
Disabled because we've observed that this is introducing noise on G5.
Also added perftuning to the other Pixel 2 tests.
Differential Revision: https://phabricator.services.mozilla.com/D83808
These scripts don't call `build-clang.py`, they just repackage artifacts from other tasks that do.
I went with `repack` over `repackage` since that seems to be the established pattern in `taskcluster/scripts/misc/`.
Differential Revision: https://phabricator.services.mozilla.com/D83532
This change doesn't seem to affect anything now, but when I was pulling out
some bits into `job-defaults`, the lack of a `job-name` resulted in some
extremely cryptic errors from inside taskcluster code.
Differential Revision: https://phabricator.services.mozilla.com/D83127
The explanatory comment at the top of the new file provides some context.
The larger motivating context here is wanting to use `job-defaults` to clean
up the toolchain fetches for the Linux build jobs. But if we did that, we
run into the problem that such fetches are merged with whatever the
individual job(s) specify for their toolchain fetches, not replaced. So
we'd wind up with multiple clang toolchains being downloaded, or similar,
and things would go downhill from there, as workers attempt to unpack those
things to the same location (which is bad) or perhaps error out (which is
also bad).
This change therefore paves the way for providing `job-defaults` in linux.yml.
Differential Revision: https://phabricator.services.mozilla.com/D83158
Currently the macosx-cross toolchain build pulls in a linux64-clang toolchain, uses it to build a Mac native toolchain, and then clobbers the result with pieces of the Linux toolchain. This means that the same version of LLVM is used to build the Mac pieces and be part of the final artifact. This will become a problem with upcoming LLVM 11 where we'll want to build the Mac pieces with LLVM 9 but otherwise repackage an LLVM 11 Linux toolchain.
So this commit makes the macosx-cross workflow look more like the win-cross one: take two compilers built elsewhere and just merge them.
Differential Revision: https://phabricator.services.mozilla.com/D83378
If we are generating only a part of the graph, to given kind, don't fail if a
build is packaging tests and there is no corresponding test task, as the tests
may not have been generated.
Differential Revision: https://phabricator.services.mozilla.com/D82097
LLVM 11 introduces a hard requirement for SDK 10.12 in order to build for Mac. We want to keep building older LLVMs with 10.11 though, so this patch adds some flexibility so that build-clang can make use of whatever SDK package a particular task pulls from tooltool (but still requesting a deployment target of 10.11).
Differential Revision: https://phabricator.services.mozilla.com/D82621
Changes:
- removes the ability to schedule web-platform-tests-reftest-backlog and web-platform-tests-backlog from non-opt variants of each platform.
- in addition, removes the ability to schedule the above from windows7-32, regardless of the variant.
- clean up references to now-deprecated platforms in backlog task definitions.
Differential Revision: https://phabricator.services.mozilla.com/D83172
We could make a new task for this, but `mozwebidlcodegen` depends on code in `mozbuild`, and vice-versa, so there doesn't really seem to be any meaningful advantage to that.
Differential Revision: https://phabricator.services.mozilla.com/D83187
We already have `value = source[key]` thanks to iteration; we should be
using `value` directly rather than re-doing `source[key]`.
Differential Revision: https://phabricator.services.mozilla.com/D83157
The `plain` builds are intended to make sure that building with an empty mozconfig doesn't break. Since an empty mozconfig build wouldn't have wasm sandboxing, neither should the `plain` builds.
While here, also remove wasi/lucet dependencies from the base-toolchain tasks, since they don't enable sandboxing either.
Differential Revision: https://phabricator.services.mozilla.com/D83011
The unit test is broken under Python 3 on macOS, so I haven't included any macOS version of this task; one should probably be added if/when that's fixed.
Differential Revision: https://phabricator.services.mozilla.com/D83169
Add an action that will trigger a task that runs
`mach release push-scriptworker-canary`
to test a new scriptworker deployment.
Differential Revision: https://phabricator.services.mozilla.com/D82821
Adds some additional assertions against what would be ran with |mach try auto|.
This also fixes a legit bug that the 'test_important_manifests_only' test
caught.
Differential Revision: https://phabricator.services.mozilla.com/D81404
This creates a general pytest plugin under "config/mozunit" which implements
the ability to mark tests as "slow". Slow tests will be skipped by default
unless "--run-slow" is passed in.
Differential Revision: https://phabricator.services.mozilla.com/D81402
This adds the support for backfilling of manifest scheduled tasks while
preserving the existing features (e.g. schedule a specific test path).
Remove temporary new-backfill action.
Differential Revision: https://phabricator.services.mozilla.com/D82599
This optional mode to specify a test path when backfilling has been broken for a while
and is not being used to make it worth fixing. It is a mode
originally designed to be used with the test-verify tasks.
Differential Revision: https://phabricator.services.mozilla.com/D82722
In Bug 1641971 (e7d4c490c31a) we changed `taskgraph.create.create_tasks` to
always get the decision task id to use. However, the call in create-interactive
passed `None`, expecting to have on generated. This changes that to use the
`create-interactive`s action task as the task id.
Differential Revision: https://phabricator.services.mozilla.com/D82652
This patch adds all desktop pageload tests to browsertime. It adds taskcluster tasks for these but doesn't enable them in production. They can only be run when --full is used.
Differential Revision: https://phabricator.services.mozilla.com/D82432
This patch adds an option to disable the firefox window recorder in browsertime, it also adds it to the browsertime-mobile taskcluster config since it is disabled there (but not on desktop).
Differential Revision: https://phabricator.services.mozilla.com/D82431
We were building a context with a top-level directory matching the image name
during graph generation. This is not accepted by docker build. This didn't
cause issues since we only used it for digest generation. Since we are
switching to use the decision generated context, change this to match the
context being generated when building an image.
Differential Revision: https://phabricator.services.mozilla.com/D77841
To support using kaniko[1] for building images, we need to generate the docker
contexts in a seprate task from the docker-image task. Since we use the hash of
the context as the cached-task digest, we generate the context in the decision
task already, so this adds a way to write that out to be used by downstream
tasks.
[1] https://github.com/GoogleContainerTools/kaniko
Differential Revision: https://phabricator.services.mozilla.com/D77839
I'm going to change this from being autodected, to specified explicitly, and so
`use-sccache` better captures the meaning.
Differential Revision: https://phabricator.services.mozilla.com/D82483
CLOSED TREE
Backed out changeset 93b955e5c048 (bug 1637544)
Backed out changeset be0717d76643 (bug 1637544)
Backed out changeset 447fea64b68d (bug 1637544)
I'm going to change this from being autodected, to specified explicitly, and so
`use-sccache` better captures the meaning.
Differential Revision: https://phabricator.services.mozilla.com/D82483
The current version hits OOM errors when dsymutil-ing files created by clang 10 and 11.
The choice of clang-10 here is somewhat arbitrary in that it's a fetch job that we conveniently already had in the tree. It doesn't have to be exactly version 10 specifically.
Differential Revision: https://phabricator.services.mozilla.com/D82453
Fixes the bustage described in 4e1faecce8
In theory I could leave this task alone and fork a new toolchain for use by clang 11, but this seems internal-enough that existing builds shouldn't be negatively affected by this.
Differential Revision: https://phabricator.services.mozilla.com/D82444
We already do this for e.g. `linux64-clang-9`; this patch extends that pattern everywhere.
This will make it easy to do try runs with other clangs: just move the `toolchain-alias` lines from the `9` tasks to the `trunk` tasks.
Also, this makes dependencies more explicit: for example the gn task specifically requests a clang-9 compiler, so it will also need a clang-9-based cctools-port, rather than whatever moving value the alias represents.
Differential Revision: https://phabricator.services.mozilla.com/D82441
Some cleanup before I add more copies of this task.
Since this is based on a repo called `cctools-port` it seems like it would be better to keep that substring intact.
Differential Revision: https://phabricator.services.mozilla.com/D82439
We use the symbol of the original task we backfill and suffix the revision of the push
from which we backfill. This makes it a bit more clear as to from which push we requested
the backfill.
We also modify the group symbol (if available) to group backfilled tasks.
***
***
Differential Revision: https://phabricator.services.mozilla.com/D81379
This backfill action schedules a secondary action per push. This secondary action
schedules the intended task on behalf of the primary action.
This introduces logic on how to trigger a hook by an action.
Differential Revision: https://phabricator.services.mozilla.com/D79351
This is the only place where we override the default of `required=False`.
Change it to match the rest of the schemas, so as not to confuse people who
don't happen to notice the final line of the schema.
Differential Revision: https://phabricator.services.mozilla.com/D81657
We don't ever set `depname` in any of the kinds that use these transforms.
Since we don't apply default values in schemas, `Required(..., default=...)`
behaves equivalently to `Optional(...)`, so we can just remove these options.
Differential Revision: https://phabricator.services.mozilla.com/D81654
This could go further and test for m-c or the trunk/integration aliases, etc. but I opted for not letting this increase try-syntax jobs.
Differential Revision: https://phabricator.services.mozilla.com/D81756
Right now setup_perftest_test_date adds --test-date yesterday to all perftest
runs. we want that only for the ones doing batches
Differential Revision: https://phabricator.services.mozilla.com/D81562
Right now we are running `Coverity Static Analysis` for each merge to `mozilla-central`,
this is suboptimal since the resources required to do so are very expensive. It's
enough to run it once a day.
Differential Revision: https://phabricator.services.mozilla.com/D81510
Changes:
- add new flag in `web_platform_tests.py`, with name `backlog` that is a boolean value.
- simplify the timeout multiplier conditional to check for presence of the `backlog` flag.
- add the new flag to the `mozharness_extra_options` flag in `web-platform.yml`
Differential Revision: https://phabricator.services.mozilla.com/D81184
Changes:
- added tests that exercise manifest loading, mozinfo guessing and the overall process of chunking.
- tests added for both web-platform and traditional mochitest/xpcshell suites.
Differential Revision: https://phabricator.services.mozilla.com/D80985
And add suppressions for the new errors that this adds to the valgrind
run. It's not clear if it's a legitimate thing that LLVM does that
valgrind doesn't cope with, like many others, but the fact is running
valgrind on a PGO/LTO build doesn't show these errors, so we're not
actually shipping them anyways (but does show _different_ errors that we
don't see on the build the valgrind jobs do)
Differential Revision: https://phabricator.services.mozilla.com/D81016
Changes:
- install `xwit` package in the ubuntu1804 docker image setup phase.
- in `test-linux.sh` call xwit to reposition the cursor to a coodinate of 0,0
Differential Revision: https://phabricator.services.mozilla.com/D80698
Using a higher confidence threshold means manifest scheduling (i.e just |mach
try auto| for now), will schedule fewer manifests. We have data suggests that
this bump should reduce the number of manifests scheduled by more than half,
while only very slightly reducing the regression detection rate (still above
95%).
Differential Revision: https://phabricator.services.mozilla.com/D81085
There isn't a Debian package for that verson yet, so keep using the code
that pulls from the git repository, but use the revision that was
released as 3.16.1.
Differential Revision: https://phabricator.services.mozilla.com/D80957
We currently base this on the cb_name, but it would make be better to make this
explicit, so that different actions can share a permisssion. In particular, we are
adding a pair of new backfill actions that needs a singlle non-default permission.
Differential Revision: https://phabricator.services.mozilla.com/D80940
We are seeing a pretty broad distribution of results for the applink tests.
This bug increases the number of browsertime iterations from 10 to 14.
Differential Revision: https://phabricator.services.mozilla.com/D80363
Changes:
- change default return value for `get_runtimes` method to an empty dictionary if data is not found.
- add in fixtures and tests for `get_runtimes` and `chunk_manifests` method for web-platform-tests and subsuites.
Differential Revision: https://phabricator.services.mozilla.com/D80250
Changes:
- remove web-platform-tests from the test-set `macosx1014-tests`
- create new group `web-platform-tests-backlog` and move all backlog types into that group
- add the new group into test-platforms where applicable
Differential Revision: https://phabricator.services.mozilla.com/D80555
Changes:
Previous patch to apply the reduced timeout multiplier was only applicable to vanilla `web-platform-tests`. This patch reduces the chunk count for `web-platform-tests-reftest` and applies the reduction multiplier.
Differential Revision: https://phabricator.services.mozilla.com/D80520
This patch does a few small things to clean up our CI tests:
(1) The GPT cron is simplified so that it can be more manageable. In this change I also include a new helper method to make sure we run our android tests on the build variants that we should be running them on (and no others).
(2) The geckoview browsertime warm tests were running on 1 platform too many - this is no longer the case.
(3) Fennec vismet tests were previously excluded but are now included.
(4) All fennec tests that were in GPT cron have been moved to the Fennec_v68 cron so they only run once a week.
Differential Revision: https://phabricator.services.mozilla.com/D79891
It's required to properly setup psutil in the python3 virtualenv, and
psutil is required for build resources tracking by mach.
It's not clear to me why this hasn't been a problem since `mach build`
switched to python3, though. ¯\_(ツ)_/¯
Differential Revision: https://phabricator.services.mozilla.com/D80343
This patch adds a `--test-date` option to mozperftest that lets us specify the date that we are testing on. This lets us trigger multi-commit tests on past dates, and also ensures that retriggers of the multi-commit tests use the same day that they were pushed on (giving us reproducibility).
Differential Revision: https://phabricator.services.mozilla.com/D80173
It looks like genHPKPStaticPins.js can fail and still exit 0, so try and
make sure its output is plausible. The last thing it writes is the
expiration time, so check that it's there, which should mean everything
else got generated as well.
Differential Revision: https://phabricator.services.mozilla.com/D80206
Since `@memoize` on a method is per-instance, use a single instance for all
tests, rather than a new one for each tasks. This gives a ~4x speedup in test
task generation.
Differential Revision: https://phabricator.services.mozilla.com/D79764
Remove the BinAST tests from following jobs:
- SM(p)
- SM(cgc)
Remove the BinAST performance tests:
- binast-instagram
Mark the WPT BinAST tests as backlog.
Differential Revision: https://phabricator.services.mozilla.com/D77919
AFAIK there's no reason to pin to clang-9 specifically. In fact I already landed this change in the bouncy attempts at bug 1616692. I'm just separating it out in order to have fewer things in the air since I'm currently juggling clangs 9, 10, and 11 locally.
Differential Revision: https://phabricator.services.mozilla.com/D80018
Remove the BinAST tests from following jobs:
- SM(p)
- SM(cgc)
Remove the BinAST performance tests:
- binast-instagram
Mark the WPT BinAST tests as backlog.
Differential Revision: https://phabricator.services.mozilla.com/D77919
Changes:
- discontinue ability to run web-platform-tests-backlog on ccov/devedition platforms, including `web-platform-tests-reftest`.
Differential Revision: https://phabricator.services.mozilla.com/D79867
Also, make a change to force the installation of a more recent version
of clippy & rustfmt until we have a proper fix
Differential Revision: https://phabricator.services.mozilla.com/D79735
Changes:
- stop using runtime information for the chunking process of web-platform-tests
- remove the wpt_group_translation workaround as it is no longer necessary
- simplify chunking process to iterate and assign test paths to chunks in order
Simply assigning test paths (stand-in for manifests in other suites) is now possible because other patches in the stack now prevent the web-platform-tests harness (when an external process handles the grouping) from loading all tests in a given path.
Differential Revision: https://phabricator.services.mozilla.com/D78321
Changes:
- add new environment variable `TESTS_BY_MANIFEST_URL` referencing the tests-by-manfiest.json.gz artifact generated by decision task.
- add new method `download_manifest` in mozharness/web_platform_tests.py to download the artifact, then remove non-web-platform-test manifests.
- the processed artifact is then saved as JSON file containing only web-platform-tests for downstream harnesses to use.
Differential Revision: https://phabricator.services.mozilla.com/D78318
There are various binaries that are included in test archives that are
necessary for artifact builds to be repackaged. Don't generate artifact build
variants of tasks without them.
Differential Revision: https://phabricator.services.mozilla.com/D79575
This deliberately excludes builds that are implemented using the artifact build machinery,
but are not primarly intended to short-circuit build time. In particular the Windows aarch64
builds are not marked this way.
Differential Revision: https://phabricator.services.mozilla.com/D79549
This brings the `android-build` image very close to other build
images, paving the way for it to be folded in completely. It also
makes us more resilient in the face of upstream service interruptions.
Differential Revision: https://phabricator.services.mozilla.com/D78945
Changes:
- stop using runtime information for the chunking process of web-platform-tests
- remove the wpt_group_translation workaround as it is no longer necessary
- simplify chunking process to iterate and assign test paths to chunks in order
Simply assigning test paths (stand-in for manifests in other suites) is now possible because other patches in the stack now prevent the web-platform-tests harness (when an external process handles the grouping) from loading all tests in a given path.
Differential Revision: https://phabricator.services.mozilla.com/D78321
Changes:
- add new environment variable `TESTS_BY_MANIFEST_URL` referencing the tests-by-manfiest.json.gz artifact generated by decision task.
- add new method `download_manifest` in mozharness/web_platform_tests.py to download the artifact, then remove non-web-platform-test manifests.
- the processed artifact is then saved as JSON file containing only web-platform-tests for downstream harnesses to use.
Differential Revision: https://phabricator.services.mozilla.com/D78318
This patch disables and removes the raptor-webext fennec68 tests and replaces them with the browsertime variants. The raptor tests which ran through a cron are also swapped for the browsertime tests.
Differential Revision: https://phabricator.services.mozilla.com/D77674
This patch is the final patch for the raptor to browsertime geckoview migration. It disables all raptor-webext geckoview pageload tests running in production.
Differential Revision: https://phabricator.services.mozilla.com/D79157
Andreea Pavel
Marco Castelluccio
Ricky Stewart
Geoff Brown
Dorel Luca
Tom Prince
Andrew Creskey
David Major
Narcis Beleuzu
Andrew Halberstadt
Butkovits Atila
Sebastian Hengst
Jeff Gilbert
Mike Hommey
Cosmin Sabou
Nathan Froyd
Gregory Mierzwinski
Razvan Maries
Tarek Ziadé
Kartikaya Gupta
Johan Lorenzo
Mihai Alexandru Michis
Edwin Takahashi
Sylvestre Ledru
Kris Maglione
Mitchell Hentges
Armen Zambrano G.
Andi-Bogdan Postelnicu
Florin Strugariu
Chris AtLee
emilfs
Christian Holler
alexandru.ionescu
Justin Wood
James Graham
Mihai Tabara
Simon Fraser
Ben Hearsum
Martin Stransky
Henrik Skupin
Emilio Cobos Álvarez
Coroiu Cristina
VandanRogheliya
Rob Lemley
Marian Raiciof
Rail Aliiev
Julien Cristau
Gijs Kruitbosch
Ted Campbell
Csoregi Natalia
Nick Alexander
Bogdan Tara
Adam Gashlin
Joel Maher
Tooru Fujisawa