Test manifests may be included by multiple other manifests, optionally
with additional variables below the `[include:...]` section header.
These additional variables are specific to the manifest that contained
the "include" section, and should not inadvertently be shared with other
manifests that also happen to include this manifest.
To achieve that, store the defaults for included manifests in a (path to
parent manifest, path to included manifest) tuple instead of just the
included manifest.
Differential Revision: https://phabricator.services.mozilla.com/D18086
--HG--
extra : moz-landing-system : lando
Test manifests may be included by multiple other manifests, optionally
with additional variables below the `[include:...]` section header.
These additional variables are specific to the manifest that contained
the "include" section, and should not inadvertently be shared with other
manifests that also happen to include this manifest.
To achieve that, store the defaults for included manifests in a (path to
parent manifest, path to included manifest) tuple instead of just the
included manifest.
Differential Revision: https://phabricator.services.mozilla.com/D18086
--HG--
extra : moz-landing-system : lando
The following python-test paths are being moved out of 'make check' and into their own task:
- python/mozlint
- testing/mozbase
- tools/lint
The following python-test paths previously did not run on Windows:
- python/mozterm
- testing/marionette
- testing/raptor
- tools/tryselect
Differential Revision: https://phabricator.services.mozilla.com/D10759
--HG--
extra : moz-landing-system : lando
The following python-test paths are being moved out of 'make check' and into their own task:
- python/mozlint
- testing/mozbase
- tools/lint
The following python-test paths previously did not run on Windows:
- python/mozterm
- testing/marionette
- testing/raptor
- tools/tryselect
MozReview-Commit-ID: C07FANaYzf7
Depends on D10758
Differential Revision: https://phabricator.services.mozilla.com/D10759
--HG--
extra : moz-landing-system : lando
The previous code split a lot of logic between the update and download
parts, and exposed two different mach commands. In order to simplify
things it's better to have a single entry point for manifest download
and update, and ensure that's called consistently.
Differential Revision: https://phabricator.services.mozilla.com/D7497
These regexes are used for things like determining which tasks to run given a
"path" int |mach try|. Previously, we used patterns like:
mochitest-chrome-(?:e10s)?(?:-1)?$
This would match both e10s and non-e10s versions of a task with either no
chunks, or only selecting chunk 1. But we keep adding other configurations, e.g
-gpu, -no-accel, -sw, etc. Each time we create a new possibility we need to
remember to update these task regexes (or else lose test coverage when using
paths with |mach try|).
Instead of individually listing every possibility, let's use a pattern like
this:
mochitest-chrome($|.*(-1|[^0-9])$)
This also selects tasks that are either chunk 1 or don't have any chunks. But
it allows for arbitrary strings in-between. This regex doesn't need to be
updated when we add configurations like -sw.
Depends on D7119
Differential Revision: https://phabricator.services.mozilla.com/D7120
--HG--
extra : moz-landing-system : lando
I almost forgot to update the regexes in moztest.resolve when creating the -sw
variant of task. This adds a test to make sure we don't forget more things in
the future.
Differential Revision: https://phabricator.services.mozilla.com/D7119
--HG--
extra : moz-landing-system : lando
mach try fuzzy has some stronger requirements compared to mach test.
Depends on D6673
Differential Revision: https://phabricator.services.mozilla.com/D6674
--HG--
extra : moz-landing-system : lando
Changes the wpt manifest path to the topobjdir instead so it can be moved out of tree.
Other changes so that the manifest download and update, and |mach wpt| and |mach test <wpt-test>| work with the new path.
The manifest is also downloaded and updated when creating the tests-archive to ensure that it exists when we run tests on TC.
MozReview-Commit-ID: Fp6UsKJjhTU
Differential Revision: https://phabricator.services.mozilla.com/D5312
--HG--
extra : moz-landing-system : lando
Changes the wpt manifest path to the topobjdir instead so it can be moved out of tree.
Other changes so that the manifest download and update, and |mach wpt| and |mach test <wpt-test>| work with the new path.
The manifest is also downloaded and updated when creating the tests-archive to ensure that it exists when we run tests on TC.
MozReview-Commit-ID: Fp6UsKJjhTU
Differential Revision: https://phabricator.services.mozilla.com/D5312
--HG--
extra : moz-landing-system : lando
It's unclear how this situation can actually happen, but it has caused a couple of
intermittents.
Differential Revision: https://phabricator.services.mozilla.com/D2355
--HG--
extra : moz-landing-system : lando
Also include webgl2-deqp, which we would like to run eventually, but not yet.
MozReview-Commit-ID: CY4hYCI95ws
--HG--
extra : rebase_source : 9973df0f905bb65d2e8b8c66a6a57e8869e527c1
Also include webgl2-deqp, which we would like to run eventually, but not yet.
MozReview-Commit-ID: FDWdu1J0end
--HG--
extra : rebase_source : a47d88cb2c5eb82e4dfaa9e58d76acbf0736d35d
Instead of downloading the build artifacts (rather hackily) in moztest.fixtures, this now happens
directly in the taskgraph module via the run-task script.
For now extraction and setup happens in the task's command key. It might be a good idea to figure
out a syntax to tell run-task to do this extraction, e.g something like:
run:
using-artifacts:
build:
target.tar.bz2:
extract: true
path: /home/worker/build
name: firefox
But for now I wanted to avoid this extra complexity, so maybe it could be done in a follow-up.
MozReview-Commit-ID: KOhFFpFdP7Y
--HG--
extra : rebase_source : dcea36661fa9c6442c76c850ccc67f8f6d924fda
This isn't strictly related to this bug, but it is a change made to mozbase in
the raptor repo that is worth backporting here. Figured it's easiest to land it
alongside the other mozbase backports.
MozReview-Commit-ID: DW7I2zKZZNk
--HG--
extra : rebase_source : a000d27774f224d37f981d6683d96c65846b8a32
There were two issues:
1) The mach command name in resolve.py was wrong.
2) The marionette harness uses deepcopy on the passed in kwargs and sometimes
the 'log' argument that testing/mach_commands.py was passing in can be a class
instance (which can't be deepcopied).
MozReview-Commit-ID: 5gPxuiHs3dY
--HG--
extra : rebase_source : 63bc9c84fdcb540862f1dcbc2654bf5729e0dec8
Since we're adding specific 'task_regexes' for each new suite definition,
this will allow us to schedule tests of these subsuites with
|mach try fuzzy <path>|.
MozReview-Commit-ID: 2mDSneV95lG
--HG--
extra : rebase_source : 467b9d885e92c1c855ed547f2a7496b1062f2dc2
The end goal here is to be able to use |mach try fuzzy <path>| with tests that
belong to a subsuite. To do this, we need a unique 'task_regex' value for each
subsuite so that we can map a test path back to a set of tasks.
This removes the TEST_FLAVORS dict (which was mostly just a redefinition of the
data in TEST_SUITES), and instead provides two new private mappings:
<flavor> -> suite definition
(<flavor>, <subsuite>) -> suite definition
To retrieve a suite definition given a flavor/subsuite, consumers can now call
get_suite_definition.
MozReview-Commit-ID: 2pe1v1IHUVy
--HG--
extra : rebase_source : 6fff947ba214112ccf16c894174a6a0e2487111a
This removes a lot of redundant alias definitions by calling lower() on the
user input. It also adds a couple of new aliases that look like they might
be useful.
MozReview-Commit-ID: 3Aix4LPB8wg
--HG--
extra : rebase_source : c4bdc327bd737a18f03952bb360af35608d091f1
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
This enables the syntax like:
./mach try fuzzy dom/indexedDB
This will open up the fzf interface like normal, except only tasks
that have tests under dom/indexedDB will be selectable (and there
will only be one chunk per configuration).
This can be combined with -q/--query like normal:
./mach try fuzzy dom/indexedDB -q "!pgo !cov !asan"
When the tasks get scheduled, only the tests under the specified
path(s) will run within the harness.
MozReview-Commit-ID: IHRXXi5mB4G
--HG--
extra : rebase_source : 8a89f255591e6dfa31b1420196c4698f2015d10c
This removes the subcommands for "./mach geckodriver", reverting
it back to have the meaning of running the geckodriver binary.
The build- and test commands are now integrated with mach, which
means you can run "./mach build testing/geckodriver" and "./mach
test testing/geckodriver" to run tests. This is backed by a new
top-level "./mach geckodriver-test" command, which we will not be
announcing.
MozReview-Commit-ID: CiQsfNqrvIp
--HG--
extra : rebase_source : 6c492b7e1128e4858e42ae4bb35ab4b29564dbeb
Rob Wu
Csoregi Natalia
Narcis Beleuzu
Wes Kocher
Andrew Halberstadt
Marco Castelluccio
Ciure Andrei
Andreea Pavel
James Graham
Ahilya Sinha
shindli
Chris AtLee
Bogdan Tara
Jeff Gilbert
Gurzau Raul
Dave Hunt
Joel Maher
Geoff Brown
Sylvestre Ledru
Noemi Erli
Andreas Tolfsen
Tom Prince