Changes:
- require `--full` keyword for `./mach try fuzzy` in order to schedule android-hw jobs to hopefully reduce the backlog
Differential Revision: https://phabricator.services.mozilla.com/D33834
--HG--
extra : moz-landing-system : lando
- sm-shell: Selects shell only test cases that shouldn't require a full browser build.
- sm-all: Selects test cases that may require a full browser build.
Differential Revision: https://phabricator.services.mozilla.com/D28994
--HG--
extra : moz-landing-system : lando
Bug 1484243 removed the --git argument and made the VCS auto-detected,
so we should no longer suggest passing in that argument in error
messages.
Differential Revision: https://phabricator.services.mozilla.com/D30257
--HG--
extra : moz-landing-system : lando
Since we need to generate the full_task_set as a prereq to the target_task_set,
and since getting from full_task_set -> target_task_set is trivial.. we might
as well save both computed sets to the cache while we have them. This means users
that run:
$ ./mach try fuzzy
and then run:
$ ./mach try fuzzy --full
(or vice versa)
Will only incur task generation once. It also means that the 'watchman' trigger
will cache both taskgraphs.
Differential Revision: https://phabricator.services.mozilla.com/D29557
--HG--
extra : moz-landing-system : lando
This adds a 'watchman.json' file to /tools/tryselect and some documentation on
how to use it. Tl;dr, install watchman and then:
$ cd path/to/gecko
$ watchman -j < tools/tryselect/watchman.json
Differential Revision: https://phabricator.services.mozilla.com/D28771
--HG--
extra : moz-landing-system : lando
It turns out there are several places where the change to suite 'jittest-chunked'
causes problem. I am abandoning that approach.
Desktop uses this trick, and this returns android '-chunked' handling to a state
similar to what it was before I started messing around!
Differential Revision: https://phabricator.services.mozilla.com/D28897
--HG--
extra : moz-landing-system : lando
This skips a number of talos jobs that are unlikely to be affected
by due a browser-chrome specific change
Run with: `./mach try fuzzy --preset perf-chrome`
Differential Revision: https://phabricator.services.mozilla.com/D27911
--HG--
extra : moz-landing-system : lando
This officially makes 'moztest.resolve' the source of truth when it comes to
suite names. It aligns that file with the names used in both the
desktop_unittest and android_emulator_unittest scripts.
Differential Revision: https://phabricator.services.mozilla.com/D27555
--HG--
extra : moz-landing-system : lando
Currently we have the concept of a "suite" and a "flavour" in our task
configuration. Typically, the "suite" refers to the high-level test harness
like "mochitest" or "reftest", whereas the flavour is more specific, e.g
"browser-chrome-instrumentation" or "crashtest". However the line between suite
and flavour is not applied with any semblance of consistency which results in
inconsistent naming throughout the tree.
This patch gets rid of the concept of "flavours" entirely (at least when it
comes to task configuration). A suite is a type of test run, for example:
- mochitest-plain
- mochitest-devtools-chrome
- mochitest-browser-chrome-instrumentation
- jsreftest
- reftest
- firefox-ui-functional-remote
etc
There is no confusion here between suites and flavours because flavours don't
exist. However, there are a couple of places where we *do* need to know what
"test harness" is used to run a suite. These cases are:
1. For SCHEDULES moz.build rules
2. For the desktop_unittest.py mozharness script which takes arguments like
--mochitest-suite=browser (this is not a compelling use of this information
and should be refactored to work more like the android_emulator_unittest.py
script)
So to get this information, this patch introduces a new concept of a "category"
which is the overall "test harness" that runs the suite. For many suites, the
"category" is identical to the suite name. Unlike flavours, "categories" have
no bearing on how we call or refer to the suite.
Differential Revision: https://phabricator.services.mozilla.com/D27554
--HG--
extra : moz-landing-system : lando
This was regressed by bug 1544816 but the test never ran on the push that regressed.
This patch also updates the 'files-changed' for the tryselect task.
Differential Revision: https://phabricator.services.mozilla.com/D28386
--HG--
extra : moz-landing-system : lando
Changes:
- added windows10-aarch64 to the filter for fuzzy, to require `--full` in order to trigger jobs
- return False for any test tasks that contain windows10-aarch64 to prevent users using old try syntax from overwhelming the limited number of hardware
Differential Revision: https://phabricator.services.mozilla.com/D27590
--HG--
extra : moz-landing-system : lando
Since e10s is the default configuration, we shouldn't explicitly mark things
with the "-e10s" suffix. Instead we should mark things that *don't* run with
'e10s. This patch removes '-e10s' from all treeherder group symbols and task
labels, adds the "-1proc" suffix to tasks that are non-e10s.
Differential Revision: https://phabricator.services.mozilla.com/D25958
--HG--
extra : moz-landing-system : lando
This helps run the tests locally if fzf is normally installed in
$HOME/.mozbuild. Since the tests set MOZBUILD_STATE_PATH to a temporary
directory, fzf_bootstrap() can't find fzf in the HOME location unless it
is added to PATH.
Differential Revision: https://phabricator.services.mozilla.com/D27194
--HG--
extra : moz-landing-system : lando
Add Android 7.0 gtests, opt and debug, running against the geckoview
TestRunnerActivity.
Differential Revision: https://phabricator.services.mozilla.com/D27016
--HG--
extra : moz-landing-system : lando
There are a few windows/mac-only suites that were missed since we were only
reading the linux variant.
Depends on D25401
Differential Revision: https://phabricator.services.mozilla.com/D25432
--HG--
extra : moz-landing-system : lando
While running presets + intersection queries works with 'mach try fuzzy
--preset <foo>', it is still broken with 'mach try --preset <foo>'.
Differential Revision: https://phabricator.services.mozilla.com/D25298
--HG--
extra : moz-landing-system : lando
Some of these were working with the '<flavor>-<subsuite>' mechanism that was
previously being used, but better to be explicit wherever possible.
Depends on D25077
Differential Revision: https://phabricator.services.mozilla.com/D25078
--HG--
extra : moz-landing-system : lando
Mozharness appends -chunked/-coverage to some suites, but the build system/test
resolver don't have any concept of these things. We need to normalize these out
for the purposes of MOZHARNESS_TEST_PATHS.
Differential Revision: https://phabricator.services.mozilla.com/D25015
--HG--
extra : moz-landing-system : lando
It turns out bug 1489100 regressed the ability to specify test paths for most
suites by naively assuming that mozharness uses suite names that look like:
<flavor>-<subsuite>
In reality, there is no consistency to how suite names are generated. This test
does a few things:
1) Patches the moztest.TestResolver to return a list of all possible
suites/subsuites (assuming the lists in moztest.resolve are up to date).
2) Finds all the suites defined in the mozharness configs (e.g
linux_unittest.py), and uses these are parametrized inputs.
3) Checks that for each test suite,
DesktopUnittest._get_mozharness_test_paths() returns something.
I've marked all of the test suites that currently fail as expected. This way I
have a good sense of what needs to be fixed, and can validate my changes as I
move through the list.
Differential Revision: https://phabricator.services.mozilla.com/D25014
--HG--
extra : moz-landing-system : lando
The graph contains some extra things like toolchains, fetches and packaging
tasks that people will almost never want to run on their own. This change gets
them out of the default fuzzy selection interface, and makes it so --full is
needed to schedule them.
Differential Revision: https://phabricator.services.mozilla.com/D24187
--HG--
extra : moz-landing-system : lando
Not only is it best practice to pin dependencies and check hashes, but this
will allow tests to install these dependencies as well.
Depends on D22784
Differential Revision: https://phabricator.services.mozilla.com/D22785
--HG--
extra : moz-landing-system : lando
This forces us to be more strict with what gets passed into the try selector.
This change means test_preset.py will error if someone makes a typo in an
in-tree preset.
Differential Revision: https://phabricator.services.mozilla.com/D22024
--HG--
extra : moz-landing-system : lando
Edwin Gao
Brian Grinstead
Tom Prince
Matthew Gaudet
Julian Descottes
Mike Shal
Matthew Noorenberghe
Andrew Halberstadt
Joel Maher
Geoff Brown
neha
Justin Wood
Cosmin Sabou
Dustin J. Mitchell
Jason Laster