Marionette requirements fail to install with the new pip resolver due
to a conflict between `h2` and `wptserve`.
Use the legacy resolver with Marionette until the dependencies are
reorganized by the Marionette maintainers.
Depends on D99940
Differential Revision: https://phabricator.services.mozilla.com/D100811
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Lack of support for specialPowers is often cited as a reason for
developers to write mochitests rather than wpt. Although using
specialPowers means that the tests can't be upstreamed, having it
available in gecko-only tests should reduce the need to write
mochitests and potentially means that a patch which previously would
be written with just mochitests would instead be written with just
wpt, a fraction of which would be unsuitable for sharing.
Nevertheless a gecko-only specialPowers-using test should be
considered the option of last resort when cross-browser APIs such as
testdriver aren't sufficient.
This patch installs the specialPowers extension when the path to the
XPI is provided via a --specialpowers-path command line argument. To
ensure this only happens for gecko-only tests, the url_base is added
to the Test object, and the firefox settings() method is updated to
contain a specialpowers setting, which uses the url_base of the test
to determine if this is a gecko-only test.
Because the extension has to be enabled using marionette, we also need
to route this setting through the to executor, along with the provided
XPI path.
Differential Revision: https://phabricator.services.mozilla.com/D92035
Lack of support for specialPowers is often cited as a reason for
developers to write mochitests rather than wpt. Although using
specialPowers means that the tests can't be upstreamed, having it
available in gecko-only tests should reduce the need to write
mochitests and potentially means that a patch which previously would
be written with just mochitests would instead be written with just
wpt, a fraction of which would be unsuitable for sharing.
Nevertheless a gecko-only specialPowers-using test should be
considered the option of last resort when cross-browser APIs such as
testdriver aren't sufficient.
This patch installs the specialPowers extension when the path to the
XPI is provided via a --specialpowers-path command line argument. To
ensure this only happens for gecko-only tests, the url_base is added
to the Test object, and the firefox settings() method is updated to
contain a specialpowers setting, which uses the url_base of the test
to determine if this is a gecko-only test.
Because the extension has to be enabled using marionette, we also need
to route this setting through the to executor, along with the provided
XPI path.
Differential Revision: https://phabricator.services.mozilla.com/D92035
Lack of support for specialPowers is often cited as a reason for
developers to write mochitests rather than wpt. Although using
specialPowers means that the tests can't be upstreamed, having it
available in gecko-only tests should reduce the need to write
mochitests and potentially means that a patch which previously would
be written with just mochitests would instead be written with just
wpt, a fraction of which would be unsuitable for sharing.
Nevertheless a gecko-only specialPowers-using test should be
considered the option of last resort when cross-browser APIs such as
testdriver aren't sufficient.
This patch installs the specialPowers extension when the path to the
XPI is provided via a --specialpowers-path command line argument. To
ensure this only happens for gecko-only tests, the url_base is added
to the Test object, and the firefox settings() method is updated to
contain a specialpowers setting, which uses the url_base of the test
to determine if this is a gecko-only test.
Because the extension has to be enabled using marionette, we also need
to route this setting through the to executor, along with the provided
XPI path.
Differential Revision: https://phabricator.services.mozilla.com/D92035
Changes:
- add new flag in `web_platform_tests.py`, with name `backlog` that is a boolean value.
- simplify the timeout multiplier conditional to check for presence of the `backlog` flag.
- add the new flag to the `mozharness_extra_options` flag in `web-platform.yml`
Differential Revision: https://phabricator.services.mozilla.com/D81184
Changes:
Previous patch to apply the reduced timeout multiplier was only applicable to vanilla `web-platform-tests`. This patch reduces the chunk count for `web-platform-tests-reftest` and applies the reduction multiplier.
Differential Revision: https://phabricator.services.mozilla.com/D80520
Changes:
If the web-platform-tests runtype is `backlog` (as noted by the lack of `skip-implementation: backlog`), set a timeout multiplier of 0.25.
Differential Revision: https://phabricator.services.mozilla.com/D79356
Changes:
If the web-platform-tests runtype is `backlog` (as noted by the lack of `skip-implementation: backlog`), set a timeout multiplier of 0.25.
Differential Revision: https://phabricator.services.mozilla.com/D79356
Changes:
- if MOZHARNESS_TEST_PATHS is defined, write the contents of that directory to a JSON file.
- append `--run-by-dir` only if `--test-groups` flag is not set.
Differential Revision: https://phabricator.services.mozilla.com/D78820
Changes:
- add new environment variable `TESTS_BY_MANIFEST_URL` referencing the tests-by-manfiest.json.gz artifact generated by decision task.
- add new method `download_manifest` in mozharness/web_platform_tests.py to download the artifact, then remove non-web-platform-test manifests.
- the processed artifact is then saved as JSON file containing only web-platform-tests for downstream harnesses to use.
Differential Revision: https://phabricator.services.mozilla.com/D78318
Changes:
- if MOZHARNESS_TEST_PATHS is defined, write the contents of that directory to a JSON file.
- append `--run-by-dir` only if `--test-groups` flag is not set.
Differential Revision: https://phabricator.services.mozilla.com/D78820
Changes:
- add new environment variable `TESTS_BY_MANIFEST_URL` referencing the tests-by-manfiest.json.gz artifact generated by decision task.
- add new method `download_manifest` in mozharness/web_platform_tests.py to download the artifact, then remove non-web-platform-test manifests.
- the processed artifact is then saved as JSON file containing only web-platform-tests for downstream harnesses to use.
Differential Revision: https://phabricator.services.mozilla.com/D78318
In all mozharness scripts running test-verify (or per-test coverage), use distinct
file names for errorsummary and raw logs. This usually means that the command needs
to be built inside the per-test loop.
Differential Revision: https://phabricator.services.mozilla.com/D77746
This is very similar to wcosta's earlier patch, with a few adjustments to overcome
special cases:
- remove the wrench override of adb path; handle in config instead
- fix taskcluster config syntax in a couple of places
- add android-sdk-linux fetch for test-verify
Differential Revision: https://phabricator.services.mozilla.com/D73157
This runs wpt with --processes=2 in CI, excluding wdspec tests where this
causes obvious test failures and android where we can't have more than a
single instance. This should provide some speedup at the cost of possibly
increased instability. So following landing this it will be necessary to
look out for an increased rate of intermittents.
Differential Revision: https://phabricator.services.mozilla.com/D68885
--HG--
extra : moz-landing-system : lando
The definition for reftest/crashtest variants of web-platform-tests in web-platform.yml has a trailing `s` which is technically incorrect.
This causes problems when querying ActiveData for the runtimes for these subsuites since the suite name recorded in ActiveData has the trailing s.
Changes:
- remove the trailing `s` from the definitions
Differential Revision: https://phabricator.services.mozilla.com/D68069
--HG--
extra : moz-landing-system : lando
This runs wpt with --processes=2 in CI, excluding wdspec tests where this
causes obvious test failures and android where we can't have more than a
single instance. This should provide some speedup at the cost of possibly
increased instability. So following landing this it will be necessary to
look out for an increased rate of intermittents.
Differential Revision: https://phabricator.services.mozilla.com/D68885
--HG--
extra : moz-landing-system : lando
This will allow us to generate flamegraph-style output to see where wpt jobs
spend time
Differential Revision: https://phabricator.services.mozilla.com/D63066
--HG--
extra : moz-landing-system : lando
This minor optimization is easily implemented: If there are no tests to verify, call fatal(0).
However, I encountered a minor obstacle: If the task exits before creating the upload directory,
the task fails, regardless of exit value; the remaining changes overcome this by creating the
upload directory earlier.
Differential Revision: https://phabricator.services.mozilla.com/D51684
--HG--
extra : moz-landing-system : lando
Mitchell Hentges
Bob Clary
Ricky Stewart
Bogdan Tara
Joel Maher
Dorel Luca
James Graham
Noemi Erli
Tom Prince
Henrik Skupin
Narcis Beleuzu
Edwin Takahashi
Razvan Maries
Geoff Brown
Stephen Donner
Mihai Alexandru Michis
Daniel Varga