In mochitest, we use a test's manifest to:
1. Log the test groups at suite_start
2. Ensure all tests in a manifest have the same prefs/env
3. Implement run-by-manifest mode
However, we don't factor in a test's ancestor manifest. This can cause problems
if the same manifest is imported more than once by different parent manifests.
This patch ensures that if a test has an ancestor manifest, we use the key
'<ancestor_manifest>:<manifest>' for the purposes listed above.
Differential Revision: https://phabricator.services.mozilla.com/D57828
--HG--
extra : moz-landing-system : lando
Instead of setting MOZ_QUIET to hide the DOMWINDOW and DOCSHELL log messages, you
now must set a regular logging module to enable them. They are automatically enabled
on tests that rely on these messages are leak checking.
That module is DocShellAndDOMWindowLeak:3
One disadvantage of this change is that you cannot set MOZ_QUIET to hide these
messages when running those tests (primarily browser-chrome).
Differential Revision: https://phabricator.services.mozilla.com/D52413
--HG--
extra : moz-landing-system : lando
As a side-effect this will also update runtime data for all suites using
'--chunk-by-runtime'.
This change simultaneously:
1. Stores runtime data from all suites
2. Stores runtime data from all tests (no more percentile)
3. Stores distinct data for android, unix (osx/linux) and windows
4. Reduces the size of 'testing/runtimes' from 408k -> 168k
The chunks look more balanced from my unscientific glance (especially on Windows).
Differential Revision: https://phabricator.services.mozilla.com/D53702
--HG--
extra : moz-landing-system : lando
As a side-effect this will also update runtime data for all suites using
'--chunk-by-runtime'.
This change simultaneously:
1. Stores runtime data from all suites
2. Stores runtime data from all tests (no more percentile)
3. Stores distinct data for android, unix (osx/linux) and windows
4. Reduces the size of 'testing/runtimes' from 408k -> 168k
The chunks look more balanced from my unscientific glance (especially on Windows).
Differential Revision: https://phabricator.services.mozilla.com/D53702
--HG--
extra : moz-landing-system : lando
This leaves the testharness files, because they are used in various mochitests.
Differential Revision: https://phabricator.services.mozilla.com/D49132
--HG--
extra : moz-landing-system : lando
This leaves the testharness files, because they are used in mochitest-chrome
tests in dom/animation/test.
Differential Revision: https://phabricator.services.mozilla.com/D49132
--HG--
extra : moz-landing-system : lando
This leaves the testharness files, because they are used in mochitest-chrome
tests in dom/animation/test.
Differential Revision: https://phabricator.services.mozilla.com/D49132
--HG--
extra : moz-landing-system : lando
Due to limitations in the test harness, the "serviceworker_e10s" (for mochitest
and xpcshell) and "sw-e10s" (for WPT) values will match mozinfo's "nightly_build"
value, unless overridden by "--setpref dom.serviceWorkers.parent_intercept=..."
provided at the CLI.
Differential Revision: https://phabricator.services.mozilla.com/D43170
--HG--
extra : moz-landing-system : lando
Due to limitations in the test harness, the "serviceworker_e10s" (for mochitest
and xpcshell) and "sw-e10s" (for WPT) values will match mozinfo's "nightly_build"
value, unless overridden by "--setpref dom.serviceWorkers.parent_intercept=..."
provided at the CLI.
Differential Revision: https://phabricator.services.mozilla.com/D43170
--HG--
extra : moz-landing-system : lando
Test manifest annotations like "skip-if = verify" are normally handled in mozharness:
When a TV task finds a modified test that needs verification, but that test is skipped
in verify mode, mozharness discards the test from the list of tests to be run.
When running tests locally (mach mochitest, etc), these annotations should also be
respected, and are currently handled in wpt and reftest harnesses, but not in
mochitest or xpcshell -- rectified by this patch. mochitest and xpcshell had
neglected to set mozinfo["verify"] when running in verify mode -- easily corrected.
It should be noted that when running tests locally and a single test is requested,
most test harnesses run the requested test even if it is skip-annotated. Thus,
"mach test <test> --verify" will continue to run tests annotated "skip-if = verify"
and this patch only changes the mochitest/xpcshell behavior of verify-skipped
tests when run with "mach test <directory> --verify" -- a long-running test mode
with complex logging never used in continuous integration.
Differential Revision: https://phabricator.services.mozilla.com/D40486
--HG--
extra : moz-landing-system : lando
Test manifest annotations like "skip-if = verify" are normally handled in mozharness:
When a TV task finds a modified test that needs verification, but that test is skipped
in verify mode, mozharness discards the test from the list of tests to be run.
When running tests locally (mach mochitest, etc), these annotations should also be
respected, and are currently handled in wpt and reftest harnesses, but not in
mochitest or xpcshell -- rectified by this patch. mochitest and xpcshell had
neglected to set mozinfo["verify"] when running in verify mode -- easily corrected.
It should be noted that when running tests locally and a single test is requested,
most test harnesses run the requested test even if it is skip-annotated. Thus,
"mach test <test> --verify" will continue to run tests annotated "skip-if = verify"
and this patch only changes the mochitest/xpcshell behavior of verify-skipped
tests when run with "mach test <directory> --verify" -- a long-running test mode
with complex logging never used in continuous integration.
Differential Revision: https://phabricator.services.mozilla.com/D40486
--HG--
extra : moz-landing-system : lando
I don't know if this covers all the things that use mozinfo (probably not)
but it covers all the suites that use mozinfo and have webrender conditions
in the test manifests (i.e. mochitest and wpt variants).
Differential Revision: https://phabricator.services.mozilla.com/D35869
--HG--
extra : moz-landing-system : lando
Improve support for Python 3 by running modernize over adb.py and a few other related changes.
Differential Revision: https://phabricator.services.mozilla.com/D19306
--HG--
extra : moz-landing-system : lando
Previously we would silently change the value of "e10s" from False to True.
This can cause confusion and lead people to falsely think mochitest-chrome/a11y
work with e10s (they do not).
Now we explicitly error out in this case. This might be slightly less
convenient for the developer (e.g they might need to re-run the command), but
the downside of needing to rerun a test command is less than the risk of
misunderstanding what is being tested.
Note: when running |mach test| or |mach mochitest| on a directory that contains
both chrome/a11y and another suite, we'll still do the right thing and
implicitly set "e10s=False".
Differential Revision: https://phabricator.services.mozilla.com/D28538
--HG--
extra : moz-landing-system : lando
Set the `crashreporter` value in mozinfo to `False`.
This requires the setting of a new `options.crashreporter` value from `mochitest_options.py`, when it detects windows10-aarch64 configuration.
Differential Revision: https://phabricator.services.mozilla.com/D25503
--HG--
extra : moz-landing-system : lando
Not specifying an explicit host causes pywebsocket to listen on the
default address, which may be 0.0.0.0. This triggers the firewall
on macOS, and causes the following prompt to be shown when mochitests
are run:
> Do you want the application "Python.app"
> to accept incoming network connections?
The dialog is a nuisance because it is always on top. Since denying
the access does not change the outcome of tests, it should be safe
to only listen on localhost.
Differential Revision: https://phabricator.services.mozilla.com/D25364
--HG--
extra : moz-landing-system : lando
Unifies the mochitest MOZ_LOG file naming and archiving across desktop/android.
Notably the inclusion of pid and uuid for android log files helps generate
appropriate files for e10s on geckoview.
Differential Revision: https://phabricator.services.mozilla.com/D24262
--HG--
extra : moz-landing-system : lando
Perry Jiang
Andrew Halberstadt
Tom Ritter
Cosmin Sabou
Geoff Lankow
Christian Holler
Ms2ger
shindli
Daniel Varga
Nika Layzell
Gijs Kruitbosch
Gurzau Raul
Edwin Takahashi
Geoff Brown
Csoregi Natalia
Julian Descottes
Kris Maglione
Kartikaya Gupta
Gabriele Svelto
Marian Raiciof
Kevin Jacobs
Bob Clary
Rob Wu
Florian Quèze
Byron Campen [:bwc]
Brindusan Cristian