When MOZHARNESS_TEST_PATHS is set, the test suite mozharness scripts
will run the paths specified there instead of the normal chunking
and/or default manifest. Paths should be separated by a ':' character.
In the case of web_platform_tests.py, we have to make the test paths
relative to 'testing/web-platform'.
MozReview-Commit-ID: IHRXXi5mB4G
--HG--
extra : rebase_source : 17b31ec19a64ab16918d0bd80d19d9bb496cbe37
This uploads a JSON summary of the results, without extra logging or
expectation data or anything. It is mostly useful for comparing the
results from two runs e.g. in a dashboard.
MozReview-Commit-ID: Ac45NVBxhy8
--HG--
extra : rebase_source : 2c7a82d6e46b8c9f7af8ad34559e51d84ba4e4ba
Ahem.ttf is copied to $(DIST)/bin/firefox/fonts, but this file is broken due to text mode copy. So we should use binary mode instead.
MozReview-Commit-ID: KP7yNyPiejU
--HG--
extra : rebase_source : 2de749f458a6d4650f9044f1912ff97835c5b795
Currently these are enabled for Linux and Windows, using the corresponding WPT
symbols with an "H" suffix to match the Mn->MnH convention followed by the
Marionette tests.
For Linux, the tests are set to run with a simulated 1600x1200 screen
resolution, which matches the Docker configuration. The default 1366x768 for
headless mode results in some unexpected failures due to windows not being
able to reach the full sizes the web platform tests expect.
For Windows, the simulated screen resolution is set to 1024x768, which is what
I've determined to be the resolution that Windows VMs run under within our
testing infrastructure. Setting a different screen resolution causes some web
platform tests to unexpectedly pass (in particular, several in
open-features-non-integer-height.html).
A "headless" variable is added to the environment that the WPT metadata
(.ini) files can access, indicating whether or not the test is running in
headless mode. This is used to mark the reftest bdi-neutral-wrapped.html as an
expected pass on Linux under headless mode: it normally fails (spuriously) due
to differences in scrollbar sizing, but scrollbars aren't rendered in headless
mode.
MozReview-Commit-ID: 3usazOoxx3q
--HG--
extra : rebase_source : c036070920391c30b13813e2d6282bd407a96715
This allows syntax like --try-test-paths web-platform-tests-reftests:path/to/test
MozReview-Commit-ID: uAet1ilPVy
--HG--
extra : rebase_source : 447277e47701435186ad87dfc089bd21f2bd1907
Flip around the logic of bug 1356122, so that the default from Stylo runs is the
parallel traversal, but we can opt in to single traversal as desired.
This ensures that for testing on other desktop platforms, we use parallel
traversal as the default.
MozReview-Commit-ID: KoBe1ltHP52
--HG--
extra : rebase_source : ad96f24c9760544c5812060f26e9ca18e5bf2ba8
`try-test-paths` is set up to map anything under testing/web-platform
to the web-platform-tests flavour. By default, the web-platform-tests flavour
refers to the testharness test type for wptrunner, so we need to account for
reftest and wdspec test types.
This change causes mozharness to omit the test-type argument to wptrunner when
try-test-paths is being used, therefore making wptrunner determine the
appropriate test type for each requested test.
MozReview-Commit-ID: 7TDAShdDM4g
--HG--
extra : rebase_source : fde6ec219f574cd1e536764c0128a6816834f533
Allow running |mach wpt| on one click loaners in order to run
web-platform-tests tests.
This implementation is just like the one for other testsuites using
thee packaged tests rather than the checkout that we get with wpt, at
least on Linux. That's also where the tests run from so it seems
reasonable for now. Moving to the checkout in the future could remove
some of the logic here by using a fake mozbuild environment so that
the testsuite itself doesn't have to implement anything much.
MozReview-Commit-ID: CaewrdjJ2ef
--HG--
extra : rebase_source : 491b8014d48f06ff5bd41b28cc985608981fbdf4
Instead of fetching geckodriver from tooltool, we ask mozharness to
pick up the geckodriver binary from the common test archive. As it
is packaged under /bin along with other test-relevant binaries such as
wptserve, we can retrieve it from the abs_test_install_dir.
It would also be possible to use ScriptMixin.query_exe for this purpose
after specifying the binary in the "exes" section of the different
mozharness configs, but this seems needlessly complicated.
Because we also do not yet have geckodriver on all platforms, we only
want to look for it if requested to run the wdspec test type.
MozReview-Commit-ID: 7jLuBeDiQNE
--HG--
extra : rebase_source : ea8c1be063be0f40ff4c8f8c3a77b1b57580829d
This allows running tests with WebRender enabled, using builds that have WebRender
built-in, but not enabled by default.
MozReview-Commit-ID: HkFgB09J7gT
We're starting to execute mozharness scripts from source checkouts
in automation. This means we can stop downloading and extracting
files that are already available in the source checkout and just
reference files from the checkout.
Web platform tests (WPT) are a logical place to start because they
are pretty well isolated. This commit modifies the mozharness script
for WPT execution to use files from a source checkout (previous
commits have guaranteed that a pristine source checkout is available
to the test execution environment).
As part of this, we also need to define an explicit mozinfo.json
path because previously it was relying on parent directory traversal
to find it (WPT tests were under a directory containing the
mozinfo.json file).
MozReview-Commit-ID: C1dlKC4eSzr
--HG--
extra : rebase_source : af6f5ad0f88739efc5f5e0c74106a50e845564f6
4 WPT config files all contained the same config options boilerplate.
Inline it into the WPT mozharness script.
MozReview-Commit-ID: 5Lba8QeKMTA
--HG--
extra : rebase_source : a99f8e25d04a3e3344db1cf280fd79c47177ccb4
list.extend() is favored over +=. Also use single quotes, fixup
indentation, and factor out a common variable.
MozReview-Commit-ID: 3qVDGrkYhVe
--HG--
extra : rebase_source : 7573dca951eca0611c86cbb42e3cf53ccae078f4
We add a mozharness action to the TestingMixin base class that ensures
we're running from a VCS checkout and we add this action to the WPT
script.
This ensures that we always have a source checkout available, even
in buildbot. (Before, we only had a source checkout in TaskCluster.)
MozReview-Commit-ID: 26NxwDZywXr
--HG--
extra : rebase_source : 6aea0390b0c9ff43972ef6fe6592f104401cc3fc
Downloads geckodriver from tooltool when wdspec tests are being run,
and adds the --webdriver-binary argument
MozReview-Commit-ID: AJeP0YDk7Yl
--HG--
extra : rebase_source : 497f25c5af32b1851adf3a6f0b90a20640b6ccc6
We're starting to execute mozharness scripts from source checkouts
in automation. This means we can stop downloading and extracting
files that are already available in the source checkout and just
reference files from the checkout.
Web platform tests (WPT) are a logical place to start because they
are pretty well isolated. This commit modifies the mozharness script
for WPT execution to use files from a source checkout (previous
commits have guaranteed that a pristine source checkout is available
to the test execution environment).
As part of this, we also need to define an explicit mozinfo.json
path because previously it was relying on parent directory traversal
to find it (WPT tests were under a directory containing the
mozinfo.json file).
MozReview-Commit-ID: C1dlKC4eSzr
--HG--
extra : rebase_source : fad7905abafaf126329aa25a96efafc40b01051f
4 WPT config files all contained the same config options boilerplate.
Inline it into the WPT mozharness script.
MozReview-Commit-ID: 5Lba8QeKMTA
--HG--
extra : rebase_source : c613db4107128ad2fd4ba568836ec27e0a9a92a8
list.extend() is favored over +=. Also use single quotes, fixup
indentation, and factor out a common variable.
MozReview-Commit-ID: 3qVDGrkYhVe
--HG--
extra : rebase_source : 0c5faa30f5be09165f61943300ecfb2e9c7080f4
We add a mozharness action to the TestingMixin base class that ensures
we're running from a VCS checkout and we add this action to the WPT
script.
This ensures that we always have a source checkout available, even
in buildbot. (Before, we only had a source checkout in TaskCluster.)
MozReview-Commit-ID: 26NxwDZywXr
--HG--
extra : rebase_source : d9c0fade450ab14c0b52be674c3c92bf670d2d3b
Given that we have a universal unpack method now do not keep 'unzip' in method names.
Also adapt arguments to be better understandable.
MozReview-Commit-ID: ClDB5mSVcI2
--HG--
extra : rebase_source : a98bb26748536115d254842df8257ba050ec8eac
Henrik Skupin
Marco Castelluccio
Andrew Halberstadt
James Graham
Geoff Brown
Steve Armand
Sebastian Hengst
J. Ryan Stinnett
Kartikaya Gupta
Makoto Kato
Michael Smith
Maja Frydrychowicz
Ryan VanderMeulen
Wes Kocher
Shing Lyu
Andreas Tolfsen
Madeleine Chercover
Gregory Szorc
Phil Ringnalda
Andrew Comminos