Gecko was trying to pass formatter options in by default, which
doesn't work if the user overrides the formatter. Instead pass in the
default options explicitly using the designed mechanism, which
previously wasn't exposed in wpt.
Differential Revision: https://phabricator.services.mozilla.com/D47117
--HG--
extra : moz-landing-system : lando
wptrunner.start() negates the return value of wptrunner.run() so
that false is used to indicate that the test run was successful.
This is then (wrongly) negated again by WebPlatformTestsRunner.run()
and coerced into an integer.
Quick demonstration:
>>> unexpected_total = 0
>>> run_tests_rv = unexpected_total == 0
>>> start_rv = not run_tests_rv
>>> run_rv = int(not start_rv)
>>> run_rv
1
This causes git-bisect(1) (specifically "git bisect run ./mach wpt")
to not work because it relies on the exit code to tell if a commit
is good or bad.
This patch offers the least invasive fix for the problem by not
altering the API of the wptrunner test harness.
Differential Revision: https://phabricator.services.mozilla.com/D45190
--HG--
extra : moz-landing-system : lando
When running non-Firefox products in wpt we go through the run.py
script from upstream. Therefore we need to initialise the logger in
that module as well as the one in wptrunner now that it's been
properly converted to use logging rather than print statements.
In addition, we change the run.py module to allow passing in logging
defaults for the case where nothing is specified on the command line
since the behaviour for wpt upstream is different to the behaviour for
gecko.
Differential Revision: https://phabricator.services.mozilla.com/D13547
--HG--
extra : moz-landing-system : lando
The previous code split a lot of logic between the update and download
parts, and exposed two different mach commands. In order to simplify
things it's better to have a single entry point for manifest download
and update, and ensure that's called consistently.
Differential Revision: https://phabricator.services.mozilla.com/D7497
Before we were using a different logger for the manifest download and the actual test run.
This caused timestamps to get reset in a confusing way. Now create the logger early and
share it for all the subseteps.
Depends on D7171
Differential Revision: https://phabricator.services.mozilla.com/D7172
--HG--
extra : moz-landing-system : lando
This allows running web-platform-tests on Fennec given a running emulator.
(Which is how we expect the tests to run in automation as well -- the
android_emulator_unittest mozharness script takes care of emulator
start-up.) It also hooks up ./mach wpt.
wptrunner sets up a profile for Fennec, forwards the marionette port
and starts up Fennec, etc.
= Usage =
Set your mozconfig to build fennec.
Start an emulator: `./mach android-emulator --version x86`
Install fennec: `./mach build && ./mach package && ./mach install`
Run the tests:
```
./mach wpt --product=fennec --testtype=testharness
--certutil-binary path/to/host/os/certutil path/to/some/tests
```
Differential Revision: https://phabricator.services.mozilla.com/D1587
Upstream wpt changed from having a wptrun script to a wpt script with
a run subcommand. This involved some internal movement of code which
broke the `mach wpt` command when used with a non-firefox
product. This commit changes the mach integration to be compatible
with the new upstream API.
MozReview-Commit-ID: 1hvmZedNHSX
--HG--
extra : rebase_source : decd24551c2c5eaaae21eb64c7c376110b3de5b7
Allow running |mach wpt| on one click loaners in order to run
web-platform-tests tests.
This implementation is just like the one for other testsuites using
thee packaged tests rather than the checkout that we get with wpt, at
least on Linux. That's also where the tests run from so it seems
reasonable for now. Moving to the checkout in the future could remove
some of the logic here by using a fake mozbuild environment so that
the testsuite itself doesn't have to implement anything much.
MozReview-Commit-ID: CaewrdjJ2ef
--HG--
extra : rebase_source : 491b8014d48f06ff5bd41b28cc985608981fbdf4
Tooru Fujisawa
Botond Ballo
James Graham
Andreas Tolfsen
Maja Frydrychowicz
Brindusan Cristian