It's uncelar if this should be mapped to CRASH or ERROR, but
it probably doesn't matter much; it seems to be a fatal error
that terminates the rest of the run, but we don't get a stack.
I arbitarily chose CRASH.
Differential Revision: https://phabricator.services.mozilla.com/D66753
--HG--
extra : moz-landing-system : lando
Use mozlog to handle logging from the `mach puppeteer-test`
command. This makes the output compatible with treeherder and allows
generating error summaries that can be used by the push-health
feature.
Presently the integration is based on parsing the output from mocha
with regex since that's easy to implement without digging into the
many layers of js code too much. In time it might make sense to
replace this with a custom output formatter integrated into mocha
itself.
We also add the ability to store and use expected results, so that we
are able to detect regressions even when not all test are passing. The
format of expected results is Dict[String, List[String]], where the
keys are test names and the values are lists of statuses. If the list
has more than one value the remaining values are treated as possible
intermittent statuses.
The updated expected statues can be generated by passing the
--write-results flag to the mach command.
Differential Revision: https://phabricator.services.mozilla.com/D65700
--HG--
extra : moz-landing-system : lando
This introduces a -v[v] flag to the "./mach puppeteer-test" command
for increasing the logging verbosity of the remote agent in Gecko.
This can be used as a shorthand for --setpref="remote.log.level=<level>".
A single -v will enable log messages up to debug level, and two
-vv will enable all log messages up to trace level.
Differential Revision: https://phabricator.services.mozilla.com/D58619
--HG--
extra : moz-landing-system : lando
The "./mach puppeteer-test" command now takes a new flag,
--enable-fission, which will set the fission.autostart preference to true.
Differential Revision: https://phabricator.services.mozilla.com/D58512
--HG--
extra : moz-landing-system : lando
Puppeteer accepts an environmental variable EXTRA_LAUNCH_OPTIONS
that is a JSON encoded object as a string, containing additional
configuration to use when running tests.
This takes an extraPrefsFirefox key which is an object
mapping preference names to values. This is extracted in
remote/test/puppeteer/lib/Launcher.js:390 and later written to a
fresh profile in :515.
It appears remote/mach_commands.py has left out the "Firefox" appendix.
Differential Revision: https://phabricator.services.mozilla.com/D58511
--HG--
extra : moz-landing-system : lando
This takes advantage of the browser-selection patch proposed to Puppeteer.
A --product option allows choice between 'firefox' and 'chrome'.
Puppeteer takes care of profile creation for Firefox. Additional Puppeteer
Launcher options can be passed along with --setopt.
Depends on D52313
Differential Revision: https://phabricator.services.mozilla.com/D52314
--HG--
extra : moz-landing-system : lando
Similar to all the other test jobs also the Puppeteer jobs should
not run in headless by default. Instead it should be an opt-in.
Differential Revision: https://phabricator.services.mozilla.com/D48823
--HG--
extra : moz-landing-system : lando
Introduces "./mach puppeteer-test" command for running the Puppeteer
tests against the remote agent. This has to be a top-level command
because the automatic test detection system in mach does not allow
us to delegate to a subcommand such as "./mach remote puppeteer-test".
The tests run against a fork of Puppeteer with hotfixes needed for
it to work with the CDP implementation in Firefox. This fork is
located at https://github.com/andreastt/puppeteer/tree/firefox, and
vendored under remote/test/puppeteer/ in a previous commit in this series.
Differential Revision: https://phabricator.services.mozilla.com/D37009
--HG--
extra : moz-landing-system : lando
Introduces "./mach remote vendor-puppeteer" for vendoring the
Puppeteer client without dependencies into remote/test/puppeteer/.
The particular checkout of Puppeteer is
https://github.com/andreastt/puppeteer/tree/firefox, which contains a
couple of hotfixes we need for the client to work with the Firefox
implementation of CDP.
The remote agent targets a specific version of Puppeteer, so it is
not suitable for this to be vendored under third_party/. We also
wouldn't want other code in central to accidentally use a patched fork.
The vendoring process is not part of "./mach vendor" because it does
not yet have Node.js support, and implementing that for mach is outside
the scope of getting the Puppeteer tests running with the remote agent.
Differential Revision: https://phabricator.services.mozilla.com/D37007
--HG--
extra : moz-landing-system : lando
Introduces "./mach puppeteer-test" command for running the Puppeteer
tests against the remote agent. This has to be a top-level command
because the automatic test detection system in mach does not allow
us to delegate to a subcommand such as "./mach remote puppeteer-test".
The tests run against a fork of Puppeteer with hotfixes needed for
it to work with the CDP implementation in Firefox. This fork is
located at https://github.com/andreastt/puppeteer/tree/firefox, and
vendored under remote/test/puppeteer/ in a previous commit in this series.
Differential Revision: https://phabricator.services.mozilla.com/D37009
--HG--
extra : moz-landing-system : lando
Introduces "./mach remote vendor-puppeteer" for vendoring the
Puppeteer client without dependencies into remote/test/puppeteer/.
The particular checkout of Puppeteer is
https://github.com/andreastt/puppeteer/tree/firefox, which contains a
couple of hotfixes we need for the client to work with the Firefox
implementation of CDP.
The remote agent targets a specific version of Puppeteer, so it is
not suitable for this to be vendored under third_party/. We also
wouldn't want other code in central to accidentally use a patched fork.
The vendoring process is not part of "./mach vendor" because it does
not yet have Node.js support, and implementing that for mach is outside
the scope of getting the Puppeteer tests running with the remote agent.
Differential Revision: https://phabricator.services.mozilla.com/D37007
--HG--
extra : moz-landing-system : lando
James Graham
Henrik Skupin
Tooru Fujisawa
Maja Frydrychowicz
Andreas Tolfsen
Ciure Andrei