Today we don't require that `mach` `CommandProvider`s subclass from any particular parent class and we're very lax about the requirements they must meet. While that's convenient in certain circumstances, it has some unfortunate implications for feature development.
Today the only requirements that we have for `CommandProvider`s are that they have an `__init__()` method that takes either 1 or 2 arguments, the second of which must be called `context` and is populated with the `mach` `CommandContext`. Again, while this flexibility is occasionally convenient, it is limiting. As we add features to `mach`, having a better idea what the shape of our `CommandProvider`s are and how we can instantiate them and use them is increasingly important, and this gives us additional control when having `mach` configure `CommandProvider`s based on data that is only available at the `mach` level. In particular, we plan to leverage this in bugs 985141 and 1654074.
Here we add validation to the `CommandProvider` decorator to ensure all classes inherit from `MachCommandBase`, update all `CommandProvider`s in-tree to inherit from `MachCommandBase`, and update source and test code accordingly.
Follow-up work: we now require (de facto) that the `context` be populated with a `topdir` attribute by the `populate_context_handler` function, since instantiating the `MachCommandBase` requires a `topdir` be provided. This is fine for now in the interest of keeping this patch reasonably sized, but some additional refactoring could make this cleaner.
Differential Revision: https://phabricator.services.mozilla.com/D86255
The architecture of `mach` does not make this easy to do from within one process, not least of all due to persistent global, mutable state. :( There may be another way to do this, I'm not saying it's completely impossible (and obviously with a significant enough refactor there's a way to make this happen), but subprocessing is a foolproof way to accomplish the same.
Differential Revision: https://phabricator.services.mozilla.com/D77374
It's uncelar if this should be mapped to CRASH or ERROR, but
it probably doesn't matter much; it seems to be a fatal error
that terminates the rest of the run, but we don't get a stack.
I arbitarily chose CRASH.
Differential Revision: https://phabricator.services.mozilla.com/D66753
--HG--
extra : moz-landing-system : lando
Use mozlog to handle logging from the `mach puppeteer-test`
command. This makes the output compatible with treeherder and allows
generating error summaries that can be used by the push-health
feature.
Presently the integration is based on parsing the output from mocha
with regex since that's easy to implement without digging into the
many layers of js code too much. In time it might make sense to
replace this with a custom output formatter integrated into mocha
itself.
We also add the ability to store and use expected results, so that we
are able to detect regressions even when not all test are passing. The
format of expected results is Dict[String, List[String]], where the
keys are test names and the values are lists of statuses. If the list
has more than one value the remaining values are treated as possible
intermittent statuses.
The updated expected statues can be generated by passing the
--write-results flag to the mach command.
Differential Revision: https://phabricator.services.mozilla.com/D65700
--HG--
extra : moz-landing-system : lando
This introduces a -v[v] flag to the "./mach puppeteer-test" command
for increasing the logging verbosity of the remote agent in Gecko.
This can be used as a shorthand for --setpref="remote.log.level=<level>".
A single -v will enable log messages up to debug level, and two
-vv will enable all log messages up to trace level.
Differential Revision: https://phabricator.services.mozilla.com/D58619
--HG--
extra : moz-landing-system : lando
The "./mach puppeteer-test" command now takes a new flag,
--enable-fission, which will set the fission.autostart preference to true.
Differential Revision: https://phabricator.services.mozilla.com/D58512
--HG--
extra : moz-landing-system : lando
Puppeteer accepts an environmental variable EXTRA_LAUNCH_OPTIONS
that is a JSON encoded object as a string, containing additional
configuration to use when running tests.
This takes an extraPrefsFirefox key which is an object
mapping preference names to values. This is extracted in
remote/test/puppeteer/lib/Launcher.js:390 and later written to a
fresh profile in :515.
It appears remote/mach_commands.py has left out the "Firefox" appendix.
Differential Revision: https://phabricator.services.mozilla.com/D58511
--HG--
extra : moz-landing-system : lando
This takes advantage of the browser-selection patch proposed to Puppeteer.
A --product option allows choice between 'firefox' and 'chrome'.
Puppeteer takes care of profile creation for Firefox. Additional Puppeteer
Launcher options can be passed along with --setopt.
Depends on D52313
Differential Revision: https://phabricator.services.mozilla.com/D52314
--HG--
extra : moz-landing-system : lando
Similar to all the other test jobs also the Puppeteer jobs should
not run in headless by default. Instead it should be an opt-in.
Differential Revision: https://phabricator.services.mozilla.com/D48823
--HG--
extra : moz-landing-system : lando
Introduces "./mach puppeteer-test" command for running the Puppeteer
tests against the remote agent. This has to be a top-level command
because the automatic test detection system in mach does not allow
us to delegate to a subcommand such as "./mach remote puppeteer-test".
The tests run against a fork of Puppeteer with hotfixes needed for
it to work with the CDP implementation in Firefox. This fork is
located at https://github.com/andreastt/puppeteer/tree/firefox, and
vendored under remote/test/puppeteer/ in a previous commit in this series.
Differential Revision: https://phabricator.services.mozilla.com/D37009
--HG--
extra : moz-landing-system : lando
Introduces "./mach remote vendor-puppeteer" for vendoring the
Puppeteer client without dependencies into remote/test/puppeteer/.
The particular checkout of Puppeteer is
https://github.com/andreastt/puppeteer/tree/firefox, which contains a
couple of hotfixes we need for the client to work with the Firefox
implementation of CDP.
The remote agent targets a specific version of Puppeteer, so it is
not suitable for this to be vendored under third_party/. We also
wouldn't want other code in central to accidentally use a patched fork.
The vendoring process is not part of "./mach vendor" because it does
not yet have Node.js support, and implementing that for mach is outside
the scope of getting the Puppeteer tests running with the remote agent.
Differential Revision: https://phabricator.services.mozilla.com/D37007
--HG--
extra : moz-landing-system : lando
Introduces "./mach puppeteer-test" command for running the Puppeteer
tests against the remote agent. This has to be a top-level command
because the automatic test detection system in mach does not allow
us to delegate to a subcommand such as "./mach remote puppeteer-test".
The tests run against a fork of Puppeteer with hotfixes needed for
it to work with the CDP implementation in Firefox. This fork is
located at https://github.com/andreastt/puppeteer/tree/firefox, and
vendored under remote/test/puppeteer/ in a previous commit in this series.
Differential Revision: https://phabricator.services.mozilla.com/D37009
--HG--
extra : moz-landing-system : lando
Introduces "./mach remote vendor-puppeteer" for vendoring the
Puppeteer client without dependencies into remote/test/puppeteer/.
The particular checkout of Puppeteer is
https://github.com/andreastt/puppeteer/tree/firefox, which contains a
couple of hotfixes we need for the client to work with the Firefox
implementation of CDP.
The remote agent targets a specific version of Puppeteer, so it is
not suitable for this to be vendored under third_party/. We also
wouldn't want other code in central to accidentally use a patched fork.
The vendoring process is not part of "./mach vendor" because it does
not yet have Node.js support, and implementing that for mach is outside
the scope of getting the Puppeteer tests running with the remote agent.
Differential Revision: https://phabricator.services.mozilla.com/D37007
--HG--
extra : moz-landing-system : lando
Ricky Stewart
Anny Gakhokidze
Maja Frydrychowicz
James Graham
Henrik Skupin
Tooru Fujisawa
Andreas Tolfsen
Ciure Andrei