5da943f86c
Depends on D188104 > TestExpectations.json > Launcher specs Puppeteer Puppeteer.launch should launch Chrome properly with Those tests can't work on Mozilla CI because we don't perform a regular install of puppeteer there. > TestExpectations.json > Launcher specs Puppeteer Puppeteer.launch should work with no default arguments Same thing. > TestExpectations.json > [proxy.spec] request proxy should respect proxy bypass list This test frequently fails on our CI because the responseCompleted event is received too late (ie after navigate has resolved). Filed https://bugzilla.mozilla.org/show_bug.cgi?id=1852891 about this. > TestExpectations.json > [network.spec] network Response.fromCache should work I am not sure how this one passes on the puppeteer CI. Locally I never get a second request for the CSS file which we expect to get from the cache. Maybe we are completely missing the request in this case, but I don't know how this can pass on the puppeteer CI. > TestExpectations.json > [waittask.spec] waittask specs Frame.waitForFunction should work when resolved right before execution context disposal The polling mechanism used in this test always fail for me locally and on CI. It might pass on puppeteer CI if it has slightly different timings? > evaluation.spec.ts > should not throw an error when evaluation does a navigation Is ending too quickly on our CI, and the next test is trying to use the browsing context which is navigating, failing frequently. > page.spec.ts > Page.Events.PageError should fire I frequently receive other errors for the same context during the test, which make the test fail. I'm proposing to wait for the specific error message here to avoid the issue. > target.spec.ts > should timeout waiting for a non-existent target Depending on which test ran before, there could be a `server.EMPTY_PAGE` target available, and for some reason that lead the test to immediately fail with a TIMEOUT. Not quite sure why, but I think using an obviously not matching URL is safer? Differential Revision: https://phabricator.services.mozilla.com/D188105 |
||
|---|---|---|
| .. | ||
| cdp | ||
| components | ||
| doc | ||
| marionette | ||
| server | ||
| shared | ||
| test/puppeteer | ||
| webdriver-bidi | ||
| .gitignore | ||
| README.md | ||
| jar.mn | ||
| mach_commands.py | ||
| moz.build | ||
README.md
The Firefox remote agent is a low-level debugging interface based on the CDP protocol.
With it, you can inspect the state and control execution of documents running in web content, instrument Gecko in interesting ways, simulate user interaction for automation purposes, and debug JavaScript execution.
This component provides an experimental and partial implementation of a remote devtools interface using the CDP protocol and transport layer.
See https://firefox-source-docs.mozilla.org/remote/ for documentation.
It is available in Firefox and is started this way:
% ./mach run --remote-debugging-port
Puppeteer
Puppeteer is a Node library which provides a high-level API to control Chrome, Chromium, and Firefox over the Chrome DevTools Protocol. Puppeteer runs headless by default, but can be configured to run full (non-headless) browsers.
To verify that our implementation of the CDP protocol is valid we do not only run xpcshell and browser-chrome mochitests in Firefox CI but also the Puppeteer unit tests.
Expectation Data
With the tests coming from upstream, it is not guaranteed that they
all pass in Gecko-based browsers. For this reason it is necessary to
provide metadata about the expected results of each test. This is
provided in a manifest file under test/puppeteer-expected.json.
For each test of the Puppeteer unit test suite an equivalent entry will exist
in this manifest file. By default tests are expected to PASS.
Tests that are intermittent may be marked with multiple statuses using a list of possibilities e.g. for a test that usually passes, but intermittently fails:
"Page.click should click the button (click.spec.ts)": [
"PASS", "FAIL"
],
Disabling Tests
Tests are disabled by using the manifest file test/puppeteer-expected.json.
For example, if a test is unstable, it can be disabled using SKIP:
"Workers Page.workers (worker.spec.ts)": [
"SKIP"
],
For intermittents it's generally preferable to give the test multiple expectations rather than disable it.
Autogenerating Expectation Data
After changing some code it may be necessary to update the expectation
data for the relevant tests. This can of course be done manually, but
mach is able to automate the process:
mach puppeteer-test --write-results
By default it writes the output to test/puppeteer-expected.json.
Given that the unit tests run in Firefox CI only for Linux it is advised to download the expectation data (available as artifact) from the TaskCluster job.