This introduces a -v[v] flag to the "./mach puppeteer-test" command
for increasing the logging verbosity of the remote agent in Gecko.
This can be used as a shorthand for --setpref="remote.log.level=<level>".
A single -v will enable log messages up to debug level, and two
-vv will enable all log messages up to trace level.
Differential Revision: https://phabricator.services.mozilla.com/D58619
--HG--
extra : moz-landing-system : lando
The "./mach puppeteer-test" command now takes a new flag,
--enable-fission, which will set the fission.autostart preference to true.
Differential Revision: https://phabricator.services.mozilla.com/D58512
--HG--
extra : moz-landing-system : lando
Puppeteer accepts an environmental variable EXTRA_LAUNCH_OPTIONS
that is a JSON encoded object as a string, containing additional
configuration to use when running tests.
This takes an extraPrefsFirefox key which is an object
mapping preference names to values. This is extracted in
remote/test/puppeteer/lib/Launcher.js:390 and later written to a
fresh profile in :515.
It appears remote/mach_commands.py has left out the "Firefox" appendix.
Differential Revision: https://phabricator.services.mozilla.com/D58511
--HG--
extra : moz-landing-system : lando
Bug 1605722 already added those helpers to the global head.js file.
As such they can be removed from the target's head.js file.
Differential Revision: https://phabricator.services.mozilla.com/D58163
--HG--
extra : moz-landing-system : lando
This feature is experimental and not required to get implemented
on our end as of now. As such adding a place-holder will be enough.
Differential Revision: https://phabricator.services.mozilla.com/D59006
--HG--
extra : moz-landing-system : lando
Similarly to bug 1603451, the name of the function passed to the
specialised add_agent_task() is lost because of the anonymous
function wrapper inside.
Differential Revision: https://phabricator.services.mozilla.com/D58967
--HG--
extra : moz-landing-system : lando
When remote.log.level is Log.Level.Info or above, verbose logging
is enabled and we pretty-print JSON payloads in requests to the
HTTPD in JSONHandler.
This patch matches the behaviour of the JSONHandler logging, where
JSON payloads are logged to stdout with special formatting before
being transmitted across WebSocket connections.
Differential Revision: https://phabricator.services.mozilla.com/D58506
--HG--
extra : moz-landing-system : lando
The CDP protocol expects consistent types to be returned. By filtering
out null values and strings of zero length we break this promise.
Differential Revision: https://phabricator.services.mozilla.com/D58620
--HG--
extra : moz-landing-system : lando
Upgrading to http 0.2 will give us bytes ^0.5 which removes
a dependency on iovec. iovec is problematic for us because it
depends on a version of winapi that does not compile on Windows AArch64.
The effect of this should be that the remote agent is now possible
to compile on Windows AArch64.
Differential Revision: https://phabricator.services.mozilla.com/D58765
--HG--
extra : moz-landing-system : lando
nsIHttpServer.registerPathHandler() is designed to take null as
the second argument to unregister path handlers. This saves us
from having to modify its internal state, and fixes a minor TODO
in the remote agent initialisation code.
Differential Revision: https://phabricator.services.mozilla.com/D58605
--HG--
extra : moz-landing-system : lando
We were lacking documentation on how to run inidividual Puppeteer
tests. It turns out this is not possible yet to do on a per-file
basis, like other test harnesses.
Instead, suggest use of fdescribe()/fit() and recommend some useful
Mocha flags.
Differential Revision: https://phabricator.services.mozilla.com/D58618
--HG--
extra : moz-landing-system : lando
This reverts the commit:
git: d640ab64b993431099cd9205297bfff464d76927
hg: 9f6327c0d8298ec58a6c9f8d51526872d01d765e
Differential Revision: https://phabricator.services.mozilla.com/D58503
--HG--
extra : moz-landing-system : lando
This reverts the commit:
git: d640ab64b993431099cd9205297bfff464d76927
hg: 9f6327c0d8298ec58a6c9f8d51526872d01d765e
Differential Revision: https://phabricator.services.mozilla.com/D58503
--HG--
extra : moz-landing-system : lando
Errors that arise from use of the protocol, for example by the client
passing in the wrong argument or an operationg being unsupported, are
returned to the client. Since these are not inherent implementation
problems there is no point to double-logging them in the server.
This is a left-over from when the remote agent was in a prototype
stage and we couldn't always rely on the Puppeteer client to
interact with Firefox correctly.
Differential Revision: https://phabricator.services.mozilla.com/D58508
--HG--
extra : moz-landing-system : lando
When remote.log.level is Log.Level.Info or above, verbose logging
is enabled and we pretty-print JSON payloads in requests to the
HTTPD in JSONHandler.
This patch matches the behaviour with JSON payloads being transmitted
across WebSocket connections.
Differential Revision: https://phabricator.services.mozilla.com/D58506
--HG--
extra : moz-landing-system : lando
Moving the JSON payload sanitisation function to Protocol.jsm
means we can share it across modules.
The patch also adds new tests.
Differential Revision: https://phabricator.services.mozilla.com/D58505
--HG--
extra : moz-landing-system : lando
This patch adds basic support for retrieving cookies,
which means that it returns the cookies for the currently
active target.
Hereby it has the following limitations:
1. It does not walk the frame tree, and as such only returns
the cookies from the top-level frame. Support for that will
be added once frames can correctly be handled, which means
once support for the JSWindowActor API has been landed.
2. The "urls" parameter is not supported because it is
unclear right now what it actually does. More investigation
is necessary before any implementation can happen.
3. There is no support for the file:// protocol yet.
4. Dot domains aren't taken care of yet.
Differential Revision: https://phabricator.services.mozilla.com/D57614
--HG--
extra : moz-landing-system : lando
The browser-chrome test name is derived from the function name.
The function name of the original task is lost when wrapped inside
the add_task() override in remote/test/browser/head.js.
This fixes the problem by rewriting the taskFn name on the new wrapper.
Differential Revision: https://phabricator.services.mozilla.com/D57079
--HG--
extra : moz-landing-system : lando
The documentation for the overridden add_task() is quite poor.
Document it as to not confuse new developers why it is behaving
differently to the default bc add_task().
DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D57078
--HG--
extra : moz-landing-system : lando
This patch implements the IO.read() method to allow
reading streams for files and blobs.
Therefor all the methods in the IO domain need a registry
for streams. Those have to be stored globally because
they need to be kept existent across different client
connections.
Differential Revision: https://phabricator.services.mozilla.com/D55968
--HG--
extra : moz-landing-system : lando
This is generally pretty straightforward, and rewrites nearly all calls. It
skips the ones that it can detect using frame script globals like
`sendAsyncMessage`, though.
Differential Revision: https://phabricator.services.mozilla.com/D53740
--HG--
extra : moz-landing-system : lando
Adds browser-chrome tests for the main entry point of the remote
agent, nsIRemoteAgent.
add_agent_task() wraps add_plain_task() (which again wraps the
original add_task()) so we can ensure that the agent isn't listening
before a test runs, and that any modified state such as preferences
is reset afterwards.
The tests rely heavily on the assumption that nsIRemoteAgent.close()
behaves as a no-op when the server is not listening.
This patch is a follow-up to bug 1590828.
Differential Revision: https://phabricator.services.mozilla.com/D55456
--HG--
extra : moz-landing-system : lando
This is generally pretty straightforward, and rewrites nearly all calls. It
skips the ones that it can detect using frame script globals like
`sendAsyncMessage`, though.
Differential Revision: https://phabricator.services.mozilla.com/D53740
--HG--
extra : moz-landing-system : lando
This does not support the grantUniversalAccess parameter.
It just creates a new execution context.
Differential Revision: https://phabricator.services.mozilla.com/D55169
--HG--
extra : moz-landing-system : lando
As noted in Bug 1602083, the CDP viewer implies that execution context management
is independent of the Runtime domain, which makes sense with the behaviour of methods
like Page.createIsolatedWorld.
This is a first step in that direction.
Depends on D55168
Differential Revision: https://phabricator.services.mozilla.com/D56279
--HG--
extra : moz-landing-system : lando
Dismantle the assumption that there is one ExecutionContext per
inner window and generate a fresh id for each ExecutionContext
rather than reusing the inner window id.
Differential Revision: https://phabricator.services.mozilla.com/D55168
--HG--
extra : moz-landing-system : lando
This is generally pretty straightforward, and rewrites nearly all calls. It
skips the ones that it can detect using frame script globals like
`sendAsyncMessage`, though.
Differential Revision: https://phabricator.services.mozilla.com/D53740
--HG--
extra : moz-landing-system : lando
This is generally pretty straightforward, and rewrites nearly all calls. It
skips the ones that it can detect using frame script globals like
`sendAsyncMessage`, though.
Differential Revision: https://phabricator.services.mozilla.com/D53740
--HG--
extra : moz-landing-system : lando
Now that nsIRemoteAgent propagates errors correctly to Rust we can
report errors back to the user when something terrible happens.
The effect of all this is that the startup handler can stop Firefox
when the remote agent fails to listen.
Differential Revision: https://phabricator.services.mozilla.com/D55178
--HG--
extra : moz-landing-system : lando
The nsIRemoteAgent interface introduced with bug 1543115 did
not change the exceptions thrown internally by the class. To ensure
better interaction with consumers over XPIDL we should ensure the
JS implementation throws NS exceptions.
Differential Revision: https://phabricator.services.mozilla.com/D55177
--HG--
extra : moz-landing-system : lando
As nsIHttpServer.registerPathHandler now checks that the path is
not empty we can drop our pre-check when target-created is handled.
If target.path does happen to be empty an error is printed to the console.
Differential Revision: https://phabricator.services.mozilla.com/D55161
--HG--
extra : moz-landing-system : lando
Interfaces exposed over XPIDL cannot be marked async, otherwise their
return values get lost. This patch makes nsIRemoteAgent.listen()
synchronous by removing the use of async/await.
Unfortunately Rust does not yet support deserialising promises sent across
XPIDL, documented in https://bugzilla.mozilla.org/show_bug.cgi?id=1512319.
In an effort to retain the original API internally the function
now returns a promise. This means the function can be awaited in
JS only, which we make use of in remote/test/browser/head.js.
Differential Revision: https://phabricator.services.mozilla.com/D55173
--HG--
extra : moz-landing-system : lando
Interfaces exposed over XPIDL cannot be marked async, otherwise their
return values get lost. This patch makes nsIRemoteAgent.close()
synchronous by removing the use of async/await.
Unfortunately Rust does not yet support deserialising promises sent
across XPIDL, so in an effort to retain the original API internally
the function now returns a promise. This means the function can be
awaited in JS only, which we make use of in remote/test/browser/head.js.
Differential Revision: https://phabricator.services.mozilla.com/D55172
--HG--
extra : moz-landing-system : lando
Adds browser-chrome tests for the main entry point of the remote
agent, nsIRemoteAgent.
add_agent_task() wraps add_plain_task() (which again wraps the
original add_task()) so we can ensure that the agent isn't listening
before a test runs, and that any modified state such as preferences
is reset afterwards.
The tests rely heavily on the assumption that nsIRemoteAgent.close()
behaves as a no-op when the server is not listening.
This patch is a follow-up to bug 1590828.
Differential Revision: https://phabricator.services.mozilla.com/D55456
--HG--
extra : moz-landing-system : lando
Now that the security review in bug 1542229 and the follow-up work
that came out of it is complete, we should go ahead and flip the
remote.enabled preference.
This patch causes the remote agent to be available to users on the
Firefox Nightly release channel. This meansusing --remote-debugger
will no longer cause a fatal error
Differential Revision: https://phabricator.services.mozilla.com/D55137
--HG--
extra : moz-landing-system : lando
As long as we do not validate incoming messages against the
Chrome DevTools JSON schema, all incoming commands would be
executed. To prevent clients from changing internal data by
calling internal commands (prefixed with "_"), deny their
execution.
Differential Revision: https://phabricator.services.mozilla.com/D55411
--HG--
extra : moz-landing-system : lando
With bug 1543115 we saw a rewrite of the startup code path in the
remote agent. It forgot to update the build instructions to mention
that changes to the Rust code requires you to have a full build available.
DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D55130
--HG--
extra : moz-landing-system : lando
This moves the Chromium remote debugging protocol implementation
in Firefox from using sessionstore-windows-restored to a new
remote-startup-requested notification observer.
This change makes it possible for applications to explicitly decide
at which stage it is considered safe to initialise the remote agent
listener. It is safe to emit this notification at any time since
the observer for it is only registered after the --remote-debugger
flag is is used.
The remote debugger only supports Firefox, so this patch does not
take care of GeckoView/Android.
Differential Revision: https://phabricator.services.mozilla.com/D54485
--HG--
extra : moz-landing-system : lando
This event is currently sent for each and every message,
and as such duplicates every log entry as displayed when
having the Puppeteer debug logs turned on.
Until we can fix it to behave correctly, it should be
disabled.
Differential Revision: https://phabricator.services.mozilla.com/D54273
--HG--
extra : moz-landing-system : lando
Full SHA-1: 2fff4ea08428842ad252a72bb3af30ab107fbd6c
Contains two commits reverting the stderr workaround, now that bug
1590826 prints to stderr.
Differential Revision: https://phabricator.services.mozilla.com/D54271
--HG--
extra : moz-landing-system : lando
This bootstraps the remote agent from Rust so that we have access
to write to stderr using the eprintln!() macro. There is a future
intention to expand Rust usage in the remote agent by delegating
CDP and WebDriver Bi-Di protocol schema validation to serde.
The Rust port is faithful to the JS version in terms of functionality,
and in some places improves on the original design by enforcing
a strict division between flag handling code on one hand, and the
remote agent server on the other.
Differential Revision: https://phabricator.services.mozilla.com/D50289
--HG--
extra : moz-landing-system : lando
This change makes it possible to access the remote agent service
from C++ and Rust.
Differential Revision: https://phabricator.services.mozilla.com/D50288
--HG--
extra : moz-landing-system : lando
Adds an XPIDL interface for the remote agent which we will later use to
initialise and start it from a new command-line handler written in Rust.
Differential Revision: https://phabricator.services.mozilla.com/D50287
--HG--
extra : moz-landing-system : lando
Crafting nsIURIs in Rust is complicated. Allow RemoteAgent.listen()
to accept both strings and nsIURIs when called in JavaScript.
Differential Revision: https://phabricator.services.mozilla.com/D50286
--HG--
extra : moz-landing-system : lando
When calling RemoteAgent.listen() across XPIDL the function is run
asynchronously. In order to find out when the remote agent has started
listening we introduce a "remote-listening" system observer notification.
Differential Revision: https://phabricator.services.mozilla.com/D50285
--HG--
extra : moz-landing-system : lando
Although it currently makes no difference, we should ensure the
required preferences are set sooner, in case any of the internal
remote agent features depend on it.
Because we also cannot control when the nsICommandLineHandler for
the remote agent is invoked, setting it sooner rather than later,
seems a lot safer.
Differential Revision: https://phabricator.services.mozilla.com/D50283
--HG--
extra : moz-landing-system : lando
close() is meant to be failsafe in the sense that it should be
possible to call without side-effects.
We are currently setting up a lot of state in listen() that is not
cleaned up if the server eventually fails to start. Calling close()
when this happens will ensure any state listen() has accrued is reset.
Differential Revision: https://phabricator.services.mozilla.com/D50282
--HG--
extra : moz-landing-system : lando
Having init() as a separate function leads to inconsistencies about
how the required state is checked.
init() prevents the remote agent from being loaded when the
remote.enabled preference is false or it is attempted loaded into a
child process, but listen() already manipulates state before these
checks are run. This is probably not the intention, but an easy
mistake to make when the code flow is not crystal clear.
Since we never have a need to call init() independently, this patch
merges init() into listen().
Differential Revision: https://phabricator.services.mozilla.com/D50281
--HG--
extra : moz-landing-system : lando
All other XPCOM component manifests are named components.conf and
this makes the remote agent's conform with those.
It will also become apparent in a later patch in this changeset
that we need to register two XPCOM components: one implementing
the remote agent service, and one for handling command-line arguments.
Differential Revision: https://phabricator.services.mozilla.com/D50280
--HG--
rename : remote/RemoteAgent.conf => remote/components.conf
extra : moz-landing-system : lando
This bootstraps the remote agent from Rust so that we have access
to write to stderr using the eprintln!() macro. There is a future
intention to expand Rust usage in the remote agent by delegating
CDP and WebDriver Bi-Di protocol schema validation to serde.
The Rust port is faithful to the JS version in terms of functionality,
and in some places improves on the original design by enforcing
a strict division between flag handling code on one hand, and the
remote agent server on the other.
Differential Revision: https://phabricator.services.mozilla.com/D50289
--HG--
extra : moz-landing-system : lando
This change makes it possible to access the remote agent service
from C++ and Rust.
Differential Revision: https://phabricator.services.mozilla.com/D50288
--HG--
extra : moz-landing-system : lando
Adds an XPIDL interface for the remote agent which we will later use to
initialise and start it from a new command-line handler written in Rust.
Differential Revision: https://phabricator.services.mozilla.com/D50287
--HG--
extra : moz-landing-system : lando
Crafting nsIURIs in Rust is complicated. Allow RemoteAgent.listen()
to accept both strings and nsIURIs when called in JavaScript.
Differential Revision: https://phabricator.services.mozilla.com/D50286
--HG--
extra : moz-landing-system : lando
When calling RemoteAgent.listen() across XPIDL the function is run
asynchronously. In order to find out when the remote agent has started
listening we introduce a "remote-listening" system observer notification.
Differential Revision: https://phabricator.services.mozilla.com/D50285
--HG--
extra : moz-landing-system : lando
Although it currently makes no difference, we should ensure the
required preferences are set sooner, in case any of the internal
remote agent features depend on it.
Because we also cannot control when the nsICommandLineHandler for
the remote agent is invoked, setting it sooner rather than later,
seems a lot safer.
Differential Revision: https://phabricator.services.mozilla.com/D50283
--HG--
extra : moz-landing-system : lando
close() is meant to be failsafe in the sense that it should be
possible to call without side-effects.
We are currently setting up a lot of state in listen() that is not
cleaned up if the server eventually fails to start. Calling close()
when this happens will ensure any state listen() has accrued is reset.
Differential Revision: https://phabricator.services.mozilla.com/D50282
--HG--
extra : moz-landing-system : lando
Having init() as a separate function leads to inconsistencies about
how the required state is checked.
init() prevents the remote agent from being loaded when the
remote.enabled preference is false or it is attempted loaded into a
child process, but listen() already manipulates state before these
checks are run. This is probably not the intention, but an easy
mistake to make when the code flow is not crystal clear.
Since we never have a need to call init() independently, this patch
merges init() into listen().
Differential Revision: https://phabricator.services.mozilla.com/D50281
--HG--
extra : moz-landing-system : lando
All other XPCOM component manifests are named components.conf and
this makes the remote agent's conform with those.
It will also become apparent in a later patch in this changeset
that we need to register two XPCOM components: one implementing
the remote agent service, and one for handling command-line arguments.
Differential Revision: https://phabricator.services.mozilla.com/D50280
--HG--
rename : remote/RemoteAgent.conf => remote/components.conf
extra : moz-landing-system : lando
The method under the Network domain is a redirect to
Emulation only. It might be deprecated, but that's unclear
in the current documentation.
Differential Revision: https://phabricator.services.mozilla.com/D52929
--HG--
extra : moz-landing-system : lando
This patch implements setting the user agent only. Support for
accepted language, and platform will be done in a follow-up bug.
Differential Revision: https://phabricator.services.mozilla.com/D52831
--HG--
extra : moz-landing-system : lando
By using loadSubScript() instead of injecting a script node
into the hidden window, the content of chrome-remote-interface.js
can be cached. Only the first load will take about 150ms, each
test afterward will only have to spend about 30ms in getCDP().
Differential Revision: https://phabricator.services.mozilla.com/D53073
--HG--
extra : moz-landing-system : lando
Using the minimized version reduces the file size from 880kB
to 550kB, which means lesser data to load and to cache.
Imported revision: f412dd1ed0028bcf9116918879b99924039d52e2
Differential Revision: https://phabricator.services.mozilla.com/D53072
--HG--
extra : moz-landing-system : lando
This takes advantage of the browser-selection patch proposed to Puppeteer.
A --product option allows choice between 'firefox' and 'chrome'.
Puppeteer takes care of profile creation for Firefox. Additional Puppeteer
Launcher options can be passed along with --setopt.
Depends on D52313
Differential Revision: https://phabricator.services.mozilla.com/D52314
--HG--
extra : moz-landing-system : lando
This sub-test has races which were exacerbated by making data: URLs load asynchronously. Disable it until the race can be resolved.
MANUAL PUSH: bustage fix on a CLOSED TREE
--HG--
extra : amend_source : 76c3d8486213b5829615059587f44f0db93b9041
The actual screenshot data should be returned via the "data"
property of an object. Also the data URL prefix has to be
stripped off, so that only the base64 encoded data will be
transmitted.
Differential Revision: https://phabricator.services.mozilla.com/D52144
--HG--
extra : moz-landing-system : lando
This implements an all-or-nothing insecure sweeping override that
bypasses security exceptions when loading documents with invalid
or otherwise bad TLS certificates.
Differential Revision: https://phabricator.services.mozilla.com/D50838
--HG--
extra : moz-landing-system : lando
Currently when browser chrome tests are failing the open tabs, client,
and Remote Agent will never be closed, and as such each failing test
causes massive memory leaks.
Therefore the teardown logic needs to be moved out of the tests into
the "add_task()" function. Only that way we can make sure to run
all the clean-up steps independent of the test success state.
Differential Revision: https://phabricator.services.mozilla.com/D50233
--HG--
extra : moz-landing-system : lando
Henrik Skupin
David Burns
Maja Frydrychowicz
Noemi Erli
Coroiu Cristina
Andreas Tolfsen
Khushal Sahni
Brindusan Cristian
Mohit Singh
Kris Maglione
shindli
Daniel Varga
Gurzau Raul
Mihai Alexandru Michis
Sylvestre Ledru
danwalsh99
Oana Pop Rus
JaStenson
Ryan VanderMeulen
Thomas
Cosmin Sabou