This requires passing the test_environment into the get_executor_kwargs function
so that in the firefox wdspec case we can add a cleanup function to the environment
when running wdspec tests. That seems reasonable since we were previously using
a variety of data in the environment to setup the kwargs anyway
Depends on D99698
Differential Revision: https://phabricator.services.mozilla.com/D101768
In `verify_android_device`, when `debugger` is true, we setup `lldb-server`. We
also add a new method, `run_lldb_server`, for actually starting `lldb-server`.
Both cases call into a new function, `_setup_or_run_lldb_server`.
For setup functionality, this function copies `lldb-server` to the device and
then sets up its perms within the target app's data directory.
For run functionality, this function sets some environment variables and
(re)creates directories and log files as needed. Finally it starts the server
and returns to the caller the absolute path to the socket file on the device.
The client provides that path to `lldb` in order to connect to the server.
Both cases use `ADBDevice.batch_execute` to run several commands. The
`LLDB_SERVER_INSTALL_COMMANDS_SCRIPT` and `LLDB_SERVER_START_COMMAND_SCRIPT`
commands are intended to produce the same result as the `start_lldb_server.sh`
script that is provided by Android SDK.
Differential Revision: https://phabricator.services.mozilla.com/D94383
The change in behaviour of the gc in Python 3 means that __del__ is
being called in cases where it previously wasn't, and this can have
unfortunate side effects where __del__ is being used to clean up
resources.
In particular with web-platform-tests running on Android, we end up
closing the test application unexpectedly. That causes widespread test
failures.
In general using __del__ for this kind of cleanup is problematic
because it's not defined exactly when, or if, it's called. It's much
better to require explcit cleanup (and better still to make that
scope-based, using a context manager, although this patch doesn't do
so).
Therefore we add an option to only do explicit cleanup. This is
currently opt-in to make the breakage as small as possible, but
ultimately ought to be the default and only behaviour. We also change
the wpt android runner to use this behaviour.
Differential Revision: https://phabricator.services.mozilla.com/D99256
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Today we don't require that `mach` `CommandProvider`s subclass from any particular parent class and we're very lax about the requirements they must meet. While that's convenient in certain circumstances, it has some unfortunate implications for feature development.
Today the only requirements that we have for `CommandProvider`s are that they have an `__init__()` method that takes either 1 or 2 arguments, the second of which must be called `context` and is populated with the `mach` `CommandContext`. Again, while this flexibility is occasionally convenient, it is limiting. As we add features to `mach`, having a better idea what the shape of our `CommandProvider`s are and how we can instantiate them and use them is increasingly important, and this gives us additional control when having `mach` configure `CommandProvider`s based on data that is only available at the `mach` level. In particular, we plan to leverage this in bugs 985141 and 1654074.
Here we add validation to the `CommandProvider` decorator to ensure all classes inherit from `MachCommandBase`, update all `CommandProvider`s in-tree to inherit from `MachCommandBase`, and update source and test code accordingly.
Follow-up work: we now require (de facto) that the `context` be populated with a `topdir` attribute by the `populate_context_handler` function, since instantiating the `MachCommandBase` requires a `topdir` be provided. This is fine for now in the interest of keeping this patch reasonably sized, but some additional refactoring could make this cleaner.
Differential Revision: https://phabricator.services.mozilla.com/D86255
This adds the ability to fetch avd images from toolchain tasks, but doesn't update the configuration to use that feature yet. AVDs continue to be fetched with tooltool.
Differential Revision: https://phabricator.services.mozilla.com/D77157
There have been a few backwards incompatible changes, so bumping major version.
This also updates the minimum mozprocess to 1.2.0 to pick up the latest Py3
fixes.
Depends on D82974
Differential Revision: https://phabricator.services.mozilla.com/D82975
`./mach run` or `./mach install` for GekcoView uses `android_device.py`, but
`android_device.py` isn't compatible with Python 3.
So, wWhen checking kvm (-accel-check), the following warning occurs.
```
WARNING: sequence item 0: expected str instance, int found
```
Also, when verifying emulator using telnet, the following exception occurs.
```
WARNING: Trying again after unexpected exception
```
Differential Revision: https://phabricator.services.mozilla.com/D75743
Because the web platform test harness doesn't recognize them and annotates them
with `CRITICAL`, which is disconcerting, even though those errors are typically
not harmful.
Differential Revision: https://phabricator.services.mozilla.com/D71485
Because the web platform test harness doesn't recognize them and annotates them
with `CRITICAL`, which is disconcerting, even though those errors are typically
not harmful.
Differential Revision: https://phabricator.services.mozilla.com/D71485
Currently AWSY-with-DMD doesn't work on Windows. This is because `fix-stacks`
is initialized lazily, and by the time the initialization happens some file
descriptors for files are open, and that leads to some major Python2-on-Windows
sadness as described in the big comment in the commit.
To fix the problem, this commit adds an `init` function to `fix_stacks.py` so
that `fix-stacks` can be initialized eagerly, hopefully before any file
descriptors for files are open.
For `dmd.py`, other than fixing the AWSY problems, this has little effect,
because `fix-stacks` is always initialized.
For `utils.py`, which is used to process the output of most tests, this has a
more noticeable effect: the `fix-stacks` process is always spawned, rather than
being spawned only when needed. If no stack traces appear in the test output,
this means that `fix-stacks` is spawned unnecessarily. But it's cheap to spawn;
the expensive part only happens when stack traces start getting fixed. So I
think this change in behaviour is acceptable.
Furthermore, the commit adds a `finish` function to `fix_stacks.py`, so that
the `fix-stacks` process can be explicitly shut down. This has never been done
for processes spawned for any of the stack fixing scripts. It's never caused
problems on Linux/Mac, but it seems to be necessary on Windows to avoid
similar "this file is locked" problems with the test_dmd.js test.
The commit also renames some things to more standard Python style, e.g.
`json_mode` instead of `jsonMode`.
Finally, Android tests use `utils.py` from the repository but `fix_stacks.py`
from the Android host utils. Because the two scripts must be updated in tandem,
this commit also updates the Android host utils to a version that contains the
updated `fix_stacks.py`. Thanks to aerickson for packaging up the new Android
host utils and providing the change to the `hostutils.manifest` file.
Differential Revision: https://phabricator.services.mozilla.com/D69478
Add a --gpu option to 'mach android-emulator' as a convenient way to over-ride the -gpu argument
passed to the emulator.
Also check the environment for MOZ_EMULATOR_COMMAND_ARGS to allow for additional over-rides
of the emulator command line.
Differential Revision: https://phabricator.services.mozilla.com/D70914
--HG--
extra : moz-landing-system : lando
Update code in a few places to work with Python 3, especially with regard to how we invoke subprocesses with text output. Note that the Marionette test harness code needs to be updated so that text is written out in the correct encoding (see bug 1615748).
Differential Revision: https://phabricator.services.mozilla.com/D69012
--HG--
extra : moz-landing-system : lando
Currently AWSY-with-DMD doesn't work on Windows. This is because `fix-stacks`
is initialized lazily, and by the time the initialization happens some file
descriptors for files are open, and that leads to some major Python2-on-Windows
sadness as described in the big comment in the commit.
To fix the problem, this commit adds an `init` function to `fix_stacks.py` so
that `fix-stacks` can be initialized eagerly, hopefully before any file
descriptors for files are open.
For `dmd.py`, other than fixing the AWSY problems, this has little effect,
because `fix-stacks` is always initialized.
For `utils.py`, which is used to process the output of most tests, this has a
more noticeable effect: the `fix-stacks` process is always spawned, rather than
being spawned only when needed. If no stack traces appear in the test output,
this means that `fix-stacks` is spawned unnecessarily. But it's cheap to spawn;
the expensive part only happens when stack traces start getting fixed. So I
think this change in behaviour is acceptable.
The commit also renames some things to more standard Python style, e.g.
`json_mode` instead of `jsonMode`.
Differential Revision: https://phabricator.services.mozilla.com/D69478
--HG--
extra : moz-landing-system : lando
imoraru
James Graham
Aaron Klotz
Bob Clary
Sylvestre Ledru
Ian Moody
Ricky Stewart
Bogdan Tara
Dorel Luca
Simon Giesecke
Narcis Beleuzu
Mihai Alexandru Michis
Geoff Brown
Andrew Halberstadt
Sarah Bird
Andrew Erickson
Makoto Kato
Nicholas Nethercote
Csoregi Natalia
Cosmin Sabou
Oana Pop Rus
William Lachance