Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Today we don't require that `mach` `CommandProvider`s subclass from any particular parent class and we're very lax about the requirements they must meet. While that's convenient in certain circumstances, it has some unfortunate implications for feature development.
Today the only requirements that we have for `CommandProvider`s are that they have an `__init__()` method that takes either 1 or 2 arguments, the second of which must be called `context` and is populated with the `mach` `CommandContext`. Again, while this flexibility is occasionally convenient, it is limiting. As we add features to `mach`, having a better idea what the shape of our `CommandProvider`s are and how we can instantiate them and use them is increasingly important, and this gives us additional control when having `mach` configure `CommandProvider`s based on data that is only available at the `mach` level. In particular, we plan to leverage this in bugs 985141 and 1654074.
Here we add validation to the `CommandProvider` decorator to ensure all classes inherit from `MachCommandBase`, update all `CommandProvider`s in-tree to inherit from `MachCommandBase`, and update source and test code accordingly.
Follow-up work: we now require (de facto) that the `context` be populated with a `topdir` attribute by the `populate_context_handler` function, since instantiating the `MachCommandBase` requires a `topdir` be provided. This is fine for now in the interest of keeping this patch reasonably sized, but some additional refactoring could make this cleaner.
Differential Revision: https://phabricator.services.mozilla.com/D86255
Remove some exclusions so that more files are linted. These exclusions had
been made to allow for code that was not py3 compatible, but with recent
py3 efforts, the exclusions can be removed. (Linting subsequently found
a few small issues which needed to be fixed.)
Differential Revision: https://phabricator.services.mozilla.com/D84393
Various updates to the custom retrigger action so that, without any custom changes to
parameters, the retriggered task runs with the same parameters as the original task.
Several issues were found and corrected, notably:
- parameters like --allow-software-gl-layers were ignored
- MOZHARNESS_TEST_PATHS was ignored
- many parameter customizations in the desktop mozharness configs were ignored
- mochitest suite/subsuite/flavor selection was not always correct
- using repeat=1 by default meant that each test ran twice
Differential Revision: https://phabricator.services.mozilla.com/D70457
Add test package mach support for gtest and hook into the custom retrigger
action. Some existing custom retrigger features, like setting gecko prefs,
are not (easily) applicable to gtest, which doesn't use mozprofile; for
this reason, use a separate action context with items suitable for gtest.
Differential Revision: https://phabricator.services.mozilla.com/D67384
--HG--
extra : moz-landing-system : lando
Convert the gtest option parser from optparse to argparse. mochitest, reftest,
and other suites use argparse. Using argparse will simplify the integration
of gtest with the custom retrigger action.
Differential Revision: https://phabricator.services.mozilla.com/D66429
--HG--
extra : moz-landing-system : lando
Whereas:
- desktop tests don't make this check;
- the check for directory existence has been troublesome and almost never useful;
- bug classification of this condition has been troublesome;
- if a startup crash actually did occur before crashreporter init, there would still be an indication in logcat and possibly a tombstone, and the "No test summary found" check would definitely be triggered;
Let's stop checking for minidumps directory creation.
Differential Revision: https://phabricator.services.mozilla.com/D54755
--HG--
extra : moz-landing-system : lando
Improve gtest diagnostics in an effort to improve our understanding
of this intermittent failure.
Differential Revision: https://phabricator.services.mozilla.com/D50654
--HG--
extra : moz-landing-system : lando
In Android gtest, instead of pushing the gtest libxul.so to the
(assumed and potentially incorrect) location of the test application's
libraries, push libxul.so to /data/local/gtest, and direct the linker
to use that instead.
Differential Revision: https://phabricator.services.mozilla.com/D34942
--HG--
extra : moz-landing-system : lando
A possible explanation for the infrequent observation of an incomplete log is that
the log is still being written when the harness detects that the process has completed.
Waiting for one extra polling interval may avoid this.
Differential Revision: https://phabricator.services.mozilla.com/D38577
--HG--
extra : moz-landing-system : lando
Including the traceback info will include the exception name, like "ADBError" which
should trigger the android-em task retry; it also provides more debugging info.
Differential Revision: https://phabricator.services.mozilla.com/D36116
--HG--
extra : moz-landing-system : lando
This allows me to run 'mach gtest "APZ*"' and other short runs of gtest
that were previously failing.
Differential Revision: https://phabricator.services.mozilla.com/D34764
--HG--
extra : moz-landing-system : lando
Increasing the timeout avoids a perma-fail on linux64/ccov and leaves room for the
test suite to grow on all platforms.
Differential Revision: https://phabricator.services.mozilla.com/D30083
--HG--
extra : moz-landing-system : lando
Add basic support for 'mach gtest' on Android.
Handling of Android-only and desktop-only options is awkward; I hope to
re-visit this after bug 1519369.
Differential Revision: https://phabricator.services.mozilla.com/D28129
--HG--
extra : moz-landing-system : lando
Adds remotegtests.py, like rungtests.py, but for Android. Unlike some other test harnesses, remotegtests is not a sub-class of rungtests: There wasn't much code that could be re-used.
The code to wait for the test app and collect and display the log is derived from and similar to existing code in remoteautomation.py: I didn't want to increase reliance on remoteautomation; I don't mind a bit of code duplication.
There is a TODO item related to libxul.so. gtest runs in the browser, but with tests compiled into a gtest-specific version of libxul.so. remotegtests.py patches the installed app by pushing the libxul.so to the installation directory of the app -- root required and fragile, but works fine on the emulator. I think the "right" way to do this is to create a gtest-specific apk; I plan to file a follow-up bug and petition build folks to help sort that out.
I have been running this locally with:
mach gtest (builds gtests, then errors out)
mach python testing/gtest/remotegtests.py --libxul=/home/gbrown/objdirs/x86_64/dist/bin/gtest/libxul.so
Work-in-progress on 'mach gtest' will make this much simpler.
Differential Revision: https://phabricator.services.mozilla.com/D26835
--HG--
extra : moz-landing-system : lando
Desktop gtest creates minidumps in the current working directory. That is
problematic on Android, since the test app's cwd may not be writable, or
may not be readable by the test harness. This patch allows the test harness
to specify an alternate minidump path with environment variable
MOZ_GTEST_MINIDUMPS_PATH.
Differential Revision: https://phabricator.services.mozilla.com/D26833
--HG--
extra : moz-landing-system : lando
On Android, update mozilla gtest logging so that logging appears in the Android logcat.
Also, when MOZ_GTEST_LOG_PATH is defined in the environment, create the named file
and direct logging to that file. Android gtest will use this to collect gtest logging from
the device and copy it to the test log.
Differential Revision: https://phabricator.services.mozilla.com/D26610
--HG--
extra : moz-landing-system : lando
Before this change, testing/gtest had 19 errors. These errors have been resolved.
Bug 1506611 - Fixed flake8 errors in testing/gtest
Differential Revision: https://phabricator.services.mozilla.com/D15074
--HG--
extra : moz-landing-system : lando
Dorel Luca
Ricky Stewart
Simon Giesecke
Bob Clary
Geoff Brown
Narcis Beleuzu
Mihai Alexandru Michis
Gabriele Svelto
Sylvestre Ledru
Kartikaya Gupta
Chris Manchester
Gijs Kruitbosch
Csoregi Natalia
Ryan VanderMeulen
Boris Zbarsky
Ehsan Akhgari
terrameijar