As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).
Differential Revision: https://phabricator.services.mozilla.com/D109650
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).
Differential Revision: https://phabricator.services.mozilla.com/D109650
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).
Differential Revision: https://phabricator.services.mozilla.com/D109650
***
Bug 1698838 - Restrict the list of linters to valid formatters r=ahal
***
Bug 1698838 - Abort formatting even if one of the linters passed is invalid r=ahal
***
***
Differential Revision: https://phabricator.services.mozilla.com/D109231
Before, on Windows, this resulted in installing the package in the parent environment (not the `virtualenv`). We fix this by passing down the `virtualenv_manager` so linters can install packages they need using that object's helper methods.
Differential Revision: https://phabricator.services.mozilla.com/D95738
The logic the `black` and `flake8` linters were using to find the location of the appropriate binaries for linting was wrong in certain cases given how `mach lint` uses subprocesses to batch work. Instead, we allow the option to override the old janky behavior with a known-good path.
Differential Revision: https://phabricator.services.mozilla.com/D95396
Adds --config-paths parameter to mach mozlint where test configuration
files (.yml) can be located. The default path "tools/lint" is set automatically
and additional paths have preference over the default.
This allows Thunderbird to run mozlint-based tests in Taskcluster with a
different configuration.
Differential Revision: https://phabricator.services.mozilla.com/D67077
--HG--
extra : moz-landing-system : lando
This allows lints to "condition" themselves on having a build
environment or a specific build application. It also adds the "name"
parameter, so that setup functions can be shared across lints.
`MozbuildObject` cannot be used as parameters to functions distributed
via multiprocessing, since they cannot be pickled (due, currently, to
internal terminal handles). Therefore we extract just a few key
parts of the environment to expose.
Differential Revision: https://phabricator.services.mozilla.com/D35274
--HG--
extra : moz-landing-system : lando
When using globs in exclude directorives, FileFinder will return every *file*
that gets matched. This is can be thousands of files in the case of an objdir.
While we now collapse these files down to highest possible directories, this
collapse operation can still take a noticeable amount of time (0.6s). This
simply scans topsrcdir for files that start with 'obj' to avoid the glob.
This also moves the '_activate_virtualenv' call to the top of the function
because in CI, this will cause an objdir to be created (to store the
virtualenv). If this happens *after* calculating the global excludes, we won't
catch it since it doesn't exist yet. This will result in the objdir's
virtualenv being linted and erroneous failures.
Depends on D7739
Differential Revision: https://phabricator.services.mozilla.com/D7740
--HG--
extra : moz-landing-system : lando
Often we specify globs in our exclude patterns, e.g:
exclude:
- **/node_modules
- obj*
However, these globs get expanded out to *every* file that matches them. This
can sometimes be thousands or even tens of thousands of files.
We then pass these paths on to the underlying linters and tell them to
exclude them all. This causes a lot of overhead and slows down performance.
This commit implements a "collapse" function. Given a set of paths, it'll
collapse them into the smallest set of parent directories that contain the
original set, and that don't contain any extra files.
For example, given a directory structure like this:
a
-- foo.txt
-- b
-- bar.txt
-- baz.txt
-- c
-- ham.txt
-- d
-- spam.txt
Then the following will happen:
>>> collapse(['a/foo.txt', 'a/b/bar.txt', 'a/c/ham.txt', 'a/c/d/spam.txt'])
['a/foo.txt', 'b/bar.txt', 'c']
Since all files under directory 'c' are specified by the original set (both
'c/ham.txt' and 'c/d/spam.txt'), we can collapse it down to just 'c'. However
not all files under 'b' are specified (we're missing 'a/b/baz.txt'), so we
can't collapse 'b' (and by extension also can't collapse 'a').
If we had included 'a/b/baz.txt':
>>> collapse(['a/foo.txt', 'a/b/bar.txt', 'a/b/baz.txt', 'a/c/ham.txt', 'a/c/d/spam.txt'])
['a']
In both cases, the smallest set of paths that contains the original set (and
only the original set) is computed.
The collapse function has a little bit of overhead but it's not too bad.
For example collapsing all files matched by '**/node_modules' takes ~0.015s.
Collapsing two full objdirs, takes ~0.6s. But a follow up commit is planned to
make sure we stop using 'obj*' to reduce that overhead.
Depends on D7738
Differential Revision: https://phabricator.services.mozilla.com/D7739
--HG--
extra : moz-landing-system : lando
If scandir is already present on the system the attempt to import the
c helper library will currently find the c helper from the system
install which may well be an outdated verion, so causing mach to
break. To solve this this patch does two things:
* Stops importing scandir in files that are run unconditionally when
invoking mach. This is generally considered good for performance
reasons.
* Installs the vendored scandir into the virtualenv for `mach lint`
rather than trying to import it directly from the source tree and so
not getting the c helper library.
Differential Revision: https://phabricator.services.mozilla.com/D8379
--HG--
extra : moz-landing-system : lando
When using globs in exclude directorives, FileFinder will return every *file*
that gets matched. This is can be thousands of files in the case of an objdir.
While we now collapse these files down to highest possible directories, this
collapse operation can still take a noticeable amount of time (0.6s). This
simply scans topsrcdir for files that start with 'obj' to avoid the glob.
This also moves the '_activate_virtualenv' call to the top of the function
because in CI, this will cause an objdir to be created (to store the
virtualenv). If this happens *after* calculating the global excludes, we won't
catch it since it doesn't exist yet. This will result in the objdir's
virtualenv being linted and erroneous failures.
Depends on D7739
Differential Revision: https://phabricator.services.mozilla.com/D7740
--HG--
extra : moz-landing-system : lando
This essentially tests tools/lint/python/flake8.py. Though it also adds
a basic framework for testing all the other linters as well. Getting this
added now will allow others to collaborate on adding more tests without
needing to get to 100% coverage for all linters right off the bat.
All python tests under tools/lint/test will run as part of the 'ml' task
on Linux, and the build task on Windows (OSX coverage is currently missing
for python tests).
The flake8 linter currently has a bug where the 'exclude' argument is
ignored. This is why we need to also exclude 'tools/lint/test/files' in
topsrcdir/.flake8, even though it is already listed in the
'mach_commands.py'. Other linters shouldn't need to do this, the exclusion
in 'mach_commands.py' should be good enough. See bug 1277851 for more
details.
MozReview-Commit-ID: 9ho8C83eeuj
--HG--
extra : rebase_source : 0adb01aa81c06f26cb891ae9f92a17f8f9fdf111
Changed flake8 installation strategy, try to update it using testings/lint/flake8_requirements.txt
moved flake8_requirements to tools/lint/flake8 directory
extracted common pip invocations to _run_pip function
minor fixes
fixed calls to check_process (concatenating list to tuple)
removed update_pip function and its error message variable
if flake8 installation reports and error print pip command output - replaced out with e.output in print function
MozReview-Commit-ID: Fmq22aL8dMq
--HG--
extra : rebase_source : 2e2f942c1ffab012795c632ceacf6debf0dc1407
This integrates 'eslint' with the mozlint framework. The old |mach eslint| command is kept around
for backwards compatibility and will simply dispatch to |mach lint|. But |mach lint| should be
preferred as the old command may eventually be removed.
The old |mach eslint| command should be mostly backwards compatible with a few exceptions:
1. Can no longer define --extensions on the command line, this is instead hardcoded into
eslint.lint.
2. No longer using eslint formatters. However, the default mozlint formatter should be
identical to the default eslint formatter, so developers should not notice a change.
This does mean that non-default eslint formatters can no longer be used unless a mozlint
copy of them is created.
3. Installs dependencies automatically without prompting the user. This was necessary due to
python multiprocessing limitations, but is actually also a better UX. Because the npm
dependencies aren't global anymore, there isn't really any reason *not* to install them
automatically.
Apart from that, any difference from the old |mach eslint| I'd consider a bug.
The main eslint implementation now lives in tools/lint/eslint.lint instead of
tools/lint/mach_commands.py.
MozReview-Commit-ID: KYhC6SEySC3
--HG--
extra : rebase_source : 36d06b1a3fced764e17154cc53091d7722919b67
This removes the in-tree plugin from the tooltool archive and uses that code
directly from the Gecko checkout instead.
For automation, we now get ESLint and external plugins from tooltool and then
symbolic link to the in-tree plugin.
For local development, we install ESLint and external plugins following the
shrinkwrap file created from the last change to the tooltool archive. The local
plugin is then installed.
This change also removes the list of module versions from mach_commands.py, so
there is only one place to update module versions for the future.
MozReview-Commit-ID: AhbZ8lVPmN4
The file base/content/browser-fullScreen.js was renamed. This patch updates the reference
to this filename in the esling rule import-browserjs-globals.js and bumps the version to
0.1.1.
MozReview-Commit-ID: 9hibfDnbtp
This adds two parameters, --rev and --workdir. Each works both with mercurial and git (though the syntax for
specifying revisions is different between them). The value is simply forwarded to either |hg log| or |git diff|
so syntax like |mach lint -r .~4::.| or |mach lint -r "HEAD~4 HEAD"| will work as expected.
MozReview-Commit-ID: aOGp2Yrncs
--HG--
extra : rebase_source : d2cb834d4cc1a083171a3551af4e72c8a7d14021
There is currently no built-in user interface to mozlint. The only existing interface is the
external cli provided by |mach lint|. However, in the future mozlint may need to be used in a
context where mach isn't readily available (i.e version-control-tools). This patch basically
just moves the cli logic out of mach_commands.py, and into mozlint core. That way it can be
re-used in other places without needing to be re-implemented.
The |mach lint setup| subcommand was removed because apparently subcommands don't work with
the parser attribute. Nothing was using it yet anyway, so I removed it for now. It may get
re-added in some form in the future.
MozReview-Commit-ID: aOGp2Yrncs
--HG--
extra : rebase_source : 8f7530de96e5c131d2ed5bfcdd7a159329401e5b
This makes it easier to lint a path that otherwise wouldn't have been linted due to the include/exclude
directives. Now, you can pass in -n/--no-filter instead of needing to modify the linter configuration file.
MozReview-Commit-ID: GMJuE2C1NyY
--HG--
extra : rebase_source : 03627e930f76903ad629cb01b58c4ae7372e4bb1
Alex Lopez
Butkovits Atila
Cosmin Sabou
Nisarg
Steve Fink
Ricky Stewart
Andrew Halberstadt
Sylvestre Ledru
Mihai Alexandru Michis
Razvan Maries
Rob Lemley
Oana Pop Rus
Nick Alexander
Csoregi Natalia
Narcis Beleuzu
James Graham
Tom Prince
Mark Banner
Carsten "Tomcat" Book
Francesco Pischedda
Philipp Kewisch
Matthew Noorenberghe
Jared Wein
J. Ryan Stinnett
Tom Tromey
Julian Descottes