Rather than using .lint.py files that contain a LINTER object, linter definitions are now in
standalone .yml files. In the case of external linters that need to run python code, the payload
is now of the form:
<module path>:<object path>
The <module path> is the import path to the module, and <object path> is the callable object to
use within that module. It is up to the consumer of mozlint to ensure the <module path> lives on
sys.path. For example, if an external lint's function lives in package 'foo', file 'bar.py' and
function 'lint', the payload would read:
foo.bar:lint
This mechanism was borrowed from taskcluster.
MozReview-Commit-ID: AIsfbVmozy4
--HG--
rename : python/mozlint/test/linters/badreturncode.lint.py => python/mozlint/test/linters/badreturncode.yml
rename : python/mozlint/test/linters/explicit_path.lint.py => python/mozlint/test/linters/explicit_path.yml
rename : python/mozlint/test/linters/external.lint.py => python/mozlint/test/linters/external.yml
rename : python/mozlint/test/linters/invalid_exclude.lint.py => python/mozlint/test/linters/invalid_exclude.yml
rename : python/mozlint/test/linters/invalid_extension.lnt => python/mozlint/test/linters/invalid_extension.ym
rename : python/mozlint/test/linters/invalid_include.lint.py => python/mozlint/test/linters/invalid_include.yml
rename : python/mozlint/test/linters/invalid_type.lint.py => python/mozlint/test/linters/invalid_type.yml
rename : python/mozlint/test/linters/missing_attrs.lint.py => python/mozlint/test/linters/missing_attrs.yml
rename : python/mozlint/test/linters/missing_definition.lint.py => python/mozlint/test/linters/missing_definition.yml
rename : python/mozlint/test/linters/raises.lint.py => python/mozlint/test/linters/raises.yml
rename : python/mozlint/test/linters/regex.lint.py => python/mozlint/test/linters/regex.yml
rename : python/mozlint/test/linters/string.lint.py => python/mozlint/test/linters/string.yml
rename : python/mozlint/test/linters/structured.lint.py => python/mozlint/test/linters/structured.yml
extra : rebase_source : bda3926712234123355c5af71c6453ce869b19fc
Back when the class was written, for the packaging code, it made sense
that the default was True. But now that it's used all over the place,
and that the vast majority of uses are with find_executables=False, it
makes more sense for that to be the default.
--HG--
extra : rebase_source : ff813735fc0d53093f348f20eb77ee03e9b09d4e
This is an educated guess in an attempt to solve a hard to reproduce intermittent. This
also makes sure the original SIGINT handler is restored after running the linters.
MozReview-Commit-ID: HRo0GRlSN5L
--HG--
extra : rebase_source : 7b8bf4b5818d741b5ab6a38c8c56f4eb53a801c8
This deprecates PYTHON_UNIT_TESTS and replaces it with PYTHON_UNITTEST_MANIFESTS.
In the build system, this means python unittests will be treated the same as all
other test suites that use manifestparser. New manifests called 'python.ini' have
been created for all test directories containing python unittests.
MozReview-Commit-ID: IBHG7Thif2D
--HG--
extra : rebase_source : 11a92a2bc544d067946bbd774975140147458caa
This fixes a UnicodeDecodeError when sys.stdout's encoding can't handle unicode.
MozReview-Commit-ID: 3INna8MRje5
--HG--
extra : rebase_source : bb8f3b5cbdbb6601b6946d5fbb7396848f22b5c6
This replaces the "return_code" property on the LintRoller object with a list of "failed"
linters. This is a bit more useful as it lets us know exactly which linters had a problem
(whereas previously we just knew *something* went wrong). This patch pushes determining
the return code back into cli.py, which I think is fine.
In addition, we now pass the list of failed linters into the formatter. This allows us to
clarify exactly how many linters hit a failure which is a lot better than a seemingly
"successful" summary message.
Finally I also removed the "no files to lint" message because I've seen several people
confuse it for an error. I'll probably add it back as a debug log message when we switch
to using mozlog for output.
MozReview-Commit-ID: 4wyCeOZdOf8
--HG--
extra : rebase_source : 23810a8ab8dd9cbbad6b9e965ccff7214f947fbc
If a linter returns a status code instead of a list of results, mozlint will throw away
that status code and move on to the next linter. We should make sure that status code
bubbles all the way up to the cli.
MozReview-Commit-ID: 9fXpmtlUUT1
--HG--
extra : rebase_source : 84c80d1b24d7233d0c4c6e38b23e775110776d1f
This avoids printing harmless (but confusing) errors to the log. For instance, git users
will see a '.hg not found!' error in the output even though not finding an hg repo is
expected in that case.
MozReview-Commit-ID: DBPOabcV7PA
--HG--
extra : rebase_source : a40de185cd4bf0c58917065d1331e0e663b92b94
Versions of mercurial older than 3.0 don't support the -T shorthand for
the --template option. While most people should be using something
newer, it can still happen that some run an older version, and it's
still trivial to support them by using the long option.
--HG--
extra : rebase_source : 1507aea436779495045df48b044a58f4af1382be
This is mostly to maintain backwards compatibility with the |mach eslint| command. But it's also going to be used in
automation. The 'mozlint-eslint' task will need to pass in --quiet to eslint.
Maybe in the future we should remove this ability and only allow well-defined arguments in mozlint. But for now it's
convenient and allows us to land the eslint->mozlint patch series quicker.
MozReview-Commit-ID: KYhC6SEySC3
--HG--
extra : rebase_source : ad7f0cdeb57ed8a99becb90d761d73ef45ce853a
Some linters, such as flake8, will lint invalid file extensions if you explicitly pass them in. E.g,
|flake8 foobar.js| will result in flake8 attempting to lint a JS file. This is a problem because passing
in files explicitly is exactly what the --rev/--workdir options do. If a developer modifies a JS file
then runs |mach lint -l flake8 -w|, that JS file will get linted.
To prevent this, mozlint needs to handle file extensions instead of relying on the underlying linter to
do it. This patch adds an "extensions" config option to the LINTER dict, and will filter these files out
as part of the 'filterpaths' steps.
MozReview-Commit-ID: KYhC6SEySC3
--HG--
extra : rebase_source : 6fea2942b2db1bea7deca1d6738546362b6ebd65
This will make specifying terminal colours cleaner and provide fallbacks for terminals
with only 8 colours. It also tweaks the 'grey' color to match an update to the eslint
stylish formatter.
MozReview-Commit-ID: KYhC6SEySC3
--HG--
extra : rebase_source : 8d3242edc54bf6505aa2e60cdb08cb0dd9743dd3
It's important to use absolute paths so lint errors for the same files don't show up in two
different places. For example, eslint will absolutize a relative path, whereas flake8 will
not.
This patch also makes sure all include/exclude paths are joined to the mozlint 'root' that
was previously calculated from vcs.
MozReview-Commit-ID: KYhC6SEySC3
--HG--
extra : rebase_source : 9b88ee70a997d3f7b6cc0eb849e47931d5726f20
Previously, vcs related stuff was resolved in the cli.py module. But it's possible
for consumers to bypass the cli and instantiate a LintRoller directly. In fact this
is what the mozlint tests do.
Now that we always try to find the vcs root, calling into vcs is no longer optional.
This patch moves the VCSFiles class to a new vcs.py module and makes LintRoller
responsible for instantiating it instead of cli.py.
MozReview-Commit-ID: 5yA3gDZ1UGM
--HG--
extra : rebase_source : 96adc0ca01e8220b0432b64c96c478fc526db756
extra : source : 709c1e521ad3eb466c3434000aed7f71ebf37365
This makes sure we return absolute paths from the VCSFiles class. This is done so we don't accidentally normalize
the relative paths onto $CWD in the event the relative path is valid in both places.
MozReview-Commit-ID: 2QIuR2YqFos
--HG--
extra : rebase_source : 2e82b4a5cf9a18856bbcb8d04fdc916ff7e72deb
This makes it easier to pass configuration to the filterpaths method. In
the future, 'root' and 'extensions' will both be used here.
MozReview-Commit-ID: KYhC6SEySC3
--HG--
extra : rebase_source : 19ddf511ebd24a1a6c21f7c06b67ab64e78e7903
Some linters, such as flake8, will lint invalid file extensions if you explicitly pass them in. E.g,
|flake8 foobar.js| will result in flake8 attempting to lint a JS file. This is a problem because passing
in files explicitly is exactly what the --rev/--workdir options do. If a developer modifies a JS file
then runs |mach lint -l flake8 -w|, that JS file will get linted.
To prevent this, mozlint needs to handle file extensions instead of relying on the underlying linter to
do it. This patch adds an "extensions" config option to the LINTER dict, and will filter these files out
as part of the 'filterpaths' steps.
MozReview-Commit-ID: KYhC6SEySC3
--HG--
extra : rebase_source : 46807a4913660f33e691b864c2c59c448902dfa5
This removes the unnecessary setting of c-basic-offset from all
python-mode files.
This was automatically generated using
perl -pi -e 's/; *c-basic-offset: *[0-9]+//'
... on the affected files.
The bulk of these files are moz.build files but there a few others as
well.
MozReview-Commit-ID: 2pPf3DEiZqx
--HG--
extra : rebase_source : 0a7dcac80b924174a2c429b093791148ea6ac204
This adds two parameters, --rev and --workdir. Each works both with mercurial and git (though the syntax for
specifying revisions is different between them). The value is simply forwarded to either |hg log| or |git diff|
so syntax like |mach lint -r .~4::.| or |mach lint -r "HEAD~4 HEAD"| will work as expected.
MozReview-Commit-ID: aOGp2Yrncs
--HG--
extra : rebase_source : d2cb834d4cc1a083171a3551af4e72c8a7d14021
There is currently no built-in user interface to mozlint. The only existing interface is the
external cli provided by |mach lint|. However, in the future mozlint may need to be used in a
context where mach isn't readily available (i.e version-control-tools). This patch basically
just moves the cli logic out of mach_commands.py, and into mozlint core. That way it can be
re-used in other places without needing to be re-implemented.
The |mach lint setup| subcommand was removed because apparently subcommands don't work with
the parser attribute. Nothing was using it yet anyway, so I removed it for now. It may get
re-added in some form in the future.
MozReview-Commit-ID: aOGp2Yrncs
--HG--
extra : rebase_source : 8f7530de96e5c131d2ed5bfcdd7a159329401e5b
This makes it easier to lint a path that otherwise wouldn't have been linted due to the include/exclude
directives. Now, you can pass in -n/--no-filter instead of needing to modify the linter configuration file.
MozReview-Commit-ID: GMJuE2C1NyY
--HG--
extra : rebase_source : 03627e930f76903ad629cb01b58c4ae7372e4bb1
This is a really simple and ugly formatter that is compatible with
treeherder's error highlighting mechanism. It is designed to be identical
to the current eslint output on treeherder:
https://dxr.mozilla.org/mozilla-central/rev/4d63dde701b47b8661ab7990f197b6b60e543839/tools/lint/eslint-formatter.js
Eventually eslint will also use this and we can remove that file. Once
bug 1276486 is fixed, we can make this look a little nicer. But for now
it gets the job done.
MozReview-Commit-ID: CwfWPcwWFxF
--HG--
extra : transplant_source : %F3PJ%CB%27%A5%82U%D2%CF%B3%9E%A7%9F%0F%A4%F4%E9%5D%BB
This is a really simple and ugly formatter that is compatible with
treeherder's error highlighting mechanism. It is designed to be identical
to the current eslint output on treeherder:
https://dxr.mozilla.org/mozilla-central/rev/4d63dde701b47b8661ab7990f197b6b60e543839/tools/lint/eslint-formatter.js
Eventually eslint will also use this and we can remove that file. Once
bug 1276486 is fixed, we can make this look a little nicer. But for now
it gets the job done.
MozReview-Commit-ID: CwfWPcwWFxF
--HG--
extra : rebase_source : 8dd39aefec1064e0836c847c6d223db43df4755b
Currently a bug in python (https://bugs.python.org/issue8296) is preventing a KeyboardInterrupt from
reaching the parent process, meaning we can't kill the process with SIGINT. There is a workaround to
this bug, but instead I decided to ignore SIGINT in the parent process completely. Now, each child
process is responsible for handling SIGINT on its own. Since child processes should all shutdown
relatively quickly anyway, this effectively also ends the parent process.
The benefit of doing it this way is that each child process can return the results they have collected
to date. So when a developer hits Ctrl-C, they'll still see some (but not all) formatted lint output.
The downside is that a poorly implemented external linter could block the parent process from exiting
quickly, but if this happens we should just fix the linter.
MozReview-Commit-ID: 2tRJgtmoPYP
--HG--
extra : rebase_source : e72282bcc29c9b22690168dc973a817236cdf8ca
For now, only the following two directories will be linted:
python/mozlint
tools/lint
New directories can be added by adding them to the 'include'
directive in tools/lint/flake8.lint. They all default to the
configuration specified in topsrcdir/.flake8. Subdirectories
can override this configuration by creating their own .flake8
file.
MozReview-Commit-ID: Eag48Lnkp3l
--HG--
extra : rebase_source : 6d98c9fef80055a48cc2622848aa04c3045b747e
The current algorithm for filtering down tests is too naive. For example, given the following
directory structured:
parent
- foo
- bar
- baz
And the following include/exclude directives:
include = ['foo']
exclude = ['foo/bar']
Then running ./mach lint parent and ./mach lint foo/baz should both lint all files in baz but
no files in bar. This provides a nice way to include/exclude directories, while allowing the
underlying linters to find appropriate files to lint *within* those directories.
tl;dr - Straight file paths (no globs) will be passed straight to the underlying linter as is.
While paths with globs will first be resolved to a list of matching file paths.
MozReview-Commit-ID: Eag48Lnkp3l
--HG--
extra : rebase_source : 18ce2231bc4198b1e811d39f9803f0d5e03d982e
Mozlint provides two main benefits:
1. A common system for defining lints across multiple languages
2. A common interface and result format for running them
This commit only adds the core library, it does not add any consumers of mozlint just yet.
MozReview-Commit-ID: CSQzq5del5k
--HG--
extra : rebase_source : b520b96177281a1b1770edf53a01cbc2196f494f
Andrew Halberstadt
Mark Banner
Carsten "Tomcat" Book
Mike Hommey
Francesco Pischedda
James Graham
Iris Hsiao
Tom Tromey