The excluded directories aren't being properly handled in the py2/py3 compat
linters. In order for FileFinder to apply the exclusions properly they need
to either be relative to or contained by the base.
This means that currently the following will work:
./mach lint -l py2 <topsrcdir>
./mach lint -l py2 testing/mochitest
But this is broken:
./mach lint -l py2 testing
This change fixes the compat linters so exclude paths will be made relative
to the FileFinder base before passing them in. Any exclude not contained by
the base is simply discarded as it won't be relevant to that FileFinder
instance anyway.
MozReview-Commit-ID: LJx97TvKlSa
--HG--
extra : rebase_source : b8f0cb20ec5a88a33c26ace699ed9216b070f443
check_compat.py was adapted from gps' check-py3-compat.py in mercurial:
https://www.mercurial-scm.org/repo/hg/file/tip/contrib/check-py3-compat.py
The py3 linter simply runs ast.parse(f) for each file being linted. Any syntax errors
are formatted as mozlint results and dumped to stdout as json. I looked into also
importing the file (using 3.5+'s importlib.util.spec_from_file_location), but there
were too many problems:
1. Lots of false positives (e.g module not found)
2. Some files seemed to run indefinitely on import
I decided to punt on importing for now, we can always investigate in a follow-up.
The py2 linter runs ast.parse(f), and also checks that the file has:
from __future__ import absolute_import, print_function
Initially every python file in the tree is excluded from the py2 check, though
at least this makes it easy to find+fix, and new files in un-excluded
directories will automatically be linted.
MozReview-Commit-ID: ABtq9dnPo9T
--HG--
extra : rebase_source : 60762937284d498514cd020b90cbfd2ba23f0b70
This will allow us keep python related linting files in the same place.
MozReview-Commit-ID: ABtq9dnPo9T
--HG--
rename : tools/lint/flake8_/__init__.py => tools/lint/python/__init__.py
rename : tools/lint/flake8_/__init__.py => tools/lint/python/flake8.py
rename : tools/lint/flake8_/flake8_requirements.txt => tools/lint/python/flake8_requirements.txt
extra : rebase_source : 2568bc0bf8f4adbf8e0be73a54d5da068a8d81b0
This adds a new morph template for modifying a task's env and the corresponding glue
to specify it from |mach try fuzzy|. It can be used like:
./mach try fuzzy --env FOO=1 --env BAR=baz
This will simply set those environment variables in *all* tasks. We could add the
ability to only specify it for a subset of tasks in the future, but that seems like
a hard problem that probably isn't worth it.
MozReview-Commit-ID: C4sokv886PU
--HG--
extra : rebase_source : 64c9105848e6becd33fae63aeabc3423cfda3bdf
To use this, create a ~/.mozbuild/machrc file and add:
[try]
default = fuzzy
Now, running |mach try| without a subcommand will default to the fuzzy selector. The syntax selector can still be
run with |mach try syntax|.
MozReview-Commit-ID: Ai6f8cW3Swq
--HG--
extra : rebase_source : 6a107478f7fbd7d8deb4882f0892963fed755b33
This allows .flake8 files to override one another, and fixes a pretty bad known
bug with our flake8 implementation. For example, say we have a .flake8 file at:
/foo/.flake8
Before this patch, if we ran |mach lint foo/bar|, the configuration defined in
that .flake8 file wouldn't get picked up. It would only work if running the
specific directory that contains it, e.g |mach lint foo|.
This change additionally allows multiple .flake8 files to be used. So if
there's one defined at both:
/.flake8
/foo/.flake8
Then running |mach lint foo/bar| will first apply the root .flake8, then the
one under /foo (overriding earlier configuration).
This bug still doesn't make flake8 configuration perfect though. Any directory
containing a .flake8 file still needs to be explicitly listed in the "include"
section of /tools/lint/flake8.yml. Otherwise in the example above, if running
|mach lint /|, it wouldn't be able to find /foo/.flake8. This is a hard problem
and is likely best solved by fixing flake8's upstream configuration handling.
Unfortunately this means we still can't switch from a whitelist to a blacklist.
MozReview-Commit-ID: 3DZAi1QHYYo
--HG--
extra : rebase_source : 51298c5847f6c2792581d9b312c87b70fa716ee1
This fixes a regression from bug 1384593. I was going to add a test for this, but it would be
a better use of time to fix bug 1185599 and add a test there. We already have vcs tests in other
parts of the tree so consolidating them into a single module is likely easier than standing up
another one-off vcs tester.
MozReview-Commit-ID: E51Tb1qC9Wb
--HG--
extra : rebase_source : f957b1857d162fd229dcef0e87668dd270512cf7
This reduces the time taken to upload the Firefox docs from ~30s to
~5s (per invocation).
MozReview-Commit-ID: DxOrvxvVn42
--HG--
extra : rebase_source : b170fb728267c43484573e0a57ebc96512eb0e7c
Previously, we uploaded the main Firefox tree docs to /.
In reality, there are multiple Sphinx projects in the repo. In
addition, it is sometimes desirable to access docs for an older
version of Firefox.
In this commit, we add support for specifying the S3 key prefix
for uploads. Then we change the upload code to upload to multiple
locations:
* <project>/latest (always)
* <project>/<version> (if a version is defined in the Sphinx config)
* / (for the main Sphinx docs project)
For the Firefox docs, ``version`` corresponds to a sanitized value from
``milestone.txt``. Currently, it resolves to ``57.0``.
While we're here, we add support for declaring an alternate project
name in the Sphinx conf.py file. If ``moz_project_name`` is defined,
we use that as the project name. For Firefox, we set it to ``main``.
This means our paths (local and uploaded) are now ``main`` instead of
``Mozilla_Source_Tree_Docs``. That's much more pleasant.
MozReview-Commit-ID: 8Gl6l2m6uU4
--HG--
extra : rebase_source : e56885092c12eb8cc76e5e7300f938be566e3e5a
extra : intermediate-source : 8509af1e135177a93460270b27f263c10a62d996
extra : source : 71b4f32caf209fe9dffc340c0b8ccb51ac79c7de
We now have an --upload flag to control whether upload is performed.
We don't inline it because we want to maintain a "firewall" between
regular docs and all the extra packages and imports needed for S3.
MozReview-Commit-ID: DVKhsS545gp
--HG--
extra : rebase_source : 7fee832145189be882db0f0ca057eda6158e0492
Instead of doing the file finding inside s3_upload(), the function now
takes the output of distribution_files().
The new code is much simpler.
MozReview-Commit-ID: 43i2Alvyu5i
--HG--
extra : rebase_source : 29d840c65acec794749edfda0e47193806f387f4
extra : source : 5a758defcf2f1a17de851e703388469300789ac5
Alexandre Poirot
Steve Armand
Sebastian Hengst
Andi-Bogdan Postelnicu
Andrew Halberstadt
Wes Kocher
Sylvestre Ledru
Kris Maglione
Ionut Goldan
Chris Manchester
Dustin J. Mitchell
Cosm
Mark Hammond
Ralph Giles
Marco Castelluccio
Markus Stange
Xavier ALT
Phil Ringnalda
Tom Ritter
Gregory Szorc