Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
`mach configure` currently runs the equivalent to `make -f client.mk`.
This is history, and essentially does the following:
- Create `configure` and `js/src/configure` from `configure.in` and
`js/src/configure.in` respectively.
- Create the objdir.
- Run `configure` from the objdir.
The `configure` script is, nowadays, only really used as a means to set
OLD_CONFIGURE (and also for people who want to run `configure`,
literally, as in the `configure; make` workflow). `mach configure`
actually doesn't need it. Neither does recursing into `js/src` require
`js/src/configure`, since bug 1520340 (and now as of bug 1669633, we
don't even recurse).
Because configure.py can actually derive OLD_CONFIGURE on its own
(except for `js/src/configure`, but `mach configure` doesn't run that),
we don't really need `configure` for `mach configure`.
So all in all, we're at a point in history where it's straightforward to
just initiate configure.py from mach configure, so we just do that.
And in the hypothetical case where the `mach configure` code is somehow
running in python2, we get the mach virtualenv python3 and use it to
execute `configure.py`.
Differential Revision: https://phabricator.services.mozilla.com/D93741
`mach configure` currently runs the equivalent to `make -f client.mk`.
This is history, and essentially does the following:
- Create `configure` and `js/src/configure` from `configure.in` and
`js/src/configure.in` respectively.
- Create the objdir.
- Run `configure` from the objdir.
The `configure` script is, nowadays, only really used as a means to set
OLD_CONFIGURE (and also for people who want to run `configure`,
literally, as in the `configure; make` workflow). `mach configure`
actually doesn't need it. Neither does recursing into `js/src` require
`js/src/configure`, since bug 1520340 (and now as of bug 1669633, we
don't even recurse).
Because configure.py can actually derive OLD_CONFIGURE on its own
(except for `js/src/configure`, but `mach configure` doesn't run that),
we don't really need `configure` for `mach configure`.
So all in all, we're at a point in history where it's straightforward to
just initiate configure.py from mach configure, so we just do that.
And in the hypothetical case where the `mach configure` code is somehow
running in python2, we get the mach virtualenv python3 and use it to
execute `configure.py`.
Differential Revision: https://phabricator.services.mozilla.com/D93741
Instead, we now run js/src/old-configure from the top-level configure
after having run old-configure and extracted a few variables to inherit
from it.
Because we're now running from the top-level, $_objdir is always the
top-level objdir, which simplifies some things. The topobjdir in
js/src/config.status, however, needs to stay in js/src because of the
build frontend expecting it there.
When running js/src/old-configure, we used to need some special
treatment for a large number of variables for historic reasons, where
we'd take values from the assigned values before running old-configure
for some, or from AC_SUBSTs after running old-configure.
Now that both old-configure and js/src/old-configure get the same
assignments from old-configure.vars, we don't need anything special for
the former. And only a few remaining variables still need manual work
for the latter.
One notable difference, though, is that the new code doesn't try to
avoid running js subconfigure, which added complexity, and was actually
error-prone.
Differential Revision: https://phabricator.services.mozilla.com/D92725
In order to be able to run both old-configure and js/src/old-configure
from the same python configure run, we need to stop setting the items
set by old-configure into the global sandbox config, and instead store
them to be later handled by configure.py.
Differential Revision: https://phabricator.services.mozilla.com/D92718
I think they're remnants from the past that we don't really need anymore.
And they're making things more complicated for some pending work of mine.
Differential Revision: https://phabricator.services.mozilla.com/D89687
I think they're remnants from the past that we don't really need anymore.
And they're making things more complicated for some pending work of mine.
Differential Revision: https://phabricator.services.mozilla.com/D89687
Rather than always printing instructions at the end of the bootstrap phase, we will now create a mozconfig
file if one doesn't exist and there's configuration to be written.
Differential Revision: https://phabricator.services.mozilla.com/D78417
This also does a few remaining python 2 incompatible changes to
.configure files.
Differential Revision: https://phabricator.services.mozilla.com/D69538
--HG--
extra : moz-landing-system : lando
Many values we get out from configure are of types that look like
lists/tuples, that write_indented_repr will serialize as lists... but
only in python 2, because the alternative implementation for python 3
is not doing that. So sanitize first.
Differential Revision: https://phabricator.services.mozilla.com/D69949
--HG--
extra : moz-landing-system : lando
If configure is invoked manually, clobber.py is bypassed and the CLOBBER
file doesn't get touched. The clobber check in Makefile.in gets
triggered causing the build to stop.
Moving the objdir/CLOBBER file creation into configure.py should cause
it to be created regardless of how configure is invoked.
Depends on D53290
Differential Revision: https://phabricator.services.mozilla.com/D53291
--HG--
extra : moz-landing-system : lando
Now that there is no bytes strings in it, we don't need to store
config.status in the system encoding to keep those valid. Moreover, the
system encoding is lossy, which utf-8 is not.
Differential Revision: https://phabricator.services.mozilla.com/D42731
--HG--
extra : moz-landing-system : lando
With bug 1575135, we ensure nothing gets out of the configure sandbox
as a bytes string. We can thus now avoid the encode() pass in
configure.py. We still need, however, to normalize the configuration
so that it doesn't contain unexpected types, and conformning to what
indented_repr does to the configuration in config.status.
While here, convert some obj.iteritems() to six.iteritems(obj).
And remove the now unused encode function.
Differential Revision: https://phabricator.services.mozilla.com/D42630
--HG--
extra : moz-landing-system : lando
Make it a hard error when the sandbox returns non-unicode strings.
This should help quickly catch any remaining non-unicode string that
are not caught by automation.
Differential Revision: https://phabricator.services.mozilla.com/D42607
--HG--
extra : moz-landing-system : lando
This is necessary for any attempt at @import('six') or doing any similar py2+py3 work
in the configure sandbox.
Differential Revision: https://phabricator.services.mozilla.com/D29604
--HG--
extra : moz-landing-system : lando
Since js configure is also python configure, we can actually create
a ConfigureSandbox directly, with the right environment and arguments.
Depends on D16668
Differential Revision: https://phabricator.services.mozilla.com/D16669
--HG--
extra : moz-landing-system : lando
Since MozbuildObject.from_environment() reads from mozinfo.json, tup
picks up that file as a dependency for anything that imports buildconfig
(eg: all generated files). Using FileAvoidWrite when creating
mozinfo.json will help avoid unnecessary work after re-running configure
in the tup backend.
MozReview-Commit-ID: EEOPQYJA1MV
--HG--
extra : rebase_source : 136a0579090776dc55ea5cee870574f13cc27c58
So the file can be more easily consumed without this variable defined.
MozReview-Commit-ID: DF3ASwx4SZP
--HG--
extra : rebase_source : 7d760a71c8f0fa389c2f31f1b383a1f298220545
The config.statusd directory is created alongside config.status, which
contains the same information but is split across many files instead of
all in a single file. This allows the build system to track dependencies
on individual configure values.
MozReview-Commit-ID: 2DbwKCJuNSX
--HG--
extra : rebase_source : 8b6257fd9c75cff3e4b6513d69048c0e3fdda5f4
This generalizes the monkey-patching of the main module added
in bug 914563 to allow multiprocessing to be used from config.status
(which triggers the same bug as when it's used with `mach` as main).
MozReview-Commit-ID: AdOdpKzmbsp
As of Python 3, decimal notations of octal values for permission modes
are no longer permitted and will result in a `SyntaxError` exception
(`invalid token`).
Using the proper octal notation which is also Python 2.7 compatible will
fix this issue.
--HG--
extra : rebase_source : 2e897c51f04ad0ee69071f84b98df224f3af72d3
We want local source directory paths to take precedence over system
paths so system installed packages don't get used.
MozReview-Commit-ID: FIZ0RxqZj78
--HG--
extra : rebase_source : e8d484ea0c8a0bdb1df10375317f6b2c349b9162
The mozconfig detection logic has bitten us on many occasions in the
past. The following changes are made to tentatively improve the
situation:
- The API is modified such that autodetection of the mozconfig has
to be a conscious decision made by the caller, and not triggered
any time there is no mozconfig given, which could be a conscious
decision of the opposite.
- mozinfo.json now stores the actual mozconfig (or lack thereof) used
during configure.
--HG--
extra : rebase_source : c7a632afd414f25daf7bbe7e1a66c3736c26e039
Historically, a mozinfo for js standalone build has not been necessary,
but with the move towards a) having things work with mach and b)
buildconfig using the MozbuildObject.from_environment in next patch,
mozinfo becomes necessary (it's required for
MozbuildObject.from_environment to find the directory is an objdir).
Interestingly, hazard builds do both a js standalone build and a desktop
Firefox build at the same time, both of which are done with MOZCONFIG
set in the environment... with the Firefox mozconfig. The result of now
writing a mozinfo for js standalone builds is that in that case, they
end up with a reference to the mozconfig, which the build system then
reads, and finds a MOZ_OBJDIR, which then doesn't match the js
standalone build objdir. So for those builds, reset MOZCONFIG.
This allows to use True and False as values given to
set_config/set_define in moz.configure files, while postponing having to
deal with the long tail of things depending on the values from substs
and defines.
Ideally, everything would handle the bools just fine, but there are too
many things involved to deal with this right now: scripts using
buildconfig.{substs,defines}, scripts using ConfigEnvironment (e.g.
process_define_files.py), ConfigEnvironment itself, etc.
Now that MOZ_BUILD_APP is set to js when building js/src, we can
distinguish those builds with MOZ_BUILD_APP==js instead of BUILDING_JS.
Consequently, remove BUILDING_JS.
Now that MOZ_BUILD_APP is set to js when building js/src, we can
distinguish those builds with MOZ_BUILD_APP==js instead of BUILDING_JS.
Consequently, remove BUILDING_JS.
Ricky Stewart
Bogdan Tara
Dorel Luca
Mike Hommey
Narcis Beleuzu
Razvan Maries
Mitchell Hentges
Anthony Ramine
Mike Shal
Justin Wood
Justin Wood
Chris Manchester
Gregory Szorc
Sylvestre Ledru
Elias Probst