Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
As all versions of zlib >= 1.2.3.1 have a pkg-config file, and 1.2.3.1
is close to 14 years old, let's drop 1.2.3 and just use pkg-config, which
simplifies what we need to do dramatically.
Differential Revision: https://phabricator.services.mozilla.com/D77404
Now that there are only two uses of it, and there won't be more, remove
them and just use host_for_sub_configure and target_for_sub_configure
directly.
Differential Revision: https://phabricator.services.mozilla.com/D73411
When running e.g. check_symbols with extra flags like when checking
for vpx_codec_dec_init_ver when building against system libvpx, in some
configurations, the test can fail when the library flags (-l) appear
before the source file path.
The reason is that in some configurations, the compiler passes
--as-needed to the linker before both the flags and the object file
path, and the object file path is in the same position as the source
file path was. With --as-needed, -l flags are dropped if the library
wasn't needed for any of the linked code that appears *before* the flag.
So linking with `--as-needed -lfoo foo.o`, is equivalent to linking with
`foo.o` only in practice, while `--as-needed foo.o -lfoo` is equivalent
to `foo.o -lfoo`.
Differential Revision: https://phabricator.services.mozilla.com/D71456
This matches up to what we were doing before the Python 3 switch. Subprocess.Popen's default encoding doesn't necessarily match up to the system encoding as reported by mozbuild.util.system_encoding, and pre-Python 3.6 the constructor doesn't have `encoding` or `errors` parameters for no apparent reason. In the meantime, do the decoding manually.
Differential Revision: https://phabricator.services.mozilla.com/D71193
When running e.g. check_symbols with extra flags like when checking
for vpx_codec_dec_init_ver when building against system libvpx, in some
configurations, the test can fail when the library flags (-l) appear
before the source file path.
The reason is that in some configurations, the compiler passes
--as-needed to the linker before both the flags and the object file
path, and the object file path is in the same position as the source
file path was. With --as-needed, -l flags are dropped if the library
wasn't needed for any of the linked code that appears *before* the flag.
So linking with `--as-needed -lfoo foo.o`, is equivalent to linking with
`foo.o` only in practice, while `--as-needed foo.o -lfoo` is equivalent
to `foo.o -lfoo`.
Differential Revision: https://phabricator.services.mozilla.com/D71456
This also does a few remaining python 2 incompatible changes to
.configure files.
Differential Revision: https://phabricator.services.mozilla.com/D69538
--HG--
extra : moz-landing-system : lando
This gets rid of the last use of the 'which' module left in the tree. So not
only will this help 'configure' become a little more Python 3 compatible, but
we can now remove 'third_party/python/which'.
Differential Revision: https://phabricator.services.mozilla.com/D37427
--HG--
extra : moz-landing-system : lando
This gets rid of the last use of the 'which' module left in the tree. So not
only will this help 'configure' become a little more Python 3 compatible, but
we can now remove 'third_party/python/which'.
Differential Revision: https://phabricator.services.mozilla.com/D37427
--HG--
extra : moz-landing-system : lando
The configure sandbox has wrapped subprocess methods to add its own
encoded environment if none is provided, since bug 1520394. It only
makes sense that it normalizes the environment that comes in too,
avoiding caller in the configure sandbox to have to do it themselves.
OTOH, and while we're here, none of get_cmd_output, old_configure or the
sandbox were actually using the right encoding for this conversion, so
fix the configure sandbox to use the right one, and make it stop using
encode(), which does deep recursion that is not necessary here, and that
I'm trying to remove entirely.
Also while here, remove an unused import of encode().
Differential Revision: https://phabricator.services.mozilla.com/D42608
--HG--
extra : moz-landing-system : lando
Because most calling places in python configure don't actually want to
deal with encodings, although in practical terms they should, make
get_cmd_output handle it itself.
Places that explicitly do want bytes can keep using subprocess directly.
Differential Revision: https://phabricator.services.mozilla.com/D42604
--HG--
extra : moz-landing-system : lando
Bug 1520394 changed things such that the configure sandbox is using a
copy of os.environ. So when mozconfig injects environment changes, they
only affect the sandbox. Which means when the which module uses
os.environ to get PATH, it gets the original unmodified environment.
So instead of relying on the which module getting PATH itself, we feed
it with it. It's worth noting that the which module adds `.` on Windows,
but we don't copy this behavior, because in the context of configure,
it's actually not important (`.` would be the topobjdir).
Differential Revision: https://phabricator.services.mozilla.com/D17771
--HG--
extra : moz-landing-system : lando
Running rustc with an open fd 4 sometimes causes it to fail with an obscure
error when it tries to connect to a job server on that fd. Closing the fd
solves the problem.
close_fds does what we need, and is the default value in Python 3, but not
in Python 2.
Differential Revision: https://phabricator.services.mozilla.com/D8167
--HG--
extra : rebase_source : d94447019e3e698d7cf2c293b0a9b99769cf316a
Bug 1282866 removed the only configuration where it did something.
Since then, using it only leads to a configure error.
Differential Revision: https://phabricator.services.mozilla.com/D7691
--HG--
extra : moz-landing-system : lando
Currently, --enable-lto just implies --enable-linker=lld, which
essentially only works on Linux, and assumes one can't do lto with
anything other than lld. Which is not true. As a matter of fact, even
ld.bfd can do lto, as long as the gold plugin for llvm is available,
or when doing lto with GCC instead of clang.
Anyways, to allow more lto setups, we adapt the --enable-linker setup
to:
- work on macOS, which it currently doesn't, and add support for the mac
linker (ld64), which, unfortunately, doesn't have a clean way to be
detected, so work around that.
- default to lld if lto is enable, no linker was explicitly given, the
compiler is clang *and* the build target is not macOS.
--HG--
extra : rebase_source : 1dab2ad6230d18e7f4285943e76352e23b987d4e
LineIO currently attempts to convert bytes to unicode using the
system preferred encoding. This can obviously fail for some byte
sequences.
In many cases, the conversion is happening for purposes of logging.
In these cases, Unicode validation shouldn't be that important to
us.
So, this commit teaches LineIO to accept an argument defining its
Unicode error mode. We also teach relevant users of LineIO doing
logging to switch the default mode from "strict" to "replace" so
invalid Unicode byte sequences can be logged with the Unicode
replacement character.
During review, glandium pointed out that it may make better sense
to defer Unicode decoding to the logging layer instead of in LineIO.
I agree with this idea. But it can be implemented in another bug:
for now we should unbust the build system.
MozReview-Commit-ID: 7miuSDY8Xzv
--HG--
extra : rebase_source : 40d5d97d908d00b410b49d03729a34568e5bfb1a
Often you only want to evaluate a function if all its dependencies
are true. Expressing this in a "when" can be difficult. So let's
add a convenience decorator for it.
The existing code for @depends_if() was refactored to take an
evaluation function as its first argument. This prevents some
duplicate code and turns @depends_if() and @depends_all() into
one-liners.
MozReview-Commit-ID: Jbugvf0lioM
--HG--
extra : rebase_source : 177741b80ac4fbfb547d6b36a6f5777fe514d91a
extra : source : 0c2bc12f4ebe44428385745266d2fd158e0c3382
Often you only want to evaluate a function if all its dependencies
are true. Expressing this in a "when" can be difficult. So let's
add a convenience decorator for it.
The existing code for @depends_if() was refactored to take an
evaluation function as its first argument. This prevents some
duplicate code and turns @depends_if() and @depends_all() into
one-liners.
MozReview-Commit-ID: Jbugvf0lioM
--HG--
extra : rebase_source : 38b4af7b668830589126e8a83f5d5ab73914d07c
The values that we need to find in the registry can be inconsistent across
different installations, so we retrieve values from both views in our search
for a valid SDK. This also ensures this works for 32-bit and 64-bit python.
And include code to work around a bug on older Python versions.
MozReview-Commit-ID: 4pBnMQQJOGB
--HG--
extra : rebase_source : 6f7c5784230bd37b3496b9bb1781e8d342f741b4
And include code to work around a bug on older Python versions.
MozReview-Commit-ID: 4pBnMQQJOGB
--HG--
extra : rebase_source : 6f7c5784230bd37b3496b9bb1781e8d342f741b4
In bug 1296530, we made @depends take a when argument, it can now replace
all uses of @depends_when.
--HG--
extra : rebase_source : d090723fcbf3a56e06bd9c2defc3301cac04d8b0
Ricky Stewart
Bogdan Tara
Dorel Luca
Mike Hommey
Stefan Hindli
Csoregi Natalia
Cosmin Sabou
Andrew Halberstadt
Coroiu Cristina
Kris Maglione
Ivan Pozdeev
Gregory Szorc
Sylvestre Ledru
Sebastian Hengst
Bob Owen
Ryan VanderMeulen