Credit: mars for making the shell POSIX compliant
This embeds a blacklist of every mach command that needs to run with Python 2
directly in the mach driver itself. Initially this is every mach command. We
then use a bit of shell to determine whether the command being run needs Python
2 or 3.
While this approach may seem a bit hacky, it has several benefits:
1. No need to add complex machinery in mach's registration code.
2. No need to spawn two separate Python interpreters in the event a different
Python from the original interpreter is needed.
3. Perf impact is negligible.
4. New commands are Python 3 by default.
It is also only a temporary hack. Once all commands are running with Python 3,
we can revert back to the original mach driver.
Differential Revision: https://phabricator.services.mozilla.com/D36103
--HG--
extra : moz-landing-system : lando
common.gypi uses hard-coded absolute path for echo command, but this path
is invalid on Windows.
But this lines are unused, so we should remove it.
Differential Revision: https://phabricator.services.mozilla.com/D42779
--HG--
extra : moz-landing-system : lando
Also don't pass universal_newlines to check_cmd_output. That was added
to make python 3 happier when check_cmd_output still returned bytes, to
hint subprocess to return a unicode string, but now that
check_cmd_output does the decoding, that's not needed anymore.
Differential Revision: https://phabricator.services.mozilla.com/D43205
--HG--
extra : moz-landing-system : lando
The discrepancy of features used for syn between jsrust and gkrust
triggered syn and its reverse dependencies up to cranelift to be
built once for jsrust and once for gkrust, while they are time consuming
to build.
Differential Revision: https://phabricator.services.mozilla.com/D43209
--HG--
extra : moz-landing-system : lando
This change looks a lot more complicated than it actually is.
The motivating issue is that on x86-64 android, the NDK uses gold as its
default linker, and on this platform gold has some bugs in how it handles
`.note.*` sections that lead to `strip` and `objcopy` produce invalid
binaries...or at least invalid binaries according to `elfhack`. When
`elfhack` complains, the build stops, which is suboptimal. Instead, let's
use bfd `ld`, which doesn't have these problems.
Except that when we make the change to provide `--enable-linker` with a
platform-specific default, the lint configure tests complain that the
provided help string is not using the proper format. So we tack on the
sekret `{|}` string to satisfy the linter.
Differential Revision: https://phabricator.services.mozilla.com/D43098
--HG--
extra : moz-landing-system : lando
For some reason, cargo incremental doesn't work as well as it should,
and doesn't perform as well as sccache does. So when building with
sccache, disable cargo incremental. This brought a no-change clobber
build with 100% cache from 3:50 to 2:05 on a beefy AWS instance I was
testing this with.
Differential Revision: https://phabricator.services.mozilla.com/D43188
--HG--
extra : moz-landing-system : lando
Bug 1575135 changed check_cmd_output to return unicode strings, but a
couple places were already trying to do their own decoding, which now
can fail. Remove those.
Interesting the decoding was previously broken on Windows, this
actually fixes it (the output of hg config is not actually utf-8 on
Windows).
Differential Revision: https://phabricator.services.mozilla.com/D43044
--HG--
extra : moz-landing-system : lando
The configure sandbox has wrapped subprocess methods to add its own
encoded environment if none is provided, since bug 1520394. It only
makes sense that it normalizes the environment that comes in too,
avoiding caller in the configure sandbox to have to do it themselves.
OTOH, and while we're here, none of get_cmd_output, old_configure or the
sandbox were actually using the right encoding for this conversion, so
fix the configure sandbox to use the right one, and make it stop using
encode(), which does deep recursion that is not necessary here, and that
I'm trying to remove entirely.
Also while here, remove an unused import of encode().
Differential Revision: https://phabricator.services.mozilla.com/D42608
--HG--
extra : moz-landing-system : lando
There are a few problems with the strategy currently used to find the
rust target. For example, we don't find a target for arm freebsd, and we
pick the wrong target for armel linux. Both are related to how things
currently work when multiple targets have the same (cpu, endianness,
os).
So, to derive the rust target, we now use a more fine-grained approach.
Differential Revision: https://phabricator.services.mozilla.com/D41481
--HG--
extra : moz-landing-system : lando
Currently, we have no real visibility on the time spent after the build
finished, despite the fact that a large chunk is actually happening via
make check (although thankfully more and more of it is moving out to
separate tasks).
Also, the mozharness machinery for make check dates from when we were
running in buildbot and takes care of turning builds orange instead of
red in case of failure, etc. which doesn't do anything useful anymore.
Differential Revision: https://phabricator.services.mozilla.com/D42806
--HG--
extra : moz-landing-system : lando
The automation tiers all have a dependency:
automation/%: automation-start/%
The dependency runs the command for BUILDSTATUS, making the resource
monitor aware of what tiers start. Unfortunately, Make decides to run
all these automation-start/% targets way before starting the
automation/% targets, because of how the interdependencies are laid out.
Use a slightly better way, albeit still biased, making the
automation-start/% target for a tier depend on the dependencies, which
at least ensures that the resource monitor doesn't see a tier starting
before its dependencies are finished.
Differential Revision: https://phabricator.services.mozilla.com/D42805
--HG--
extra : moz-landing-system : lando
This warning, new in clang 9, is noisy about patterns like:
```
unsigned long x = ...
if (x > 4294967295)
```
The condition is always false on 32-bit and LLP64 systems, but on LP64 it's a valid test.
Differential Revision: https://phabricator.services.mozilla.com/D42346
--HG--
extra : moz-landing-system : lando
Disable -ftrivial-auto-var-init for DllBLocklistWin.cpp with clang-cl because the file's interceptions happen so early in the main process that the loader hasn't yet resolved the import of memset (used by -ftrivial-auto-var-init) from vcruntime140.dll.
Disable -ftrivial-auto-var-init on Linux32 because it causes some xpcshell test failures.
Differential Revision: https://phabricator.services.mozilla.com/D42273
--HG--
extra : moz-landing-system : lando
As a consequence, we can replace the encoded_open function that did the
same in an opt-in manner.
Differential Revision: https://phabricator.services.mozilla.com/D42605
--HG--
extra : moz-landing-system : lando
Because most calling places in python configure don't actually want to
deal with encodings, although in practical terms they should, make
get_cmd_output handle it itself.
Places that explicitly do want bytes can keep using subprocess directly.
Differential Revision: https://phabricator.services.mozilla.com/D42604
--HG--
extra : moz-landing-system : lando
This is not something that happens under normal circumstances, but it
can happen when you go fancy and run multiple configures in parallel
with different objdirs, and old-configure doesn't exist in the first
place ; then one configure may overwrite old-configure while another is
starting to execute it, resulting in the latter nor executing
old-configure completely.
Differential Revision: https://phabricator.services.mozilla.com/D42428
--HG--
extra : moz-landing-system : lando
This definition is now unnecessary, given that the source code is
fetched for us from elsewhere.
Depends on D41368
Differential Revision: https://phabricator.services.mozilla.com/D41369
--HG--
extra : moz-landing-system : lando
Otherwise we wind up with clownshoes like:
```
froydnj@hawkeye:/opt/build/froydnj/tmp/clang$ ls -l lib/libstdc++.so*
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so.6
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so.6.0.24
```
and have duplicate copies of shared libraries in our toolchain packages,
which are not exactly small.
Differential Revision: https://phabricator.services.mozilla.com/D41368
--HG--
extra : moz-landing-system : lando
In some libstdc++ these appear in the inheritance hierarchy of __atomic_base, which is already whitelisted.
Differential Revision: https://phabricator.services.mozilla.com/D42348
--HG--
extra : moz-landing-system : lando
Andrew Halberstadt
Emilio Cobos Álvarez
Chris Manchester
Makoto Kato
Mihai Alexandru Michis
Mike Hommey
Tom Ritter
Sylvestre Ledru
Nicholas Nethercote
Razvan Maries
Nathan Froyd
Dustin J. Mitchell
Bogdan Tara
David Major
Daniel Varga
Chris Peterson
Cameron Kaiser