It's nicer to have everything in one place, and because we support
clang-cl, we can have a single definition for the error flag too.
Differential Revision: https://phabricator.services.mozilla.com/D45705
--HG--
extra : moz-landing-system : lando
The option was removed entirely several months ago, and this bit was missed.
Differential Revision: https://phabricator.services.mozilla.com/D46296
--HG--
extra : moz-landing-system : lando
Getting rid of unneeded code and having fewer global defines are both
worthy things.
Differential Revision: https://phabricator.services.mozilla.com/D45726
--HG--
extra : moz-landing-system : lando
A previous patch defined `_LIBCPP_HIDE_FROM_ABI` to the empty string.
This definition worked for normal (opt/debug) builds, but
PGO-instrumented builds fell over with some linker errors. A closer
look at the definition of `_LIBCPP_HIDE_FROM_ABI`:
```
#ifndef _LIBCPP_HIDE_FROM_ABI_PER_TU
# ifndef _LIBCPP_HIDE_FROM_ABI_PER_TU_BY_DEFAULT
# define _LIBCPP_HIDE_FROM_ABI_PER_TU 0
# else
# define _LIBCPP_HIDE_FROM_ABI_PER_TU 1
# endif
#endif
#ifndef _LIBCPP_HIDE_FROM_ABI
# if _LIBCPP_HIDE_FROM_ABI_PER_TU
# define _LIBCPP_HIDE_FROM_ABI _LIBCPP_HIDDEN _LIBCPP_INTERNAL_LINKAGE
# else
# define _LIBCPP_HIDE_FROM_ABI _LIBCPP_HIDDEN _LIBCPP_EXCLUDE_FROM_EXPLICIT_INSTANTIATION
# endif
#endif
```
says that there are two macros involved there: one to set hidden visibility
(which we don't want), and another to exclude the function definition
from explicit instantiation. The semantics of this are not obvious (the
clang documentation only documents this attribute in terms of C++
templates; the particular case we're interested in sets the attribute on
normal function definitions). But roughly, what this does is that it
forces the compiler to see that an equivalent function definition is
provided external to the compilation unit, so it doesn't have to keep
the annotated function definition around.
Differential Revision: https://phabricator.services.mozilla.com/D45073
--HG--
extra : moz-landing-system : lando
If the run task generates bad profile data, the merge step in the
profile-use task will fail. However, retrying the profile-use task
doesn't fix the problem, and there isn't a straightforward way to retry
the run task in this situation. Instead we can add a clang toolchain to
all the run tasks, and perform the merge there.
This means the output from the run task will always be a successfully
merged file called 'merged.profdata', and we no longer need to perform
the merge as part of the profile-use build as a GENERATED_FILES step.
Depends on D45262
Differential Revision: https://phabricator.services.mozilla.com/D45263
--HG--
extra : moz-landing-system : lando
This is needed in the following patch, but the diff is more clear with
this split out.
Differential Revision: https://phabricator.services.mozilla.com/D45262
--HG--
extra : moz-landing-system : lando
One less thing for configurations to vary is a good thing. This change
also means that when we do NDK bumps that introduce some kind of weird
incompatible change, configuration with a newly-pulled tree and an old
NDK halts at configure, rather than giving weird build errors.
Differential Revision: https://phabricator.services.mozilla.com/D44705
--HG--
extra : moz-landing-system : lando
We define some libc++-internal macros to avoid conflicts between the way
we do symbol visibility and the way libc++ expects visibility to work.
Newer NDK versions use a newer libc++ which changes the way the macro we
were using works. To enable building with newer libc++ versions, let's
also define macros appropriate to those versions as well.
Differential Revision: https://phabricator.services.mozilla.com/D44510
--HG--
extra : moz-landing-system : lando
We have an `ndk_version_major` and an `ndk_version_minor` that have a
lot of duplicated code in them. We could factor them into a single
function, but it seems better to just pull their logic into `ndk_version`
directly, rearranging that function to be IMHO more understandable.
Differential Revision: https://phabricator.services.mozilla.com/D44506
--HG--
extra : moz-landing-system : lando
Also don't pass universal_newlines to check_cmd_output. That was added
to make python 3 happier when check_cmd_output still returned bytes, to
hint subprocess to return a unicode string, but now that
check_cmd_output does the decoding, that's not needed anymore.
Differential Revision: https://phabricator.services.mozilla.com/D43205
--HG--
extra : moz-landing-system : lando
This change looks a lot more complicated than it actually is.
The motivating issue is that on x86-64 android, the NDK uses gold as its
default linker, and on this platform gold has some bugs in how it handles
`.note.*` sections that lead to `strip` and `objcopy` produce invalid
binaries...or at least invalid binaries according to `elfhack`. When
`elfhack` complains, the build stops, which is suboptimal. Instead, let's
use bfd `ld`, which doesn't have these problems.
Except that when we make the change to provide `--enable-linker` with a
platform-specific default, the lint configure tests complain that the
provided help string is not using the proper format. So we tack on the
sekret `{|}` string to satisfy the linter.
Differential Revision: https://phabricator.services.mozilla.com/D43098
--HG--
extra : moz-landing-system : lando
For some reason, cargo incremental doesn't work as well as it should,
and doesn't perform as well as sccache does. So when building with
sccache, disable cargo incremental. This brought a no-change clobber
build with 100% cache from 3:50 to 2:05 on a beefy AWS instance I was
testing this with.
Differential Revision: https://phabricator.services.mozilla.com/D43188
--HG--
extra : moz-landing-system : lando
Bug 1575135 changed check_cmd_output to return unicode strings, but a
couple places were already trying to do their own decoding, which now
can fail. Remove those.
Interesting the decoding was previously broken on Windows, this
actually fixes it (the output of hg config is not actually utf-8 on
Windows).
Differential Revision: https://phabricator.services.mozilla.com/D43044
--HG--
extra : moz-landing-system : lando
The configure sandbox has wrapped subprocess methods to add its own
encoded environment if none is provided, since bug 1520394. It only
makes sense that it normalizes the environment that comes in too,
avoiding caller in the configure sandbox to have to do it themselves.
OTOH, and while we're here, none of get_cmd_output, old_configure or the
sandbox were actually using the right encoding for this conversion, so
fix the configure sandbox to use the right one, and make it stop using
encode(), which does deep recursion that is not necessary here, and that
I'm trying to remove entirely.
Also while here, remove an unused import of encode().
Differential Revision: https://phabricator.services.mozilla.com/D42608
--HG--
extra : moz-landing-system : lando
There are a few problems with the strategy currently used to find the
rust target. For example, we don't find a target for arm freebsd, and we
pick the wrong target for armel linux. Both are related to how things
currently work when multiple targets have the same (cpu, endianness,
os).
So, to derive the rust target, we now use a more fine-grained approach.
Differential Revision: https://phabricator.services.mozilla.com/D41481
--HG--
extra : moz-landing-system : lando
This warning, new in clang 9, is noisy about patterns like:
```
unsigned long x = ...
if (x > 4294967295)
```
The condition is always false on 32-bit and LLP64 systems, but on LP64 it's a valid test.
Differential Revision: https://phabricator.services.mozilla.com/D42346
--HG--
extra : moz-landing-system : lando
Disable -ftrivial-auto-var-init for DllBLocklistWin.cpp with clang-cl because the file's interceptions happen so early in the main process that the loader hasn't yet resolved the import of memset (used by -ftrivial-auto-var-init) from vcruntime140.dll.
Disable -ftrivial-auto-var-init on Linux32 because it causes some xpcshell test failures.
Differential Revision: https://phabricator.services.mozilla.com/D42273
--HG--
extra : moz-landing-system : lando
As a consequence, we can replace the encoded_open function that did the
same in an opt-in manner.
Differential Revision: https://phabricator.services.mozilla.com/D42605
--HG--
extra : moz-landing-system : lando
Because most calling places in python configure don't actually want to
deal with encodings, although in practical terms they should, make
get_cmd_output handle it itself.
Places that explicitly do want bytes can keep using subprocess directly.
Differential Revision: https://phabricator.services.mozilla.com/D42604
--HG--
extra : moz-landing-system : lando
This is not something that happens under normal circumstances, but it
can happen when you go fancy and run multiple configures in parallel
with different objdirs, and old-configure doesn't exist in the first
place ; then one configure may overwrite old-configure while another is
starting to execute it, resulting in the latter nor executing
old-configure completely.
Differential Revision: https://phabricator.services.mozilla.com/D42428
--HG--
extra : moz-landing-system : lando
Just like C++14 sized deallocation support, we don't want to support
this. We shouldn't be using `new` on over-aligned types anyway.
Differential Revision: https://phabricator.services.mozilla.com/D41819
--HG--
extra : moz-landing-system : lando
Some recent changes to how we set cross-language LTO for Windows
resulted in compilation-time decreases and small performance regressions
on a few benchmarks. The changes intended to remove explicit enablement
of cross-language LTO for all builds, but rely on shippable builds being
built with PGO and moz.configure's clever defaulting of cross-language
LTO for PGO'd builds on Windows, which would then enable cross-language
LTO for only shippable builds.
Obviously something went wrong with those changes.
The problem was our defaulting wasn't visible to moz.configure's logic
for how to pass command-line options to the JS subconfigure. We set the
value (`cross`) after the value for `--enable-lto` has been determined,
and the default value is off (that is, `--disable-lto`). Since
moz.configure is very thorough in passing configure options down into
JS, it dutifully looked at what the default value of `--enable-lto` was
supposed to be, and passed `--disable-lto` to JS's configure.
There's some evidence that we knew our defaulting was a little sketchy:
we'd only attempt cross-language LTO when we were performing the PGO use
phase, and only if the value of `--enable-lto` wasn't explicitly passed.
Which was a fine idea--you don't want to override what the user was
trying to do--but in the case of JS backfired on us: the value *was*
coming from the explicitly-passed command-line option of
`--disable-lto`. So JS didn't enable any kind of LTO, with attendant
consequences.
This problem *didn't* happen before the aforementioned change because we
were explicitly specifying that `--enable-lto=cross` should be passed in
the mozconfig, which ensured that the correct setting was passed into
JS. We were just setting `--enable-lto=cross` for *all* builds, which
was less than desirable.
The easiest way to fix all this is simply to put the
`--enable-lto=cross` setting in the Windows mozconfigs, conditional on
`MOZ_PGO_PROFILE_USE`. That placement captures the intent of the
previous attempt at defaulting, but without the troubles described
above: the option explicitly appears on the command line, and
moz.configure will correctly pass it through to the JS subconfigure.
This also makes our Windows configuration closer to our Linux
configuration (the Linux configuration enables cross-language LTO for
both PGO phases, which is arguably a bug).
Differential Revision: https://phabricator.services.mozilla.com/D41080
--HG--
extra : moz-landing-system : lando
This patch makes BuildEnvironmentNotFoundException a subclass of AttributeError as well, because hasattr in
python3 no longer catches all tracebacks but only AttributeError, and we use both hasattr and
BuildEnvironmentNotFoundException to guard against a handful of buildconfig variables in a few places
where it is OK to not have a buildenvironment.
We also use universal_newlines in real_host in init.configure (since I found
that fix while working on the AttributeError one) so that we get the right string type back from the process call
Lastly this patch also uses BytesIO for calling into a ReducedConfigureSandbox as its stdout and stderr pipes,
This ensures that related code handling the sandbox doesn't complain about getbuffer() missing in StringIO in py3.
Differential Revision: https://phabricator.services.mozilla.com/D36605
--HG--
extra : moz-landing-system : lando
By default, the linker chooses a "generic" 32-bit CPU to optimize for,
and LLVM's "generic" 32-bit CPU model doesn't include some features that
are helpful for performance on microbenchmarks. We explicitly specify a
CPU model to ensure the model we want is selected.
On x86-64, we explicitly force a generically good processor model, even
though the automatically selected one didn't seem to hurt benchmarks.
Differential Revision: https://phabricator.services.mozilla.com/D40479
--HG--
extra : moz-landing-system : lando
Chris Manchester
Cosmin Sabou
Marco Castelluccio
Dorel Luca
Nathan Froyd
Mike Shal
Sylvestre Ledru
Emilio Cobos Álvarez
Andrew Halberstadt
Mike Hommey
Nicholas Nethercote
Bogdan Tara
David Major
Chris Peterson
Cameron Kaiser
Justin Wood