Now that we ship builds using clang on all platforms, pick it during
configure. It is still possible to opt-in to building other compilers by
setting CC/CXX (or even only CC) to the desired compiler.
Depends on D5829
Differential Revision: https://phabricator.services.mozilla.com/D5637
--HG--
extra : moz-landing-system : lando
Before this change, we'd derive a host compiler and handle things as if
HOST_CC/HOST_CXX had been passed. With this change, we change the list
of compilers that are tested with check_prog instead.
Depends on D5828
Differential Revision: https://phabricator.services.mozilla.com/D5829
--HG--
extra : moz-landing-system : lando
Currently, when building for Android x86-64 on Linux x86-64, we drop the
'--target' flag, which causes the build to fail. This patch adds a check
for OS mismatch, so we keep the '--target' flag in this situation.
Differential Revision: https://phabricator.services.mozilla.com/D4483
It's nicer to have all that logic in one place, and to be able to common
up the Unix-y flags setting. The Makefile constructs in string values
is gross, but it's no worse than we had before.
Add extra preferences for the profileserver script on-top of
the ones as inheritted from unittests.
--HG--
extra : rebase_source : 02d3297134e82c2425fc4c6c939cc5d0cc4a2c13
extra : histedit_source : 07f699b0ff8bec2a6eda7705a0fa13fb3b352f98
Currently mozconfig.cache overrides a few build options for sccache.
This patch moves them into toolchain.configure so that the build system
will set them properly when sccache is in use. Additionally,
{CC,CXX}_WRAPPER are set in config.mk, so just avoid setting them when
sccache is in use.
Now that Linux PGO builds also do LTO and all the Linux builds use
clang, there's not much use for separate LTO builds.
Differential Revision: https://phabricator.services.mozilla.com/D5632
Currently, sccache doesn't work at all with clang-cl (PGO or not, bug
1476604), and newer versions of sccache that do the right thing for PGO
flags with clang have other problems (see bug 1484897).
The benefit of sccache on PGO+LTO builds is however limited (the
profile-use half of the build doesn't benefit at all, and since it does
LTO as well, it's the most time-consuming part of the build), so in
order not to further block switching Linux builds to clang, let's just
disable sccache on PGO builds for now (the earlier we can switch to
clang, the more time it leaves to catch any problem that causes).
Differential Revision: https://phabricator.services.mozilla.com/D5509
--HG--
extra : moz-landing-system : lando
Nothing is using the xpt module anymore, which means we can remove it,
as well as the runtests.py script that runs its test, and the
integration of those tests in the build system.
Depends on D5221
Differential Revision: https://phabricator.services.mozilla.com/D5223
--HG--
extra : moz-landing-system : lando
Not waiting for rc3 or final from llvm because the issue is pretty severe
Differential Revision: https://phabricator.services.mozilla.com/D5124
--HG--
extra : moz-landing-system : lando
Last attempt, a few years ago, blatantly failed because nautilus (the
GNOME file manager) can't start PIE executables, which look like shared
libraries, and that it thus considers not being executables.
Downstreams don't actually have the problem, because users won't be
launching Firefox from a file manager, but for mozilla.org builds, it is
a problem because users would download, then extract, and then likely
try to run the Firefox executable from a file manager.
So for mozilla.org builds, we still need to find a way around the
nautilus problem.
A .desktop file could be a solution, but .desktop files have not
actually been designed for this use case, which leads to:
- having to use an awful one-liner shell wrapper to derive the path
to the executable from that of the .desktop file,
- not even being able to associate an icon,
- the .desktop file not being copiable to a location where .desktop
files would normally go, because it would then fail to find the
executable.
Another possibility is to go back to using a shell wrapper, but that's
not entirely appealing.
What we chose here is similar, where we have a small `firefox` wrapper
that launches the real `firefox-bin` (which is still leftover from those
old times where we had a shell wrapper, for reasons).
The small `firefox` wrapper is a minimalist C executable that just
finds the path to the `firefox-bin` executable and executes it with the
same args it was called with. The wrapper is only enabled when the
MOZ_NO_PIE_COMPAT environment variable is set, which we only take into
account on Linux. The variable is only really meant to be used for
mozilla.org builds, for the nautilus problem. Downstreams will just pick
the default, which is changed to build PIE.
On other platforms, PIE was already enabled by default, so we just
remove the --enable-pie configure flag.
Differential Revision: https://phabricator.services.mozilla.com/D5109
When building executables as PIE, and because we use -Bsymbolic, which
symbols are exported from an executable varies based on the libraries
it's directly linked against, to fulfil their symbol needs. So when a
library depends on e.g. ASAN runtime symbols, and the linker finds that,
it will keep those ASAN symbols in the executable for the library. And
drop the other, unused symbols.
But when the executable then dlopen()s a library (e.g. shlibsign loading
libfreebl) that uses another set of ASAN symbols, including symbols that
none of the direct dependencies of the executable need, dlopen() fails
because of the missing symbols.
It's not currently an apparent problem because we don't enable PIE, and
we build Gecko executables with -rdynamic already (for mozjemalloc). But
we don't build non-Gecko executables this way (like shlibsign).
Differential Revision: https://phabricator.services.mozilla.com/D5108
Bug 1487330 made mozconfig.asan use mozconfig.linux, but while that
worked out fine, it turns out that we do have mac builds using
e.g. mozconfig.asan, and that those can break when doing linux-specific
changes to mozconfig.linux.
--HG--
rename : build/unix/mozconfig.linux => build/unix/mozconfig.unix
As per bug 1487622, annotations have side effects on codegen, one of
which changes a leak signature that the wpt harness doesn't match
against its whitelist anymore.
Interestingly, while looking at the issue, I found multiple moz_*
annotations that weren't actually looked for by the clang plugin,
making them useless. Even more interestingly, removing them changes
codegen in such a way that the leak signature mentioned above is not
altered anymore.
Differential Revision: https://phabricator.services.mozilla.com/D5000
As described in c2b5cf7bde83, it is still preferable to build with BFD
ld when doing clang LTO, and one of the reasons we defaulted to lld in
the first place is that we didn't have the LLVM gold plugin on
automation, which, as of bug 1488307, we now have.
Differential Revision: https://phabricator.services.mozilla.com/D4987
At least, when that combination doesn't work. This requires passing the
LTO flags to the compiler invocations for the test, which in turn
requires actually calling the linker directly on Windows, instead of
directly through the compiler, as when actually building.
Differential Revision: https://phabricator.services.mozilla.com/D4739
--HG--
extra : moz-landing-system : lando
As much as it's tempting to use LLD for LTO, it still causes some
subtle problems with the build, and it's still better to keep using
BFD ld for the time being. Doing so requires the gold plugin, which
only requires to pass cmake the directory where the binutils headers
are, and they are part of the gcc toolchain headers.
Differential Revision: https://phabricator.services.mozilla.com/D4896
Back before expandlibs was removed in bug 1429875, not using list files
was supported, but nowadays, it just leads to errors like the following:
*** No rule to make target 'libfake_so.list', needed by 'libfake.so'. Stop.
We might as well error out during configure, rather than much later
during the build.
Differential Revision: https://phabricator.services.mozilla.com/D4738
All the Linux builds using GCC uses the binutils bundled with GCC. This
gives us some leeway to update the binutils used for clang builds (using
the binutils toolchain as of bug 1486998) separately.
Since we only ship builds using GCC, we're more free to upgrade
binutils for clang builds, without worrying about the next merge.
This upgrades to the last released version of binutils, and applies the
patch from https://sourceware.org/bugzilla/show_bug.cgi?id=23591 on top,
so that asan fuzzing builds don't fail.
The GPG key used to sign the upstream tarball is unfortunately not
connected to the web of trust. I verified the contents matched what's in
the Debian archive (which has a different tarball, because some files
are removed/modified in Debian for license reasons ; there were no
differences besides those).
Differential Revision: https://phabricator.services.mozilla.com/D4748
--HG--
extra : moz-landing-system : lando
Last time it was updated is bug 1436208, and the crashes we patched it
for back then has been fixed upstream a few months later.
For some reason, they renamed the executable from llvm-dsymutil to
dsymutil.
Differential Revision: https://phabricator.services.mozilla.com/D4741
--HG--
extra : moz-landing-system : lando
Those mozconfigs do things in common that are already in
mozconfig.linux. Let's just use that.
This will make the build have the binutils toolchain in PATH rather than
passing it to CC/CXX, which is better because it also makes the build
use tools such as ar, ranlib, objcopy from the binutils toolchain,
rather than the old system ones.
Differential Revision: https://phabricator.services.mozilla.com/D4649
Back before expandlibs was removed in bug 1429875, not using list files
was supported, but nowadays, it just leads to errors like the following:
*** No rule to make target 'libfake_so.list', needed by 'libfake.so'. Stop.
We might as well error out during configure, rather than much later
during the build.
Differential Revision: https://phabricator.services.mozilla.com/D4738
As much as it's tempting to use LLD for LTO, it still causes some
subtle problems with the build, and it's still better to keep using
BFD ld for the time being. Doing so requires the gold plugin, which
only requires to pass cmake the directory where the binutils headers
are, and they are part of the gcc toolchain headers.
Differential Revision: https://phabricator.services.mozilla.com/D4896
Those mozconfigs do things in common that are already in
mozconfig.linux. Let's just use that.
This will make the build have the binutils toolchain in PATH rather than
passing it to CC/CXX, which is better because it also makes the build
use tools such as ar, ranlib, objcopy from the binutils toolchain,
rather than the old system ones.
Differential Revision: https://phabricator.services.mozilla.com/D4649
With patches from other bugs in place to use the right C compiler and cflags,
we can enable geckodriver on cross-compiles for macOS.
MozReview-Commit-ID: 5wqBiA6UCf
The telemetry gathering code is currently split in two places, so move it
all to a common place. Followup patches will rewrite most of this code.
--HG--
extra : rebase_source : afd14940f7175f7ca730a3b90ad770f143bcb6c7
The x86 linux builds originally were performed in x86 build environment.
That was a long time ago. Since then, they moved to x86-64 hosts with
x86 headers and libraries. But for reasons that might have to do with
the build system not really supporting cross-compilation nicely back
then, the build is still explicitly passing both --host and --target,
making those builds non-cross builds.
Since the toolchains used to build are for x86-64, the fact that --host
is for x86 actually prevents enabling the clang plugin, so we need to
turn these builds in actual cross builds if we want to enable the clang
plugin.
The x86 linux builds originally were performed in x86 build environment.
That was a long time ago. Since then, they moved to x86-64 hosts with
x86 headers and libraries. But for reasons that might have to do with
the build system not really supporting cross-compilation nicely back
then, the build is still explicitly passing both --host and --target,
making those builds non-cross builds.
Since the toolchains used to build are for x86-64, the fact that --host
is for x86 actually prevents enabling the clang plugin, so we need to
turn these builds in actual cross builds if we want to enable the clang
plugin.
The GNU_AS check in old-configure depended on running with the value
of $AS before it gets reset to just be the C compiler, which breaks when
we move setting AS into moz.configure.
This patch moves the GNU_AS check to toolchain.configure and changes it
so that it works when the assembler is the C compiler. We do have to
fix things slightly for clang, because the previous check was
succeeding, but not because of clang: it was detecting the presence of
"GNU" in the output for GNU ld/gold and a message about the GNU GPL.
This is a fairly straightforward port of the AS tool checks from old-configure
to toolchain.configure. AS is a little quirky in that we currently do a
normal-looking check for it, but then override that value to be the C compiler
for non-Windows builds, and ml[64]/armasm64 for Windows builds.
After migrating those checks, the only things left in the MOZ_DEFAULT_COMPILER
macro in compiler-opts.m4 were some unused bits, so I removed them:
* Setting of CPP/CXXCPP, which are set in toolchain.configure now
* Setting HOST_LDFLAGS to empty, which doesn't seem particularly useful.
There was also a quirky old test that the assembler was ml[64] when js-ctypes
is enabled that I removed, I don't think it provides any value since this
patch will ensure that we're using the right assembler for Windows builds.
This setup seems to work well enough to enable me to link
HOST_SIMPLE_PROGRAMS with an AArch64-cross setup. Necessary library
paths are passed to the linker via -LIBPATH and HOST_LDFLAGS rather than
letting MSVC fish them out of the environment. The change to
HOST_SIMPLE_PROGRAMS to pass HOST_LDFLAGS was necessary for this to
work, in addition to the HOST_LINKER changes.
-Wtautological-overlap-compare is an opt-in warning added in clang 3.5. It warns about overlapping comparisons that are always true or false, such as:
if (x > 4 || x < 10) {} // warning! always true
int b = x < 2 && x > 5; // warning! always false
return x > 4 || x < 10; // warning! always true
https://clang.llvm.org/docs/DiagnosticsReference.html#wtautological-overlap-compare
There are currently no -Wtautological-overlap-compare warnings in mozilla-central.
Differential Revision: https://phabricator.services.mozilla.com/D3477
--HG--
extra : rebase_source : e6aa211a338633e5fb7507b3a8f341709d72d6c9
This opt-in warning catches bugs where a constructor modifies a constructor parameter that shadows member variable name. The code probably intended to change the member variable value, not the paramter. There are currently no -Wshadow-field-in-constructor warnings in mozilla-central.
https://clang.llvm.org/docs/DiagnosticsReference.html#wshadow-field-in-constructor-modified
Differential Revision: https://phabricator.services.mozilla.com/D3822
--HG--
extra : rebase_source : 604ff90e036d536955db41351ee9ee97e8424d92
extra : intermediate-source : 712f538afc160f4efae2383130727edbd1dd70f6
extra : source : 300fff9a58515c7cac0647afc9ccc0e69bc8c5cb
-Wfloat-overflow-conversion detects when a constant floating point value is converted to an integer type and will overflow the target type.
https://clang.llvm.org/docs/DiagnosticsReference.html#wfloat-overflow-conversion
-Wfloat-zero-conversion detects when a non-zero floating point value is converted to a zero integer value.
https://clang.llvm.org/docs/DiagnosticsReference.html#wfloat-zero-conversion
There are currently no -Wfloat-overlap-conversion warnings in mozilla-central. There is one -Wfloat-zero-conversion warning in a webrtc test. It doesn't block enabling this check because the webrtc tests are not compiled with warnings-as-errors.
media/webrtc/trunk/webrtc/modules/audio_coding/audio_network_adaptor/frame_length_controller_unittest.cc:255:54 [-Wfloat-zero-conversion] implicit conversion from 'const float' to 'int' changes non-zero value from 0.045000002 to 0
We can't enable all -Wfloat-conversion warnings (for any implicit conversion of a floating-point number into an integer) because there are currently over 1400 warnings. I spot checked a few of these -Wfloat-conversion warnings. I didn't find any obvious bugs, but there is some suspicious code, such as implicit conversions of floats to bools.
Differential Revision: https://phabricator.services.mozilla.com/D3476
--HG--
extra : rebase_source : 589be9e593749c7000b1f89ca155e4f95a487f8f
Warn about C++ constructs whose meaning change in C++2a.
https://clang.llvm.org/docs/DiagnosticsReference.html#wc-2a-compat
So far the only -Wc++2a-compat check that I know of is for valid pre-C++2a code that inadvertently parses as C++2a's new <=> "spaceship" comparison operator. `f<&A::operator<=>();` is an example of a warning reported for a real project on GitHub. That code can be rewritten as `f< &operator<= >();`.
There are currently no -Wc++2a-compat warnings in mozilla-central.
Differential Revision: https://phabricator.services.mozilla.com/D3478
--HG--
extra : rebase_source : 4ec8147ad3fe2d1ef7c543d100c660a000385786
Our version check is removed as well, as the current required rustc version for
Firefox includes all features required to build with tup.
Differential Revision: https://phabricator.services.mozilla.com/D3847
--HG--
extra : moz-landing-system : lando
When valgrind prints out backtraces, it prints raw addresses and symbol
names, but that doesn't help find the exact code that caused the errors,
because we don't know where the libraries are loaded.
With --sym-offsets=yes, it adds the offset from the symbol, which allows
to find the relevant code in the binary.
For some reason, clang 6 crashes with a stack overflow on PGO + LTO on
Linux 64 bits. Clang 7 doesn't, but has other problems.
After some bisecting, I found the following:
- r322684 is the first revision that is broken on the release_60 branch.
- that revision is a cherry pick of r322313 from trunk, which is
similarly broken.
- trunk was fixed by r322325, which, funnily enough, predates when
r322313 was cherry-picked.
While the change from r322325 is relatively large, mixing multiple
different changes in a single commit, there also haven't been
significant changes to the same file on trunk since (one macro name
change, one documentation change, and a change related to debug info),
which would tend to indicate the change is not going to break anything,
or at least not more than upgrading to clang 7 would.
The exact part that fixes the issue could probably be found in this
large commit, but I didn't feel like digging into it further considering
the above.
In order to support operator==() for tagged enum, we have to bump the version to
0.6.2.
Differential Revision: https://phabricator.services.mozilla.com/D3932
--HG--
extra : moz-landing-system : lando
yasm doesn't support aarch64, and trying to use GNU as with an MSVC
build seems like sadness waiting to happen. Instead, we'll generate our
own assembly file that armasm64 will accept.
This change switches most CI builds to clang, with a few exceptions:
- valgrind builds, until bug 1481670 is figured out.
- PGO and nightly builds, until that's fully tested.
- coverage builds, per bug 1471339 comment 17.
- base toolchain builds, to keep some builds on GCC even when we're
fully switched to clang.
- any build that doesn't use build/unix/mozconfig.linux (e.g. probably
all those driven by autospider.py, maybe others).
This change adds a voluptuous schema for build system telemetry, replacing
the existing json schema file. Using voluptuous will make it easier to work
with the schema from Python code in the build system. A future commit will
use a Python module to provide a mach command to convert the voluptuous
schema to json schema format for consumption by other systems.
Differential Revision: https://phabricator.services.mozilla.com/D2840
--HG--
extra : rebase_source : 067995385334d1dbc123f2db4245ef4e69d076c3
In some rare cases, it is possible for one of the eh_frame sections'
original address to be larger than the address of the injected code
section, which is added before the first executable section. Namely,
this happens in the rare case where that eh_frame section is smaller
than the injected code section, and is adjacent to the first executable
section. We obviously want to move the eh_frame sections in that case,
since one of them is in the way.
We're currently using NDK r15c, which is rather old, and happens to come
with a buggy gold linker. Let's use a more recent NDK, with a fixed
linker.
Unfortunately, we're currently at NDK API level 9, which the newer NDK
doesn't provide for x86 anymore. But that corresponds to Gingerbread
(2.3), which we've long stopped supporting. On the SDK side, we already
dropped support of versions before Jelly Bean, so we can do the same on
the NDK side. That corresponds to API level 16. So let's just use that
as a baseline.
Another change in the newer NDK is that the target-name changed from
i386-linux-android to i686-linux-android, so adjust for that in the
android x86 mozconfigs.
We're currently using NDK r15c, which is rather old, and happens to come
with a buggy gold linker. Let's use a more recent NDK, with a fixed
linker.
Unfortunately, we're currently at NDK API level 9, which the newer NDK
doesn't provide for x86 anymore. But that corresponds to Gingerbread
(2.3), which we've long stopped supporting. On the SDK side, we already
dropped support of versions before Jelly Bean, so we can do the same on
the NDK side. That corresponds to API level 16. So let's just use that
as a baseline.
Another change in the newer NDK is that the target-name changed from
i386-linux-android to i686-linux-android, so adjust for that in the
android x86 mozconfigs.
This change switches most CI builds to clang, with a few exceptions:
- valgrind builds, until bug 1481670 is figured out.
- PGO and nightly builds, until that's fully tested.
- coverage builds, per bug 1471339 comment 17.
- base toolchain builds, to keep some builds on GCC even when we're
fully switched to clang.
- any build that doesn't use build/unix/mozconfig.linux (e.g. probably
all those driven by autospider.py, maybe others).
We already copy the 64-bits libraries, but don't copy the 32-bits
libraries, which prevents building for linux32 by default.
Incidentally, this also makes the clang build system build the 32-bits
compiler-rt libraries, allowing e.g. 32-bits PGO.
Bug 1423822 moved the injected code section before the .text section.
When linking with lld, the text section is usually page aligned, and
starting a PT_LOAD. We inject code at the beginning of the PT_LOAD,
which means the PT_LOAD is going to be extended at least a page
downwards. And it means the preceding PT_LOAD can't finish in that same
page, so the overhead of the injected code is needs to account for the
page alignment.
If the .eh_frame_hdr and .eh_frame sections are not between the elfhack
relocation and elfhack code sections, it's not going to change anything
to try to move it, so don't even try.
While here, adjust the adjacency test to error out when the section name
doesn't match, and account for the fact that the eh_frame_hdr section
might appear after eh_frame.
--HG--
extra : rebase_source : 7d3525abe75b5a014b39ce0bd406e8f78592ec39
All CI builds for linux32 have been running on 64-bits hosts for a long
while now, and there are no 32-bits hosts left. We can remove the
special-casing that makes the linux32 builds behave differently
depending on the bit-width of the host.
Furthermore, configure has been able to deal with adding -m32 on its
own, so we don't need to add it to $CC/$CXX manually anymore.
We already copy the 64-bits libraries, but don't copy the 32-bits
libraries, which prevents building for linux32 by default.
Incidentally, this also makes the clang build system build the 32-bits
compiler-rt libraries, allowing e.g. 32-bits PGO.
We only enable libstdc++ compat on Android for the host parts, and
in practice, the target part has it not enabled at the moment because
libstdc++ can't be found. But making the clang toolchain capable of
building for x86 changes the deal for Android x86.
If the .eh_frame_hdr and .eh_frame sections are not between the elfhack
relocation and elfhack code sections, it's not going to change anything
to try to move it, so don't even try.
While here, adjust the adjacency test to error out when the section name
doesn't match.
--HG--
extra : rebase_source : 4b31712576fd3472bb94a2b9ab9542253f04cba8
Somehow, when building with LTO, clang can end up creating a eh_frame
section with only one, empty, entry (which just looks like a 4-bytes
long section full of 0x00).
--HG--
extra : rebase_source : 385c05c7e447fe1c4bc261b79c7d56138e268458
We need to add mappings for target.cpu to MSVC's arm64 name for
determining various paths, and we need to add an extra case to
get_vc_paths so the compiler can find all the necessary DLLs.
This patch adds JaCoCo as a dependency for the geckoview androidTest configurations, as well as
the `mach android archive-geckoview-coverage-artifacts` command, and the `--enable-java-coverage`
mozconfig flag.
MozReview-Commit-ID: 36jNAzK44g3
--HG--
extra : rebase_source : 9edc37913a3929ad045270c601c77791d122e363
For the same reason test-array.c and test-ctors.c need to be built
explicitly without LTO.
--HG--
extra : rebase_source : d037ef7cf1dd2d278c2918dbfee5b4f4c213e408
Mike Hommey
shindli
Brindusan Cristian
Robert Bartlensky
Jim Chen
Chris Peterson
Thomas P.
Masatoshi Kimura
Nathan Froyd
Henrik Skupin
Ted Mielczarek
Sylvestre Ledru
Daniel Varga
David Major
Cosmin Sabou
Andreas Tolfsen
Margareta Eliza Balazs
Tom Ritter
Robert Longson
Bogdan Tara
Ciure Andrei
Sebastian Hengst
ui.manish
Coroiu Cristina
Gurzau Raul
Myk Melez
Hiroyuki Ikezoe
Landry Breuil
Dorel Luca
arthur.iakab
Andi-Bogdan Postelnicu
Chris Manchester
svoisen
Andreea Pavel
Boris Chiou
Jan Beich
Csoregi Natalia
Emilio Cobos Álvarez
Bob Clary
Dan Mosedale
Gabriele Svelto
Tiberius Oros
Narcis Beleuzu
Julian Seward
Dan Mosedale
Noemi Erli
David Heiberg
Jesse Schwartzentruber
Geoff Brown
Panos Astithas
Tudor-Gabriel Vîjială