libclang 3.9 has a bug that makes bindgen unable to distinguish some typedefs
from the underlying type, which matters for bug 1523071.
We have had quite a few workarounds for this bug and I don't really want to add
more, since in this case it is non-trivial. I think requiring libclang 4.0+ is
reasonable at this point.
Of the distros that can't build Firefox out of the box with clang, dropping support
for clang 3.9 would only break Ubuntu 14.04 LTS, which support ends April 2019,
right before we release 67.
Differential Revision: https://phabricator.services.mozilla.com/D18889
--HG--
rename : build/build-clang/clang-3.9-linux64.json => build/build-clang/clang-4.0-linux64.json
rename : taskcluster/scripts/misc/build-clang-3.9-linux.sh => taskcluster/scripts/misc/build-clang-4.0-linux.sh
extra : moz-landing-system : lando
Firefox uses multiple processes. It has intentional leaks, and when
running with ASAN, we have suppressions to eliminate those. When running
ASAN builds through CI tests, when Firefox exits, each of the processes
(parent and child) exits and goes through its leaks and when there are
(which is a given), the ASAN runtime runs llvm-symbolizer to symbolicate
and match against suppressions. So each process runs llvm-symbolizer. At
the same time.
Some of the addresses to symbolicate are in libxul. Which contains all
DWARF info, making it a ~1GB monster. Oh, and because you're lucky,
things align perfectly such that libxul size is a multiple of the page
size. That makes llvm-symbolizer pread() the file instead of mmap()ing
it. Did I say there are multiple processes? So suddenly you have n
processes simultaneously allocating and filling 1GB of memory each, on
CI machines that have enough memory for the job they usually run, but
not enough for a sudden rush of n GB.
And things go awry. When you're lucky and the OOM killer didn't take
care of killing the CI entirely, symbolication couldn't happen and the
suppressions are not matched, and leaks are reported.
This all turns out it originates in how llvm-symbolicate chooses between
pread() and mmap(), which turns out is just defaults not being made for
binary files.
Differential Revision: https://phabricator.services.mozilla.com/D16010
--HG--
extra : moz-landing-system : lando
This matches more closely cross toolchains prefixes (as can be seen in
e.g. media/libvpx/libvpx/README for x86_64-darwin*-gcc), and leaves it
to the build system to figure out the right --target to pass to clang on
its own.
Differential Revision: https://phabricator.services.mozilla.com/D14376
This removes several patches that have been upstreamed.
--HG--
extra : rebase_source : 8462b42b8342935cfb5ecb912fac06c67b9165b9
extra : amend_source : 271b5d4a5b37fa74555ef87cb0c4444f8f7468f0
This patch is based on the cmake cache files for Android checked in to the
clang repo.
Differential Revision: https://phabricator.services.mozilla.com/D14004
--HG--
extra : moz-landing-system : lando
Bug 1492663 upgraded those builds to clang 7, and bug 1503330 brought
them back to clang 6 because of speedometer regressions.
With the previous change, the regression doesn't happen any more,
allowing to upgrade again.
Depends on D12394
Differential Revision: https://phabricator.services.mozilla.com/D12395
--HG--
extra : moz-landing-system : lando
The cctools-port linker links against libraries from clang (for LTO),
which have different SONAMEs depending on the clang version. Which means
the linker needs to be used along the same version of clang it was built
against. Thus we also make it depend on linux64-clang-7.
But changing the dependency is not enough, cf. bug 1471905, so also
touch its build script, which it turns out, we need to do anyways
because llvm-dsymutil was renamed to dsymutil.
Relatedly, all toolchains that are built using cctools-port need to use
linux64-clang-7 too.
Building compiler-rt 7 with the OSX 10.11 SDK fails because of some
newer APIs being used in compiler-rt for xray, but this is not a feature
we use, so disable that.
Differential Revision: https://phabricator.services.mozilla.com/D6766
The cctools-port linker links against libraries from clang (for LTO),
which have different SONAMEs depending on the clang version. Which means
the linker needs to be used along the same version of clang it was built
against. Thus we also make it depend on linux64-clang-7.
But changing the dependency is not enough, cf. bug 1471905, so also
touch its build script, which it turns out, we need to do anyways
because llvm-dsymutil was renamed to dsymutil.
Relatedly, all toolchains that are built using cctools-port need to use
linux64-clang-7 too.
Building compiler-rt 7 with the OSX 10.11 SDK fails because of some
newer APIs being used in compiler-rt for xray, but this is not a feature
we use, so disable that.
Differential Revision: https://phabricator.services.mozilla.com/D6766
With libLLVM being a shared library exporting many symbols, all internal
calls using those symbols default to go through the PLT, which is
unnecessary (and costly) overhead. Using -Bsymbolic-functions makes
internal calls go directly to the right place without going through the
PLT.
Differential Revision: https://phabricator.services.mozilla.com/D7029
With libLLVM being a shared library exporting many symbols, all internal
calls using those symbols default to go through the PLT, which is
unnecessary (and costly) overhead. Using -Bsymbolic-functions makes
internal calls go directly to the right place without going through the
PLT.
Differential Revision: https://phabricator.services.mozilla.com/D7029
The cctools-port linker links against libraries from clang (for LTO),
which have different SONAMEs depending on the clang version. Which means
the linker needs to be used along the same version of clang it was built
against. Thus we also make it depend on linux64-clang-7.
But changing the dependency is not enough, cf. bug 1471905, so also
touch its build script, which it turns out, we need to do anyways
because llvm-dsymutil was renamed to dsymutil.
Relatedly, all toolchains that are built using cctools-port need to use
linux64-clang-7 too.
Building compiler-rt 7 with the OSX 10.11 SDK fails because of some
newer APIs being used in compiler-rt for xray, but this is not a feature
we use, so disable that.
Differential Revision: https://phabricator.services.mozilla.com/D6766
--HG--
rename : build/build-clang/clang-6-macosx64.json => build/build-clang/clang-7-macosx64.json
rename : taskcluster/scripts/misc/build-clang-6-linux-macosx-cross.sh => taskcluster/scripts/misc/build-clang-7-linux-macosx-cross.sh
extra : moz-landing-system : lando
The ASan runtime wasn't designed to be unloaded, so pin it in memory.
Differential Revision: https://phabricator.services.mozilla.com/D6943
--HG--
extra : rebase_source : 843fc91b3150f3b264e321ab89bf723a01fc8b3c
Doing so changes the size of the compressed toolchain archive from ~280M
to ~120M, and the decompressed size from ~1500M to ~675M. This will
reduce the overhead of decompression during builds.
As we ship llvm-symbolizer as part of ASan builds, we do need it to
still statically link against LLVM, which we do with a small patch.
With LLVM as a shared library, libLTO, which is used by cctools-port for
the linker, is dynamically linked to LLVM, and the cctools-port
configure script fails to link against libLTO. So we add a -rpath-link
to make it find the LLVM library. This happens to force a rebuild of
cctools-port, but for future cases where we might need a rebuild because
of some clang changes, we add a comment to ease the process, and avoid
a newer cctools-port taking the cache spot of an older one.
Ideally, mac cctools-port would need something similar, but it needs a
mac libLTO.dylib, which is not there anyways (and the mac cctools-port
thus already didn't support LTO).
Also, with LLVM built as a shared library, all its symbols are exported
with a LLVM_x.y version. Combined with -static-libstdc++ that is used
during the clang build, this causes problems (see
https://bugzilla.mozilla.org/show_bug.cgi?id=1492037#c7). But it turns
out things have evolved since -static-libstdc++ has been added to the
clang build script, and things work without now, so remove it (as well
as -static-libgcc).
Differential Revision: https://phabricator.services.mozilla.com/D6117
Doing so changes the size of the compressed toolchain archive from ~280M
to ~120M, and the decompressed size from ~1500M to ~675M. This will
reduce the overhead of decompression during builds.
As we ship llvm-symbolizer as part of ASan builds, we do need it to
still statically link against LLVM, which we do with a small patch.
With LLVM as a shared library, libLTO, which is used by cctools-port for
the linker, is dynamically linked to LLVM, and the cctools-port
configure script fails to link against libLTO. So we add a -rpath-link
to make it find the LLVM library. This happens to force a rebuild of
cctools-port, but for future cases where we might need a rebuild because
of some clang changes, we add a comment to ease the process, and avoid
a newer cctools-port taking the cache spot of an older one.
Ideally, mac cctools-port would need something similar, but it needs a
mac libLTO.dylib, which is not there anyways (and the mac cctools-port
thus already didn't support LTO).
Also, with LLVM built as a shared library, all its symbols are exported
with a LLVM_x.y version. Combined with -static-libstdc++ that is used
during the clang build, this causes problems (see
https://bugzilla.mozilla.org/show_bug.cgi?id=1492037#c7). But it turns
out things have evolved since -static-libstdc++ has been added to the
clang build script, and things work without now, so remove it (as well
as -static-libgcc).
Differential Revision: https://phabricator.services.mozilla.com/D6117
Doing so changes the size of the compressed toolchain archive from ~280M
to ~120M, and the decompressed size from ~1500M to ~675M. This will
reduce the overhead of decompression during builds.
As we ship llvm-symbolizer as part of ASan builds, we do need it to
still statically link against LLVM, which we do with a small patch.
CCTools-port needs to be rebuilt in some cases of clang changes, which
this one of, so touch the script so that it happens.
Differential Revision: https://phabricator.services.mozilla.com/D6117
Not waiting for rc3 or final from llvm because the issue is pretty severe
Differential Revision: https://phabricator.services.mozilla.com/D5124
--HG--
extra : moz-landing-system : lando
As much as it's tempting to use LLD for LTO, it still causes some
subtle problems with the build, and it's still better to keep using
BFD ld for the time being. Doing so requires the gold plugin, which
only requires to pass cmake the directory where the binutils headers
are, and they are part of the gcc toolchain headers.
Differential Revision: https://phabricator.services.mozilla.com/D4896
As much as it's tempting to use LLD for LTO, it still causes some
subtle problems with the build, and it's still better to keep using
BFD ld for the time being. Doing so requires the gold plugin, which
only requires to pass cmake the directory where the binutils headers
are, and they are part of the gcc toolchain headers.
Differential Revision: https://phabricator.services.mozilla.com/D4896
For some reason, clang 6 crashes with a stack overflow on PGO + LTO on
Linux 64 bits. Clang 7 doesn't, but has other problems.
After some bisecting, I found the following:
- r322684 is the first revision that is broken on the release_60 branch.
- that revision is a cherry pick of r322313 from trunk, which is
similarly broken.
- trunk was fixed by r322325, which, funnily enough, predates when
r322313 was cherry-picked.
While the change from r322325 is relatively large, mixing multiple
different changes in a single commit, there also haven't been
significant changes to the same file on trunk since (one macro name
change, one documentation change, and a change related to debug info),
which would tend to indicate the change is not going to break anything,
or at least not more than upgrading to clang 7 would.
The exact part that fixes the issue could probably be found in this
large commit, but I didn't feel like digging into it further considering
the above.
We already copy the 64-bits libraries, but don't copy the 32-bits
libraries, which prevents building for linux32 by default.
Incidentally, this also makes the clang build system build the 32-bits
compiler-rt libraries, allowing e.g. 32-bits PGO.
We already copy the 64-bits libraries, but don't copy the 32-bits
libraries, which prevents building for linux32 by default.
Incidentally, this also makes the clang build system build the 32-bits
compiler-rt libraries, allowing e.g. 32-bits PGO.
https://github.com/rust-lang/rust/issues/52694 is a miscompilation I
found in rust when it uses system llvm 5 or 6, that was fixed 5 months
ago in the llvm rust bundles. This may or may not affect clang, but
considering it was also reported to upstream llvm independently of rust,
it's better to side with caution.
It doesn't affect 3.9, and bug 1478919 got rid of the last use of clang
5 (except for clang-tidy, but that's not used to compile).
The patches come from the llvm trunk from 5 months ago, so they're
already in our clang 7 snapshots.
Windows static analysis builds are still using an old trunk, but are
stuck on bug 1427808. They're "only" for static analysis, though.
--HG--
extra : rebase_source : f4fce69eb7c69b6245518a1bad37e04236c7075b
Instead of clang 4, which they were the last to use, so remove the
clang 4 toolchain.
--HG--
extra : rebase_source : d03a083e9217aeb6c1d2c91decb978426f0e8d1a
David Major
Kartikaya Gupta
Emilio Cobos Álvarez
Mike Hommey
Chris Manchester
Cosmin Sabou
Ciure Andrei
Narcis Beleuzu
Tom Ritter
Dorel Luca
Jacek Caban
Marco Castelluccio
Noemi Erli
Coroiu Cristina
Nathan Froyd
Andi-Bogdan Postelnicu
Brindusan Cristian
Sylvestre Ledru
arthur.iakab
Bogdan Tara
Daniel Varga
Christian Holler
Devika Sugathan