Since we've upgraded to clang 9, clang-format changed and now uses dynamic libraries
for the clang tooling lib that it leverages.
Differential Revision: https://phabricator.services.mozilla.com/D47265
--HG--
extra : moz-landing-system : lando
This definition is now unnecessary, given that the source code is
fetched for us from elsewhere.
Depends on D41368
Differential Revision: https://phabricator.services.mozilla.com/D41369
--HG--
extra : moz-landing-system : lando
Otherwise we wind up with clownshoes like:
```
froydnj@hawkeye:/opt/build/froydnj/tmp/clang$ ls -l lib/libstdc++.so*
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so.6
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so.6.0.24
```
and have duplicate copies of shared libraries in our toolchain packages,
which are not exactly small.
Differential Revision: https://phabricator.services.mozilla.com/D41368
--HG--
extra : moz-landing-system : lando
What this means is that the sources for clang/llvm are downloaded
separately from the toolchain build (which also means we finally only
download a given version of clang once for all platforms).
In turn, this means the build-clang.py script needs to start with an
existing llvm-project tree, and we choose to make build-clang.py expect
that it's run from the llvm-project root directory.
This also means we don't need to download git for the windows toolchain
task.
Differential Revision: https://phabricator.services.mozilla.com/D40402
Bug 1546136 wrapped subprocess execution output to capture cmake's, but
at the detriment of other output, and hiding everything unless an error
occurs.
So instead, we only capture the output when the called process is cmake,
and even when it is cmake, we don't pipe stderr at all (since we only
care about cmake's stdout) and we print out stdout as it comes in rather
than later. We then later check the output for hints at the more useful
cmake logs and dump them.
While here, add verbosity to ninja output (which gives the command
lines, rather than generic "Building foo.o" output).
Differential Revision: https://phabricator.services.mozilla.com/D40142
Credit: Callek for figuring out an issue in 'make check' making the binary
absolute in mozbuild.base.
Differential Revision: https://phabricator.services.mozilla.com/D37319
--HG--
extra : moz-landing-system : lando
Our current OS X builds use `--target=x86_64-darwin11` (which
corresponds to OS X 10.7). This target is problematic for two reasons:
* We're actually targeting for OS X 10.9 (`MACOSX_DEPLOYMENT_TARGET`);
* It's slightly different from the default Rust target.
Let's address these problems in reverse order: differences from the Rust
target are bad, because the `--target` we provide to `clang` and the
Rust target find their way into LLVM bitcode files and the linker will
refuse to link together bitcode files that have incompatible targets.
Why are the two incompatible? The current `--target` doesn't have a
"vendor" in triple-speak, whereas the Rust one has "apple" as the
vendor (`x86_64-apple-darwin`) We therefore need to change the
`--target` we pass to `clang` to have a vendor of "apple".
This need is behind the {init,toolchain}.configure changes,
but it has ramifications elsewhere, because `clang` looks for
`--target`-prefixed build tools. So we have to change the `--target`
for cctools to get the right tool prefixes and we have to change the
`--target` for building clang ourselves so that *those* builds can find
the newly renamed cctools.
Once we've done, that's really enough; we don't *need to address the
first problem: While the `--target` might be `x86_64-apple-darwin11`,
both `clang` and `rustc` will dynamically choose the target triple that
eventually lands in LLVM bitcode files based on
`MACOSX_DEPLOYMENT_TARGET`, which we set in all builds. But the current
target is slightly misleading, and the cctools don't need to be prefixed
with a particular Darwin version, since they work for all Darwin
targets. Let's just drop the "11" from the `--target` and eliminate a
little bit of confusion.
Differential Revision: https://phabricator.services.mozilla.com/D31128
--HG--
extra : moz-landing-system : lando
Our current OS X builds use `--target=x86_64-darwin11` (which
corresponds to OS X 10.7). This target is problematic for two reasons:
* We're actually targeting for OS X 10.9 (`MACOSX_DEPLOYMENT_TARGET`);
* It's slightly different from the default Rust target.
Let's address these problems in reverse order: differences from the Rust
target are bad, because the `--target` we provide to `clang` and the
Rust target find their way into LLVM bitcode files and the linker will
refuse to link together bitcode files that have incompatible targets.
Why are the two incompatible? The current `--target` doesn't have a
"vendor" in triple-speak, whereas the Rust one has "apple" as the
vendor (`x86_64-apple-darwin`) We therefore need to change the
`--target` we pass to `clang` to have a vendor of "apple".
This need is behind the {init,toolchain}.configure changes,
but it has ramifications elsewhere, because `clang` looks for
`--target`-prefixed build tools. So we have to change the `--target`
for cctools to get the right tool prefixes and we have to change the
`--target` for building clang ourselves so that *those* builds can find
the newly renamed cctools.
Once we've done, that's really enough; we don't *need to address the
first problem: While the `--target` might be `x86_64-apple-darwin11`,
both `clang` and `rustc` will dynamically choose the target triple that
eventually lands in LLVM bitcode files based on
`MACOSX_DEPLOYMENT_TARGET`, which we set in all builds. But the current
target is slightly misleading, and the cctools don't need to be prefixed
with a particular Darwin version, since they work for all Darwin
targets. Let's just drop the "11" from the `--target` and eliminate a
little bit of confusion.
Differential Revision: https://phabricator.services.mozilla.com/D31128
--HG--
extra : moz-landing-system : lando
Our current OS X builds use `--target=x86_64-darwin11` (which
corresponds to OS X 10.7). This target is problematic for two reasons:
* We're actually targeting for OS X 10.9 (`MACOSX_DEPLOYMENT_TARGET`);
* It's slightly different from the default Rust target.
Let's address these problems in reverse order: differences from the Rust
target are bad, because the `--target` we provide to `clang` and the
Rust target find their way into LLVM bitcode files and the linker will
refuse to link together bitcode files that have incompatible targets.
Why are the two incompatible? The current `--target` doesn't have a
"vendor" in triple-speak, whereas the Rust one has "apple" as the
vendor (`x86_64-apple-darwin`) We therefore need to change the
`--target` we pass to `clang` to have a vendor of "apple".
This need is behind the {init,toolchain}.configure changes,
but it has ramifications elsewhere, because `clang` looks for
`--target`-prefixed build tools. So we have to change the `--target`
for cctools to get the right tool prefixes and we have to change the
`--target` for building clang ourselves so that *those* builds can find
the newly renamed cctools.
Once we've done, that's really enough; we don't *need to address the
first problem: While the `--target` might be `x86_64-apple-darwin11`,
both `clang` and `rustc` will dynamically choose the target triple that
eventually lands in LLVM bitcode files based on
`MACOSX_DEPLOYMENT_TARGET`, which we set in all builds. But the current
target is slightly misleading, and the cctools don't need to be prefixed
with a particular Darwin version, since they work for all Darwin
targets. Let's just drop the "11" from the `--target` and eliminate a
little bit of confusion.
Differential Revision: https://phabricator.services.mozilla.com/D31128
--HG--
extra : moz-landing-system : lando
This will match the compiler version Tor would like. We backport several
llvm-objcopy patches that landed right after the 8 branch though. We
also grab some upstream changes from mingw-clang in the build script
Differential Revision: https://phabricator.services.mozilla.com/D31347
--HG--
extra : moz-landing-system : lando
Analogously to the existing `linux64-clang-8-android-cross` build, this
build is a linux x86-64 build with runtime library support for aarch64.
Depends on D28405
Differential Revision: https://phabricator.services.mozilla.com/D28406
--HG--
extra : moz-landing-system : lando
This change enables us to build compiler-rt and related
libraries (e.g. address sanitizer, etc.) for whatever targets we like,
assuming that we have an accessible sysroot for the target on the build
machine.
Depends on D28404
Differential Revision: https://phabricator.services.mozilla.com/D28405
--HG--
extra : moz-landing-system : lando
CMake errors can be pretty opaque, especially if CMake is being run
inside the Ninja build process. Let's try to surface those errors to
make problems easier to debug.
Differential Revision: https://phabricator.services.mozilla.com/D28360
--HG--
extra : moz-landing-system : lando
It seems better to set switches enabling runtime libraries and switches
enabling runtime libraries to build in different places, as future
changes might only enable runtime libraries for certain targets, but not
need any special switches for building.
Depends on D27594
Differential Revision: https://phabricator.services.mozilla.com/D27595
--HG--
extra : moz-landing-system : lando
`android_targets` here is a dict, not a sequence, and while `iter` on a
dict object implicitly means `dict.iterkeys()`, that's not really
obvious. We should instead be explicit about what we're doing here.
Depends on D27593
Differential Revision: https://phabricator.services.mozilla.com/D27594
--HG--
extra : moz-landing-system : lando
We don't need them and we might as well be explicit about not building them.
Depends on D27592
Differential Revision: https://phabricator.services.mozilla.com/D27593
--HG--
extra : moz-landing-system : lando
The setup for compiler-rt is currently done before the stage 2 build,
which happens to be the final stage for our android runtime libraries
build. But we may also want to build runtime libraries on 3-stage
bootstrap builds, in which case we don't want compiler-rt to be active
for the second stage. Move the setup into build_one_stage so that the
setup is controllable by is_final_stage, which is set in all the place
that we care about.
Differential Revision: https://phabricator.services.mozilla.com/D27592
--HG--
extra : moz-landing-system : lando
As of clang 8, llvm-config doesn't return all flags clang was built
with, and omits some flags that do impact the libclang ABI,
-stdlib=libc++ being one of them (it might well be the only one).
Building clang with LLVM_ENABLE_LIBCXX=ON does build it with
-stdlib=libc++, but is unrelated to whether or not libc++ is built and
shipped with clang, which still happens without it.
So while versions older than clang 8 are not really affected, it doesn't
hurt to build clang without -stdlib=libc++ (especially when it
currently only applies to the clang used to cross build android with
PGO, not even the other android cross builds), in preparation for
switching to clang 8.
Differential Revision: https://phabricator.services.mozilla.com/D25031
--HG--
extra : moz-landing-system : lando
We do this to encourage clang to find an new-enough linker instead of
the system one.
Differential Revision: https://phabricator.services.mozilla.com/D22881
--HG--
extra : moz-landing-system : lando
We want our clang bootstrap to use the GCC headers we're building with,
not whatever sysroot it happens to find on the server we're building on.
The -gcc-toolchain argument we specify when building clang will also be
picked up by llvm-config, so we need to strip it out when building the
plugin. Otherwise, we will get peculiar failures about not being able to
find C++ header files.
Differential Revision: https://phabricator.services.mozilla.com/D22880
--HG--
extra : moz-landing-system : lando
Updating clang indicates that 32-bit compilation is substantially longer
than 64-bit compilation, perhaps due to swapping. The compilation
process is hitting the timeout limit shortly before the compilation
process completes (~3681/3695 tasks according to ninja).
We could tweak our clang build process to accommodate this job. But we
don't support building on 32-bit Windows anymore, and we don't produce a
32-bit Windows clang either. So we shouldn't support a 32-bit Windows
clang-tidy job either. Let's get rid of it.
Differential Revision: https://phabricator.services.mozilla.com/D23517
--HG--
extra : moz-landing-system : lando
We do this to encourage clang to find an new-enough linker instead of
the system one.
Differential Revision: https://phabricator.services.mozilla.com/D22881
--HG--
extra : moz-landing-system : lando
We want our clang bootstrap to use the GCC headers we're building with,
not whatever sysroot it happens to find on the server we're building on.
The -gcc-toolchain argument we specify when building clang will also be
picked up by llvm-config, so we need to strip it out when building the
plugin. Otherwise, we will get peculiar failures about not being able to
find C++ header files.
Depends on D22879
Differential Revision: https://phabricator.services.mozilla.com/D22880
--HG--
extra : moz-landing-system : lando
libclang 3.9 has a bug that makes bindgen unable to distinguish some typedefs
from the underlying type, which matters for bug 1523071.
We have had quite a few workarounds for this bug and I don't really want to add
more, since in this case it is non-trivial. I think requiring libclang 4.0+ is
reasonable at this point.
Of the distros that can't build Firefox out of the box with clang, dropping support
for clang 3.9 would only break Ubuntu 14.04 LTS, which support ends April 2019,
right before we release 67.
Differential Revision: https://phabricator.services.mozilla.com/D18889
--HG--
rename : build/build-clang/clang-3.9-linux64.json => build/build-clang/clang-4.0-linux64.json
rename : taskcluster/scripts/misc/build-clang-3.9-linux.sh => taskcluster/scripts/misc/build-clang-4.0-linux.sh
extra : moz-landing-system : lando
Firefox uses multiple processes. It has intentional leaks, and when
running with ASAN, we have suppressions to eliminate those. When running
ASAN builds through CI tests, when Firefox exits, each of the processes
(parent and child) exits and goes through its leaks and when there are
(which is a given), the ASAN runtime runs llvm-symbolizer to symbolicate
and match against suppressions. So each process runs llvm-symbolizer. At
the same time.
Some of the addresses to symbolicate are in libxul. Which contains all
DWARF info, making it a ~1GB monster. Oh, and because you're lucky,
things align perfectly such that libxul size is a multiple of the page
size. That makes llvm-symbolizer pread() the file instead of mmap()ing
it. Did I say there are multiple processes? So suddenly you have n
processes simultaneously allocating and filling 1GB of memory each, on
CI machines that have enough memory for the job they usually run, but
not enough for a sudden rush of n GB.
And things go awry. When you're lucky and the OOM killer didn't take
care of killing the CI entirely, symbolication couldn't happen and the
suppressions are not matched, and leaks are reported.
This all turns out it originates in how llvm-symbolicate chooses between
pread() and mmap(), which turns out is just defaults not being made for
binary files.
Differential Revision: https://phabricator.services.mozilla.com/D16010
--HG--
extra : moz-landing-system : lando
This matches more closely cross toolchains prefixes (as can be seen in
e.g. media/libvpx/libvpx/README for x86_64-darwin*-gcc), and leaves it
to the build system to figure out the right --target to pass to clang on
its own.
Differential Revision: https://phabricator.services.mozilla.com/D14376
This removes several patches that have been upstreamed.
--HG--
extra : rebase_source : 8462b42b8342935cfb5ecb912fac06c67b9165b9
extra : amend_source : 271b5d4a5b37fa74555ef87cb0c4444f8f7468f0
This patch is based on the cmake cache files for Android checked in to the
clang repo.
Differential Revision: https://phabricator.services.mozilla.com/D14004
--HG--
extra : moz-landing-system : lando
Bug 1492663 upgraded those builds to clang 7, and bug 1503330 brought
them back to clang 6 because of speedometer regressions.
With the previous change, the regression doesn't happen any more,
allowing to upgrade again.
Depends on D12394
Differential Revision: https://phabricator.services.mozilla.com/D12395
--HG--
extra : moz-landing-system : lando
David Major
Andi-Bogdan Postelnicu
Brindusan Cristian
Nathan Froyd
Sylvestre Ledru
Daniel Varga
Mike Hommey
Bogdan Tara
Andrew Halberstadt
Andreea Pavel
Coroiu Cristina
Tom Ritter
Gabriele Svelto
Jesse Schwartzentruber
Chris Manchester
Christian Holler
Noemi Erli
Ciure Andrei
Nathan Froyd
Gurzau Raul
Kartikaya Gupta
Emilio Cobos Álvarez
Cosmin Sabou
Narcis Beleuzu
Dorel Luca
Jacek Caban
Marco Castelluccio