This change switches most CI builds to clang, with a few exceptions:
- valgrind builds, until bug 1481670 is figured out.
- PGO and nightly builds, until that's fully tested.
- coverage builds, per bug 1471339 comment 17.
- base toolchain builds, to keep some builds on GCC even when we're
fully switched to clang.
- any build that doesn't use build/unix/mozconfig.linux (e.g. probably
all those driven by autospider.py, maybe others).
This change adds a voluptuous schema for build system telemetry, replacing
the existing json schema file. Using voluptuous will make it easier to work
with the schema from Python code in the build system. A future commit will
use a Python module to provide a mach command to convert the voluptuous
schema to json schema format for consumption by other systems.
Differential Revision: https://phabricator.services.mozilla.com/D2840
--HG--
extra : rebase_source : 067995385334d1dbc123f2db4245ef4e69d076c3
In some rare cases, it is possible for one of the eh_frame sections'
original address to be larger than the address of the injected code
section, which is added before the first executable section. Namely,
this happens in the rare case where that eh_frame section is smaller
than the injected code section, and is adjacent to the first executable
section. We obviously want to move the eh_frame sections in that case,
since one of them is in the way.
We're currently using NDK r15c, which is rather old, and happens to come
with a buggy gold linker. Let's use a more recent NDK, with a fixed
linker.
Unfortunately, we're currently at NDK API level 9, which the newer NDK
doesn't provide for x86 anymore. But that corresponds to Gingerbread
(2.3), which we've long stopped supporting. On the SDK side, we already
dropped support of versions before Jelly Bean, so we can do the same on
the NDK side. That corresponds to API level 16. So let's just use that
as a baseline.
Another change in the newer NDK is that the target-name changed from
i386-linux-android to i686-linux-android, so adjust for that in the
android x86 mozconfigs.
We're currently using NDK r15c, which is rather old, and happens to come
with a buggy gold linker. Let's use a more recent NDK, with a fixed
linker.
Unfortunately, we're currently at NDK API level 9, which the newer NDK
doesn't provide for x86 anymore. But that corresponds to Gingerbread
(2.3), which we've long stopped supporting. On the SDK side, we already
dropped support of versions before Jelly Bean, so we can do the same on
the NDK side. That corresponds to API level 16. So let's just use that
as a baseline.
Another change in the newer NDK is that the target-name changed from
i386-linux-android to i686-linux-android, so adjust for that in the
android x86 mozconfigs.
This change switches most CI builds to clang, with a few exceptions:
- valgrind builds, until bug 1481670 is figured out.
- PGO and nightly builds, until that's fully tested.
- coverage builds, per bug 1471339 comment 17.
- base toolchain builds, to keep some builds on GCC even when we're
fully switched to clang.
- any build that doesn't use build/unix/mozconfig.linux (e.g. probably
all those driven by autospider.py, maybe others).
We already copy the 64-bits libraries, but don't copy the 32-bits
libraries, which prevents building for linux32 by default.
Incidentally, this also makes the clang build system build the 32-bits
compiler-rt libraries, allowing e.g. 32-bits PGO.
Bug 1423822 moved the injected code section before the .text section.
When linking with lld, the text section is usually page aligned, and
starting a PT_LOAD. We inject code at the beginning of the PT_LOAD,
which means the PT_LOAD is going to be extended at least a page
downwards. And it means the preceding PT_LOAD can't finish in that same
page, so the overhead of the injected code is needs to account for the
page alignment.
If the .eh_frame_hdr and .eh_frame sections are not between the elfhack
relocation and elfhack code sections, it's not going to change anything
to try to move it, so don't even try.
While here, adjust the adjacency test to error out when the section name
doesn't match, and account for the fact that the eh_frame_hdr section
might appear after eh_frame.
--HG--
extra : rebase_source : 7d3525abe75b5a014b39ce0bd406e8f78592ec39
All CI builds for linux32 have been running on 64-bits hosts for a long
while now, and there are no 32-bits hosts left. We can remove the
special-casing that makes the linux32 builds behave differently
depending on the bit-width of the host.
Furthermore, configure has been able to deal with adding -m32 on its
own, so we don't need to add it to $CC/$CXX manually anymore.
We already copy the 64-bits libraries, but don't copy the 32-bits
libraries, which prevents building for linux32 by default.
Incidentally, this also makes the clang build system build the 32-bits
compiler-rt libraries, allowing e.g. 32-bits PGO.
We only enable libstdc++ compat on Android for the host parts, and
in practice, the target part has it not enabled at the moment because
libstdc++ can't be found. But making the clang toolchain capable of
building for x86 changes the deal for Android x86.
If the .eh_frame_hdr and .eh_frame sections are not between the elfhack
relocation and elfhack code sections, it's not going to change anything
to try to move it, so don't even try.
While here, adjust the adjacency test to error out when the section name
doesn't match.
--HG--
extra : rebase_source : 4b31712576fd3472bb94a2b9ab9542253f04cba8
Somehow, when building with LTO, clang can end up creating a eh_frame
section with only one, empty, entry (which just looks like a 4-bytes
long section full of 0x00).
--HG--
extra : rebase_source : 385c05c7e447fe1c4bc261b79c7d56138e268458
We need to add mappings for target.cpu to MSVC's arm64 name for
determining various paths, and we need to add an extra case to
get_vc_paths so the compiler can find all the necessary DLLs.
This patch adds JaCoCo as a dependency for the geckoview androidTest configurations, as well as
the `mach android archive-geckoview-coverage-artifacts` command, and the `--enable-java-coverage`
mozconfig flag.
MozReview-Commit-ID: 36jNAzK44g3
--HG--
extra : rebase_source : 9edc37913a3929ad045270c601c77791d122e363
For the same reason test-array.c and test-ctors.c need to be built
explicitly without LTO.
--HG--
extra : rebase_source : d037ef7cf1dd2d278c2918dbfee5b4f4c213e408
When linking with ld.bfd or gold, this changes the PT_LOAD in which the
elfhack code section ends up, making it go in the same one as .init, .text,
etc. rather than .rel.*. When linking with lld, this completely
avoids doing a PT_LOAD split, because lld already splits .rel.* and
.text.
--HG--
extra : rebase_source : 1f69b8f4b48b055892ea24eaa6226859cc4ffd50
We treat segments overlapping as a fatal error, rather than a condition
to do nothing, because it happening is usually the result of some bad
assumptions on the input ELF, and we don't want to silently ignore
those.
However, there are cases where a setup /could/ lead to overlapping
segments, but would be skipped because elfhack wouldn't be a win
anyways. By checking segments overlap later, we allow those to not
hard fail.
--HG--
extra : rebase_source : deca2051722aeaa959c5e4dae06642908f6d843a
The current check makes assumption wrt what PT_LOAD the injected sections
end up in, and won't work with upcoming changes.
--HG--
extra : rebase_source : b7cfb65ea13c16f977fe523aaf9f39eafeb2cdce
Summary:
The plan is to also expose perfecthash.py from this module on the python path.
This also allows us to stop using explicit module loading to load make_dafsa.py.
make_dafsa.py was moved into tools/ to avoid any extra python files from
accidentally ending up on the python path.
Reviewers: froydnj!
Tags: #secure-revision
Bug #: 1479484
Differential Revision: https://phabricator.services.mozilla.com/D2614
--HG--
rename : xpcom/ds/make_dafsa.py => xpcom/ds/tools/make_dafsa.py
When compiling C/C++ sources via Rust build scripts there's no point in
enabling warnings-as-errors as all such code is third-party code.
MozReview-Commit-ID: 5pGH6w9ZE2I
--HG--
extra : rebase_source : ad36be6aed39d55055728f85b799adf5f0cf0f4b
https://github.com/rust-lang/rust/issues/52694 is a miscompilation I
found in rust when it uses system llvm 5 or 6, that was fixed 5 months
ago in the llvm rust bundles. This may or may not affect clang, but
considering it was also reported to upstream llvm independently of rust,
it's better to side with caution.
It doesn't affect 3.9, and bug 1478919 got rid of the last use of clang
5 (except for clang-tidy, but that's not used to compile).
The patches come from the llvm trunk from 5 months ago, so they're
already in our clang 7 snapshots.
Windows static analysis builds are still using an old trunk, but are
stuck on bug 1427808. They're "only" for static analysis, though.
--HG--
extra : rebase_source : f4fce69eb7c69b6245518a1bad37e04236c7075b
This was cargo-culted from the asan/tsan mozconfigs, but is not necessary
for builds without sanitizers.
--HG--
extra : rebase_source : 41bad4761f424410cb7a099ecaecce8a86becf59
JS, at least, doesn't need bindgen at this point. If JS does start
requiring bindgen under certain build configurations, we'll have to
figure out what to do about the bindgen dependency at that point:
require bindgen always, or just require bindgen for the
configuration(s).
Summary:
This adds a clangcl flag to gyp.mozbuild and then uses that to set the same
warnings flags for clang-cl as for clang.
Reviewers: dmajor
Tags: #secure-revision
Bug #: 1478000
Differential Revision: https://phabricator.services.mozilla.com/D2427
--HG--
extra : rebase_source : 6137d2f59e5504d448d4fb32a7e734756ff76afa
Summary:
We're currently returning a fake value on all of our automation builds. Might
as well not run the script at all.
Tags: #secure-revision
Bug #: 1371485
Differential Revision: https://phabricator.services.mozilla.com/D1803
--HG--
extra : rebase_source : 20717372c15547b35759ff69088aea1ca92b7198
This adds just enough host shared library support for this one use case,
but also takes shortcuts, because fully supporting host shared library
is a deep rabbit hole I'm not ready to take just to fix --enable-lto
--enable-clang-plugin on mac builds.
One downside is that one my machine the plugin now takes > 80s to build,
instead of 15s before, thanks to the lack of unified sources.
--HG--
extra : rebase_source : bf52a72a01d4e3eb77cf52b646b19734b9273075
Currently, --enable-lto just implies --enable-linker=lld, which
essentially only works on Linux, and assumes one can't do lto with
anything other than lld. Which is not true. As a matter of fact, even
ld.bfd can do lto, as long as the gold plugin for llvm is available,
or when doing lto with GCC instead of clang.
Anyways, to allow more lto setups, we adapt the --enable-linker setup
to:
- work on macOS, which it currently doesn't, and add support for the mac
linker (ld64), which, unfortunately, doesn't have a clean way to be
detected, so work around that.
- default to lld if lto is enable, no linker was explicitly given, the
compiler is clang *and* the build target is not macOS.
--HG--
extra : rebase_source : 1dab2ad6230d18e7f4285943e76352e23b987d4e
Because we have no linking configure test in python configure (yet), we
just make old-configure tests use LINKER_LDFLAGS, and make those flags
added to LDFLAGS by old-configure at the same time.
--HG--
extra : rebase_source : 80ab7c5021c7ddd1b53d58ef76cd4372a524d3cb
The intent is for the build system to soon require Node.js to build
Firefox. But we aren't ready to make Node.js a build requirement
just yet.
The goal of this commit is to implement configure detection for
Node.js so that we can a) work out detection bugs b) give people a
means to validate system compatibility *before* we throw the switch to
require Node.js.
This commit introduces configure logic for finding a Node.js
executable, resolving its version, and validating its suitability.
By default, if Node.js cannot be found or there is an error resolving
its version, we print some warning messages and move on.
If --enable-nodejs is used (not the default), errors are raised
if Node.js cannot be found or its version isn't suitable.
Once we require Node.js, the added code can likely be simplified.
When writing the code, I went out of my way to make failures as
non-fatal as possible. e.g. normally we'd say that failures to run
`node --version` would be fatal. I'm purposefully trying to not have
this configure check break anyone's environment, even if failure
occurs. Again, the goal is to introduce the configure checks first
in a non-fatal way such that we can debug failures so the flag day
transition is simpler.
Differential Revision: https://phabricator.services.mozilla.com/D1818
--HG--
extra : moz-landing-system : lando
Added `./mach python-safety`, distinct from python-test so it doesn't have
to be run on every CI job - its errors may not depend on the area the push has changed.
Added the python/safety directory to ensure a different Pipfile is used, avoiding
conflicts with python-test.
Differential Revision: https://phabricator.services.mozilla.com/D1825
--HG--
extra : moz-landing-system : lando
We're well overdue for an upgrade of the rust compiler requirements.
Now that we're building with 1.28 (albeit a beta, due to be bumped when
it's released), we can bump the requirement away from 1.24 which is now
old. 1.27 is too new, though, so settle for the older 1.26.
--HG--
extra : rebase_source : a17aa496bf3d4af4d1349d69a637c686c6817d0f
Cosmin Sabou
Emilio Cobos Álvarez
Mike Hommey
Tiberius Oros
Dan Mosedale
Dorel Luca
Ted Mielczarek
Bogdan Tara
Narcis Beleuzu
Julian Seward
Tom Ritter
Chris Manchester
Dan Mosedale
Masatoshi Kimura
Noemi Erli
David Heiberg
Daniel Varga
Jesse Schwartzentruber
Brindusan Cristian
Geoff Brown
Margareta Eliza Balazs
Panos Astithas
Andreea Pavel
Nathan Froyd
Tudor-Gabriel Vîjială
David Major
Nika Layzell
Christian Holler
Mike Shal
Devika Sugathan
Mike Shal
Robert Bartlensky
Kartikaya Gupta
Bob Clary
Dan Minor
Jacek Caban
Ivan Pozdeev
Marco Castelluccio
Gurzau Raul
Ciure Andrei
Ms2ger
Gregory Szorc
Simon Fraser
Dave Hunt
Henri Sivonen