developer_options (!--enable-release) implies to search for and prefer
the gold linker. As part of porting gold detection to moz.configure
in d0e782180741 (bug 1351109), the logic inadvertently changed to
require gold when either --enable-gold or !--enable-release were
present.
This commit relaxes the requirement to find gold when only --enable-gold
is true.
MozReview-Commit-ID: HTBicWNUkvy
--HG--
extra : rebase_source : dd3938a7914f5db6c315fb775e7cc5ea177bf600
check_prog, when used with a `when` argument, doesn't work the same way
as putting it under a `with only_when()` block, while it should. The
difference comes from the fact that `with only_when()` applies the
`when` to every option and depends used in the block (which check_prog
calls a bunch of).
So, we "manually" apply the `when` to all option and depends in
check_prog. An alternative solution would be to put the whole function
under a `with only_when()` block, but that would mean reindenting the
whole function.
Either way, as a consequence, this requires the `when` to have a
dependency on --help for "non-trivial" functions, which fortunately,
there's only one of.
--HG--
extra : rebase_source : d91eca9e303c7062394d92a526983714ef3e612f
LLVM_CONFIG, per the contents of toolkit/moz.configure, is tied to
--enable-stylo, but it currently is set on all types of builds. It
currently happens to work, but it's actually not meant to, and sure
enough, the fix for bug 1374727 exacerbates that.
So we create a new mozconfig.stylo file that enables stylo and sets
LLVM_CONFIG, such that only build types that do enable stylo have
LLVM_CONFIG set.
--HG--
extra : rebase_source : 01277a79951888046c0b8e29c61cfc3b049ee0f0
For reasons unknown to me, Windows CI is periodically failing to find
the Mercurial binary.
In addition, we've also reimplemented various VCS logic throughout
the build system. There is room to cut down on code complexity by
e.g. recording VCS info in configure instead of determining it
at run-time.
Also, for forensic purposes it is sometimes desirable to know which
VCS tool is in use by a build and which version of that tool is being
used.
This commit adds VCS type detection, binary searching, and version
resolution to configure.
substs now contains VCS_CHECKOUT_TYPE, HG, and GIT, which can be
consulted by downstream consumers.
If the Mercurial or Git versions could not be resolved, all variables
are not set. Otherwise, VCS_CHECKOUT_TYPE and one of HG or GIT is set.
If MOZ_AUTOMATION is set, we require that the VCS info be resolved.
This helps prevents weirdness in automation due to unexpected
environment configuration.
MozReview-Commit-ID: AMLy0Hfx5rD
--HG--
extra : rebase_source : edef9165d32dc47308a14b0fbabce3c1d3d28176
And include code to work around a bug on older Python versions.
MozReview-Commit-ID: 4pBnMQQJOGB
--HG--
extra : rebase_source : 6f7c5784230bd37b3496b9bb1781e8d342f741b4
This enables apm logging by setting the apm_debug_dump variable in gyp.mozbuild.
Prior to this change, some files were including apm_data_dumper.h with logging
enabled and some were not.
This also removes the AEC* C interface and calls into webrtc::Trace directly.
Whatever historical reasons for having a C interface into these calls no
longer seems to apply. In addition reserving a buffer for the base file name
and then ensuring it was null terminated caused an ASAN "stack-buffer-overflow"
while testing. This was because it was not handling an empty base file
name properly. This would not normally happen if AEC logging was enabled through
about:webrtc, but it still seems safer to use std::string.
MozReview-Commit-ID: Ikz5xO74syA
--HG--
extra : rebase_source : 8e0c59117135fadb75f4a7e6be5588af1404533d
When investigating why a particular build got a particularly high number
of cache misses, it is useful to know exactly *what* had cache misses,
and you can't get the missing information after the fact.
We used to have some stats stored in a log file uploaded as artifact
with the old python sccache per bug 1005495 (and before that inline in
the build log), and we actually still have remains of that in the form
of a EXTRA_UPLOAD_FILES in build/mozconfig.cache.
Let's reuse that, and get some useful logs from sccache. Ideally sccache
would give better logs, but those are sufficient for now. We'll figure
things out eventually. https://github.com/mozilla/sccache/issues/151
--HG--
extra : rebase_source : a2e7093697a5b720ab28f155560211d6c0a4347e
Rustc 1.20 gained support for make jobserver. But when rustc is invoked
through sccache, sccache needs to have inherited the make jobserver file
descriptor to pass it down to rustc. But what will be running sccache
first during the build is not really deterministic. It might as well end
up being a script invoking the preprocessor that ends up doing it. And
we can't add the necessary + prefix to all commands that potentially can
end up invoking sccache.
So we manually start the sccache server at the beginning of the build,
through preflight.
Note this works on automation, but not for local builds, but few people
are using sccache locally for rust (yet). Let's figure that out in a
followup.
--HG--
extra : rebase_source : eda5853d7247be247a1bec1a5d7e84b3846fd2c1
This patch does two things:
- add a Gradle-only ANDROID_COMPILE_SDK_VERSION substitution;
- uses it while uniformizing all of the Gradle Android SDK version
configurations.
The approach is fairly standard (and we were using it already); see,
for example
https://medium.com/@ali.muzaffar/gradle-configure-variables-for-all-android-project-modules-in-one-place-5a6e56cd384e
This will make bumping the Gradle configuration versions forward
easier.
MozReview-Commit-ID: 1j5siCvR5qt
--HG--
extra : rebase_source : 07afb00de0e4a72af4026eb19ff4f2530c119336
This is just a small piece in the larger project of minimizing the set
of global AC_DEFINEs. None of these are relevant to C/C++ code;
they're mobile/android specific, so we should get them out of
mozilla-config.h.
In addition, this replaces hard-coded values with values from the
build system throughout mobile/android.
MozReview-Commit-ID: 9IJlsm38LFK
--HG--
extra : rebase_source : 0e55c0967cbb7f0df0d297eb330fc615e5759346
Enable stylo in Firefox Nightly builds for macOS so we can
get testing feedback until we have addressed issues blocking
enabling it for developer builds.
MozReview-Commit-ID: 3y0QT4oiBpt
--HG--
extra : rebase_source : 531162da334a6d124dcd7328f45d677cb4933551
When cross-compiling, rust-bindgen needs the -isysroot
flag we pass to the C++ compiler to find the correct
headers. Add a new BINDGEN_CFLAGS environment variable
for passing this and other relevant options, and reformat
its contents in toolchain.configure so we can use autoconf-
style template substitution to poke it into a bindgen.toml
file to be read by build scripts like build_gecko.rs.
Set this variable from the macosx/cross-mozconfig.common
to the same extra flags we pass to CXX so automation
builds work correctly with --enable-stylo.
MozReview-Commit-ID: 7wabObiFtVb
--HG--
extra : rebase_source : eeba30e3d64112da65e2e6830ef5fc1b54965529
The PT_PHDR segment is optional, but the Android toolchain decides to
create one in some cases, and places it first. When that happens, the
work around for bug 1233963 fails, because the fake phdr section has not
been adjusted yet (it only happens when we see a PT_LOAD).
So we adjust the fake phdr section when we see a PT_PHDR segment (and
avoid re-updating it when we see a subsequent PT_LOAD).
--HG--
extra : rebase_source : 2190ec2f20ba9d144b8828874f9f8d70dd5ad2f6
Somehow, with the Android toolchain, we end up with
non-empty-but-really-empty DT_INIT_ARRAYs.
In practical terms, they are arrays with no relocations, and content
that is meaningless:
$ objdump -s -j .init_array libnss3.so
libnss3.so: file format elf32-little
Contents of section .init_array:
1086e0 00000000 ....
$ readelf -r libnss3.so | grep 1086e0
$ objdump -s -j .init_array libplugin-container-pie.so
libplugin-container-pie.so: file format elf32-little
Contents of section .init_array:
4479c ffffffff 00000000 ffffffff 00000000 ................
$ readelf -r libplugin-container-pie.so | grep 4479c
Because so far, elfhack expected meaningful DT_INIT_ARRAYs, it bailed out
early in that case.
--HG--
extra : rebase_source : 217aacb42fdfabb466ed1f8149dfaeb4a595eda8
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: zehcGJrUQX
--HG--
extra : rebase_source : e9ce75ecf18e7d4c829e369f5fbd1df8d6449edf
This gives us source file names with repository info in our generated
symbol files, so that crash reports on crash-stats can link to the
correct source files for files from the Rust standard library.
I've hardcoded the source paths that the Rust project uses, which is
not my favorite thing, but there's no simple way to get this information
otherwise.
MozReview-Commit-ID: 6SeaMqH8xfc
--HG--
extra : rebase_source : 227a52db42b3f86378d744e68b4d88d2d8ba9757
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.
It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.
Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.
MozReview-Commit-ID: C5ofhfJdpCS
--HG--
extra : rebase_source : 952043a441b41b2f58ec4abc51ac15fa71fc142f
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.
It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.
Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.
MozReview-Commit-ID: C5ofhfJdpCS
--HG--
extra : rebase_source : 64aa804965d24bb90b103c00c692a2ac6859e408
Reduce development drag by requiring the most-recent-but-one
stable Rust release. This version is packaged for most
distros, but lets us use more recent library and language
features and spend less time finding work-arounds.
MozReview-Commit-ID: 4W3vkjlKoTu
--HG--
extra : rebase_source : 3828a8be964081bedb2857f9f71f4cd99f75c8be
_LIBCPP_ALWAYS_INLINE and its EXCEPT_GCC49 counterpart are broken in the
same fashion as _LIBCPP_INLINE_VISIBILITY and can be fixed in exactly
the same way.
For parts of configuring Stylo, we need information about the library
extensions on all of our platforms, and this change is a reasonable way
to get at that information without duplicating it in two places. Plus
moving more things to moz.configure is more better.
Before, MSVS was set in old-configure and could only be unset or
"2015." We move the definition of the variable to moz.configure
and support defining its value as "2017" when VS2017 is being used.
As part of this, I discovered that GYP barfs with a "2017" value.
This is likely a limitation of the legacy version of GYP we have
vendored. Rather than go down the rabbit hole of upgrading GYP,
I added code to convert the value to "2015." This preserves existing
behavior and unblocks us from setting MSVS_VERSION properly. A
warning is emitted to remind us to remove this hack once GYP is
upgraded.
After this commit, we now generate native VS2017 solutions and
projects when building with VS2017.
MozReview-Commit-ID: BvNJX3F8qCn
--HG--
extra : rebase_source : 13a495856a83d9ca7afbb4770ebab1cc7f651cae
Gregory Szorc
Mike Hommey
Ryan VanderMeulen
Dan Minor
Bob Owen
Jean-Yves Avenard
Sylvestre Ledru
mozilla@mozilla-23773
Nick Alexander
Chris Manchester
Ralph Giles
Christian Holler
Wes Kocher
rforbes
Carsten "Tomcat" Book
Bruce Sun
Wander Lairson Costa
Ted Mielczarek
Bryce Van Dyk
Randell Jesup
Nathan Froyd
André Bargull
Andrew Osmond