The desired outcome of this change is that we'll set
`-Wl,--version-script` based on linker kind and not on the output of
`$LINKER -v`.
This is a cheap way to address a simple problem that has a complicated
ideal solution. The underlying issue is that in some situations, when
targeting Android, a macOS system `ld` is interrogated to determine if
a cross-compiling linker "is GNU ld" and a particular linker feature
is set in that situation. The macOS system `ld` doesn't pass the "is
GNU ld" test, and the linker feature isn't set; that causes link
failures, even though the actual linker has nothing to do with the
system `ld`.
The ideal solution is to test for linker capabilities dynamically. We
do a lot of that in old-configure.in, and we don't do any of that in
toolchain.configure. Rather than start testing in
toolchain.configure, we hard-code: a cheap solution to the immediate
problem.
Differential Revision: https://phabricator.services.mozilla.com/D8471
--HG--
extra : moz-landing-system : lando
While we do have some uses of @depends-function comparison in some
templaces, related to host/target, we ought to be using `is` comparisons
rather than `==` anyways, so we switch those, and prevent other kinds of
comparisons being used at all.
This unveils the one noted in
https://phabricator.services.mozilla.com/D7713?id=21357#inline-30414
(and surprisingly only that one), that we remove entirely since it was
doing nothing in practice. Bug 1492305 will have to add it back in a
proper form.
Differential Revision: https://phabricator.services.mozilla.com/D8501
--HG--
extra : moz-landing-system : lando
This commit adds some documentation about build telemetry
to the build system documentation. This doc should be used
as a reference during opt-in to build system telemetry,
and includes paths to the relevant files for users to audit
independently. Documentation regarding the schema types and
object properties was generated by the `jsonschema2rst` tool,
available on PyPI.
Differential Revision: https://phabricator.services.mozilla.com/D8030
--HG--
extra : moz-landing-system : lando
If scandir is already present on the system the attempt to import the
c helper library will currently find the c helper from the system
install which may well be an outdated verion, so causing mach to
break. To solve this this patch does two things:
* Stops importing scandir in files that are run unconditionally when
invoking mach. This is generally considered good for performance
reasons.
* Installs the vendored scandir into the virtualenv for `mach lint`
rather than trying to import it directly from the source tree and so
not getting the c helper library.
Differential Revision: https://phabricator.services.mozilla.com/D8379
--HG--
extra : moz-landing-system : lando
This also requires the 64-bits rust compiler and some build system
tweaks.
And since we make the 32-bits builds cross-compiles on CI, we also need
to adjust the MSVC build mozconfigs such that the host compiler points
to the right MSVC cl. Likewise, the DIA SDK is used for host things, so
use the 64-bits version or it.
Differential Revision: https://phabricator.services.mozilla.com/D7845
--HG--
extra : moz-landing-system : lando
Running rustc with an open fd 4 sometimes causes it to fail with an obscure
error when it tries to connect to a job server on that fd. Closing the fd
solves the problem.
close_fds does what we need, and is the default value in Python 3, but not
in Python 2.
Differential Revision: https://phabricator.services.mozilla.com/D8167
--HG--
extra : rebase_source : d94447019e3e698d7cf2c293b0a9b99769cf316a
This patch adds two new actions to virtualenv_packages.txt processing:
windows and !windows. The former processes the rest of the action only on
Windows, and the latter processes it only on non-Windows.
These new features are used in virtualenv_packages.txt to use the
path to the unpacked Windows psutil wheel when on Windows, and build psutil
from source and use that path on other platforms.
This fixes the long-standing problem of not having psutil available on most
Windows systems (since they don't have the right set of Visual C++ build tools).
Differential Revision: https://phabricator.services.mozilla.com/D3437
--HG--
extra : moz-landing-system : lando
Scandir is a faster implementation of os.listdir since it caches file metadata
as it works. Using listdir and then calling things like os.path.isfile after
the fact will result in multiple system calls. Whereas with scandir, there will
only be a single system call per file.
Scandir is part of the stdlib in Python 3.6+, so the following can be used for
Python 2/3 compatible code:
try
from os import scandir, walk
except ImportError:
from scandir import scandir, walk
Differential Revision: https://phabricator.services.mozilla.com/D7738
--HG--
extra : moz-landing-system : lando
The incorrect LEGACY_BUILD_ID will be fixed in a subsequent changeset.
We must add https://www.mozilla.org/ to server-locations.txt and regenerate the mochitest certificates [1] because the new navigator.buildID test pretends to load content from https://www.mozilla.org/.
[1] https://searchfox.org/mozilla-central/source/build/pgo/certs/README
Differential Revision: https://phabricator.services.mozilla.com/D7982
--HG--
rename : dom/tests/mochitest/bugs/test_bug351601.html => dom/tests/mochitest/bugs/test_navigator_buildID.html
extra : rebase_source : 1deb142930f1a7a570cf719c4cb2bed8adfeabe2
extra : source : 408bff32f9623513a271cdf043d11ba6d1318e03
The cctools-port linker links against libraries from clang (for LTO),
which have different SONAMEs depending on the clang version. Which means
the linker needs to be used along the same version of clang it was built
against. Thus we also make it depend on linux64-clang-7.
But changing the dependency is not enough, cf. bug 1471905, so also
touch its build script, which it turns out, we need to do anyways
because llvm-dsymutil was renamed to dsymutil.
Relatedly, all toolchains that are built using cctools-port need to use
linux64-clang-7 too.
Building compiler-rt 7 with the OSX 10.11 SDK fails because of some
newer APIs being used in compiler-rt for xray, but this is not a feature
we use, so disable that.
Differential Revision: https://phabricator.services.mozilla.com/D6766
Bug 1282866 removed the only configuration where it did something.
Since then, using it only leads to a configure error.
Differential Revision: https://phabricator.services.mozilla.com/D7691
--HG--
extra : moz-landing-system : lando
We need to sign parts of the contents of the archives, so the mar's that we
ship get built as part of the repackage task. Thus, there is no reason to also
create and upload as part of the build, just to throw them away.
Differential Revision: https://phabricator.services.mozilla.com/D6213
--HG--
extra : moz-landing-system : lando
The cctools-port linker links against libraries from clang (for LTO),
which have different SONAMEs depending on the clang version. Which means
the linker needs to be used along the same version of clang it was built
against. Thus we also make it depend on linux64-clang-7.
But changing the dependency is not enough, cf. bug 1471905, so also
touch its build script, which it turns out, we need to do anyways
because llvm-dsymutil was renamed to dsymutil.
Relatedly, all toolchains that are built using cctools-port need to use
linux64-clang-7 too.
Building compiler-rt 7 with the OSX 10.11 SDK fails because of some
newer APIs being used in compiler-rt for xray, but this is not a feature
we use, so disable that.
Differential Revision: https://phabricator.services.mozilla.com/D6766
--HG--
rename : build/build-clang/clang-6-macosx64.json => build/build-clang/clang-7-macosx64.json
rename : taskcluster/scripts/misc/build-clang-6-linux-macosx-cross.sh => taskcluster/scripts/misc/build-clang-7-linux-macosx-cross.sh
extra : moz-landing-system : lando
This incorporates some fixes for ccache, icecream, and a better display
output (decreasing count instead of an increasing count). It may also
let us experiment with incremental rust support, though I think that
feature still needs some more work.
MozReview-Commit-ID: 7zPGiL9Ob6N
Differential Revision: https://phabricator.services.mozilla.com/D7152
--HG--
extra : moz-landing-system : lando
Some build flags are being passed by the build system: they're passed in a text
file called extra-bindgen-flags.in that's filled at configure time.
Other flags have to be inferred from the current target/host combination, in
Cranelift's build script directly. This is mostly cargo-culted from the
ServoBindings.toml file, and should probably be merged in the build system at
some point.
Some Windows-specific adjustments were needed to provide access to libclang for
bindgen support, by adding clang-cl to the plain Spidermonkey Windows builds.
--HG--
extra : rebase_source : 0bda40b1d1eb38c2657593f094c951013711d00a
extra : histedit_source : aad930a5f9099e299d385ae4de2deb81aed9b6d5
The ASan runtime wasn't designed to be unloaded, so pin it in memory.
Differential Revision: https://phabricator.services.mozilla.com/D6943
--HG--
extra : rebase_source : 843fc91b3150f3b264e321ab89bf723a01fc8b3c
With libLLVM being a shared library exporting many symbols, all internal
calls using those symbols default to go through the PLT, which is
unnecessary (and costly) overhead. Using -Bsymbolic-functions makes
internal calls go directly to the right place without going through the
PLT.
Differential Revision: https://phabricator.services.mozilla.com/D7029
With libLLVM being a shared library exporting many symbols, all internal
calls using those symbols default to go through the PLT, which is
unnecessary (and costly) overhead. Using -Bsymbolic-functions makes
internal calls go directly to the right place without going through the
PLT.
Differential Revision: https://phabricator.services.mozilla.com/D7029
This extracts the current logic for finding nodejs into its own module in mozbuild. Configure and ESLint then use it.
For ESLint, this will change the first location it looks for nodejs to be the .mozbuild directory.
Differential Revision: https://phabricator.services.mozilla.com/D6430
--HG--
extra : moz-landing-system : lando
This change prevents javascript:-URLs from being passed at the command
line. This restriction was already applied to every URL but the first.
Even the first URL did not result in any visible effect when Firefox is
started. Yet somehow the PGO profiler script managed to rely on it.
This commit ensures that "javascript:" URLs are not activated regardless of
position in the command line, and switches to a data:-URL for the PGO script to
achieve the (previously) desired effect of quitting the browser on startup.
Depends on D4929
Differential Revision: https://phabricator.services.mozilla.com/D5958
--HG--
extra : moz-landing-system : lando
This patch gets us building an updater binary that always embeds the dep certificates (instead of release or nightly), and builds a new tests package that includes it.
This was originally D5900, but that was backed out due to busting artifact builds. I've fixed that by removing the Makefile that Ted pointed out is unnecessary.
Differential Revision: https://phabricator.services.mozilla.com/D6418
--HG--
extra : moz-landing-system : lando
ParmVarDecl being a subclass of VarDecl, using two matchers then caused
ScopeChecker::check to be called twice for ParmVarDecl nodes, once for
each match. But the code in ScopeCheck::check is written with the
assumption that it's called only once for such nodes.
Somehow, this didn't cause problems with clang up to version 6, but
makes the plugin spuriously warn about already_AddRefed not being used
as temporaries when used as argument in function declarations, with
clang 7.
Differential Revision: https://phabricator.services.mozilla.com/D6360
--HG--
extra : moz-landing-system : lando
Doing so changes the size of the compressed toolchain archive from ~280M
to ~120M, and the decompressed size from ~1500M to ~675M. This will
reduce the overhead of decompression during builds.
As we ship llvm-symbolizer as part of ASan builds, we do need it to
still statically link against LLVM, which we do with a small patch.
With LLVM as a shared library, libLTO, which is used by cctools-port for
the linker, is dynamically linked to LLVM, and the cctools-port
configure script fails to link against libLTO. So we add a -rpath-link
to make it find the LLVM library. This happens to force a rebuild of
cctools-port, but for future cases where we might need a rebuild because
of some clang changes, we add a comment to ease the process, and avoid
a newer cctools-port taking the cache spot of an older one.
Ideally, mac cctools-port would need something similar, but it needs a
mac libLTO.dylib, which is not there anyways (and the mac cctools-port
thus already didn't support LTO).
Also, with LLVM built as a shared library, all its symbols are exported
with a LLVM_x.y version. Combined with -static-libstdc++ that is used
during the clang build, this causes problems (see
https://bugzilla.mozilla.org/show_bug.cgi?id=1492037#c7). But it turns
out things have evolved since -static-libstdc++ has been added to the
clang build script, and things work without now, so remove it (as well
as -static-libgcc).
Differential Revision: https://phabricator.services.mozilla.com/D6117
Doing so changes the size of the compressed toolchain archive from ~280M
to ~120M, and the decompressed size from ~1500M to ~675M. This will
reduce the overhead of decompression during builds.
As we ship llvm-symbolizer as part of ASan builds, we do need it to
still statically link against LLVM, which we do with a small patch.
With LLVM as a shared library, libLTO, which is used by cctools-port for
the linker, is dynamically linked to LLVM, and the cctools-port
configure script fails to link against libLTO. So we add a -rpath-link
to make it find the LLVM library. This happens to force a rebuild of
cctools-port, but for future cases where we might need a rebuild because
of some clang changes, we add a comment to ease the process, and avoid
a newer cctools-port taking the cache spot of an older one.
Ideally, mac cctools-port would need something similar, but it needs a
mac libLTO.dylib, which is not there anyways (and the mac cctools-port
thus already didn't support LTO).
Also, with LLVM built as a shared library, all its symbols are exported
with a LLVM_x.y version. Combined with -static-libstdc++ that is used
during the clang build, this causes problems (see
https://bugzilla.mozilla.org/show_bug.cgi?id=1492037#c7). But it turns
out things have evolved since -static-libstdc++ has been added to the
clang build script, and things work without now, so remove it (as well
as -static-libgcc).
Differential Revision: https://phabricator.services.mozilla.com/D6117
This commit updates submit_telemetry_data.py to send data
to the Telemetry pipeline. The script assumes the presence
of a "telemetry" directory within the statedir, and an
"outgoing" directory within the "telemetry" directory (otherwise
there is no data to submit). The script will create a
"submitted" directory and "telemetry.log" file if absent,
making the assumption that this is the first build telemetry
submission for that user. UUID values for submitted data points
are seeded from the filename, without the ".json" suffix.
Differential Revision: https://phabricator.services.mozilla.com/D5687
--HG--
extra : moz-landing-system : lando
This patch gets us building an updater binary that always embeds the dep certificates (instead of release or nightly), and builds a new tests package that includes it.
I had a lot of trouble getting the test package generated correctly due to the fact that things in "_tests" won't be included for test packages that aren't "common". My fix for that isn't ideal - I'm open to something better.
Differential Revision: https://phabricator.services.mozilla.com/D5900
--HG--
extra : moz-landing-system : lando
Nick Alexander
Mike Hommey
David Major
Csoregi Natalia
Connor Sheehan
Narcis Beleuzu
James Graham
Tiberius Oros
Kris Maglione
Dave Hunt
Ted Mielczarek
Andrew Halberstadt
Emilio Cobos Álvarez
Sylvestre Ledru
Brindusan Cristian
Chris Peterson
Noemi Erli
Ciure Andrei
Guillaume GARDET
Kartikaya Gupta
shindli
Mike Shal
Gurzau Raul
Nathan Froyd
Tom Prince
Coroiu Cristina
Chris Manchester
Benjamin Bouvier
Mark Banner
Andi-Bogdan Postelnicu
Rob Wu
Ben Hearsum
Thomas P.
Andreea Pavel
Margareta Eliza Balazs
Bogdan Tara
arthur.iakab