To get an accurate backtrace if the run task crashes, we need the
symbols from the instrumented build.
Depends on D53716
Differential Revision: https://phabricator.services.mozilla.com/D53717
--HG--
extra : moz-landing-system : lando
Having a full VPATH for the srcdir sometimes causes make to grab the
wrong prerequisite for a rule, in particular if we have a file in the
srcdir and also generate a file of the same name in the objdir. We don't
really need VPATH anymore though, since most of the information comes
from mozbuild, where we can explicitly list the path to the srcdir or
objdir as necessary.
Differential Revision: https://phabricator.services.mozilla.com/D42968
--HG--
extra : moz-landing-system : lando
-Fd is the parameter accepted by clang. -PDB is the parameter accepted by lld-link. Additionally, to produce a pdb you apparently also need the -DEBUG flag
Differential Revision: https://phabricator.services.mozilla.com/D43525
--HG--
extra : moz-landing-system : lando
Make depend files related to object files are only really useful when
the corresponding build targets are being processed. OTOH, when they are
included too broadly, when the build recurses in the same directory for
different targets (e.g. target-objects, target, host-objects and/or
host), it is possible for one of them to start creating depend files
that the others would end up reading, and those reads might be
incomplete, possibly leading to build failures because Make think it's
malformed.
So, we only include the object files's depend files in when Make is
explicitly treating a target that will build objects.
Differential Revision: https://phabricator.services.mozilla.com/D42766
--HG--
extra : moz-landing-system : lando
Make the target and host targets depend on those, and flatten the
dependencies.
That is, instead of chains like:
browser/app/target: mozglue/build/target
mozglue/build/target: memory/build/target
we now have:
browser/app/target: browser/app/target-objects \
mozglue/build/target-objects \
memory/build/target-objects
Which means Make can feel free to build the object files in browser/app
before building other dependencies.
Differential Revision: https://phabricator.services.mozilla.com/D42252
--HG--
extra : moz-landing-system : lando
With the addition of toolkit/library/build because of the rust
shenanigans, bug 1573314 and bug 1572046 don't do anything useful
anymore. We're going to do something better anyways.
Differential Revision: https://phabricator.services.mozilla.com/D42251
--HG--
extra : moz-landing-system : lando
Apart from the need to link them last, they don't actually need to be
treated any different from normal static libraries.
Differential Revision: https://phabricator.services.mozilla.com/D42249
--HG--
extra : moz-landing-system : lando
When a directory, like toolkit/library, builds both a static and a
shared library, and another, like toolkit/library/gtest, depends on the
static part, it currently needs to wait for the shared library to be
finished building, preventing both libraries being built in parallel.
By separating shared libraries to a different target, we allow more
parallelism to the build.
Differential Revision: https://phabricator.services.mozilla.com/D41099
--HG--
extra : moz-landing-system : lando
We needed these rules and bits for the QT widget port, but there's no
longer a QT port in the tree, so we might as well remove them.
Differential Revision: https://phabricator.services.mozilla.com/D38886
--HG--
extra : moz-landing-system : lando
The condition that enabled ELOG to do something has been broken since
bug 279212 landed two years ago, because MAKEFLAGS doesn't contain the
dash for flags, so looking for "-s" never did anything.
Bug 1560527 changed the condition to !BUILD_VERBOSE_LOG, which as of the
previous commit, does work. This would restore the old behavior, at the
expense of build time, since it involves wrapping in a shell script,
which is not necessarily desirable (and hasn't been done for two years
now)
Differential Revision: https://phabricator.services.mozilla.com/D35967
--HG--
extra : moz-landing-system : lando
Bug 1560527 was not supposed to change verbosity for mach build, but it
turns out it did, because the ifeq it copied from one place to another
was wrong in the first place.
While here, replace a ifeq that did work with the now equivalent
BUILD_VERBOSE_LOG.
Differential Revision: https://phabricator.services.mozilla.com/D35966
--HG--
extra : moz-landing-system : lando
Windows 1-tier PGO builds only partially clobber between the
profile-generate and profile-use stages, so that exports/installed files
don't have to be reprocessed. Unfortunately we can't skip the install
manifests in 3-tier PGO because the profile-generate build happens on a
different machine, so we have to differentiate between 1-tier and 3-tier
PGO builds. A new variable, MOZ_1TIER_PGO, is used for this purpose.
Eventually this logic can be cleaned up in bug 1557788 once all PGO
builds use the 3-tier model.
Differential Revision: https://phabricator.services.mozilla.com/D34802
--HG--
extra : moz-landing-system : lando
This makes running without mach more consistent. e.g. running
`make -C $objdir/toolkit/library/rust target` makes the cargo log
verbose, and adding `-s` makes it less verbose.
Differential Revision: https://phabricator.services.mozilla.com/D35521
--HG--
extra : moz-landing-system : lando
We're moving to IR-level PGO instrumentation for clang-cl. We've also
moved to using static linker ordering files, which was the primary
application of the previous style of PGO instrumentation. We therefore
we no longer need this code.
Differential Revision: https://phabricator.services.mozilla.com/D31134
--HG--
extra : moz-landing-system : lando
We used to support specifying single Rust files for compilation,
emphasis on "used to".
Differential Revision: https://phabricator.services.mozilla.com/D32314
--HG--
extra : moz-landing-system : lando
This way compilation backends don't all have to figure out the right way
to handle symbol files.
Differential Revision: https://phabricator.services.mozilla.com/D29673
--HG--
extra : moz-landing-system : lando
This will allow us to copy the pdbs for test files into the resulting
test file archive, but doesn't do it automatically.
Bug 1505936 is for copying some of them.
Differential Revision: https://phabricator.services.mozilla.com/D29761
--HG--
extra : moz-landing-system : lando
Bug 1533425 makes Gecko try to load from $ARCH/greprefs.js, etc on
Android. This patch teaches the packager to put preferences into
those architecture-specific locations for that code to find.
Differential Revision: https://phabricator.services.mozilla.com/D24984
--HG--
extra : moz-landing-system : lando
Consequently, this removes:
- MOZ_LIBPRIO, which is now always enabled.
- non_msvc_compiler, which is now always true.
- The cl.py wrapper, since it's not used anymore.
- CL_INCLUDES_PREFIX, which was only used for the cl.py wrapper.
- NONASCII, which was only there to ensure CL_INCLUDES_PREFIX still
worked in non-ASCII cases.
This however keeps a large part of detecting and configuring for MSVC,
because we still do need it for at least headers, libraries, and midl.
Depends on D19614
Differential Revision: https://phabricator.services.mozilla.com/D19615
--HG--
extra : moz-landing-system : lando
Because we're going to change how cargo recipes are called to export
environment variables rather than by wrapping the call with `env`, to
avoid msys roundtrips, it's better to avoid the complexity when not
building rust, and including a separate file only when required helps
with that. It is also possible to wrap the entire rust section of
rules.mk in the same condition we use for the include, but using a
separate file also makes things clearer.
Differential Revision: https://phabricator.services.mozilla.com/D18180
--HG--
rename : config/rules.mk => config/makefiles/rust.mk
extra : moz-landing-system : lando
We default to -Zi for Windows debug flags, which is for generating debug info
into a PDB file, but clang-cl doesn't actually implement that so -Zi is
an alias for -Z7 in clang-cl:
http://clang.llvm.org/docs/UsersManual.html#id9
sccache has special handling for -Zi because multiple compiles writing
to the same PDB file is not cacheable, so we've always overridden -Zi with -Z7
in CI when using sccache.
Given that everyone should be using clang-cl nowadays and MSVC will no longer
be supported soon, this patch changes the default and removes some PDB
file name flag setting from rules.mk, as well as the no-longer-necessary
overrides from mozconfig.cache.
Differential Revision: https://phabricator.services.mozilla.com/D18264
--HG--
extra : moz-landing-system : lando
Now that everything is lined up nicely, we don't need those anymore.
We can fill cargo-linker.bat with HOST_LINKER and HOST_LINKER_LIBPATHS,
which, unfortunately, can't be used as is, so we add a variant for
substitution in .bat files.
Differential Revision: https://phabricator.services.mozilla.com/D18034
--HG--
extra : moz-landing-system : lando
The only use in configure itself is for a MSVC version check that is now
always true (we don't accept versions < 19.15 anymore).
The only uses in the build system are in code that could just use
CC_TYPE instead.
Differential Revision: https://phabricator.services.mozilla.com/D17207
--HG--
extra : moz-landing-system : lando
Because MSVC compilers only support one architecture, we need to search
"cl" in different toolchain search paths for each of the host and
target, especially when they are different.
Likewise for the library paths for the linker. Ideally we'd pass
-LIBPATH both for host and target, but that has implications for rust
that I don't want to have to figure just now.
Depends on D15263
Differential Revision: https://phabricator.services.mozilla.com/D15264
--HG--
extra : moz-landing-system : lando
None of the values tested against OS_TEST are actually possible per
split_triplet in build/moz.configure/init.configure, so the code is
dead in practice.
Differential Revision: https://phabricator.services.mozilla.com/D16161
--HG--
extra : moz-landing-system : lando
The build system has skipped creating target static libraries for very
long, except in very specific cases.
We can actually do the same for host static libraries, for which we
don't even need the escape hatch to still allow to create static
libraries.
Depends on D15171
Differential Revision: https://phabricator.services.mozilla.com/D15172
--HG--
extra : moz-landing-system : lando
OBJ_SUFFIX is modified during the profile-generation phase to be i_o
instead of o/obj. _OBJ_SUFFIX is the unmodified value.
We don't actually do PGO for host objects, so we don't need to build the
objects with a different suffix.
Differential Revision: https://phabricator.services.mozilla.com/D15171
--HG--
extra : moz-landing-system : lando
This patch also changes how pdbs for the ASAN job are copied:
we relax restrictions so that pdbs if present) are always copied out
and add an environment variable MOZ_COPY_PDBS to indicate when we
want to produce pdbs for copying.
- Embed vendored Rust dependencies into the source package.
- Don't use --freeze with Cargo in JS standalone builds: since all the Rust
dependencies are vendored in, it won't trigger an update of the toplevel
Cargo.lock file for a standalone build in the non-package build case.
--HG--
extra : rebase_source : 06704eb326aad36a870586694a99964ad7a4dbfa
extra : histedit_source : 1502d976bc12a3eed17efaaf5dd2d4f46d0b00d5
This makes it more apparent which directory is responsible for creating
a particular generated file, program, library, etc. It is particularly
helpful in bug 1485081 since all node.py invocations will have similarly
named make targets, but run in different directories.
MozReview-Commit-ID: HlSOiL34i3M
Differential Revision: https://phabricator.services.mozilla.com/D6822
--HG--
extra : moz-landing-system : lando
BUILD_TOOLS was only ever used for things that another variable provides
equally well. Removing BUILD_TOOLS means that we can remove win_srcdir
and WIN_TOP_SRC as well.
All but one of the current uses of DEFFILE use `SRCDIR + '/file.def'` to
get a srcdir-relative path anyway, and the other one wants an
objdir-relative path, so using Path makes everything clearer.
This makes it more straightforward to translate the paths for the WSL
build.
MSVC 15.8 linker dislikes forward slashes in the /OUT: parameter when it is generating a profile
--HG--
extra : rebase_source : 6980e212ea5a7ae21513805c22ec0e8ddeaa5dc1
MSVC 15.8 linker dislikes forward slashes in the /OUT: parameter when it is generating a profile
--HG--
extra : source : d0098304d5b1a35446781d60a9050d310b45f5bd
Previously we weren't passing the variables if we were compiling for Windows
when really we only want to do it if we're compiling with MSVC/clang-cl.
The MinGW-Clang build needs this.
Differential Revision: https://phabricator.services.mozilla.com/D3470
--HG--
extra : moz-landing-system : lando
This setup seems to work well enough to enable me to link
HOST_SIMPLE_PROGRAMS with an AArch64-cross setup. Necessary library
paths are passed to the linker via -LIBPATH and HOST_LDFLAGS rather than
letting MSVC fish them out of the environment. The change to
HOST_SIMPLE_PROGRAMS to pass HOST_LDFLAGS was necessary for this to
work, in addition to the HOST_LINKER changes.
For "real" Windows-to-Windows cross compiles, the setting of
HOST_OUTOPTION is incorrect: it assumes that if we are cross-compiling,
we'll be using `-o ` (GNU-style) rather than `-Fo` (MSVC-style). Our
normal x86 Windows automation builds are technically not
cross-compiles (host and target are both x86 Windows), so this case has
never bothered us before. But when compiling for AArch64 Windows, we
are really doing a cross-compile, and so we need to be more careful
about how we set this option; otherwise, host compilations will
mysteriously fail because they won't produce any output.
For clang-cl, we want to add code to libxul that only exists during the
PGO generation phase, so we can collect data. The most expedient way to
do that is to enable certain files in SOURCES to be marked as to only be
compiled during the PGO generation step.
This adds just enough host shared library support for this one use case,
but also takes shortcuts, because fully supporting host shared library
is a deep rabbit hole I'm not ready to take just to fix --enable-lto
--enable-clang-plugin on mac builds.
One downside is that one my machine the plugin now takes > 80s to build,
instead of 15s before, thanks to the lack of unified sources.
--HG--
extra : rebase_source : bf52a72a01d4e3eb77cf52b646b19734b9273075
Many Rust build scripts compile C/C++ source files with the cc crate, but
cargo doesn't print the output of build scripts unless you pass `-vv`, so
pass that instead of just `--verbose` to get that output in automation and
builds where verbose output was requested.
MozReview-Commit-ID: EUazlKWFsDw
--HG--
extra : rebase_source : 137882c4e970bb0e7b28cc36d7f8e436b59a8011
We perform, on the binaries we build, a series of check, that are
implemented as half-baked make commands, invoked after linking them.
- check libstdc++ symbol versions to ensure binary compatibility with
a baseline.
- check glibc symbol versions to ensure binary compatibility with a
baseline.
- check that target binaries don't contain text relocations.
- check that libmozglue is linked before libc on android.
- on libxul, check that NSModules are laid out correctly.
- on libxul, check that there is more than one PT_LOAD segment.
Those checks happen to work where they matter, but their setup is
unreliable. For example, the checks for symbol versions are supposed to
work for libclang-plugin on cross osx builds, but in fact, don't,
because the readelf path doesn't exist, and the command doesn't fail in
that case.
So move them all to a standalone script, performing the checks more
thoroughly (especially the NSModules one, where we now also check that
they are all adjacent), and more verbosely.
--HG--
extra : rebase_source : 7072e622e95f363d4a6c3a8e272d3445d998b592
The crash reporter symbol files are the easiest cross-platform way to
find static initializers. While some types of static initializers (e.g.
__attribute__(constructor) functions) don't appear there in a notable
way, the static initializers we do care the most about for tracking do
(static initializers from C++ globals). As a matter of fact, there is
only a difference of 2 compared to the currently reported count of 125
on a linux64 build, so this is a good enough approximation. And allows
us to easily track the count on Android, OSX and Windows builds, which
we currently don't do.
The tricky part is that the symbol files are in
dist/crashreporter-symbols/$lib/$fileid/$lib.sym, and $fileid is hard to
figure out. There is a `fileid` tool in testing/tools, but it is a
target binary, meaning it's not available on cross builds (OSX,
Android).
So the simplest is just to gather the data while creating the symbol
files, which unfortunately requires to go through some hoops to make it
happen for just the files we care about.
--HG--
extra : rebase_source : 458fed1ffd6f9294eefef61f10ff7a284af0d986
Now that we're no longer shipping the SDK we no longer need real libraries for
the libraries that were created by this rule.
MozReview-Commit-ID: ALATVGBayHu
--HG--
extra : rebase_source : a20905125e5ff1846ef29de12323ba7b0a58928b
The MSVC linker winds up generating import libraries when linking some of
our executables, presumably because they contain functions that are
__declspec(dllexport). By default the import libraries get written
alongside the exe, so we force them to be written to the objdir so they don't
clutter up dist/bin.
MozReview-Commit-ID: 7DTfCo3OdDQ
Historically we built all our binaries in directories in the objdir, then
symlinked them into dist/bin. Some binaries needed to be copied instead
so that certain relative path lookups work properly, so we resorted to
sprinkling `NSDISTMODE=copy` around Makefiles.
This change makes it so we build PROGRAMs (not any other sort of targets)
directly in dist/bin instead. We could do the same for our other targets
with a little more work.
There were several places in the tree that were copying built binaries to
some other place and needed fixup to match the new location of binaries.
On Windows pdb files are left in the objdir where the program was
originally linked. symbolstore.py needs to locate the pdb file both to
determine whether it should dump symbols for a binary and also to copy
the pdb file into the symbol package. We fix this by simply looking for
the pdb file in the current working directory if it isn't present next
to the binary, which matches how we invoke symbolstore.py.
MozReview-Commit-ID: 8TOD1uTXD5e
The last APK produced using the ANDROID_APK_* moz.build/Makefile.in
mechanism was Robocop, so we can get rid of these now.
MozReview-Commit-ID: 9b08ZvvOAoC
--HG--
extra : rebase_source : ac4fea057bf6e731b0f26a1b6902f17a7362076d
Now that writing the .purgecaches sentinel is done by |mach build|, we
can remove it from Make. In addition, we can cull the now-unused
app-rules.mk \o/
MozReview-Commit-ID: 6CnAqLeZwzB
--HG--
extra : rebase_source : 2a5b67e723f8e1654627e6e6f26704f6a4e57706
I very much doubt these are used, but even if we are -- we shouldn't
support this type of local customization, since it doesn't extend to
non-Make-based backends.
With the customization point removed, there's no way to set ETAGS, so
we remove what little support there was for generating Emacs tags.
MozReview-Commit-ID: IEF2Q4tISEn
--HG--
extra : rebase_source : 3bc8e651c03517edb797032db6ce60ed8852d9fa
Now that writing the .purgecaches sentinel is done by |mach build|, we
can remove it from Make. In addition, we can cull the now-unused
app-rules.mk \o/
MozReview-Commit-ID: 6CnAqLeZwzB
--HG--
extra : rebase_source : 7427ca9aecc6619f48090c6a26c9a2e0e7d37bad
Mike Shal
Ricky Stewart
Tom Ritter
Mike Hommey
Nathan Froyd
Noemi Erli
Nick Alexander
Ted Mielczarek
Chris Manchester
Marco Castelluccio
Benjamin Bouvier
Thomas P.
Andreea Pavel
arthur.iakab
Adam Gashlin
Masatoshi Kimura
Margareta Eliza Balazs
Gurzau Raul
Myk Melez
Narcis Beleuzu
Aaron Klotz
Nathan Froyd
David Major
Henri Sivonen
Xidorn Quan
Coroiu Cristina