At the same time, remove HOST_LD. It was only used for MSVC builds,
which don't support cross-compile anyways, so we can, at least for now,
use LINK for both host and target.
--HG--
extra : rebase_source : 9ee9e7e1bd3edefc043fa63d5c03f2a242f76982
The only complicating factor here is having to split out the --target
flag from cargo_build_flags, so we can pass the appropriate one
depending on our build target.
In preparation for a world where we have cargo building binaries too,
the existing rules should be renamed to reflect their library
associations. The lone Cargo invocation should be updated to explicitly
build libraries only, so libraries and binaries in the same directory
will work correctly.
Fixup a leftover comment from a previous set of changes while we're here.
Combined with the previous patch that sets MACH_STDOUT_ISATTY, the
practical effect of this patch is that cargo is invoked with
`--color=always` when mach was attached to a TTY and colorized output
is sent to the terminal.
Note: this doesn't work with Rust/Cargo 1.10 for reasons unknown to me.
It appears there was a bug with Rust/Cargo because `--color=never` still
sent colorized output on that version! Cargo/Rust 1.12.1 works fine.
MozReview-Commit-ID: 6uXS3t3413i
--HG--
extra : rebase_source : 72df6e8248cb14c96850964f694576c7fd2f5fbb
In our current Rust world, we have the following dependency structure:
xul.so --------------------------+
|
xul-gtest.so -+--> xul.a --------+-> gkrust
|
+--> gkrust-gtest
This structure results in link errors with multiply-defined symbols
between gkrust-gtest and gkrust with newer Rust releases when linking
xul-gtest.so. So we have to do something different.
Our new structure is:
xul.so --------------------------+
|
xul-gtest.so -+--> xul.a --------+-> gkrust --+-> gkrust-shared
| |
+--> gkrust-gtest --------------+
and we enforce that a given shared library can only have at most one
Rust library that it depends on. Said Rust library is assumed to
include all significant Rust dependencies of the dependent static
libraries as well. (In the above structure, gkrust is simply a wrapper
around gkrust-shared, so gkrust-gtest doesn't have to include gkrust as
a dependency.)
Currently make check loops over all directories and runs each test file in PYTHON_UNIT_TESTS
individually. This patch instead creates a single top-level check target that runs
|mach python-tests|. This should make automation more similar to running locally, be a bit
quicker and make it easier to pull python tests out of make check completely at a future date.
MozReview-Commit-ID: 4Hg4zdFyc61
--HG--
extra : rebase_source : 31d0c38a76c11b75d3bf569f2bf22a0666161726
This is relevant when attempting to build libffi with our build system
(libffi expects its assembly sources to be pre-processed).
MozReview-Commit-ID: J6mCUkYYjQh
This patch is really two separate changes.
The first change is that rust crates are large, standalone entities that
may contain multitudes of source files. It therefore doesn't make sense
to keep them in SOURCES, as we have been doing. Moving to use cargo
will require a higher-level approach, which suggests that we need a
different, higher-level representation for Rust sources in the build
system.
The representation here is to have the build system refer to things
defined in Cargo.toml files as the entities dealt with in the build
system, and let Cargo deal with the details of actually building things.
This approach means that adding a new crate to an existing library just
requires editing Rust and Cargo.toml files, rather than dealing with
moz.build, which seems more natural to Rust programmers. By having the
source files for libraries (and binaries in subsequent iterations of
this support) checked in to the tree, we can also take advantage of
Cargo.lock files.
The second is that we switch the core build system over to building via
cargo, rather than invoking rustc directly.
We also clean up a number of leftover things from the Old Way of doing
things. A number of tests are added to confirm that we'll only permit
crates to be built that have dependencies in-tree.
Through an oversight, we listed librul.a twice when linking libxul: once
as part of the "objects" we were linking, and once as a static library.
This duplication is unnecessary and would cause problems later when we
try to generate librul.a via cargo, as cargo will put it someplace
different from where we expect and the two names will conflict. Let's
have rules.mk be the single source of truth for how librul.a is named,
and then the code to link libxul can simply refer to that name.
This patch is really two separate changes.
The first change is that rust crates are large, standalone entities that
may contain multitudes of source files. It therefore doesn't make sense
to keep them in SOURCES, as we have been doing. Moving to use cargo
will require a higher-level approach, which suggests that we need a
different, higher-level representation for Rust sources in the build
system.
The representation here is to have the build system refer to things
defined in Cargo.toml files as the entities dealt with in the build
system, and let Cargo deal with the details of actually building things.
This approach means that adding a new crate to an existing library just
requires editing Rust and Cargo.toml files, rather than dealing with
moz.build, which seems more natural to Rust programmers. By having the
source files for libraries (and binaries in subsequent iterations of
this support) checked in to the tree, we can also take advantage of
Cargo.lock files.
The second is that we switch the core build system over to building via
cargo, rather than invoking rustc directly.
We also clean up a number of leftover things from the Old Way of doing
things. A number of tests are added to confirm that we'll only permit
crates to be built that have dependencies in-tree.
Through an oversight, we listed librul.a twice when linking libxul: once
as part of the "objects" we were linking, and once as a static library.
This duplication is unnecessary and would cause problems later when we
try to generate librul.a via cargo, as cargo will put it someplace
different from where we expect and the two names will conflict. Let's
have rules.mk be the single source of truth for how librul.a is named,
and then the code to link libxul can simply refer to that name.
Configure uses the value of the MT environment variable before falling back
to mt(.exe), but the build system was completely ignoring the MT
environment variable.
--HG--
extra : rebase_source : 8c9b43aeb08493ae5bd6d6361f4f18f097ea0553
Our current build system support for Rust compiles any Rust crate into a
so-called staticlib, which is a static library (.a file) that includes
the Rust runtime. That staticlib is then linked into libxul. For
supporting multiple crates, this approach breaks down, as linking
multiple copies of the Rust runtime is going to fail.
For supporting multiple crates, the approach taken here is to compile
each crate into a so-called rlib, which is essentially a staticlib
without the Rust runtime linked in. The build system takes note of
every crate destined for linking with libxul (treating them like static
libraries generated from C/C++ files), and generates a super-crate,
whimsically named "rul", that is compiled as a staticlib (so as to
include the Rust runtime) and then linked into libxul. Thus only one
copy of the Rust runtime is included, and the Rust compiler can take
care of any inter-crate dependencies.
This patch currently only supports Rust code in shared libraries, not in
binaries.
Our current build system support for Rust compiles any Rust crate into a
so-called staticlib, which is a static library (.a file) that includes
the Rust runtime. That staticlib is then linked into libxul. For
supporting multiple crates, this approach breaks down, as linking
multiple copies of the Rust runtime is going to fail.
For supporting multiple crates, the approach taken here is to compile
each crate into a so-called rlib, which is essentially a staticlib
without the Rust runtime linked in. The build system takes note of
every crate destined for linking with libxul (treating them like static
libraries generated from C/C++ files), and generates a super-crate,
whimsically named "rul", that is compiled as a staticlib (so as to
include the Rust runtime) and then linked into libxul. Thus only one
copy of the Rust runtime is included, and the Rust compiler can take
care of any inter-crate dependencies.
This patch currently only supports Rust code in shared libraries, not in
binaries. The handling for the rul crate is placed in the common
backend, with a special hook for derived backends to handle shared
library objects.
Multiple people have complained that the build output of printing the
source files being built adds little value. I agree. The extra output
doesn't give really helpful progress info because sources can be built
in non-deterministic order. Furthermore, the extra output hides useful
output like compiler warnings.
This patch makes the default build system output even less verbose. We
no longer print the individual source targets when they are built. We do
still print the targets for binaries, so some sense of progress can be
inferred.
If people like verbosity, they can export the undocumented
BUILD_VERBOSE_LOG environment variable can be set to restore the old
behavior.
MozReview-Commit-ID: KeaeJJkjPJn
--HG--
extra : rebase_source : a90b4b82eeef48d9bd896b5c12907ce05a650d4d
It's an opt-in flag that allows to display where the build is in
terminal window titles. The fact that it's opt-in and likely unknown
makes it very low-value, and the fact that it was added in an era where
builds were not very well parallelized made it have a meaning, but now
that builds are parallelized, its meaningfulness is diminished.
Let's just remove it.
Nothing uses this variable. blame suggests that it was used for HP-UX
once upon a time. The companion variable, MOZ_POST_PROGRAM_COMMAND, is
only used by HP-UX, but as we're not wholesale removing HP-UX
support (yet), we should leave MOZ_POST_PROGRAM_COMMAND alone.
The current rule is only for "backend.RecursiveMakeBackend", but, with
the current default of generating both the RecursiveMake and FasterMake
backends, the command creates/refreshes both backends. This is, in fact,
how the FasterMake backend is refreshed in most cases.
Moreover, with an hybrid backends, the generated file is not
"backend.RecursiveMakeBackend" anymore, so we need a more generic way to
handle this.
Furthermore, it's not necessarily desirable for all backends to have a
dependency file to handle the dependencies to refresh the backend, so
generate a plain list instead. This has the side effect of making `mach
build-backend --diff` more readable for changes to that file.
Finally, make the backend.* files created like any other backend file,
such that its diff appears in the `mach build-backend --diff` output.
GENERATED_FILES impacts the export tier through the config/rules.mk
definitions, now moved to the backend itself, so that everything is
close to each other.
Collecting the list of object files compiled, while not ideal, will give us
some indication of how much work was involved in the build. This will help
with analyzing the data.
--HG--
extra : rebase_source : e9861ed5c0766e3ee8038dbec0b9267022c523eb
Limit ourselves to include paths for now, because there are tricky things
involved in making this globally.
While here, use shell_quote instead of manual quoting for those paths.
MOZ_DEBUG_DEFINES are essentially defines used everywhere. So treat them as
feeding the initial value for DEFINES in each moz.build sandbox. This allows
the kind overrides that was done in the past by resetting MOZ_DEBUG_DEFINES
in Makefiles.
Currently, one needs to define DEFFILE or LD_VERSION_SCRIPT appropriately,
and somehow deal with the fact that their input format is different, which
currently relies on manual invocations of the convert_def_file script, with
awkward aggregations.
This simplifies the problem by using a simple list of symbols, with
preprocessing, allowing #includes.
Currently, only css files added through jar manifests are treated this way.
There is really no reason for the discrepancy, but there are actually no css
files added directly through moz.build, so this was never a problem.
On the other hand, it makes things simpler in a world where jar manifests are
treated as if they were entirely described in moz.build (which is where the
FasterMake backend is heading).
Using TEST_DIRS is nothing more than a shortcut for
if CONFIG['ENABLE_TESTS']:
DIRS += [...]
As such, we might as well remove it being a separate variable, and use some
Context magic to just fill DIRS when ENABLE_TESTS is set.
The security/manager/ssl/tests/unit/moz.build change ensures that the order
of DIRS before the change is kept, not because it matters, but because it
allows to confirm that nothing else is modified by this change.
GENERATED_FILES and accessible/xpcom/Makefile.in add to EXTRA_MDDEPEND_FILES, but for
targets that run during export. Export doesn't include EXTRA_MDDEPEND_FILES, so none
of them is ending up with correct dependencies. The EXTRA_EXPORT_MDDEPEND_FILES variable
could be used for this purpose, but given the circumstances this variable is removed,
and EXTRA_MDDEPEND_FILES is instead included unconditionally.
--HG--
extra : commitid : 7daRRnxfkS0
The configure option has explicitly thrown an error for more than a year now,
and it happens that the remaining way to still forcefully use it has been
broken for more than 8 months.
It was added back in
5147d5c69f
for unclear reasons (and the lack of bug number doesn't help), and hasn't been
used, as far as I can see in the gecko-dev history, other than in bug 206029,
which is the only use currently in the tree.
Bug 206029 was working around the Flash player installer modifying Firefox's
prefs file and not dealing with it properly or something depending on the line
endings. 11 years later, all prefs files except channel-prefs.js are in
omni.ja, so obviously, bug 206029 doesn't actually apply anymore.
So, let's simplify it all and get rid of this.
The flags added in toolkit/locales/Makefile.in turn out not to be actually
used, so just remove that.
The remaining uses of XULPPFLAGS are to set debug flags depending on whether
MOZ_DEBUG is set or not. Just set a dedicated variable with the right value
from configure.
Optimised Rust compilation is enabled on passing --enable-optimize to
the configure script. This sets the RUSTFLAGS output variable that gets
picked up by the compile targets RSOBJS and RSSRCS and passed to rustc.
r=glandium
--HG--
extra : commitid : 8thSkfLFXSY
extra : rebase_source : 5ec79b76a187bcbb0f09ad374cf9f763f0adb0d7
Add .rs as a recognized file extension in SOURCES.
Propagate that through to the Makefile backend and add a dependency
generated and an explicit rule to call $(RUSTC) to compile them.
rustc builds static libraries, not obj files. At least, if one
asks it to output an obj file, I'm not clear how to get all the
compiler-specific runtime libraries the code will expect to link
to. Therefore we generate a static library for each rust source
file (which must be a complete crate for the time being) and link
that. Because of the extension it ends up on the LIBS line in the
the corresponding .desc file.
Note that the static library does still depend on some system
libraries, e.g. -ldl -lpthread -lm on linux. Gecko already
links to all of those, so we don't keep track of it here.
Should we need to add explicit linkage for other targets,
rustc does print a list to stderr which can be parsed.
--HG--
extra : rebase_source : 9f66101fd15a649a952d5481cf9561416204272a
Now that moz.build can see EXTRA_*COMPONENTS and NO_JS_MANIFEST, we can
move some logic from rules.mk (executed every build) to moz.build's
emitter.py (executed only at build-backend time).
Now that the mozbuild backend knows about FINAL_TARGET, we are able to
install generated xpt files into their final location. This saves us
from copying xpt files into their final location on every build.
Original patch by gps, rebased and comments addressed by Ms2ger
--HG--
extra : transplant_source : %E2%DC%0F%E0%AD%C2%25%A1%B8%A9%FE%B0%8C%60%FF%CB%02G%25%E5
See the revisions a few steps prior for more fine-grained information. This just
needs to be relanded because our automation doesn't like non-ASCII commit
messages.
--HG--
extra : amend_source : 0ee4e0da1ea9ada815abfe989c51030db24aac56
CLOSED TREE
Backed out changeset 28abb8af2d62 (bug 1099430)
Backed out changeset 358aa39360d5 (bug 1099430)
Backed out changeset 3313e545f4f6 (bug 1099430)
With these substitutions, it is possible to have comm-central's versions of
these files merely include the mozilla-central versions and avoid having to port
small changes to these files.
This change allows us to use use MOZILLA_DIR even for changes in config.mk. The
corollary is that comm-central needs to define MOZILLA_DIR before calling into
baseconfig.mk, which it already does.
The change to the test Makefile is needed since it decides to skip including
config.mk, and hence baseconfig.mk, where it would need to grab the MOZILLA_DIR
declaration.
buildlist invocations are slow and can occur in parallel since the
underlying program obtains a lock on the modified file.
Moving the XPT-related buildlist invocation from the serial libs tier to
the parallel misc tier decreased my no-op build time on OS X from 43.5s
to 37.0s.
--HG--
extra : rebase_source : 7d274024c401b1ecfbc771424a69eb487808fcbf
The installation rule for EXTRA_JS_MODULES and EXTRA_PP_JS_MODULES
became unused after b961ba8f0892 (bug 1044162). We remove the dead code.
--HG--
extra : rebase_source : 40adf7514d15ae4ba5bbfe3ac101061657aac841
This used to be necessary to avoid the conflicting names between import
libraries and static libraries, but that's now prevented by the whole
moz.build machinery.
There are, sadly, many combinations of linkage in use throughout the tree.
The main differentiator, though, is between program/libraries related to
Gecko or not. Kind of. Some need mozglue, some don't. Some need dependent
linkage, some standalone.
Anyways, these new templates remove the need to manually define the
right dependencies against xpcomglue, nspr, mozalloc and mozglue
in most cases.
Places that build programs and were resetting MOZ_GLUE_PROGRAM_LDFLAGS
or that build libraries and were resetting MOZ_GLUE_LDFLAGS can now
just not use those Gecko-specific templates.
Ever since bug 969164, the js build system, when building gecko (not when
building standalone) uses a autoconf-js.mk file for its config.
One of the suboptimal ways we have to retrigger builds when the build
configuration changes (changes to e.g. configure.in can do that) is to
make most things depend on autoconf.mk. Which unfortunately doesn't
account for the fact the js/src subdirectory uses a different file.
In practice, this means that some classes of changes to the js build
system, not accompanied with toplevel build system changes may no
trigger the corresponding rebuilds in the js subtree on incremental
builds.
Currently, when there is both an expandlibs descriptor and an actual static
library, expandlibs picks the static library. This has the side effect that
if there are object files in the static library that aren't directly used,
they're dropped when linking, even when they export symbols that would be
exported in the final linked binary.
In most cases in the code base, files are not dropped that way. The most
notable counter-example is xpcomglue, where actually not dropping files
leads to link failure because of missing symbols those files reference
(yes, that would tend to say the glue is broken in some way).
On the opposite side, there is mozglue, which does have both a descriptor
and a static library (the latter being necessary for the SDK), and that
linking as a static library drops files that shouldn't be dropped (like
jemalloc). We're currently relying on -Wl,--whole-archive for those files
not to be dropped, but that won't really be possible without much hassle
in a world where mozglue dependencies live in moz.build land.
Switching expandlibs to use descriptors when they exist, even when there
is a static library (so, the opposite of the current behavior) allows to
drop -Wl,--whole-archive and prepare for a better future. However, as
mentioned, xpcomglue does still require to be linked through the static
library, so we need to make it a static library only.
To achieve that, we make NO_EXPAND_LIBS now actually mean no expandlibs
and use that to build the various different xpcomglues.
OS_LIBS for libraries that are not part of the gecko tree, EXTRA_LIBS for
libraries, such as NSPR, that are in the tree, but are not handled by
moz.build just yet. Those EXTRA_LIBS may also come from a system library.
However, in cases where the expanded variables are always empty for the
in-tree case, OS_LIBS is used (as for, e.g. MOZ_ZLIB_LIBS). OS_LDFLAGS is
used exclusively for non-library linker flags.
Always pass EXTRA_LIBS before OS_LIBS on linker command lines.
Forbid EXTRA_DSO_LDOPTS, SHARED_LIBRARY_LIBS and LIBS in Makefiles.
Before making top-level traverse js/src moz.build files, there was a need to
distinguish between top-level traversing e.g. top-level moz.build or
config/moz.build and js/src traversing them. With a single traversal of both
moz.build sets, we now only need to distinguish between js standalone builds
and gecko builds.
There is still, however, a need to distinguish between top-level vs. js/src
configure runs on gecko builds to make them subconfigure icu and libffi from
top-level instead of js/src in js standalone builds, or when choosing to make
js/src's config.status do something when run or not.
Every directory with a jar.mn now has JAR_MANIFESTS defined in its
moz.build file.
We also removed the may_skip special consideration of jar.mn files
because this information is now available during tier traversal by the
reader courtesy of the variables being present in moz.build files.
--HG--
extra : rebase_source : 21049b15e6bd9cf65b0805ccaccc4ba5aae93c98
extra : amend_source : 0b1ea866d725beef92d37c6f6d475369ac002e19
Before, we checked if config.status was stale in any entrant Makefile
(top level or child directory). This had undesirable side-effects for
partial tree builds, notably that if the build backend was out of date,
the current Makefile was invalidated.
With this patch, we only regenerate the build config automatically in
full/toplevel builds. If an outdated build config is detected on partial
tree builds, we error. The impact of this is mitigated by having mach
build automatically ensure the build config is current.
--HG--
extra : rebase_source : a3a2c1d2bf68d1c0727352ebd9bfb785654e45bf
Host and target linkers may not support the same kind of lists expandlibs uses.
On the other hand, lists are only really needed for libxul, where the command
line length or number of arguments may overflow the kernel support. So just
don't use them for host linkage.
However, since we don't generate depfiles when compiling assembly, explicitly exclude those.
Also, fix a few issues with included depfiles:
- The tools tier need them because it (sadly) links binaries.
- depfiles for TARGETS use the full path, whereas the depfiles for OBJS and others don't.
The existing tier traversal logic is only used in the top-level Makefile
but was present in rules.mk. This meant it was getting evaluated for
every Makefile. This was silly and thus the logic has mostly been moved into
its own .mk file and is only included for the root Makefile.
Nathan Froyd
Mike Hommey
Gregory Szorc
Aaron Klotz
Cameron McCormack
Ted Mielczarek
Andrew Halberstadt
Chris Manchester
Michael Layzell
Nathan Froyd
Wes Kocher
Mike Shal
Nathan Froyd
Sebastian Hengst
Myk Melez
Ms2ger
Jacek Caban
Dan Minor
Jon Coppeard
Andreas Tolfsen
Brian O'Keefe
Ralph Giles
Joshua Cranmer
Phil Ringnalda
Jan Beich
Ed Morley
Ian Neal
Neil Rashbrook
Steve Fink
Mukilan Thiyagarajan
Ryan VanderMeulen
Ehsan Akhgari
EKR
Sylvestre Ledru
Dan Gohman
Seth Fowler
Carsten "Tomcat" Book
Nick Alexander
Marco Castelluccio
Joey Armstrong
Cykesiopka