This is strictly a quality of life improvement when cross-compiling to
Windows targets. A common scenario is mounting an object directory
into a Windows VM, which requires additional VM configuration to
handle (absolute) symlinks. With this patch, `export NSDISTMODE=copy`
in a mozconfig sidesteps any such symlink issues.
Differential Revision: https://phabricator.services.mozilla.com/D77118
For a while, now, ICU data has been stored as raw read-only data in rather
than a separate loadable library. This means the parts of ICU that can load
such a module are not necessary.
Differential Revision: https://phabricator.services.mozilla.com/D82262
All the supported compilers support a GNU AS-like syntax, with only a
few details varying. It means we can use a single, simpler, way to
include the ICU data file, instead of 3 different ways, including one
that uses armasm64.exe, possibly wrapped with Wine.
Differential Revision: https://phabricator.services.mozilla.com/D82144
Because jar_maker is not in the libs tier, we also rename the libs-%
targets for l10n repacks to l10n-%, which make it clearer what they are
for.
And because multilocale.txt is both a GeneratedFile and a file that is
generated (and installed) via manual build rules, keeping it in the misc
target actually breaks building in toolkit/locales during l10n repacks,
so move it to libs for now.
Differential Revision: https://phabricator.services.mozilla.com/D81766
While ideally we'd just move all of them at the top-level, there are
other things that depend on them that wouldn't then get the right
dependencies. One of them is install steps in dist/something, but there
are others, and that's a rather long pole of things requiring many
changes along with chances to break things.
So instead, we go with a simpler and more limited approach, where we
still recurse into directories to run their export tier (to run whatever
else than GeneratedFiles they have), but ensure the GeneratedFiles we
moved at the top-level are built before recursing by making
directory/export depend on them.
Differential Revision: https://phabricator.services.mozilla.com/D80608
While ideally we'd just move all of them at the top-level, there are
other things that depend on them that wouldn't then get the right
dependencies. One of them is install steps in dist/something, but there
are others, and that's a rather long pole of things requiring many
changes along with chances to break things.
So instead, we go with a simpler and more limited approach, where we
still recurse into directories to run their export tier (to run whatever
else than GeneratedFiles they have), but ensure the GeneratedFiles we
moved at the top-level are built before recursing by making
directory/export depend on them.
Differential Revision: https://phabricator.services.mozilla.com/D80608
While ideally we'd just move all of them at the top-level, there are
other things that depend on them that wouldn't then get the right
dependencies. One of them is install steps in dist/something, but there
are others, and that's a rather long pole of things requiring many
changes along with chances to break things.
So instead, we go with a simpler and more limited approach, where we
still recurse into directories to run their export tier (to run whatever
else than GeneratedFiles they have), but ensure the GeneratedFiles we
moved at the top-level are built before recursing by making
directory/export depend on them.
Differential Revision: https://phabricator.services.mozilla.com/D80608
They were necessary some time ago, but:
- there is no js/xpconnect/src/export target anymore,
- accessibility/xpcom/export had a dependency on xpcom/xpidl/export for
the ply cache files, but they're not used as of bug 1633156.
- the recursive make backend was adding automatic dependencies on
xpcom/xpidl/export for the ply cache files.
Differential Revision: https://phabricator.services.mozilla.com/D80150
Replace the duplicate lists in mozglue/baseprofiler/public and js/public with
a shared list. Add this list to both moz.build files so it is published twice
which simplifies supporting different standalone configurations.
Differential Revision: https://phabricator.services.mozilla.com/D79520
IMPACTED_TESTS was added in bug 1184405, but is under-utilized in current moz.build
annotations; also bug 1516847 removed use of IMPACTED_TESTS in 'mach test'. This
patch removes remaining support for IMPACTED_TESTS.
Differential Revision: https://phabricator.services.mozilla.com/D79141
Extremely common file paths on macOS like `/Users/...` are interpreted
as `/U...` flags by `clang-cl`. This is so common that there's a
`-Wslash-u-filename` warning. Ensure that file paths are considered
paths when compiling by terminating options with `--`.
This commit handles everything except assembler invocations, because
at least `nasm` doesn't handle `--`.
Differential Revision: https://phabricator.services.mozilla.com/D77119
This is strictly a quality of life improvement when cross-compiling to
Windows targets. A common scenario is mounting an object directory
into a Windows VM, which requires additional VM configuration to
handle (absolute) symlinks. With this patch, `export NSDISTMODE=copy`
in a mozconfig sidesteps any such symlink issues.
Differential Revision: https://phabricator.services.mozilla.com/D77118
Also update documentation to suggest using the `GeneratedFile` template rather than directly referencing `GENERATED_FILES` where possible.
Differential Revision: https://phabricator.services.mozilla.com/D77496
l10n-merge creates a full merge dir for a while now, let's
simplify the build logic to only read from that directory
during repacks and langpacks.
Differential Revision: https://phabricator.services.mozilla.com/D77023
I haven't been able to confirm that this would actually fix
`--with-system-icu`, but since the new regexp
engine (`js/src/new-regexp`) explicitly includes this file and this file
isn't already present in system-headers...it seems very likely that this
change will fix `--with-system-icu`.
Differential Revision: https://phabricator.services.mozilla.com/D75809
iOS support for Gecko has not been tested in years and is most probably
out of date. The build system part of it, specifically the checks in
build/autoconf/ios.m4, are not trivial to port to python configure, and
they prevent other things from moving to python configure (because some
of them change value when MOZ_IOS is set).
The code is left alone, although it could probably be stripped off as
well, but I'll leave that as an exercise for someone else.
Differential Revision: https://phabricator.services.mozilla.com/D75463
`ply`, [by design](https://github.com/dabeaz/ply/issues/79), does not produce reproducible table files; hence bug 1633156. (Note that this was *always* true, but only became a problem once we switched to Python 3, which has more unpredictable dict iteration order than Python 2.7, at least prior to [3.7](https://docs.python.org/3/whatsnew/3.7.html#summary-release-highlights).)
In any other circumstance I would consider submitting a patch to `ply` to fix this, but as of the [in-progress version 4.0 of the library](https://github.com/dabeaz/ply/blob/master/CHANGES), it doesn't even emit this cached data any more, and indeed the [latest version of the code](1fac9fed64/ply) doesn't even call `open()` at all except to do logging or to read the text data to be parsed from `stdin`. So if we were going to pin our future on `ply` and upgrade to later versions of the library in the future, we would have to live in a world where `ply` doesn't generate cached table files for us anyway.
Emitting the cached table files so later build steps can consume them is an "optimization", but it's not clear exactly how much actual value that optimization provides overall. Quoth the `CHANGES` file from that repository:
```
PLY no longer writes cached table files. Honestly, the use of
the cached files made more sense when I was developing PLY on
my 200Mhz PC in 2001. It's not as much as an issue now. For small
to medium sized grammars, PLY should be almost instantaneous.
```
In practice, I have found this to be true; namely, `./mach build pre-export export` takes just about as long on my machine after this patch as it did before, and in a try push I performed, there's no noticeable performance regression from applying this patch. In local testing I also found that generating the LALR tables in calls to `yacc()` takes about 0.01s on my machine generally, and we generate these tables a couple dozen times total over the course of the `export` tier now. This isn't *nothing*, but in my opinion it's also not nearly long enough where it would be a concern given how long `export` already takes.
That `CHANGES` file also stresses that if caching this data is important, we have the option of doing so via `pickle`. If and when we decide that re-enabling this optimization is valuable for us, we should take control of this process and perform the generation in such a way that we can guarantee reproducibility.
Differential Revision: https://phabricator.services.mozilla.com/D73484
Linking Rust code against the updatecommon library is not well supported by our build system, but it should be possible. The changes to updatecommon's moz.build will cause updatecommon.lib to be created, which Rust can link against. The build.rs script tells cargo which shared Windows libraries are needed for linking, and where updatecommon.lib can be found. The change to config/recurse.mk enforces the dependency of the update agent on updatecommon, so that we do not attempt to link the agent before updatecommon has been built.
Differential Revision: https://phabricator.services.mozilla.com/D70207
For native sanitizer builds, we currently do not pass the linker flags
to cargo, as they were causing crashes in some build scripts. Without
this, however, the linker is unable to find libstdc++. Instead, do
tell cargo to use the linker wrapper, but omit the problematic flags
from MOZ_CARGO_WRAP_LDFLAGS.
Differential Revision: https://phabricator.services.mozilla.com/D70354
For now this generates just CACHE_IR_OPS and CACHE_IR_SHARED_OPS in CacheIROpsGenerated.h
but the plan is to use this to generate parts of the IR writer and compiler/transpiler
interface. The spewer could also potentially be improved now that each operand has a name
and more precise type.
Generating the IR writer will likely happen incrementally so that will give us
another chance to double check the precise types match what's in the YAML file.
Differential Revision: https://phabricator.services.mozilla.com/D70995
--HG--
extra : moz-landing-system : lando
Most other generated SpiderMonkey files have the 'Generated' suffix so let's
follow that convention.
Differential Revision: https://phabricator.services.mozilla.com/D70993
--HG--
extra : moz-landing-system : lando
They all fail for the same reason: the sanitizer runtime in compiler-rt installs an interceptor for `getsockname` which then contains a call to the real implementation.
Differential Revision: https://phabricator.services.mozilla.com/D70889
--HG--
extra : moz-landing-system : lando
Also enable all config/tests with python3. unit-nsinstall.py was the
last one that didn't pass with python 3.
Switch the test to using @unittest.skipIf and disable the subprocess
test because we purposely broke running nsinstall.py independently with
python 2.
Differential Revision: https://phabricator.services.mozilla.com/D70160
--HG--
extra : moz-landing-system : lando
Since the test goes through all moz.build files disregarding DIRS and
the conditions that may disable directories, in some cases, moz.builds
can fail to be evaluated properly because of missing variables in
config.status. This time (because it's not the first), it's
LLVM_DLLTOOL.
After fixing that, it turns out many of the files/directories pointed to
by Files() directives were removed or moved.
While here, make the test script python3-ready.
Differential Revision: https://phabricator.services.mozilla.com/D70157
--HG--
extra : moz-landing-system : lando
This needs a few adjustments to the autobinscope script because running
binscope currently creates an HTML file in the binscope directory, and
when multiple binscopes run at the same time (which happens during the
build with the changes to run it on all executables and libraries), all
but one fail to open the HTML file for write access.
So add a flag to create that file in a temporary directory.
While here, remove log_file_path, which hasn't been used since
bug 1448306.
Differential Revision: https://phabricator.services.mozilla.com/D67422
--HG--
extra : moz-landing-system : lando
Because most of the recent changes to irregexp were patches I contributed myself, we barely need to change any of the shim code.
The only notable change is the addition of '#define COMPILING_IRREGEXP_FOR_EXTERNAL_EMBEDDER'. This is the solution that Jakob Gruber and I eventually came up with for the question of what to do with awkward V8 code that SM doesn't want. For example, NativeRegExpMacroAssembler::Match (in regexp-macro-assembler.cc) gets down in the muck with the internal details of V8's String implementation. It would be most convenient for SM if that function just didn't exist; we aren't going to use it, and we don't want to have to define a bunch of unused string API gunk in our shim. The answer is to wrap functions we don't need in "#ifndef COMPILING_IRREGEXP_FOR_EXTERNAL_EMBEDDER", which solves our problem and is minimally disruptive upstream.
Depends on D67718
Differential Revision: https://phabricator.services.mozilla.com/D67719
--HG--
extra : moz-landing-system : lando
This needs a few adjustments to the autobinscope script because running
binscope currently creates an HTML file in the binscope directory, and
when multiple binscopes run at the same time (which happens during the
build with the changes to run it on all executables and libraries), all
but one fail to open the HTML file for write access.
So add a flag to create that file in a temporary directory.
While here, remove log_file_path, which hasn't been used since
bug 1448306.
Differential Revision: https://phabricator.services.mozilla.com/D67422
--HG--
extra : moz-landing-system : lando
Dictionary iteration under Python 3 is in an inherently unpredictable order, and while we try to keep DEFINES ordered through the use of OrderedDicts, if at any point we populate DEFINES directly or indirectly while iterating through the contents of a non-ordered dictionary, the order of the DEFINES (and therefore the contents of the output Makefile) will be nondeterministic as well. This patch makes a number of changes to ensure that we only ever populate DEFINES in a deterministic fashion. (Note that in Python 3.7 and later, the built-in dict class actually has deterministic ordering, so these changes are technically only necessary until our minimum Python version becomes 3.7.)
Differential Revision: https://phabricator.services.mozilla.com/D66089
--HG--
extra : moz-landing-system : lando
Dictionary iteration under Python 3 is in an inherently unpredictable order, and while we try to keep DEFINES ordered through the use of OrderedDicts, if at any point we populate DEFINES directly or indirectly while iterating through the contents of a non-ordered dictionary, the order of the DEFINES (and therefore the contents of the output Makefile) will be nondeterministic as well. This patch makes a number of changes to ensure that we only ever populate DEFINES in a deterministic fashion. (Note that in Python 3.7 and later, the built-in dict class actually has deterministic ordering, so these changes are technically only necessary until our minimum Python version becomes 3.7.)
Differential Revision: https://phabricator.services.mozilla.com/D66089
--HG--
extra : moz-landing-system : lando
- Remove the separate option() for MT, because it dates back from when
we needed `MT` not being an absolute path, but that hasn't been true
since bug 1290040.
- Extend what was done in bug 1617794 to MT, although the long term move
is to not rely on MT at all.
- Patch leftovers from bug 1613799.
Differential Revision: https://phabricator.services.mozilla.com/D64712
--HG--
extra : moz-landing-system : lando
Windows programs run via Wine don't like Unix absolute paths (they look
like command line arguments), so we need to use relative paths.
Mingw already run fxc2 via wine, but for some reason it doesn't care
about the Unix absolute paths. genshaders does need some adjustements to
run properly with the real fxc.
Now, on actual Windows, because the temporary directory where
tempfile.NamedTemporaryFile creates files by default is not necessarily
on the same drive as where the command runs from, a relative path can't
be constructed. So we also force the temporary file to be created in the
current (obj) directory.
There is no similar concern for other files because we only go from
objdir to srcdir, and the build system already doesn't support both
being on a separate drive.
While here, flush stdout when the genshared script writes to it, so that
the messages are printed out immediately rather than randomly, later,
after output from subprocesses.
Differential Revision: https://phabricator.services.mozilla.com/D64294
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
This change is a little gross, because we don't totally control where
the statically initialized instances of `RWLock` live, due to its uses
in third-party libraries.
Differential Revision: https://phabricator.services.mozilla.com/D62936
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
- -NOLOGO is used to hide the banner that MSVC displays by default. It's
a no-op with clang-cl, no need to keep it now that we don't support
MSVC.
- -utf-8 is only passed when the compiler is not GCC-ish and not
clang-cl, which used to mean MSVC, but now means no compiler.
Differential Revision: https://phabricator.services.mozilla.com/D62705
--HG--
extra : moz-landing-system : lando
`lucetc` by default will invoke `ld` directly, which is usually not what
we want, particularly when cross-compiling. Instead, let's invoke the
compiler, which will do the hard work of determining the correct linker
to use, and we'll also explicitly specify `LDFLAGS`, since the default
`LDFLAGS` from `lucetc` assume you're invoking the linker directly.
Depends on D62797
Differential Revision: https://phabricator.services.mozilla.com/D62798
--HG--
extra : moz-landing-system : lando
We're going to need this for handling Mac cross compiles correctly.
Depends on D62795
Differential Revision: https://phabricator.services.mozilla.com/D62796
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
By default the Rust compiler will perform a limited kind of ThinLTO on each
crate. For local builds this additional optimization is not worth the
increase in compile time.
Differential Revision: https://phabricator.services.mozilla.com/D61821
--HG--
extra : moz-landing-system : lando
Baby steps. Working towards not using $(MT), let's unsupport what we
currently don't rely on (and in case someone would try to rely on it,
throw an error).
Differential Revision: https://phabricator.services.mozilla.com/D61948
--HG--
extra : moz-landing-system : lando
At the moment, everything we're putting in a wasm sandbox is not
performance-critical, so we don't need PGO. It's also not clear that
PGO would actually work properly with code that's been run through
wasm. Let's leave figuring that out until, at the very least, the wasm
toolchain is a little more mature.
Differential Revision: https://phabricator.services.mozilla.com/D58513
--HG--
extra : moz-landing-system : lando
Turns out we were using the Linux ones. This uses the Windows ones, and
adds _HAS_EXCEPTIONS to the mingw defines so the stl_wrappers behave
correctly.
Differential Revision: https://phabricator.services.mozilla.com/D54530
--HG--
extra : moz-landing-system : lando
Turns out we were using the Linux ones. This uses the Windows ones, and
adds _HAS_EXCEPTIONS to the mingw defines so the stl_wrappers behave
correctly.
Differential Revision: https://phabricator.services.mozilla.com/D54530
--HG--
extra : moz-landing-system : lando
In 1-tier PGO builds that shared the objdir between the instrumented and
profile-use builds, the instrumented build objects used a different
suffix (.i_o) to separate them from the profile-use build (which uses
the default .o suffix). These builds are now always in separate objdirs,
and don't need special suffix rules anymore.
As a bonus, this helps fix an issue with buildid.cpp continually
rebuilding because libxul_so.list always lists the inputs as *.o, which
don't exist if we're using a .i_o suffix. Make would always re-create
buildid.cpp and therefore libxul.so in the instrumented build even when
nothing has changed.
Differential Revision: https://phabricator.services.mozilla.com/D56115
--HG--
extra : moz-landing-system : lando
This was used for Windows MSVC PGO builds that re-used the objdir. We
don't use MSVC for PGO anymore, and we don't re-use the objdir.
Differential Revision: https://phabricator.services.mozilla.com/D56114
--HG--
extra : moz-landing-system : lando
MOZ_1TIER_PGO was a temporary hack to support 1-tier PGO builds while
they were being ported to 3-tier. Now that all builds are 3-tier, it can
be removed.
Differential Revision: https://phabricator.services.mozilla.com/D56112
--HG--
extra : moz-landing-system : lando
Add backend stuff to build sandboxed wasm libraries. (Don't actually update any moz.build files to consume this yet.)
Differential Revision: https://phabricator.services.mozilla.com/D54152
--HG--
extra : moz-landing-system : lando
Axel Hecht
Nick Alexander
Mike Hommey
André Bargull
Mozilla Releng Treescript
Emilio Cobos Álvarez
Coroiu Cristina
Dorel Luca
Ricky Stewart
Ted Campbell
Geoff Brown
Nathan Froyd
Bogdan Tara
Kirk Steuber
Jamie Nicol
Nika Layzell
Jan de Mooij
David Major
Jan Beich
Sylvestre Ledru
Michal Novotny
Iain Ireland
Oana Pop Rus
William Lachance
Michael Woerister
Anthony Ramine
Narcis Beleuzu
Mike Shal
Daniel Varga
ffxbld
Greg V
Tooru Fujisawa
Marco Bonardo
Csoregi Natalia
Cosmin Sabou
Brindusan Cristian
shindli
shravanrn@gmail.com
Marco Castelluccio
Dzmitry Malyshau
Noemi Erli
Ben Iofel
Razvan Maries
Tom Ritter
Andreea Pavel
Chris Manchester