We already require Xrandr 1.3 inderectly through GTK3 (`>=1.2.99`).
This bumps the requirement to 1.4, which got released in 2012 and
is thus part of RHEL 7 and Ubuntu 14.04.
Differential Revision: https://phabricator.services.mozilla.com/D126581
Historically, client.mk was not invoked with -jn because it would create
race conditions, but that was actually mostly solved by the addition of
`.NOTPARALLEL` in bug 422986, although the mechanism of adding -jn via
`MOZ_MAKE_FLAGS` or `MOZ_PARALLEL_BUILD` has continued well past that.
Nowadays, client.mk is only invoked by mach (it will even bail out if
that's not the case) and only has one target (`build`) and no
dependencies.
This means we don't need to rely on `MOZ_PARALLEL_BUILD` to pass `-jn` in
some cases, and can just always invoke `make -f client.mk` with `-jn`, even
when we just want no parallelism, in which case we can use `-j1`.
This, in turn, allows to remove the extra allow_parallel argument to
`_run_make`, and only rely on `num_jobs`, and to remove some of the
multiple ways the `n` in `-jn` could be set.
Differential Revision: https://phabricator.services.mozilla.com/D124729
Automatically generated path that adds flag `REQUIRES_UNIFIED_BUILD = True` to `moz.build`
when the module governed by the build config file is not buildable outside on the unified environment.
This needs to be done in order to have a hybrid build system that adds the possibility of combing
unified build components with ones that are built outside of the unified eco system.
Differential Revision: https://phabricator.services.mozilla.com/D122345
MKCSHLIB, used to build WASM_LIBRARY, refers to DSO_LDOPTS, which refers
to IMPORT_LIBRARY on mingw builds, but for WASM_LIBRARY, IMPORT_LIBRARY
is never set by the build backend.
Differential Revision: https://phabricator.services.mozilla.com/D123540
The build system assumes the target compiler is of the same type as the
wasm compiler, but that's not true for wasm32-wasi, which we compile
with clang, while targeting windows, which we compile with clang-cl.
We handle the duality for host/target in a compiler-specific way, but
here, the wasm compiler is always going to be clang.
Differential Revision: https://phabricator.services.mozilla.com/D123041
The build system assumes the target compiler is of the same type as the
wasm compiler, but that's not true for wasm32-wasi, which we compile
with clang, while targeting windows, which we compile with clang-cl.
We handle the duality for host/target in a compiler-specific way, but
here, the wasm compiler is always going to be clang.
Differential Revision: https://phabricator.services.mozilla.com/D123041
The build system assumes the target compiler is of the same type as the
wasm compiler, but that's not true for wasm32-wasi, which we compile
with clang, while targeting windows, which we compile with clang-cl.
We handle the duality for host/target in a compiler-specific way, but
here, the wasm compiler is always going to be clang.
Differential Revision: https://phabricator.services.mozilla.com/D123041
Except for the following two files, no other header file is still including
"jsapi.h". Add a rule to "check_spidermonkey_style.py" to make sure we don't
regress this progress.
Exceptions:
- "vm/Compartment-inl.h" still needs to include "jsapi.h" for `JS::InformalValueTypeName()`.
- "jsapi-tests/tests.h" still needs to include "jsapi.h" for `JS_ValueToSource()`.
Differential Revision: https://phabricator.services.mozilla.com/D123209
They are equivalent, except for the fact that MACOSX_DEPLOYMENT_TARGET
may apply to more processes in a normal build. In practice, all the
processes that matter are covered through compiler flags.
On the opposite end, MACOSX_DEPLOYMENT_TARGET isn't necessary passed in
all cases (like clangd, mach static-analysis, etc.), while compiler
flags are.
Differential Revision: https://phabricator.services.mozilla.com/D122145
Sanitizers try to intercept __tls_get_addr to find DTLS blocks. In some
cases, they fail miserably with a crash. And in the cases they don't
crash, they don't seem to actually work properly anyways.
This affects both build time when running host programs compiled with
sanitizer flags (this only actually affects rust build
scripts/procedural macros on non-cross builds, not C/C++ host programs),
and execution time, e.g. when running tests.
Differential Revision: https://phabricator.services.mozilla.com/D120574
The `-/*` filter also removes the alias (`%%ALIAS`) and parent locale (`%%Parent`)
links, so we have to manually add them back when we're using `-/*`.
The ICU docs warn about this issue, but unfortunately that warning wasn't
present when we originally created our filter file, so this bug went unnoticed
for a long time.
Differential Revision: https://phabricator.services.mozilla.com/D116981
It's not practically make a difference (the flags we end up setting it
to don't do anything when linking all the wasm files into one).
Differential Revision: https://phabricator.services.mozilla.com/D116437
This last part to support "calendar" requires a bit more code.
`GetCalendarDisplayName()` performs the following steps:
1. Validate the input matches the `type` nonterminal, per
CanonicalCodeForDisplayNames, step 4.a.
2. Then convert it to lower case, because ICU expects lower case names and it
needs to be in lower case for `LanguageTag::replaceUnicodeExtensionType()`.
3. Replace calendar name aliases with the preferred name, e.g. use "ethioaa"
instead of "ethiopic-amete-alem".
4. Check the input isn't too large, just in case ICU somehow doesn't properly
handle large string inputs.
5. Retrieve the old-style value of the calendar name, for example "gregorian"
for the calendar name "gregory".
6. And finally call `uldn_keyValueDisplayName` to search for the localised name.
Our ICU data file doesn't yet include calendar names. To include calendar names
we need to enable them in "data_filter.json", rebuild the ICU data file, and
then reapply the update for tzdata 2021a. (Reapplying the tzdata changed
"intl/tzdata/GIT-INFO", but otherwise didn't change any time zone files.)
Adding the calendar names to the ICU data file results in a slight increase of
the overall data file size (79,184 bytes).
The existing "calendar.js" test file was renamed to "month-calendar.js", so
the test file for the new "calendar" type can be named "calendar.js".
Differential Revision: https://phabricator.services.mozilla.com/D116008
This adds a fallible factory method to create new NumberFormat instances. This
allows us to report initialization errors at time of initialization, rather than
when format is called, and remove internal checks in the implementation for
successful initialization.
The existing fluent code assumes that creating a NumberFormat instance always
succeeds. This patch updates that code to handle failures.
Differential Revision: https://phabricator.services.mozilla.com/D114593
This temporarily adds intl::NumberFormatterSkeleton to the public API so that
we can use it in PluralRules. This allows us to remove the SpiderMonkey
NumberFormatterSkeleton implementation when we convert SpiderMonkey to use
intl::NumberFormat.
We can remove intl::NumberFormatterSkeleton from the public API when we
implement intl::PluralRules in Bug 1704509.
Differential Revision: https://phabricator.services.mozilla.com/D111887
This adds a fallible factory method to create new NumberFormat instances. This
allows us to report initialization errors at time of initialization, rather than
when format is called, and remove internal checks in the implementation for
successful initialization.
The existing fluent code assumes that creating a NumberFormat instance always
succeeds. This patch updates that code to handle failures.
Differential Revision: https://phabricator.services.mozilla.com/D114593
This temporarily adds intl::NumberFormatterSkeleton to the public API so that
we can use it in PluralRules. This allows us to remove the SpiderMonkey
NumberFormatterSkeleton implementation when we convert SpiderMonkey to use
intl::NumberFormat.
We can remove intl::NumberFormatterSkeleton from the public API when we
implement intl::PluralRules in Bug 1704509.
Differential Revision: https://phabricator.services.mozilla.com/D111887
- We can and actually should remove the -fsanitize=% flags consistently
on all cargo targets, not only programs, but because of limitations in
cargo we can't really do that everywhere. We do need it for TSAN,
though.
- Because of the above, we actually don't need to single-out NATIVE_TSAN
builds anymore.
- The comment regarding the initial filtering of MOZ_CARGO_WRAP_LDFLAGS
somehow got separated from where it's happening, so we move it back to
where it belongs.
Differential Revision: https://phabricator.services.mozilla.com/D114579
Credit goes to dmajor.
Rust gets LLVM 12 - https://github.com/rust-lang/rust/pull/81451.
The PGO format changed between 11 and 12, and builds will have poor optimization if we mix versions in xLTO.
The above PR has landed on Rust nightly 1.52.
Until we move to 1.52 nightly we should have this landed.
In the meantime, for unblocking try builds, we can disable value-PGO in Rust.
Differential Revision: https://phabricator.services.mozilla.com/D113327
When "build targets" are used, the sccache server isn't
being used/integrated properly with make.
By moving rusttests to a separate build "application",
`client.mk` is invoked normally, and sccache is happy.
* Move rusttests to a new build "application".
* Move and denormalize rusttests mozconfigs. The new rusttests
mozconfigs should expand to be mostly identical to their old
versions, except with `MOZ_AUTOMATION_*=0` and
`--enable-application=tools/rusttests`.
They will differ a bit from their old variants because some
of them originally didn't source from their
respective `$app_mozconfigs/common`.
* Modify `baseconfig.mk` to change tiers if running rusttests.
* Adjust CI config to use new "rusttests" build application.
Differential Revision: https://phabricator.services.mozilla.com/D106294
While we could change qcms, encoding_rs and packed_simd to not emit
RUSTC_BOOTSTRAP on newer versions of rust, like we do for gkrust-shared,
it's not worth the effort (especially for those that are vendored).
Differential Revision: https://phabricator.services.mozilla.com/D105423
Bug 1690930 added sysroots that can be bootstrapped. With this change,
we allow --enable-bootstrap=install to pull the right sysroot for the
configured target, and --enable-bootstrap to update it if it was already
there.
Differential Revision: https://phabricator.services.mozilla.com/D104797
Back when the flag was added, the libffi source code didn't build for
ARM with the clang integrated assembler. That has been fixed with the
upgrade to libffi v3.3 in bug 1659906, so we can use the integrated
assembler now.
Differential Revision: https://phabricator.services.mozilla.com/D102261
* We add a config option for setting the number of content services;
* We add a config option to indicate whether content services should be isolated.
This one is just a `project_flag` since it doesn't really need the ability to
be overridden; it's something whose default we would want to flip when the
time comes;
* We set a dependency so that mobile/android/base/pre-export is executed;
* We add the `gen_from_jinja.py` script which is mostly just a dumb shim that
takes the input template and the config arguments, instantiates jinja,
generates the final output, and dumps it to the output fd;
* We add the requisite `moz.build` statements to generate the manifest overlay
and the service definitions;
* We update `build.gradle` so that Gradle knows to look for the generated files
when building the apk.
Differential Revision: https://phabricator.services.mozilla.com/D82577
* We add a config option for setting the number of content services;
* We add a config option to indicate whether content services should be isolated.
This one is just a `project_flag` since it doesn't really need the ability to
be overridden; it's something whose default we would want to flip when the
time comes;
* We set a dependency so that mobile/android/base/pre-export is executed;
* We add the `gen_from_jinja.py` script which is mostly just a dumb shim that
takes the input template and the config arguments, instantiates jinja,
generates the final output, and dumps it to the output fd;
* We add the requisite `moz.build` statements to generate the manifest overlay
and the service definitions;
* We update `build.gradle` so that Gradle knows to look for the generated files
when building the apk.
Differential Revision: https://phabricator.services.mozilla.com/D82577
With clang 12, win32 debug tests run out of stack in the 1000-deep layout crashtests.
Usually when this type of thing happens, it's because the compiler made a drastically heavier inlining decision along the recursive part of the stack, with a difference in the hundreds of bytes, and we pay that price of that space 1000 times. In those cases we normally try to snip out the heavy piece with a MOZ_NEVER_INLINE.
However, in this case, the difference between old and new compilers was only something like 16 bytes of the cyclical piece. This suggests that we're already up pretty close against the limit, and we're only a bad day away from failing tests even with existing clang. This only happens in debug builds since they have lots of logging helpers on the stack. Release builds are still well under the limit.
So I propose to increase the stack limit in debug. While we normally have to be pretty careful about address space on win32, it should be less of a concern with these non-shipping builds.
Differential Revision: https://phabricator.services.mozilla.com/D101196
Also rename `branchIfNegativeBigInt` to `branchIfBigIntIsNegative` to match
the other branch methods from part 2 and part 8.
Differential Revision: https://phabricator.services.mozilla.com/D98158
Also rename `branchIfNegativeBigInt` to `branchIfBigIntIsNegative` to match
the other branch methods from part 2 and part 8.
Differential Revision: https://phabricator.services.mozilla.com/D98158
On a very parallel debug build, I see a long time just waiting for
bindgen / style compilation / geckoservo.
Turns out that a bunch of this is just proc macros / build scripts.
Optimizing it saves between 10 and 17 seconds of my debug build. We
might want to consider running bindgen much like cbindgen rather than
rebuilding it all the time, which should help a lot more, but my guess
is that this should still help with the pretty hot custom derives that
the style crate runs.
This needs rust 1.41, so the requirement for tools/crashreporter needs
to be bumped as a consequence. To make things simpler, it was bumped
to 1.47 while we're at it.
Differential Revision: https://phabricator.services.mozilla.com/D98366
* Bumps the tsan toolchain to rust-nightly-2020-11-14 that has my patches to make -Zbuild-std work in vendored environments:
* https://github.com/rust-lang/cargo/pull/8834
* https://github.com/rust-lang/rust/pull/78790
* Passes -Zbuild-std to cargo when MOZ_TSAN is defined (mk_add_options --enable-thread-sanitizer)
* Removes generic Rust supressions and adds much more specific ones
* One presumed upstream false positive from tsan not understanding the code
* One actual upstream bug tsan found (yay!)
* One new real issue uncovered
* One issue that probably already existed intermittently but I happened to hit
Differential Revision: https://phabricator.services.mozilla.com/D97165
This makes --enable-thread-sanitizer turn on Rust tsan (-Zsanitizer=thread).
This requires changing SpiderMonkey tsan to use the tsan rust nightly.
In future changes, more Rust tsan integration will key off of MOZ_TSAN.
Differential Revision: https://phabricator.services.mozilla.com/D96453
This makes --enable-thread-sanitizer turn on Rust tsan (-Zsanitizer=thread).
This requires changing SpiderMonkey tsan to use the tsan rust nightly.
In future changes, more Rust tsan integration will key off of MOZ_TSAN.
Differential Revision: https://phabricator.services.mozilla.com/D96453
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
Depends on D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
The support code for Android, which doesn't support shm_open and can't
use the memfd backend because of issues with its SELinux policy (see bug
1670277), has been reorganized to reflect that we'll always use its own
API, ashmem, in that case.
Differential Revision: https://phabricator.services.mozilla.com/D90605
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
The support code for Android, which doesn't support shm_open and can't
use the memfd backend because of issues with its SELinux policy (see bug
1670277), has been reorganized to reflect that we'll always use its own
API, ashmem, in that case.
Differential Revision: https://phabricator.services.mozilla.com/D90605
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
This avoids a sort of duplication of work between both, because the
linker will eventually LTO-compile everything, so we technically don't
really need the extra step of the rust compiler doing an intermediate
LTO on the static libraries it produces.
Differential Revision: https://phabricator.services.mozilla.com/D94224
Rustc >= 1.44 changed the file names of the static libraries it
produces with -windows-gnu targets, to match that of mingw clang/gcc.
Considering we still build on 1.43, the best fix would be to derive the
prefix/suffix based on the version of rust, but that actually turns into
a hard-to-solve problem because of configure tests for bindgen also
depending on the prefix/suffix value to be known.
On the other hand, we're soon due to an update to 1.47, so the simpler
solution is to just push mingw builds to require 1.44 (settling for the
smallest upgrade possible for now) and to remove the split between C and
rust library prefix/suffixes.
Differential Revision: https://phabricator.services.mozilla.com/D93726
The `clobber` targets are superseded by `mach clobber`, so we don't need them for any reason. The `clean` target is meant to get you to a post-`configure` state, but it doesn't really work, and if it's necessary for you to be in that state for some reason you can just clobber and re-`configure`, so it doesn't seem worth it to get it working again. Instead, delete all of them. Also delete `everything` which is not useful when `clobber` doesn't exist.
Differential Revision: https://phabricator.services.mozilla.com/D93514
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
Differential Revision: https://phabricator.services.mozilla.com/D90605
It is only really used in js/src/devtools/rootAnalysis/Makefile.in,
and even there, the way it is used seems wrong, so fix that at the
same time (binaries have been linked into $DIST/bin directly for a
while).
Differential Revision: https://phabricator.services.mozilla.com/D92721
Bug 1573566 moved libxul from toolkit/library to toolkit/library/build,
and that should be reflected in config/recurse.mk.
It's amazing the race condition hasn't caused problems earlier than now.
Differential Revision: https://phabricator.services.mozilla.com/D92708
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
Differential Revision: https://phabricator.services.mozilla.com/D90605
Bug 1645986 solved the problem for most generated files by moving their
rules to the top-level, but we're going to add rules that will end up in
subdirectories, so we have to solve the same problem again, in the
subdirectories.
Differential Revision: https://phabricator.services.mozilla.com/D88389
Somehow we have had it be a general define, but it has not mattered...at
least it won't matter until we upgrade to clang 11, at which point the
compiler will start to complain at us about the codepaths this define
activates.
Differential Revision: https://phabricator.services.mozilla.com/D88056
All the patches previously applied, except the one from bug 1279096,
are either irrelevant (as pertaining to changes to the upstream build
system we don't use anymore), were applied upstream, or the issue they
fixed were fixed differently upstream.
Two additional patches, sent upstream as
https://github.com/libffi/libffi/pull/579 and
https://github.com/libffi/libffi/pull/580, are needed to fix our build
with, respectively, mingw-clang and GCC.
Our build system is adjusted according to upstream's configure.ac
and configure.host.
Differential Revision: https://phabricator.services.mozilla.com/D87824
All the patches previously applied, except the one from bug 1279096,
are either irrelevant (as pertaining to changes to the upstream build
system we don't use anymore), were applied upstream, or the issue they
fixed were fixed differently upstream.
Two additional patches, sent upstream as
https://github.com/libffi/libffi/pull/579 and
https://github.com/libffi/libffi/pull/580, are needed to fix our build
with, respectively, mingw-clang and GCC.
Our build system is adjusted according to upstream's configure.ac
and configure.host.
Differential Revision: https://phabricator.services.mozilla.com/D87824
This resolves a long-standing issue in development where `mach artifact` (and therefore `mach bootstrap`) would fail unpredictably if you had dirty, but ignored, files in your checkout. Resolving this problem often required unwieldy `hg purge`/`git ignore` incantations that are easy to get wrong.
This patch addresses the problem by doing what we "should" have been doing all along, and consulting the VCS to list tracked files rather than listing EVERY file on disk and applying heuristics to determine whether they should be included in the hash.
Differential Revision: https://phabricator.services.mozilla.com/D86780
This makes us use one less tool from MSVC, and removes one more use of wine
in cross builds.
We replace the call to either rc/llvm-rc or windres with a wrapper script.
While the script is not strictly needed for the latter, we use a wrapper
in that case anyway because it's one step towards fixing bug 1498414.
For llvm-rc, however, we need a wrapper because llvm-rc doesn't preprocess
on its own, so the wrapper does that too.
The wrapper script also allows to deal with the default flags passed to
llvm-rc or windres, rather than inherit them from old-configure.
We also need to explicitly pass the codepage to llvm-rc, which was not
necessary with rc (presumably, llvm-rc has a different default).
While here, remove the unused WINDRES subst from js/src/old-configure.in.
Also, while here, we remove --use-temp-file, because as described in the
linked bug and in the windres manual page, it was used to work around bugs
on Windows 98 and earlier.
Differential Revision: https://phabricator.services.mozilla.com/D86312
We are currently relying on two different ways to insert side-by-side
manifests in binaries on Windows: through resource files, or through
the use of MT. The latter is not supported on mingw builds, which is
not great.
Link.exe has options to add a manifest at link time without relying on
either method above, but that's not supported on mingw either.
So the best we can do is to move everything to using resource files.
This also avoids using MT, which, on cross builds, requires using wine.
Ideally, the manifests would be declared in moz.build, but that
complicates things for cases like TestDllInterceptor, where there are
multiple binaries in the same directory, but only one of them needs the
manifest. This keeps the status quo of getting the manifest
automatically from the source directory.
Differential Revision: https://phabricator.services.mozilla.com/D85382
We're currently not using the feature, and host programs ought only
to be used during the build, so I don't expect the feature to ever
be necessary.
Differential Revision: https://phabricator.services.mozilla.com/D86167
The resource file is always generated so being able to configure its name
is not useful. On the other hand, the way things are currently implemented,
the lack of RESFILE also makes RCFILE ignored, which we fix at the same
time.
And remove a spurious RESFILE in widget/windows/moz.build, where no binary
is produced, which means RESFILE had no meaning.
Differential Revision: https://phabricator.services.mozilla.com/D86154
Because while the original perl script was added to add version info
to Windows binaries, it does more and will do even more with upcoming
changes.
Differential Revision: https://phabricator.services.mozilla.com/D86153
It was only ever set to the same value as its default, except in
comm-central, where it is unset, but in directories that now don't link
anything (they did back when binary components were a thing).
Differential Revision: https://phabricator.services.mozilla.com/D85381
We are currently relying on two different ways to insert side-by-side
manifests in binaries on Windows: through resource files, or through
the use of MT. The latter is not supported on mingw builds, which is
not great.
Link.exe has options to add a manifest at link time without relying on
either method above, but that's not supported on mingw either.
So the best we can do is to move everything to using resource files.
This also avoids using MT, which, on cross builds, requires using wine.
Ideally, the manifests would be declared in moz.build, but that
complicates things for cases like TestDllInterceptor, where there are
multiple binaries in the same directory, but only one of them needs the
manifest. This keeps the status quo of getting the manifest
automatically from the source directory.
Differential Revision: https://phabricator.services.mozilla.com/D85382
We're currently not using the feature, and host programs ought only
to be used during the build, so I don't expect the feature to ever
be necessary.
Differential Revision: https://phabricator.services.mozilla.com/D86167
The resource file is always generated so being able to configure its name
is not useful. On the other hand, the way things are currently implemented,
the lack of RESFILE also makes RCFILE ignored, which we fix at the same
time.
And remove a spurious RESFILE in widget/windows/moz.build, where no binary
is produced, which means RESFILE had no meaning.
Differential Revision: https://phabricator.services.mozilla.com/D86154
Because while the original perl script was added to add version info
to Windows binaries, it does more and will do even more with upcoming
changes.
Differential Revision: https://phabricator.services.mozilla.com/D86153
It was only ever set to the same value as its default, except in
comm-central, where it is unset, but in directories that now don't link
anything (they did back when binary components were a thing).
Differential Revision: https://phabricator.services.mozilla.com/D85381
On Windows, with the double quotes, the command goes through the msys
shell, which then alters the command line such that empty arguments are
transformed into \ when they reach python.
With the single quotes, make invokes the command directly, without going
through the msys shell, and empty arguments are preserved.
Differential Revision: https://phabricator.services.mozilla.com/D86097
This is not a feature-for-feature rewrite. The python version removes
unused things, and simplifies some others:
- Only two command line arguments are taken in, and all the others are
dropped and the corresponding values are gotten from the buildconfig
module instead. The command line arguments are also taken as
positional arguments rather than going with a full argument parser.
- Variable expansion in module.ver used to be limited to one specific
variable to expand for a given value, which is now replaced with the
possibility to expand any of the variables that are allowed in
module.ver.
- The perl version was adding a RT_MANIFEST entry on its own if a
manifest file existed in the objdir for the given binary, but if such
a file existed, the build would fail after linking from the changes in
bug 1613799.
- The perl version was defaulting the module name to the binary name in
a branch that was never taken because the module name was assigned to
an empty string before that.
The output from the new script has been validated to being identical to
the output from the perl script, except for one extra whitespace at the
end of a comment.
Differential Revision: https://phabricator.services.mozilla.com/D85817
As the commit title suggests, this is a manual update, but we need to also
update `config/rules.mk` for the changed location of the wasi bindings.
Depends on D85408
Differential Revision: https://phabricator.services.mozilla.com/D85409
run-and-prefix.py is used as a wrapper of MAKE in moz-automation.mk, so
that the output is more clearly attributed to what part of the build is
happening, for everything that is post-build (e.g. package, upload,
etc.)
Ever since the switch to python 3, when it re-invokes MAKE, the
jobserver file descriptors would be closed and the sub-make wouldn't be
able to use it.
Differential Revision: https://phabricator.services.mozilla.com/D85051
It turns out setting CARGO_PROFILE_RELEASE_LTO has unwanted side
effects.
First it's not actually strictly equivalent to using `cargo rustc --
-Clto`. For instance, it apparently also enables cross-language LTO in
newer versions of cargo.
Second, it changes the rust computed hash for all the dependencies of
the crate being built with the variable set, which makes them diverge
from when the same dependencies are built through another crate in the
tree that is not LTOed. This effectively makes us build a _lot_ of
crates twice, many of which are not cacheable.
Since the original problem is that cargo >= 1.45 passes extra flags (`-C
embed-bitcode=no`) to rustc that are incompatible with `-Clto`, and while
it knows to adjust based on the `lto` setting in the build profile
(which CARGO_PROFILE_RELEASE_LTO overrides the default of), cargo
ignores flags passed via `cargo rustc -- ...` when making those
adjustments.
So, we need to override with `-C embed-bitcode=yes` on our own at the
same time we pass `-Clto`. But doing that through `cargo rustc -- ...`
is not enough because all the dependencies of the crate built with
`-Clto` need to be built with `-C embed-bitcode=yes`. So we need to
override with `RUSTFLAGS`, which will affect all the dependencies.
But we also need to do this consistently across all crates, not only the
dependencies of crates built with `-Clto`, otherwise we'd still end up
building crates twice (once with and once without the override).
Unfortunately, the `-C embed-bitcode=*` flag is also not supported in
versions older than 1.45, so we have to avoid adding it on older
versions.
We unfortunately support a large range of versions of rustc (albeit only
for tools/crashreporter), but we actually need to upgrade the smaller
supported version because rustc < 1.38 doesn't support our top-level
Cargo.lock. This makes the version check slightly less awful.
Differential Revision: https://phabricator.services.mozilla.com/D84652
The main reason we look at the complete symbols table is that before bug
1541792, we needed to look at that table for _NSModule symbols.
In bug 1516228, we also made everything llvm-objdump to limit the
differences cross-platform, but that's not necessary anymore per the
previous change.
llvm-objdump doesn't support getting only the dynamic symbols table, so
we go back to what we were using before bug 1516228, namely readelf,
while preserving a code path to use the complete symbols table for the
networking test on libgkrust.a, which doesn't have a dynamic symbols
table.
With this change, check_binary goes from 45s to 0.2s on my machine.
Differential Revision: https://phabricator.services.mozilla.com/D84305
The rules we create in the backend for generated files use a stub file
for the real file generation, and we do need dependencies on that
specific target for the changes in bug 1264836 to work without races.
Differential Revision: https://phabricator.services.mozilla.com/D84311
Otherwise, crates that are shared between host and target are always
rebuilt because of the difference in rust flags.
Differential Revision: https://phabricator.services.mozilla.com/D83412
Doing this means that make will correctly understand that the underlying
program has changed and therefore it needs to trigger install rules.
Differential Revision: https://phabricator.services.mozilla.com/D83326
This makes dir/target, dir/target-objects, etc. depend on
dir/pre-compile, which ensures all items in the pre-compile pseudo-tier
are built before entering in the directory they are defined in, during
the compile tier. This removes the need for EXTRA_DEPS hacks.
Differential Revision: https://phabricator.services.mozilla.com/D83036
Generated files marked as "required during compile" are limited to a
number of extensions, most of which are source file types, so they get
generated when compiling the corresponding object file. Other types,
though, are currently handled via EXTRA_DEPS, which is clunky.
As of now, these other types are limited to link-related items, but more
would be useful (e.g. .inc files, which are only included in one
location).
This however works against the static analysis tasks (coverity and
clang-tidy), which currently rely on everything they need being created
via the export tier. That excludes using EXTRA_DEPS-based hacks.
We create a "pre-compile" tier, that is not used during a normal build,
but can be invoked manually, which the static analysis tasks will do.
Differential Revision: https://phabricator.services.mozilla.com/D83035
The previous logic would check the value of the `PYTHON3` environment variable, which is not set by the Firefox build system, so this build script tends to just use the `python3` at the tip of the `PATH`, regardless of which value you've configured for it. Instead, it should prefer to read from the `OBJDIR` if one is present.
Differential Revision: https://phabricator.services.mozilla.com/D82981
This makes it much simpler for the for the static JS services cache to store
and lookup interface IDs for components.
Differential Revision: https://phabricator.services.mozilla.com/D81416
... instead of a C file. This avoids issues with sccache (via dist or
caching), because sccache doesn't do anything with assembly files,
but with a C file, it doesn't know about the incbin-included data file.
This has the unfortunate side effect of requiring some manual work to
mark the file for safeseh on 32-bits Windows.
Differential Revision: https://phabricator.services.mozilla.com/D82814
This creates a general pytest plugin under "config/mozunit" which implements
the ability to mark tests as "slow". Slow tests will be skipped by default
unless "--run-slow" is passed in.
Differential Revision: https://phabricator.services.mozilla.com/D81402
This makes it much simpler for the for the static JS services cache to store
and lookup interface IDs for components.
Differential Revision: https://phabricator.services.mozilla.com/D81416
This is strictly a quality of life improvement when cross-compiling to
Windows targets. A common scenario is mounting an object directory
into a Windows VM, which requires additional VM configuration to
handle (absolute) symlinks. With this patch, `export NSDISTMODE=copy`
in a mozconfig sidesteps any such symlink issues.
Differential Revision: https://phabricator.services.mozilla.com/D77118
For a while, now, ICU data has been stored as raw read-only data in rather
than a separate loadable library. This means the parts of ICU that can load
such a module are not necessary.
Differential Revision: https://phabricator.services.mozilla.com/D82262
All the supported compilers support a GNU AS-like syntax, with only a
few details varying. It means we can use a single, simpler, way to
include the ICU data file, instead of 3 different ways, including one
that uses armasm64.exe, possibly wrapped with Wine.
Differential Revision: https://phabricator.services.mozilla.com/D82144
Because jar_maker is not in the libs tier, we also rename the libs-%
targets for l10n repacks to l10n-%, which make it clearer what they are
for.
And because multilocale.txt is both a GeneratedFile and a file that is
generated (and installed) via manual build rules, keeping it in the misc
target actually breaks building in toolkit/locales during l10n repacks,
so move it to libs for now.
Differential Revision: https://phabricator.services.mozilla.com/D81766
While ideally we'd just move all of them at the top-level, there are
other things that depend on them that wouldn't then get the right
dependencies. One of them is install steps in dist/something, but there
are others, and that's a rather long pole of things requiring many
changes along with chances to break things.
So instead, we go with a simpler and more limited approach, where we
still recurse into directories to run their export tier (to run whatever
else than GeneratedFiles they have), but ensure the GeneratedFiles we
moved at the top-level are built before recursing by making
directory/export depend on them.
Differential Revision: https://phabricator.services.mozilla.com/D80608
Mike Hommey
Mozilla Releng Treescript
criss
shravanrn@gmail.com
Robert Mader
André Bargull
Butkovits Atila
Kagami Sascha Rosylight
Jeff Walden
Andreea Pavel
Dzmitry Malyshau
Yury Delendik
Andi-Bogdan Postelnicu
Cosmin Sabou
Csoregi Natalia
Caroline Cullen
Mike Hommey
Sandor Molnar
Henri Sivonen
Noemi Erli
Steve Fink
carolinecullen
Chris Fallin
Agi Sferro
Dan Minor
Brindusan Cristian
Dorel Luca
Iulian Moraru
Nick Alexander
Deian Stefan
Jesse Schwartzentruber
Christian Holler
Alexandru Michis
Mitchell Hentges
Masatoshi Kimura
Aaron Klotz
David Major
Kartik Gautam
Emilio Cobos Álvarez
Makoto Kato
Martin Stransky
Alexis Beingessner
Philipp Ammann
Ricky Stewart
Tom Schuster
Bogdan Tara
Jed Davis
Narcis Beleuzu
Paulo Matias
Razvan Maries
Nico Grunbaum
Nathan Froyd
Tom Prince
Kris Maglione
Andrew Halberstadt
Axel Hecht
Coroiu Cristina