When "build targets" are used, the sccache server isn't
being used/integrated properly with make.
By moving rusttests to a separate build "application",
`client.mk` is invoked normally, and sccache is happy.
* Move rusttests to a new build "application".
* Move and denormalize rusttests mozconfigs. The new rusttests
mozconfigs should expand to be mostly identical to their old
versions, except with `MOZ_AUTOMATION_*=0` and
`--enable-application=tools/rusttests`.
They will differ a bit from their old variants because some
of them originally didn't source from their
respective `$app_mozconfigs/common`.
* Modify `baseconfig.mk` to change tiers if running rusttests.
* Adjust CI config to use new "rusttests" build application.
Differential Revision: https://phabricator.services.mozilla.com/D106294
While we could change qcms, encoding_rs and packed_simd to not emit
RUSTC_BOOTSTRAP on newer versions of rust, like we do for gkrust-shared,
it's not worth the effort (especially for those that are vendored).
Differential Revision: https://phabricator.services.mozilla.com/D105423
Bug 1690930 added sysroots that can be bootstrapped. With this change,
we allow --enable-bootstrap=install to pull the right sysroot for the
configured target, and --enable-bootstrap to update it if it was already
there.
Differential Revision: https://phabricator.services.mozilla.com/D104797
Back when the flag was added, the libffi source code didn't build for
ARM with the clang integrated assembler. That has been fixed with the
upgrade to libffi v3.3 in bug 1659906, so we can use the integrated
assembler now.
Differential Revision: https://phabricator.services.mozilla.com/D102261
* We add a config option for setting the number of content services;
* We add a config option to indicate whether content services should be isolated.
This one is just a `project_flag` since it doesn't really need the ability to
be overridden; it's something whose default we would want to flip when the
time comes;
* We set a dependency so that mobile/android/base/pre-export is executed;
* We add the `gen_from_jinja.py` script which is mostly just a dumb shim that
takes the input template and the config arguments, instantiates jinja,
generates the final output, and dumps it to the output fd;
* We add the requisite `moz.build` statements to generate the manifest overlay
and the service definitions;
* We update `build.gradle` so that Gradle knows to look for the generated files
when building the apk.
Differential Revision: https://phabricator.services.mozilla.com/D82577
* We add a config option for setting the number of content services;
* We add a config option to indicate whether content services should be isolated.
This one is just a `project_flag` since it doesn't really need the ability to
be overridden; it's something whose default we would want to flip when the
time comes;
* We set a dependency so that mobile/android/base/pre-export is executed;
* We add the `gen_from_jinja.py` script which is mostly just a dumb shim that
takes the input template and the config arguments, instantiates jinja,
generates the final output, and dumps it to the output fd;
* We add the requisite `moz.build` statements to generate the manifest overlay
and the service definitions;
* We update `build.gradle` so that Gradle knows to look for the generated files
when building the apk.
Differential Revision: https://phabricator.services.mozilla.com/D82577
With clang 12, win32 debug tests run out of stack in the 1000-deep layout crashtests.
Usually when this type of thing happens, it's because the compiler made a drastically heavier inlining decision along the recursive part of the stack, with a difference in the hundreds of bytes, and we pay that price of that space 1000 times. In those cases we normally try to snip out the heavy piece with a MOZ_NEVER_INLINE.
However, in this case, the difference between old and new compilers was only something like 16 bytes of the cyclical piece. This suggests that we're already up pretty close against the limit, and we're only a bad day away from failing tests even with existing clang. This only happens in debug builds since they have lots of logging helpers on the stack. Release builds are still well under the limit.
So I propose to increase the stack limit in debug. While we normally have to be pretty careful about address space on win32, it should be less of a concern with these non-shipping builds.
Differential Revision: https://phabricator.services.mozilla.com/D101196
Also rename `branchIfNegativeBigInt` to `branchIfBigIntIsNegative` to match
the other branch methods from part 2 and part 8.
Differential Revision: https://phabricator.services.mozilla.com/D98158
Also rename `branchIfNegativeBigInt` to `branchIfBigIntIsNegative` to match
the other branch methods from part 2 and part 8.
Differential Revision: https://phabricator.services.mozilla.com/D98158
On a very parallel debug build, I see a long time just waiting for
bindgen / style compilation / geckoservo.
Turns out that a bunch of this is just proc macros / build scripts.
Optimizing it saves between 10 and 17 seconds of my debug build. We
might want to consider running bindgen much like cbindgen rather than
rebuilding it all the time, which should help a lot more, but my guess
is that this should still help with the pretty hot custom derives that
the style crate runs.
This needs rust 1.41, so the requirement for tools/crashreporter needs
to be bumped as a consequence. To make things simpler, it was bumped
to 1.47 while we're at it.
Differential Revision: https://phabricator.services.mozilla.com/D98366
* Bumps the tsan toolchain to rust-nightly-2020-11-14 that has my patches to make -Zbuild-std work in vendored environments:
* https://github.com/rust-lang/cargo/pull/8834
* https://github.com/rust-lang/rust/pull/78790
* Passes -Zbuild-std to cargo when MOZ_TSAN is defined (mk_add_options --enable-thread-sanitizer)
* Removes generic Rust supressions and adds much more specific ones
* One presumed upstream false positive from tsan not understanding the code
* One actual upstream bug tsan found (yay!)
* One new real issue uncovered
* One issue that probably already existed intermittently but I happened to hit
Differential Revision: https://phabricator.services.mozilla.com/D97165
This makes --enable-thread-sanitizer turn on Rust tsan (-Zsanitizer=thread).
This requires changing SpiderMonkey tsan to use the tsan rust nightly.
In future changes, more Rust tsan integration will key off of MOZ_TSAN.
Differential Revision: https://phabricator.services.mozilla.com/D96453
This makes --enable-thread-sanitizer turn on Rust tsan (-Zsanitizer=thread).
This requires changing SpiderMonkey tsan to use the tsan rust nightly.
In future changes, more Rust tsan integration will key off of MOZ_TSAN.
Differential Revision: https://phabricator.services.mozilla.com/D96453
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
Depends on D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
This commit also allows `memfd_create` in the seccomp-bpf policy for all
process types.
`memfd_create` is an API added in Linux 3.17 (and adopted by FreeBSD
for the upcoming version 13) for creating anonymous shared memory
not connected to any filesystem. Supporting it means that sandboxed
child processes on Linux can create shared memory directly instead of
messaging a broker, which is unavoidably slower, and it should avoid
the problems we'd been seeing with overly small `/dev/shm` in container
environments (which were causing serious problems for using Firefox for
automated testing of frontend projects).
`memfd_create` also introduces the related operation of file seals:
irrevocably preventing types of modifications to a file. Unfortunately,
the most useful one, `F_SEAL_WRITE`, can't be relied on; see the large
comment in `SharedMemory:ReadOnlyCopy` for details. So we still use
the applicable seals as defense in depth, but read-only copies are
implemented on Linux by using procfs (and see the comments on the
`ReadOnlyCopy` function in `shared_memory_posix.cc` for the subtleties
there).
There's also a FreeBSD implementation, using `cap_rights_limit` for
read-only copies, if the build host is new enough to have the
`memfd_create` function.
The support code for Android, which doesn't support shm_open and can't
use the memfd backend because of issues with its SELinux policy (see bug
1670277), has been reorganized to reflect that we'll always use its own
API, ashmem, in that case.
Differential Revision: https://phabricator.services.mozilla.com/D90605
Henri Sivonen
Mozilla Releng Treescript
Jesse Schwartzentruber
Christian Holler
Alexandru Michis
Mitchell Hentges
Mike Hommey
André Bargull
Masatoshi Kimura
Aaron Klotz
Butkovits Atila
David Major
Kartik Gautam
Cosmin Sabou
Noemi Erli
Emilio Cobos Álvarez
Makoto Kato
Martin Stransky
Alexis Beingessner
Philipp Ammann
Ricky Stewart
Tom Schuster
Bogdan Tara
Jed Davis