Creates/updates virtualenvs for some mach commands, replacing
their ad-hoc usage of `install_pip_package()`, `pip install`,
and `sys.path` modifications.
Note: The `docs` virtualenv has `Sphinx==1.1.3` installed, even
though a more modern version of `Sphinx` is used when
`./mach doc` is run. This is ok for now, since `./mach doc` will
just install the newer `Sphinx` over top of the old one. Secondarily,
when we port `./mach doc` to use the centralized system, we'll
be incentivized to make the different `doc` commands use synchonized
versions of the same packages. Success!
Also, note that manual installation of `html5lib` and `requests`
isn't ported to the `wpt` site: this is because they're already
provided by the inherited Mach site.
Differential Revision: https://phabricator.services.mozilla.com/D122902
Pointer to AudioSession object is made std::atomic to resolve crashes (bug 1755700) by atomically modifying/reading the pointer as an effort to make sure any update to the pointer is seen across other threads immediately while not using locks to maintain such concurrency and avoiding dataraces.
This patch is built upon the work done in D136377.
Differential Revision: https://phabricator.services.mozilla.com/D140741
Windows uses `;` as the path separator, Unix-based systems use `:`.
`os.pathsep` conveniently represents the current system's separator.
All OS-specific path separators have been replaced with `os.pathsep`
except for some explicitly Windows-specific mozharness configs.
Differential Revision: https://phabricator.services.mozilla.com/D140981
Updates all build-related jobs (`artifact-build`, `build` and
`instrumented-build`) tasks to explicitly set
`MACH_BUILD_PYTHON_NATIVE_PACKAGE_SOURCE=system`. This allows them
to consume `psutil` (if installed on the system) without needing
to hit PyPI.org.
Modifies `build-l10n.sh` and `build-linux.sh` to no longer
explicitly fetch `psutil` from PyPI
(`./mach python --virtualenv psutil`), since that is replaced
by Mach's "native package source" behaviour.
Differential Revision: https://phabricator.services.mozilla.com/D140257
The llvm-symbolizer tasks currently extract a llvm-symbolizer from clang
tasks. Changes in clang 14 make the hack that we have in place to keep
llvm-symbolizer statically linked to libllvm while clang uses a dynamic
libllvm not work anymore, so it's time to bite the bullet and build
llvm-symbolizer separately.
We share most of the build setup with the compiler-rt build.
Differential Revision: https://phabricator.services.mozilla.com/D140711
Only enable them when building on macOS or cross-compiling for macOS on Linux.
# catch redeclaration of ObjC method parameter name
-Wduplicate-method-arg
# catch multiple declarations of ObjC method found
-Wduplicate-method-match
# catch ObjC method with no return type specified
-Wmissing-method-return-type
# catch implicit conversions between ObjC BOOL and int
-Wobjc-signed-char-bool
# catch semicolon before ObjC method body
-Wsemicolon-before-method-body
# catch ObjC method parameter type not matching super class method
-Wsuper-class-method-mismatch
https://clang.llvm.org/docs/DiagnosticsReference.html
Differential Revision: https://phabricator.services.mozilla.com/D140598
The expected behavior during a PGO run is that a browser is started and
exited once, and then it is started again on a page that runs various
workloads and once finished, exits.
When workload happens to crash the content process, the browser is left
running indefinitely, until the taskcluster task itself times out. This
leaves us with no possibility of knowing what actually went wrong during
the run, which is about the worst thing that can happen.
With the browser shutting down on its own in case of crash, the harness
can find the minidumps and report the crash, which is more useful.
Differential Revision: https://phabricator.services.mozilla.com/D140678
Inconsistency confuses some of our tools. As part of this, I:
* Updated some docs to point to rust-minidump
* Added a fallback to mozcrash.py to try both versions
* Make mozcrash.py use --brief output when the local mdsw is used
* Remove the renaming hack from build-minidump-stackwalk.sh
This isn't as simple as a sed because we still have breakpad in tree
for minidump-analyzer. I did my best to replace the right strings.
Differential Revision: https://phabricator.services.mozilla.com/D138971
MozillaBuild 3.4's kernel name was `MINGW32_NT-6.2`, but the new
MozillaBuild's kernel name looks like `MSYS_NT-10.0-19044`.
Update existing first-party detection code to properly handle the modern
MSYS "kernel name" format.
Differential Revision: https://phabricator.services.mozilla.com/D140096
This patch introduces a new system for building IPDL unit tests, which is
roughly inspired by the old cxx unit test design, however designed to work with
gtest. It re-uses the existing IPDLUnitTest process type, using static
constructors only present in xul-gtest to register child actor constructors and
ProcessChild implementations to avoid bloating xul.
The IPDLUnitTest actor is used as a backchannel to communicate test failures
from the child process back to the parent process, and manage the test's async
lifecycle more reliably.
The IPDLUnitTest process also needed to have various properties about how it was
initialized adjusted:
* The IPDLUnitTest process should now always load xul-gtest when running
gtests, by using the "Self" process type and adding a DYLD_LIBRARY_PATH override
on macOS where the "Self" process type cannot be used.
* The IPDLUnitTest process now initializes minimal XPCOM, and runs a
frankeneventloop to allow it to use XPCOM event loop structures such as
SpinEventLoopUntil in tests.
* Support for creating IPDLUnitTest processes on Android was added, as these
tests had never run on android before.
Differential Revision: https://phabricator.services.mozilla.com/D137170
Previously, the Python virtualenv path would be `realpath`'d before the
virtualenv was activated [1].
However, now that (when going through Mach) we're calling `configure.py`
with the build virtualenv's Python binary directly, that `realpath()`
was lost.
We //could// `realpath(self.topsrcdir)` in `building.py`, but then the
virtualenv will be needlessly re-created when it's called from a
non-normpath'd context.
Instead, let's leave realpath-ing Mach's `self.topsrcdir` to another
day, and let's spot-fix this issue: when evaluating `PYTHON3` in
configure, do `realpath()` on the path we get from the running Python
process.
Note: `sys.prefix` was normpath'd instead of `(...).python_path`,
because on Linux virtualenv's `bin/python` is symlinked to the system
installation it's associated with, which we *don't* want here.
[1] https://hg.mozilla.org/mozilla-central/rev/ca4d439114f3#l1.61
Differential Revision: https://phabricator.services.mozilla.com/D138594
For WebGPU, we produce the textures in the compositor process and the
content process doesn't need to be that involved except for hooking up
the texture to the display list. Currently this is done via an external
image ID.
Given that WebGPU needs to work with OffscreenCanvas, it would be best
if its display pipeline was consistent whether it was gotten from an
HTMLCanvasElement, OffscreenCanvas on the main thread, or on a worker
thread. As such, using an AsyncImagePipeline would be best.
However there is no real need to bounce the handles across process
boundaries. Hence this patch which adds CompositableInProcessManager.
This static class is responsible for collecting WebRenderImageHost
objects backed by TextureHost objects which do not leave the compositor
process. This will allow WebGPUParent to schedule compositions directly
in future patches.
Differential Revision: https://phabricator.services.mozilla.com/D138588
`coverage` has native code, so the vendored version was only used as
source code from which the actual package could be built.
Since its always used in a context where we can `pip install` over the
network, let's do that. This cleans up our tree a bit and allows us to
leverage the `coverage` wheels.
Differential Revision: https://phabricator.services.mozilla.com/D138816
Now that mac compiler-rts are built with build-compiler-rt.sh, which has
its own hack around codesign (which was broken in bug 1755415, so fix that),
we don't need this patch anymore.
Differential Revision: https://phabricator.services.mozilla.com/D138985
This improves out-of-the-box support for cross-compiles on tier-1 build
platforms, including for asan. This also reduces the number of different
*-cross toolchains we need for CI builds.
The linux clang is now also a repack, like the others, and the insertion
of the wasm compiler-rt is moved to the repack, which also allows to
remove that part of the build-clang.py script.
Differential Revision: https://phabricator.services.mozilla.com/D138749
Instead of building compiler-rt alongside clang, we reuse the clang
repack logic to put compiler-rts in place when building clang stage2
(which requires the profiling runtime).
The repack logic is unified to handle all platforms as well, and now
that compiler-rt is not built in clang, we also need to copy everything
that is not lib/* from the compiler-rt artifacts, so we now also ensure
that they're not diverging between those compiler-rt artifacts (except
for darwin libs, which need to be unified into universal libraries).
And as a bonus, we can also cross-build the mac clang toolchains with
the stage1 clang and the clang runtime for the target platform.
Differential Revision: https://phabricator.services.mozilla.com/D138748
This simplifies the setup for cross clang repacks.
Also use `ninja install` to get all the relevant things in the artifact.
Differential Revision: https://phabricator.services.mozilla.com/D138745
This improves out-of-the-box support for cross-compiles on tier-1 build
platforms, including for asan. This also reduces the number of different
*-cross toolchains we need for CI builds.
The linux clang is now also a repack, like the others, and the insertion
of the wasm compiler-rt is moved to the repack, which also allows to
remove that part of the build-clang.py script.
Differential Revision: https://phabricator.services.mozilla.com/D138749
Instead of building compiler-rt alongside clang, we reuse the clang
repack logic to put compiler-rts in place when building clang stage2
(which requires the profiling runtime).
The repack logic is unified to handle all platforms as well, and now
that compiler-rt is not built in clang, we also need to copy everything
that is not lib/* from the compiler-rt artifacts, so we now also ensure
that they're not diverging between those compiler-rt artifacts (except
for darwin libs, which need to be unified into universal libraries).
And as a bonus, we can also cross-build the mac clang toolchains with
the stage1 clang and the clang runtime for the target platform.
Differential Revision: https://phabricator.services.mozilla.com/D138748
This simplifies the setup for cross clang repacks.
Also use `ninja install` to get all the relevant things in the artifact.
Differential Revision: https://phabricator.services.mozilla.com/D138745
Bug 1752977 attempted to do that, but that part had to be backed out as
a fixup because it broke LTO support for the linker.
To make it work, cctools needs to link against clang's libLTO, which
requires libLTO to be built with the toolchain sysroot. But system GCC
doesn't support that in a nice way, so we switch to system clang. We
explicitly don't use /usr/bin/clang++ because that would use the system
libstdc++ headers instead of the ones from the sysroot, while using
/usr/lib/llvm-11/bin/clang++ uses the ones from the sysroot.
We keep building clang 5.0 with gcc, though, because that one fails to
build with clang 11.
Differential Revision: https://phabricator.services.mozilla.com/D138712
The new task builds a stage 2 compiler and uses it to generate the
profile as stage 3, and the profile is used to feed the now separate
stage 4 clang build. This will allow us to use this same profile to
build mac clangs with PGO.
Differential Revision: https://phabricator.services.mozilla.com/D138511
The new task builds a stage 2 compiler and uses it to generate the
profile as stage 3, and the profile is used to feed the now separate
stage 4 clang build. This will allow us to use this same profile to
build mac clangs with PGO.
Differential Revision: https://phabricator.services.mozilla.com/D138511
The patch conflicts with latest clang trunk changes.
It was necessary back when we started using clang-cl instead of MSVC,
and we needed to fallback to MSVC for sources that clang-cl couldn't
build. Nowadays, with cross-compiles, that fallback can't happen
anyways, and even on native Windows builds we never use cl.exe.
Differential Revision: https://phabricator.services.mozilla.com/D138615
Mitchell Hentges
Randell Jesup
smolnar
Narcis Beleuzu
Shazib Summar
Mike Hommey
Chris Peterson
Alexis Beingessner
mvollmer
Nika Layzell
Zhao Jiazhong
Alex Ionescu
Andrew Osmond
Emilio Cobos Álvarez
Cosmin Sabou
Csoregi Natalia
Marian-Vasile Laza
Thomas Wisniewski
Jan-Erik Rediger
Butkovits Atila