Consolidate Mach virtualenv management to the front of the
Mach process. This obsoletes `./mach create-mach-environment`
and simplifies the `sh` portion of the top-level `./mach` script.
This helps ensure that the Mach virtualenv doesn't become
out-of-sync and simplifies the mental model of the Mach
virtualenv situation.
Differential Revision: https://phabricator.services.mozilla.com/D120401
`--no-virtualenv` was needed for one use case: the
`get_and_diffoscope` task, despite not needing `psutil`, would
run into failures during virtualenv-creation because we used to
unconditionally build the `psutil` package, and `get_and_diffoscope`
didn't have the environment needed for such a build.
Since we no longer build and install `psutil` into every virtualenv,
it's no longer needed for its one usage, which means that it can be
removed.
`--requirements` is replaced by `--virtualenv`, which removes an
ad-hoc pip package installation and embraces the centralized dep
system.
`--no-activate` is now implied by default: a virtualenv is only
created and activated if `--virtualenv` is provided.`
`ipython==7.16.1` was the chosen version because it is the last
one compatible with Python 3.6.
Differential Revision: https://phabricator.services.mozilla.com/D131529
This is the first step in replacing a huge pile of our breakpad-based infra
with our new implementation (rust-minidump). This stackwalker is only used
for reporting crashes in local builds and CI, so it's a good first deploy.
Although most of the work on rust-minidump has been focused on the JSON output,
this uses the --human output, because it's primarily intended for humans to
directly read. There is however some minor parsing done on this format. This
is not *strictly* supported by --human (it has no schema) but it's not something
we plan to break. (This parsing is pre-existing, just recording the facts.)
The new build configs/scripts are hybridized from fix-stacks and dump_syms,
as this basically is a hybrid of the two. In particular it needs the openssl
vendoring tricks that dump_syms uses, but is a target binary that prefers
win32 over win64 (like fix-stacks).
Technically a regression but probably just culling legacy cruft at this point:
this patchset removes support for building a local copy of minidump-stackwalk
from source. You must now download a copy built on task-cluster using mozboot.
mozboot *already* did this, which is why this feature appears to be legacy cruft
-- there was little reason to build a local copy.
However rust-minidump's minidump-stackwalk has a far better portability story,
so you can build+install your own local copy by just running:
cargo install minidump-stackwalk
Differential Revision: https://phabricator.services.mozilla.com/D131315
This tool is used to install AAB packages to devices.
AAB is the new package format for Android Apps and will supersede APK.
Differential Revision: https://phabricator.services.mozilla.com/D127318
Because the result is 99% identical to the win64-nsis toolchain, we
produce a single toolchain that can be used both on native windows and
on cross builds.
Differential Revision: https://phabricator.services.mozilla.com/D125926
The build-clang step in the clang-mingw toolchains just replicates what
we already do for clang, except with no patched applied, which could
arguably seen as an issue. Instead of rebuilding what we essentially
already have as a result of the clang-12 toolchain, we use the clang-12
toolchain directly instead.
Differential Revision: https://phabricator.services.mozilla.com/D125912
The reason the error mentioned in build-mingw32-nsis.sh happens is that
the default mode NSIS builds in is a fully-installed mode, where it
hardcodes the locations of its data files. This is why nsis needs to
be used from the same place it's built for. But there's another mode,
enabled with NSIS_CONFIG_CONST_DATA_PATH=no, that makes it relocatable,
and makes it find its data files relatively to the nsis binary.
However, there's a bug in the nsis build scripts, which makes the nsis
binary installed in the destination directory instead of a bin/
subdirectory, while the source code itself looks for data files relative
to the parent directory of the directory that contains the executable.
So we need to set PREFIX_BIN to force the executable to be installed in
a bin/ subdirectory.
There is also an issue in nsis itself when it's executed by anything
other than a shell, which we patch out.
Differential Revision: https://phabricator.services.mozilla.com/D125638
While the comment at the top of build-mingw32-nsis.sh is true, it only
applies to nsis being in the same path in the build tasks as it was
built from in the toolchain task. So we don't actually need to build
it in a mingw32/bin directory, and can ship it in a nsis/bin directory,
as long as that's where we build it.
That makes the toolchain match the expectations from bootstrap, which
also makes PATH adjustments for nsis unnecessary, no other toolchain
used in those builds providing a binary in mingw32/bin.
Differential Revision: https://phabricator.services.mozilla.com/D125637
It made sense when we did have problems with the one shipped with clang
and needed an explicitly newer version, but that hasn't been true in a
while. On the contrary, we're now using a version older than clang for
no reason other than having forgotten to update it.
Differential Revision: https://phabricator.services.mozilla.com/D124886
We leave the following ones unchanged:
- geckodriver because the results are used to releases on github.
- sixgill because the script that creates it is not in-tree.
- *-dist-toolchain because sccache is not expecting a .tar.zst.
We use native tar support in most cases, except for toolchain scripts also
used on Windows, for which we use our zstdpy script.
Differential Revision: https://phabricator.services.mozilla.com/D124733
We leave the following ones unchanged:
- geckodriver because the results are used to releases on github.
- sixgill because the script that creates it is not in-tree.
- *-dist-toolchain because sccache is not expecting a .tar.zst.
We use native tar support in most cases, except for toolchain scripts also
used on Windows, for which we use our zstdpy script.
Differential Revision: https://phabricator.services.mozilla.com/D124733
We leave the following ones unchanged:
- geckodriver because the results are used to releases on github.
- sixgill because the script that creates it is not in-tree.
- *-dist-toolchain because sccache is not expecting a .tar.zst.
We use native tar support in most cases, except for toolchain scripts also
used on Windows, for which we use our zstdpy script.
Differential Revision: https://phabricator.services.mozilla.com/D124733
Sandor Molnar
Agi Sferro
Mike Hommey
Mitchell Hentges
Marian-Vasile Laza
Sylvestre Ledru
Alexis Beingessner
Iulian Moraru
Tom Ritter
Connor Sheehan
Andi-Bogdan Postelnicu
Butkovits Atila
Cosmin Sabou
Noemi Erli