The `AUTOCLOBBER` mozconfig option is reliably causing builds to fail when
a clobber is triggered. When we auto-clobber a build we do so after running
`configure` but before running `make client.mk`. This means we destroy all
the gathered information from the `configure` step in the objdir and then
attempt to run `make` using the previously destroyed information.
This commit moves the call to `_check_clobber` to an earlier stage in the
build process, before `configure` is called, so any clobber that takes place
will happen before setting up the objdir via `configure`.
Since `_check_clobber` is only called once in the codebase, and both cases
are now adding clobber metrics one after another, we remove the metrics
gathering from `_check_clobber` and rely on callers to set metrics instead.
Also clean up some nested `if` statements that can be flattened.
Differential Revision: https://phabricator.services.mozilla.com/D100794
This function has a few code paths and has a slightly confusing return value.
Add a comment describing what it does and what the return value actually
means.
Differential Revision: https://phabricator.services.mozilla.com/D100793
Since the printed value is a `str` anyways, this causes the converted `bytes`
to be printed to the terminal as `b'/path/to/topobjdir'`. Just print the `str`
version to the screen instead.
Differential Revision: https://phabricator.services.mozilla.com/D100792
When mach errors out, an error report is sent to Sentry. This error
report contains information about the state of the interpreter during
the failure, details about the environment, installed packages and more.
Having this information available immediately when attempting to resolve
a bug report is generally desirable, instead of going through a back-and-forth
needinfo tag on Bugzilla or spending time asking the reporter questions on
Matrix.
This commit captures the Sentry ID returned from `sentry_sdk.capture_exception`
and prints it to the screen. If a user adds this line to their bug report (as
the error messages suggest) a build team member can enter this number into
Sentry to identify the exact report and debug the error. At minimum this will
reduce the amount of back-and-forth between the reporter and the assignee
required to resolve a bug. Optimally it should make bugs easier to spot and
reduce the time spent on end user support requests.
To use the Sentry ID to identify information about a specific bug report, the
bug assignee should open the Mozilla Sentry page for the `mach` project and
paste the ID into the search box, which will produce the full stack trace with
all submitted information.
Differential Revision: https://phabricator.services.mozilla.com/D100247
Knowing whether `brew` or `macports` is available isn't necessary
to generate the android mozconfig.
This should fix the generation of android mozconfig when a package
manager isn't available.
Differential Revision: https://phabricator.services.mozilla.com/D99496
pylint_requirements.txt fail to install with the new pip resolver due
to a conflict between astroid and lazy-object-proxy.
Rather than bumping those packages and handling the potential fallout,
the package-upgrade has been deferred and we will use the legacy
resolver in the interrim.
Differential Revision: https://phabricator.services.mozilla.com/D99940
Some distros include flags when they specify the location of a binary,
such as: `XARGS=xargs -r`.
This was confusing in `configure`, since:
* We require that environment variables contain only path
overrides (without flags).
* The error message thrown when configure chokes was unclear: "why
would `$ xargs -r` fail?"
This patch should make our "path-only" requirement more clear.
Differential Revision: https://phabricator.services.mozilla.com/D100044
At the moment installing Python packages with native code is done by calling
`pip install <package>` and does not enforce any SHA hash for installed
dependencies, nor does it enforce a specific version to be installed.
This commit adds `requirements.in` and `requirements.txt` files for native
packages and changes these packages to be installed by running `pip install`
and passing the requirements file for the package. This allows us to pin the
SHA of the various dependencies. The `.txt` files are generated using
`pip-compile`.
We also add the new requirements files to the sparse profile for `mach`.
Differential Revision: https://phabricator.services.mozilla.com/D99912
Knowing whether `brew` or `macports` is available isn't necessary
to generate the android mozconfig.
This should fix the generation of android mozconfig when a package
manager isn't available.
Differential Revision: https://phabricator.services.mozilla.com/D99496
Knowing whether `brew` or `macports` is available isn't necessary
to generate the android mozconfig.
This should fix the generation of android mozconfig when a package
manager isn't available.
Depends on D99482
Differential Revision: https://phabricator.services.mozilla.com/D99496
Our old version of `glean_sdk` only had pip wheels, which meant that
un-wheeled machines had no option for building the package.
Differential Revision: https://phabricator.services.mozilla.com/D98795
Native python packages will probably require python headers. For some
systems, these are available in a separate system package (such as
python[3]-dev).
If the user is bootstrapping with "--no-system-changes", we just
try to install these native packages and fail gracefully if required
headers aren't installed.
Differential Revision: https://phabricator.services.mozilla.com/D98530
The CI environment doesn't bootstrap/`create-mach-environment`, so
native modules (such as zstandard) aren't installed.
`./mach artifact install` is run in CI and needs `zstandard`, so we
must install it ad-hoc.
Differential Revision: https://phabricator.services.mozilla.com/D98794
This patch enables PSM and Firefox to use TLS 1.3 Encrypted Client Hello (draft -08). Specifically:
- Compile NSS with NSS_ENABLE_DRAFT_HPKE=1
- Add ECH "public_name" handling in SSLServerCertVerification.cpp (see: https://tools.ietf.org/html/draft-ietf-tls-esni-08#section-6.3.2)
- Adds `mIsAcceptedEch` to TransportSecurityInfo, and xpcshell tests for ECH use cases
- Adds EncryptedClientHelloServer to facilitate the xpcshell tests
- Un-ifdef Set/GetEchConfigs code in nsNSSIOLayer.cpp. Also reverted the Base64 encoding and decoding, as the data returned from DNS is already decoded (wire-format).
Differential Revision: https://phabricator.services.mozilla.com/D92651
Removes a bunch of zstandard installations that were ran from
within the mach virtualenv, which should already have zstandard
installed.
Differential Revision: https://phabricator.services.mozilla.com/D98387
This patch enables PSM and Firefox to use TLS 1.3 Encrypted Client Hello (draft -08). Specifically:
- Compile NSS with NSS_ENABLE_DRAFT_HPKE=1
- Add ECH "public_name" handling in SSLServerCertVerification.cpp (see: https://tools.ietf.org/html/draft-ietf-tls-esni-08#section-6.3.2)
- Adds `mIsAcceptedEch` to TransportSecurityInfo, and xpcshell tests for ECH use cases
- Adds EncryptedClientHelloServer to facilitate the xpcshell tests
- Un-ifdef Set/GetEchConfigs code in nsNSSIOLayer.cpp. Also reverted the Base64 encoding and decoding, as the data returned from DNS is already decoded (wire-format).
Differential Revision: https://phabricator.services.mozilla.com/D92651
The sysconfig config var of "MACOSX_DEPLOYMENT_TARGET" isn't
consistently a string, so we need to explicitly convert it
Differential Revision: https://phabricator.services.mozilla.com/D98533
The zstandard package is always installed in the mach virtualenv.
The patch assumes that zstandard is only used from the mach virtualenv,
and never the build virtualenv.
Differential Revision: https://phabricator.services.mozilla.com/D98387
Most of the deletions here come from bug 1481612, the `--with-windows-wheel` option to `mach vendor python`, which according to that commit message "is very single-purpose: it's intended to let us vendor an unpacked
wheel for psutil on Windows". Since vendoring `psutil` is something we're no longer doing, we can safely just delete that added code.
Differential Revision: https://phabricator.services.mozilla.com/D90919
Install `psutil` when setting up the `mach` `virtualenv`s and stop importing the in-tree version in the build.
Nothing in-tree currently assumes or mandates the installation of `psutil` (all uses of `psutil` are guarded with imports of the form `try : import psutil; except ImportError: psutil = None`), so there's no back-incompatibility concerns here. There will be an awkward period where telemetry will be lacking CPU/disk data for everyone until they re-run `mach bootstrap` or `mach create-mach-environment`, but that will come back as people gradually update their `virtualenv`s.
An alternative to circumvent that issue is REQUIRING that `psutil` be installed by adding an assertion in `mach` that `psutil` can be found (allowing us to remove all the conditional logic in-tree around whether `psutil` is installed), but I wouldn't claim that we're ready to do that and deal with whatever fallout might occur.
Differential Revision: https://phabricator.services.mozilla.com/D90914
It currently doesn't work for several reasons, and even if what
currently fails would start working, we don't actually want people to be
using the clang it installs, for example, because it would run through
Rosetta and be slow. So until all these details are figured out, we
should outright reject mach bootstrap and send people to the always
evolving google doc that has the instructions.
Differential Revision: https://phabricator.services.mozilla.com/D97985
Connor Sheehan
Mike Conley
Andi-Bogdan Postelnicu
Sylvestre Ledru
Cosmin Sabou
Mitchell Hentges
longsonr
Tarek Ziadé
Dorel Luca
Narcis Beleuzu
Bob Clary
Butkovits Atila
Petr Sumbera
june wilde
Kevin Jacobs
Gregory Mierzwinski
Emilio Cobos Álvarez
Razvan Maries
Mike Hommey
Ricky Stewart