Some developers hold onto a `comm` directory as a mount point
that they conditionally populate depending on whether they're working on
Firefox or Thunderbird.
The "`comm` directory exists == is a thunderbird checkout"
assumption isn't compatible with this workflow.
The fix embraced by this patch is to check if the `comm` directory has
any contents.
Differential Revision: https://phabricator.services.mozilla.com/D119193
These are enough for me to run bootstrap+configure+build.
Some touch third-party code (gyp), but per discussion in the earlier
versions of this revision that seems fine.
Differential Revision: https://phabricator.services.mozilla.com/D119080
When running `mach python-test` on a clobbered tree, we create both the
python-test and common virtualenvs, but the former should be enough. The
latter is only created when running tests that use the BaseConfigureTest
class, when it indirectly includes init.configure, which makes the
configure sandbox run the virtualenv_python3 function, which ends up
initializing the common virtualenv. We only don't end up re-executing
that virtualenv python thanks to 6680ca0acc27 which was a workaround
that would allow any virtualenv in $objdir/_virtualenvs, while still
creating a virtualenv it won't use.
Tests now tell virtualenv_python3 to not use the normal virtualenv
manager, and remove the workaround.
Differential Revision: https://phabricator.services.mozilla.com/D118775
These are enough for me to run bootstrap+configure+build.
Some touch third-party code (gyp), but per discussion in the earlier
versions of this revision that seems fine.
Differential Revision: https://phabricator.services.mozilla.com/D119080
When we customized the list of possible `applications` for M1 Macs, the
change was only applying to the displayed list, but not the actual
list that the choice was indexed against.
This is fixed by properly using `applications` instead of
`APPLICATIONS` where relevant.
Note that it's still possible to manually request an
`artifact_mode` if you provide it as an argument for `./mach bootstrap`,
but this workflow is usually only done by devs who are familiar with all
the applications, and therefore likely know what they're doing.
Differential Revision: https://phabricator.services.mozilla.com/D118625
This was added ing bug 1352595. It has only been used briefly, and we
since then have gone all the way to never compress omni.ja. Even if we
did go back to compressing, we'd probably go with zstd rather than
brotli.
The gecko-side support for this has only ever been nightly-only.
Differential Revision: https://phabricator.services.mozilla.com/D118655
`_get_infer_config()` returns 3 items, but the existing code expects 2.
Add a third `_` to capture the other unwanted item.
Differential Revision: https://phabricator.services.mozilla.com/D118535
wireless-tools-devel isn't available on Rocky Linux. Fortunately, it doesn't appear to be needed for the build anymore, and it was added almost a decade ago, so it should be safe to remove.
Differential Revision: https://phabricator.services.mozilla.com/D118532
In the existing behaviour, we would return a node/npm executable from
`find_executable()`, even if we couldn't extract a version from it or if
the extracted version was insufficient.
Now, we fail early if the detected node/npm is insufficient.
Differential Revision: https://phabricator.services.mozilla.com/D118377
As :julienw beautifully described on Bugzilla,
we no longer need to search for "nodejs" since all
packages providing Node >= v12 will provide the `node`
binary.
See his comment here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1716600#c4
Base-toolchain tasks with Node v10 should still be
happy because, unlike Debian's Node v10, they provide
a `node` binary instead of `nodejs`.
Differential Revision: https://phabricator.services.mozilla.com/D118375
(Tl;dr: The original author of this patch was axel@mozilla.com (:pike).
This patch has been modified to run in production automation.)
Cross-channel takes all the en-US strings from the shipping train branches
and consolidates them into a single repository. Originally, this ran out of
https://hg.mozilla.org/users/axel_mozilla.com/cross-channel-experimental/ .
The original version was coupled tightly with specific hg internals, making
hg upgrades fragile. Axel wrote
https://bug1659691.bmoattachments.org/attachment.cgi?id=9170636 before
handing off project ownership; this is the original patch to stop replaying
history to simplify the logic.
This patch also automates the previously manual preparation steps, allows
for running both in automation and locally, and adds retries for production
robustness.
Differential Revision: https://phabricator.services.mozilla.com/D116537
Having separate `<name>.pth` files in the virtual environments
isn't providing an advantage. We can simplify configuration
by putting all `pth` adjustments into a single file: `mach.pth`.
Differential Revision: https://phabricator.services.mozilla.com/D117710
This has two benefits:
1. `handle_package()` becomes more clear - rather than referring to
`action` and `package` with array index numbers, we now give
them real names. The benefit here is also shown in `up_to_date()`.
2. This makes the top-level parser for `packages()` less opinionated
about sub-formats: if an action has a nested structure, it should
have the flexibility to define what it looks like.
Differential Revision: https://phabricator.services.mozilla.com/D117708
After some testing in `try` and locally, the manual
`PYTHONEXECUTABLE` definitions shouldn't be needed
anymore.
There's been some work on Brew's
Python to improve its `sitecustomize` behaviour.
The most likely improvement is:
https://github.com/Homebrew/homebrew-core/pull/65297
However, I'm not fully confident in this change.
If it fails, it's more likely to affect developers
than CI. I think it's worth attempting a landing,
because if the variable is indeed obsolete, then
deleting it will avoid some spicy "action-at-a-
distance" behaviour.
Differential Revision: https://phabricator.services.mozilla.com/D117452
Vendoring wheels has three benefits:
* There's far less files, so Firefox checkouts will be smaller.
* It works around `zipp` not allowing `pip install`
from extracted source `tar.gz` files. Now, we should
be able to use the pip resolver against vendored
packages, which will be needed for future
mach virtualenv work.
* `./mach vendor python` takes far less time to execute.
Since we need the raw Python to be available to add to the `sys.path`,
we extract the wheels before putting them in tree.
Due to the structure of some wheels being less nested
than of a source `tar.gz`, `common_virtualenv_packages`
needed to be adjusted accordingly.
`install_pip_package()` had to be tweaked as well since you can't
`pip install` an extracted wheel. So, we "re-bundle" the wheel
before installing from a vendored package.
Replace python packages with wheels where possible
This contains the vendoring changes caused by the
last patch.
For reviewing, there's a couple things to note:
* A bunch of files are deleted, since there's generally
less files in a wheel than in a source archive.
* There's a new `.dist-info` directory for each
extracted wheel, so expect roughly 5 or
6 new files for each wheel'd package.
* There should be no source code changes other than
moves from package names changing from having
`-` to having `_`.
Differential Revision: https://phabricator.services.mozilla.com/D116512
As pointed out by Harry, this will help in the short-term with
onboarding new developers who will be using M1 macs.
Note that this patch doesn't leverage the "instance" classes to define
whether they're artifact-mode compatible or not, and that's because:
* Most (all except one?) of the systems support artifact mode, therefore
* Since this is a temporary workaround, it's more deletable to have
a top-level `if/else` than to add a `supports_artifact_mode` property
to each Bootstrapper.
Differential Revision: https://phabricator.services.mozilla.com/D117946
Last year, we stopped vendoring Python packages that have native
code. Since we have only had pure-python packages since, the
Windows-specific qualifier (or excluder in the case of `!windows`)
hasn't been needed.
I don't foresee us needing it again, but if anything we can peel it
back from `hg` history if this assumption is incorrect.
Differential Revision: https://phabricator.services.mozilla.com/D117468
On Windows, Python's temporary directories are placed in `%APPDATA%`.
Unfortunately, the failing tests want to assume that some "other"
path (as a subpath of a tempdir) won't be within any
`filter_args()` known top-level paths, which includes the user's home
dir.
The only reason that this assumption has passed in CI so far is due
to Windows case insensitivity: running locally had `~` evaluated
to `c:\Users\...`, while running in CI had it be `c:\users\...`.
Since the tempdir path started with `C:\Users`, `filter_args()`
would `<path omitted>` in CI but would `$HOME/...` locally.
By not basing the `other_path` on `tempdir`, we now consistently get the
same results across platforms.
Differential Revision: https://phabricator.services.mozilla.com/D117698
The `calls` parameter is expected to be an iterable
container of calls, not a singular call.
This was working in `mock-1.0.0` because `calls`
was (confusingly) allowed to be a single item
if `any_order==False`. This behaviour isn't
the same in the standard library.
Differential Revision: https://phabricator.services.mozilla.com/D117074
`./mach vendor python <package>` was already adding its new package
to `requirements.txt`, so we were getting full resolver support, which
is good. However, it caused our `requirements.in` file to start
getting out-of-date, and therefore made it harder to identify the
top-level dependencies.
Arguably, we could have `./mach vendor python <package>` automatically
update `requirements.in`, too, but then we need to solve the edge cases,
such as "What if the package is already in `requirements.in`? What if
that existing item has a different version?"
The hardest part of updating `requirements.in` was finding it, so I've
also modified the `./mach vendor python` help text to make it more
identifiable.
Differential Revision: https://phabricator.services.mozilla.com/D116386
Mitchell Hentges
Emilio Cobos Álvarez
Mike Hommey
surajeet310
Sandor Molnar
june wilde
Alexander J. Vincent
Tarik Eshaq
Marian-Vasile Laza
Aki Sasaki
Gregory Mierzwinski
Alexandru Michis
Iulian Moraru
Jeff Muizelaar
Csoregi Natalia
Butkovits Atila