The copy and deepcopy operations currently use 'setitem' which obviously fails
for 'ReadOnlyDict'. But copying the dict yields a new instance so there should
be no reason this is prevented.
Specifically, I'd like to make certain subsets of task configuration read-only.
But copying is needed when we split the task into multiple tasks (e.g for
chunking).
Differential Revision: https://phabricator.services.mozilla.com/D131284
The command site manager needs to be able to do ad-hoc pip
installations, while the Mach site manager needs to manage
the system `sys.path` and conditionally create an on-disk
virtualenv.
By splitting the class into two, we can now give each use case the
attention it deserves.
Differential Revision: https://phabricator.services.mozilla.com/D129529
Rather than requiring that consumers remember to `ensure()` before
calling `activate()`, we can do so automatically during activation.
There isn't a valid use case in which we want obsolete virtualenvs to be
activateable.
Usages of `MozSiteManager` have been updated accordingly.
Differential Revision: https://phabricator.services.mozilla.com/D129688
The existing terminology had two issues:
* `VirtualenvManager` wasn't always associated with an on-disk
`virtualenv`: for example, when running in automation, Mach
"activates" a `VirtualenvManager`, updating its import scope,
but without ever creating an on-disk `virtualenv`.
* An upcoming patch splits the `VirtualenvManager` class, pulling
"on-disk virtualenv-handling functions" from the project-wide
interface for managing Python's import scope.
After some good discussion with Ahal, I think we've struck
the terminology that handles this distinction well: we'll call
the "import scope"-handling part the "site", and we'll continue
to call on-disk virtualenvs (and their representative classes)
as, well, virtualenvs.
Differential Revision: https://phabricator.services.mozilla.com/D130391
As `_run_pip()` is being removed from `VirtualenvManager` in an upcoming
patch, its usages need to be removed. Besides, they're using an
"internal" function, which is a bit of a smell.
Note that this _could_ have been solved by exposing a public `run_pip()`
function. However, I felt like that was worse because:
* Friction here is good as we try to migrate the codebase to embrace the
"requirements definition file" technique to install dependencies.
* There could be confusion about the relationship between
`install_pip_package()` (only works if venv already activated)
and `_run_pip()`, which works "in general".
Differential Revision: https://phabricator.services.mozilla.com/D130120
Previously, we always used the last UUID in configure.sh.
This is incorrect for beta because beta and release share the same "official" branding and the beta UUIDs are specified first.
We now decide whether to use the first or last UUID based on the channel.
Differential Revision: https://phabricator.services.mozilla.com/D130833
It's possible for the `PYTHON3` config to point to a different Python
than that that is executing the configure scripts themselves.
This flexibility was needed for the Python 2->3 migration, but now that
it's complete, we can remove the extra configuration and just lean on
the Python interpreter used to run configure.
Differential Revision: https://phabricator.services.mozilla.com/D129863
This patch adds a new command line argument --aab which allows users to install
GVE as an AAB.
This will also be used in a future patch to install the test runner as AAB.
Differential Revision: https://phabricator.services.mozilla.com/D127323
The fact that the test runner app is defined inside the geckoview test package
has always felt like a hack to me. I've mistakenly thought that
TestRunnerActivity was used in GeckoView's junit tests many times (even though
that's not the case).
From what I can see, there's no way to generate an AAB package for androidTest,
so to be able to run Gecko tests as AAB we finally need to define the
TestRunner as an ordinary package instead.
Differential Revision: https://phabricator.services.mozilla.com/D127320
The optional `log_handle` argument was only used by:
* Configure, but the output was always dumped to stdout and *not*
`config.log`. The manual logging _was_ used to handle encoding issues,
but those are likely invalidated by the Python 3 migration.
* `./mach doc`, where we were putting virtualenv setup into stderr,
which seems incorrect. The commit adding it doesn't explain why it's
the case, but I'm guessing it shouldn't be too risky to remove.
Additionally, `log_handle` was used very inconsistently: for example,
running `install_pip_package()` would _not_ use `log_handle`.
So, removing `log_handle` removes a bit of abstraction leakage.
Differential Revision: https://phabricator.services.mozilla.com/D129298
Now that are prioritizing system over virtualenv site-packages, the
system `pip` is sometimes being used instead.
This is causing issues when the system pip is set up in a
distro-specific way, such as when "debundled":
https://github.com/pypa/pip/blob/9.0.1/pip/_vendor/__init__.py#L53-L61
However, if we vendor `pip`, `setuptools` and `wheel`, and ensure that
they're prioritized in the `sys.path` before anything is imported from
the system, then we can ensure that we're using a modern `pip` _and_
sidestep system-specific pip weirdness.
Note that `pip-compile`'s `--allow-unsafe` flag is not as dangerous as
it sounds.
There's confusion among maintainers about its origin:
https://github.com/jazzband/pip-tools/issues/522
Additionally, it's going to be enabled by default in a future
`pip-tools` release. So, it's not scary for us to embrace here.
Also, heads up that the "pip outdated warning" no longer needs
to be manually silenced, since pip avoids that code path when
not running from an "installed" context.
Differential Revision: https://phabricator.services.mozilla.com/D127182
The previous behaviour was to:
* Never add a `pthfile` to the Mach virtualenv, and
* Always add Mach's paths to the `sys.path` when Mach initializes
However, this meant that `pip` would needlessly install packages
that already exist in the vendored environment.
Tweak `pth` behaviour so that `pip` behaves more efficiently.
Differential Revision: https://phabricator.services.mozilla.com/D120402
As we leverage the Mach environment more, it becomes increasingly
important that it isn't out-of-date on developer machines.
Add an `up_to_date()` check during Mach initialization.
To minimize the cost to startup, I'm skipping the "pip list" check.
This change required moving `virtualenv` from `mozbuild` to `mach` to
make it available during the early stage of Mach init.
Differential Revision: https://phabricator.services.mozilla.com/D127144
Inline a variable used once, and ensure that `old_env_variables` is
defined consistently before the `finally` block runs to satisfy an
intellisense warning.
Also, ensure that a wide comment is shuffled underneath the line limit.
Differential Revision: https://phabricator.services.mozilla.com/D129296
Now that packages are consistently installed in a dedicated
`pip install` process, we can remove our funky re-execution of
`virtualenv.py`.
Since this re-execution was the only case in which `virtualenv.py`
was directly invoked by Python, we can remove the `__main__`-handling
block.
Also, now that `virtualenv.py` always is interpreted with its needed
modules in-scope, the deferred `MachEnvRequirements` import can be
moved to the top of the file.
Differential Revision: https://phabricator.services.mozilla.com/D129295
`install_pip_package()` does some logic around not running pip if a
package is already installed.
However, this is redundant, because when building a venv, no packages
are installed.
Additionally, this was the constraint forcing us to populate the
virtualenv in a separate process from the one building it.
Differential Revision: https://phabricator.services.mozilla.com/D129294
Ever since all builds we get artifacts from have been cross-compiled, those
artifacts have been for Linux. They are not of any use on Mac or Windows hosts,
so it's unnecessary to get them. And we might as well be consistent across
platforms, and not get them on Linux either, even if they are native.
Differential Revision: https://phabricator.services.mozilla.com/D129790
Ever since all builds we get artifacts from have been cross-compiled, those
artifacts have been for Linux. They are not of any use on Mac or Windows hosts,
so it's unnecessary to get them. And we might as well be consistent across
platforms, and not get them on Linux either, even if they are native.
Differential Revision: https://phabricator.services.mozilla.com/D129790
Since now we can build firefox outside of the unified environment
in the hybrid-build check-syntax should be put to rest.
Differential Revision: https://phabricator.services.mozilla.com/D129616
Bug 1729411 did something similar for the source file it generates, but
that was made redundant with changes in bug 1735455. Those changes,
however, still don't account for the header, so manually handle it.
Differential Revision: https://phabricator.services.mozilla.com/D129587
When a user has been granted access to push to hgmo but does not make
use of their access for 6 months, their access is automatically revoked
via a cronjob in LDAP. This commit adds a check to the `ssh` mach doctor
check to find this condition and notify the user about how to proceed.
Differential Revision: https://phabricator.services.mozilla.com/D128206
Before this change, if `ssh hg.mozilla.org` did not contain any of the
expected output lines, `mach doctor` would fail entirely as a check is
always assumed to return a `DoctorCheck` object. This commit adds a
fallback return as a warning, saying `ssh hg.mozilla.org` returned
something unexpected.
Differential Revision: https://phabricator.services.mozilla.com/D128205
Mike Hommey
Sylvestre Ledru
Chris H-C
Andi-Bogdan Postelnicu
Andrew Halberstadt
Alexandre Lissy
Alexandru Michis
june wilde
Mitchell Hentges
Tom Ritter
Marian-Vasile Laza
Iulian Moraru
James Teh
Geoff Lankow
Agi Sferro
Sebastian Hengst
Mike Hommey
criss
William Durand
Nick Alexander
Connor Sheehan