Our platform-specific `bootstrap` code transitively depends on
the standard library's `distutils` module, but Debian Linux and derived
distros generally split `distutils` into a separate installable OS
package from `python3` itself.
So, Python 3 being able to run isn't a sufficient guarantee that
`distutils` is available.
To catch this case, add a `distutils` check at the very beginning of
Mach initialization.
-----
Also remove an obsolete comment claiming that `mach bootstrap` doesn't
need `distutils`, which was obsoleted by Bug 1717051.
Differential Revision: https://phabricator.services.mozilla.com/D136639
Assuming that the `MOZILLABUILD` environment variable is set, allow
invoking Mach from non-MozillaBuild terminals.
Note that MozillaBuild still needs to be installed, and the
`MOZILLABUILD` environment variable will have to be set.
For future reference: when I tried setting this up with Windows
Store's Python 3.9, I encountered issues when running binaries installed
via `pip`: it would fail with `abort: failed to load Python DLL
python3x.dll`.
Differential Revision: https://phabricator.services.mozilla.com/D133936
Move Python version check as early as possible so that more code can
safely depend on modern behaviour while out-of-date Python versions still
get graceful error messages.
Without this change, Python 2 usages fail on importing `importlib.util`
before the nice "out-of-date version" warning is printed.
Differential Revision: https://phabricator.services.mozilla.com/D134185
As part of this, the shell-script part of `./mach` can be removed,
making it pure Python.
There's a change in `--profile-command` behaviour, though: it now only
profiles the specific command, rather than all of Mach.
This is because _so much of Mach_ has already been run before
CLI arguments are parsed in the Python process.
If a developer wants to profile Mach itself, they can manually run
`python3 -m cProfile -o <file> ./mach ...`
Differential Revision: https://phabricator.services.mozilla.com/D133928
Move Python version check as early as possible so that more code can
safely depend on modern behaviour while out-of-date Python versions still
get graceful error messages.
Without this change, Python 2 usages fail on importing `importlib.util`
before the nice "out-of-date version" warning is printed.
Differential Revision: https://phabricator.services.mozilla.com/D134185
As part of this, the shell-script part of `./mach` can be removed,
making it pure Python.
There's a change in `--profile-command` behaviour, though: it now only
profiles the specific command, rather than all of Mach.
This is because _so much of Mach_ has already been run before
CLI arguments are parsed in the Python process.
If a developer wants to profile Mach itself, they can manually run
`python3 -m cProfile -o <file> ./mach ...`
Differential Revision: https://phabricator.services.mozilla.com/D133928
mach will initialize from comm/build/mach_initialize.py if it's present.
See bug 1731160. The initialization function wraps the one in build/mach_initialize.py,
then extends sys.path and loads more mach_commands.py files
Differential Revision: https://phabricator.services.mozilla.com/D131869
Consolidate Mach virtualenv management to the front of the
Mach process. This obsoletes `./mach create-mach-environment`
and simplifies the `sh` portion of the top-level `./mach` script.
This helps ensure that the Mach virtualenv doesn't become
out-of-sync and simplifies the mental model of the Mach
virtualenv situation.
Differential Revision: https://phabricator.services.mozilla.com/D120401
It was in `nativecmds` to make finding `pip3` easier.
However, since we're removing the distinction between "regular" and
"native" cmds (Mach will create and activate its venv naturally during
initialization), we need to start pruning the `nativecmds` list.
Fortunately, we can use the stored "external_python" value to decide
where to install "moz-phab" to.
The new behaviour continues to support the following use cases:
* Using Mach with system Python in the `$PATH`.
* Using Mach from within a human-managed virtualenv.
* Either of the above^ in combination with `MACH_USE_SYSTEM_PYTHON`.
Differential Revision: https://phabricator.services.mozilla.com/D120397
We've overloaded "bootstrap" to mean three different things:
* The "standalone bootstrap script": `python/mozboot/bin/bootstrap.py`.
This is to freshly clone a new repo, then run `./mach bootstrap`.
* `./mach bootstrap`: Install necessary dependencies and set up the
system for development.
* "Mach bootstrap": do the in-process initialization work Mach needs
before it can run commands.
By using the term "initialize" instead, perhaps we can remove
ambiguity when discussing Mach.
I'm not attached to the name (or this change at all), but I'm interested
in reviewer thoughts :)
Differential Revision: https://phabricator.services.mozilla.com/D120410
I chose to do this at the level of the outer Python invocation because:
1. `python -m cProfile ...` handles writing the file and some other
details. It's possible to rebuild the functionality -- the tools
are there -- but the APIs are awkward.
2. this allows to profile `mach` internals, instead of just the
invoked command's implementation.
This uses the return code of the `get_command` subshell to transmit
the single bit of information "is the flag present".
The Python-level argument is required in order to have `--help` know
about the option and to avoid the `mach` shell script having to filter
arguments.
Differential Revision: https://phabricator.services.mozilla.com/D116151
Updates `./mach bootstrap` to use `--no-interactive` from global args.
Ensures all bootstrap prompts have a default option.
Differential Revision: https://phabricator.services.mozilla.com/D106814
This code is generally missing tests, but I tried it out on the wpt-sync
where it's used and with this fix it worked (previously it had broken
because the wpt libraries started to assume Py 3 semantics)
Differential Revision: https://phabricator.services.mozilla.com/D107992
Shortly after mach was introduced, bug 840588 turned mach into a small
wrapper that could be copied in a directory part of $PATH, making that
wrapper load the right thing from the right source directory.
Years later, that setup is not really supported, and it's extra
complexity that can lead to unpleasant surprises.
Differential Revision: https://phabricator.services.mozilla.com/D102376
Catch additional cases where both __PYVENV_LAUNCHER__ is set and we invoke a
virtualenv python by unsetting the variable in a prominent location.
This will specifically catch the `./mach bootstrap` => `./mach artifact` case.
Differential Revision: https://phabricator.services.mozilla.com/D101809
It's not 100% clear how long this command will live, but it doesn't look
immediately removable, and it was easy to add python3 support.
Differential Revision: https://phabricator.services.mozilla.com/D94179
This command appears to work just fine under python3 already -- I removed it from the py2 list and ran it with various options, and they all worked fine.
Differential Revision: https://phabricator.services.mozilla.com/D94174
The former is a POSIX shell construct, usually a builtin, and the latter
is an external program that is not guaranteed to be available.
When `which` is not available, a full ./mach build succeeds with this
change.
Differential Revision: https://phabricator.services.mozilla.com/D92864
Minor fix to remoteautomation.py: Increment stdoutlen before any type conversions,
to ensure that it accurately reflects the byte offset in the file.
With this last change, 'mach mochitest' appears to run correctly on Android with Python 3:
switch it over to Python 3.
Differential Revision: https://phabricator.services.mozilla.com/D91586
Mitchell Hentges
Marian-Vasile Laza
Rob Lemley
Nick Alexander
championshuttler
Andrew Halberstadt
James Graham
Joel Maher
Henrik Skupin
Jan de Mooij
Mike Hommey
Kagami Sascha Rosylight
Geoff Brown
Victor Porof
Cosmin Sabou
Ben Hearsum