2017-08-23 21:47:44 +03:00
|
|
|
[include]
|
|
|
|
|
# Various mach commands call config.guess to resolve the default objdir name.
|
|
|
|
|
path:build/autoconf/config.guess
|
2019-01-29 01:14:23 +03:00
|
|
|
path:build/autoconf/config.sub
|
|
|
|
|
path:build/moz.configure/checks.configure
|
|
|
|
|
path:build/moz.configure/init.configure
|
|
|
|
|
path:build/moz.configure/util.configure
|
2017-08-23 21:47:44 +03:00
|
|
|
# Used for bootstrapping the mach driver.
|
2021-09-03 23:46:22 +03:00
|
|
|
path:build/mach_initialize.py
|
Bug 1656993: Create and require by default global `virtualenv`s in `~/.mozbuild` for `mach` r=mhentges,ahal
In two different places we've been encountering issues regarding 1) how we configure the system Python environment and 2) how the system Python environment relates to the `virtualenv`s that we use for building, testing, and other dev tasks. Specifically:
1. With the push to use `glean` for telemetry in `mach`, we are requiring (or rather, strongly encouraging) the `glean_sdk` Python package to be installed with bug 1651424. `mach bootstrap` upgrades the library using your system Python 3 in bug 1654607. We can't vendor it due to the package containing native code. Since we generally vendor all code required for `mach` to function, requiring that the system Python be configured with a certain version of `glean` is an unfortunate change.
2. The build uses the vendored `glean_parser` for a number of build tasks. Since the vendored `glean_parser` conflicts with the globally-installed `glean_sdk` package, we had to add special ad-hoc handling to allow us to circumvent this conflict in bug 1655781.
3. We begin to rely more and more on the `zstandard` package during build tasks, this package again being one that we can't vendor due to containing native code. Bug 1654994 contained more ad-hoc code which subprocesses out from the build system's `virtualenv` to the SYSTEM `python3` binary, assuming that the system `python3` has `zstandard` installed.
As we rely more on `glean_sdk`, `zstandard`, and other packages that are not vendorable, we need to settle on a standard model for how `mach`, the build process, and other `mach` commands that may make their own `virtualenv`s work in the presence of unvendorable packages.
With that in mind, this patch does all the following:
1. Separate out the `mach` `virtualenv_packages` from the in-build `virtualenv_packages`. Refactor the common stuff into `common_virtualenv_packages.txt`. Add functionality to the `virtualenv_packages` manifest parsing to allow the build `virtualenv` to "inherit" from the parent by pointing to the parent's `site-packages`. The `in-virtualenv` feature from bug 1655781 is no longer necessary, so delete it.
2. Add code to `bootstrap`, as well as a new `mach` command `create-mach-environment` to create `virtualenv`s in `~/.mozbuild`.
3. Add code to `mach` to dispatch either to the in-`~/.mozbuild` `virtualenv`s (or to the system Python 3 for commands which cannot run in the `virtualenv`s, namely `bootstrap` and `create-mach-environment`).
4. Remove the "add global argument" feature from `mach`. It isn't used and conflicts with (3).
5. Remove the `--print-command` feature from `mach` which is obsoleted by these changes.
This has the effect of allowing us to install packages that cannot be vendored into a "common" place (namely the global `~/.mozbuild` `virtualenv`s) and use those from the build without requiring us to hit the network. Miscellaneous implementation notes:
1. We allow users to force running `mach` with the system Python if they like. For now it doesn't make any sense to require 100% of people to create these `virtualenv`s when they're allowed to continue on with the old behavior if they like. We also skip this in CI.
2. We needed to duplicate the global-argument logic into the `mach` script to allow for the dispatch behavior. This is something we avoided with the Python 2 -> Python 3 migration with the `--print-command` feature, justifying its use by saying it was only temporarily required until all `mach` commands were running with Python 3. With this change, we'll need to be able to determine the `mach` command from the shell script for the forseeable future, and committing to this forever with the cost that `--print-command` incurs (namely `mach` startup time, an additional .4s on my machine) didn't seem worth it to me. It's not a ton of duplicated code.
Differential Revision: https://phabricator.services.mozilla.com/D85916
2020-08-17 20:21:02 +03:00
|
|
|
path:build/build_virtualenv_packages.txt
|
|
|
|
|
path:build/common_virtualenv_packages.txt
|
|
|
|
|
path:build/mach_virtualenv_packages.txt
|
2020-12-17 00:02:02 +03:00
|
|
|
path:build/psutil_requirements.txt
|
|
|
|
|
path:build/zstandard_requirements.txt
|
2017-08-23 21:47:44 +03:00
|
|
|
path:mach
|
|
|
|
|
# Various dependencies. There is room to trim fat, especially in
|
|
|
|
|
# third_party/python.
|
|
|
|
|
path:python/
|
|
|
|
|
path:testing/mozbase/
|
|
|
|
|
path:third_party/python/
|
2020-06-11 23:38:04 +03:00
|
|
|
# certifi is needed for Sentry
|
|
|
|
|
path:testing/web-platform/tests/tools/third_party/certifi
|
