Historically, client.mk was not invoked with -jn because it would create
race conditions, but that was actually mostly solved by the addition of
`.NOTPARALLEL` in bug 422986, although the mechanism of adding -jn via
`MOZ_MAKE_FLAGS` or `MOZ_PARALLEL_BUILD` has continued well past that.
Nowadays, client.mk is only invoked by mach (it will even bail out if
that's not the case) and only has one target (`build`) and no
dependencies.
This means we don't need to rely on `MOZ_PARALLEL_BUILD` to pass `-jn` in
some cases, and can just always invoke `make -f client.mk` with `-jn`, even
when we just want no parallelism, in which case we can use `-j1`.
This, in turn, allows to remove the extra allow_parallel argument to
`_run_make`, and only rely on `num_jobs`, and to remove some of the
multiple ways the `n` in `-jn` could be set.
Differential Revision: https://phabricator.services.mozilla.com/D124729
We had logic in both `mach_bootstrap` and the Mach Bootstrapper to
create the state_dir.
This joins them and has the added benefit of creating the state dir
earlier in the Mach lifecycle (as will be needed for early instantiation
of the Mach virtualenv).
Differential Revision: https://phabricator.services.mozilla.com/D120400
It was originally added to "create-mach-environment" because that would
eventually be called by "./mach bootstrap".
However, as "create-mach-environment" is going away, this moves it
closer to the command that it's meant to impact.
Differential Revision: https://phabricator.services.mozilla.com/D120399
Though it's acceptable for an optional dependency to not be installed,
it *is* a problem if it's installed to the wrong version.
Differential Revision: https://phabricator.services.mozilla.com/D122886
We've overloaded "bootstrap" to mean three different things:
* The "standalone bootstrap script": `python/mozboot/bin/bootstrap.py`.
This is to freshly clone a new repo, then run `./mach bootstrap`.
* `./mach bootstrap`: Install necessary dependencies and set up the
system for development.
* "Mach bootstrap": do the in-process initialization work Mach needs
before it can run commands.
By using the term "initialize" instead, perhaps we can remove
ambiguity when discussing Mach.
I'm not attached to the name (or this change at all), but I'm interested
in reviewer thoughts :)
Differential Revision: https://phabricator.services.mozilla.com/D120410
This required migrating several strings to Fluent, and the bulk of this patch
is those migrations. The rest of the items matched up with an entry in the app
menu, so those items were switched over to use the app menu strings.
Differential Revision: https://phabricator.services.mozilla.com/D124270
There are two problems that this patch addresses:
1) The path to the VS solution file that we're using to launch it is malformed,
because os.path.join is using backslash seperators, but we're passing it a
path which already contains forward slash seperators, and mixing the two is
not valid. This is preventing VS from being launched at all.
2) We're throwing if explorer.exe does not return 0 when we call it to launch
VS, but explorer.exe always returns 1 when run this way, even if it
succeeded, so we output a spurious exception to the console.
Differential Revision: https://phabricator.services.mozilla.com/D124488
Rather than deleting the expected target directory of each package
that's being vendored, clear the whole `third_party/python` directory
and re-populate it from scratch.
As part of this, there's an "exclusion" list for packages that can't
be vendored from PyPI.
This has some benefits:
* It'll be harder to forget scraps of files and directories and leave
them in `third_party/python`.
* The exclusion list makes it more clear which packages are managed
manually, and the friction it adds to the workflow will guide
developers to use "requirements.in" instead.
The `test_up_to_date_vendor` test will verify that the vendor directory
is always clean.
Differential Revision: https://phabricator.services.mozilla.com/D123124
Note that, as part of adding this packages to the automated vendoring
system, some dependencies were automatically added - most notably,
dependencies of `taskcluster` that become visible with Python 3.6+.
Also, adds `**/.git` to the exclusions because:
* `.git` is part of our `.hgignore`, but
* `.git` is part of the `aiohttp` `tar.gz` file.
Since the file isn't needed for `pip install`-ing `aiohttp`,
and since we want `./mach vendor python` to be a no-op when there's
no requirement changes, we exclude it.
Differential Revision: https://phabricator.services.mozilla.com/D123122
In 'gen_test_backend.py' we fallback to an 'EmptyConfig' in the event there is
no build environment. We do this for the benefit of tests that don't need a
Firefox build (like Python unittests). Without it, we wouldn't be able to run
e.g |mach python-test python/mozversioncontrol| unless you had a build.
However since this 'EmptyConfig' doesn't have 'ENABLE_TESTS' set, it means any
manifests defined in a 'moz.build' behind the 'TEST_DIRS' variables will not be
discovered.
Differential Revision: https://phabricator.services.mozilla.com/D120386
Now that `piptools` is no longer installed from the vendored location,
there are no remaining cases where we are doing `install_pip_package()`
with a vendored package.
Differential Revision: https://phabricator.services.mozilla.com/D123241
We've been installing `piptools` so that we can use the `pip-compile`
binary. However, we can just as easily do
`python -m piptools compile ...` instead, which also allows us to
skip a redundant install.
Differential Revision: https://phabricator.services.mozilla.com/D123240
The `tempfile.tempdir` cache value isn't always populated, but
`tempfile.gettempdir()` always is. They both provide the same result.
Differential Revision: https://phabricator.services.mozilla.com/D124261
This adds the sha256 for the ICU4X license as a variable to vendor_rust.py,
so it can be used as we add ICU4X crates. Since there are several projects
ongoing that want to use ICU4X, I think it makes sense to get the license
review done once, early, so that whoever is ready to land code that uses
ICU4X first is not blocked waiting on license review.
Differential Revision: https://phabricator.services.mozilla.com/D123776
The affected functions would cause an UnboundLocalError if they are moved
to module-level, because using an assignment inside a function makes a
variable local in Python, making it illegal to have a local variable name
be the same as a function name that is called within the same function.
The adopted solution is to rename those functions in preparation for the
future de-classing.
Differential Revision: https://phabricator.services.mozilla.com/D123867
By hybrid unified system we understand a system that encapsulates modules that are built in the unified mode
but also other modules, like `dom/Animation`, as an example, in the non unified environment.
This approach is desirable since we already have most of the modules transitioned to the non unified system but there are
still some that are not yet compatible, but in the long term this will be done by each module owner and can be also tested
locally using the build system.
If a module can't be built outside the unified method it's `moz.build` config file needs to have `REQUIRES_UNIFIED_BUILD = False`
To also enable this we need to have a flag from `mozconfig`, like:
```
ac_add_options --disable-unified-build
```
Differential Revision: https://phabricator.services.mozilla.com/D122328
By hybrid unified system we understand a system that encapsulates modules that are built in the unified mode
but also other modules, like `dom/Animation`, as an example, in the non unified environment.
This approach is desirable since we already have most of the modules transitioned to the non unified system but there are
still some that are not yet compatible, but in the long term this will be done by each module owner and can be also tested
locally using the build system.
If a module can't be built outside the unified method it's `moz.build` config file needs to have `REQUIRES_UNIFIED_BUILD = False`
To also enable this we need to have a flag from `mozconfig`, like:
```
ac_add_options --disable-unified-build
```
Differential Revision: https://phabricator.services.mozilla.com/D122328
Automatically generated path that adds flag `REQUIRES_UNIFIED_BUILD = True` to `moz.build`
when the module governed by the build config file is not buildable outside on the unified environment.
This needs to be done in order to have a hybrid build system that adds the possibility of combing
unified build components with ones that are built outside of the unified eco system.
Differential Revision: https://phabricator.services.mozilla.com/D122345
The build system assumes the target compiler is of the same type as the
wasm compiler, but that's not true for wasm32-wasi, which we compile
with clang, while targeting windows, which we compile with clang-cl.
We handle the duality for host/target in a compiler-specific way, but
here, the wasm compiler is always going to be clang.
Differential Revision: https://phabricator.services.mozilla.com/D123041
RTL flags are flags that chooses which C/C++ runtime library to use
(multi-threaded, debug, etc.). They're only relevant when targeting
Windows, not when targeting wasi.
Frame pointer flags are not relevant to wasm (as a matter of fact,
the same wasm code is generated whether using -fomit-frame-pointer
or -fno-omit-frame-pointer)
Differential Revision: https://phabricator.services.mozilla.com/D123040
The purpose of this is to remove as many docstrings from CommandProvider
classes to make the step of moving commands out of classes simpler.
Where possible, the docstring has been moved to or merged with the function.
Differential Revision: https://phabricator.services.mozilla.com/D123288
For whatever reason, ./mach doc is sourcing rewrite_mozbuild.py
causing the ast.get_source_segment function to get redefined.
(We do this redefinition because it is very easy to accidently
use get_source_segment in this code and forget it doesn't
work on Taskcluster.)
So add an exemption for sphinx to pass through to the original
function, but also fix an inifnite loop where we weren't calling
the original function.
Differential Revision: https://phabricator.services.mozilla.com/D123487
The build system assumes the target compiler is of the same type as the
wasm compiler, but that's not true for wasm32-wasi, which we compile
with clang, while targeting windows, which we compile with clang-cl.
We handle the duality for host/target in a compiler-specific way, but
here, the wasm compiler is always going to be clang.
Differential Revision: https://phabricator.services.mozilla.com/D123041
RTL flags are flags that chooses which C/C++ runtime library to use
(multi-threaded, debug, etc.). They're only relevant when targeting
Windows, not when targeting wasi.
Frame pointer flags are not relevant to wasm (as a matter of fact,
the same wasm code is generated whether using -fomit-frame-pointer
or -fno-omit-frame-pointer)
Differential Revision: https://phabricator.services.mozilla.com/D123040
Mitchell Hentges
Csoregi Natalia
Niklas Baumgardner
Mike Hommey
Katherine Patenio
Francesco Lodolo (:flod)
Kagami Sascha Rosylight
Sandor Molnar
Molly Howell
Cosmin Sabou
Andrew Halberstadt
Noemi Erli
Dan Minor
Alex Lopez
Andi-Bogdan Postelnicu
Narcis Beleuzu
Tom Ritter
Paul Adenot