For "c" (i.e., gcc, clang, and clang-cl), this configures the
`-f{debug,macro}-path-prefix` flags. We'd prefer to use
`-ffile-path-prefix`, but it seems that `clang-cl` does not recognize
that flag.
For "rust" (i.e., rustc/cargo), this configures `--remap-path-prefix`.
This is one step toward getting `sccache` hits across source and
object directories.
Differential Revision: https://phabricator.services.mozilla.com/D113065
Based on the docs and the code within `pathlib2`, it seems to focus
only on backporting the main features of `pathlib` to be available for
Python 2. It does _not_ provide features in newer Python versions
of `pathlib` (such as `is_mount`, new in Python 3.7). Therefore, it
doesn't provide anything that the standard library of Python 3.6+
doesn't have.
Differential Revision: https://phabricator.services.mozilla.com/D117071
Make mozperftest work with the new mozproxy command line
Add ability to run proxy on mobile browser
Add ability to use record and playback modes
Add unit tests
Differential Revision: https://phabricator.services.mozilla.com/D115544
Adds a `mach doctor` check to assert Python's locale is not set to
`UTF-8`, an invalid value due to a Python bug in most pre-3.8 versions.
Differential Revision: https://phabricator.services.mozilla.com/D117081
Adds a check that SSH access to hg.mozilla.org is properly configured.
Attempt to run `ssh hg.mozilla.org` which will authorize to hgmo over
SSH via `pash`, without running any commands on the service. In the case
the user has properly configured SSH we will see a success message and
information on the SCM levels granted to the user. If we don't see
a successful connection, parse the output to ensure the user is using
an email address as their username (which is required on hg.mozilla.org).
If the user couldn't connect yet does have an email address as their
username, notify them that they do not have permission to push to hgmo.
Differential Revision: https://phabricator.services.mozilla.com/D117080
This will both:
* Ensure that developers aren't accidentally using Mach with Python 2
in the year 2021.
* Confirm that CI tasks aren't still using Python 2 with Mach.
* Remove support for Python 3.5, which is EOL'd.
Differential Revision: https://phabricator.services.mozilla.com/D116484
Now that we configure pip within mach virtualenvs to turn off
its "outdated" warning, we no longer need to apply the suppression
as a CLI argument.
Differential Revision: https://phabricator.services.mozilla.com/D116891
We don't offer API splits any more, and with the separation of GeckoView with
the rest of the front-end it's increasingly unlikely that we will in the
future.
This change makes it so that the build name doesn't contain the API version so
that we can update it without breaking all the automation that relies on the
build name.
Differential Revision: https://phabricator.services.mozilla.com/D114369
We don't offer API splits any more, and with the separation of GeckoView with
the rest of the front-end it's increasingly unlikely that we will in the
future.
This change makes it so that the build name doesn't contain the API version so
that we can update it without breaking all the automation that relies on the
build name.
Differential Revision: https://phabricator.services.mozilla.com/D114369
Adds test to ensure that we don't modify files that are vendored
with PyPI.
If a modification needs to happen to a vendored library, we should
instead:
* Modify the library upstream and vendor the new version, or
* Remove the library from requirements.in and requirements.txt so that
it's now a "manually-vendored" library. Note that there many be
issues if this occurs to a library depended on by 3rd-party
libraries.
Differential Revision: https://phabricator.services.mozilla.com/D116428
Generating Python lockfiles is system-dependent, and this currently
mostly affects us just on the Python 3.6 <=> 3.7 boundary - 3.6 requires
a few additional packages like `importlib_metadata` and `iso8601`.
If a different Python version is used, the test in CI is guaranteed to
fail. By validating in advance, we help future vendoring developers
by helping them avoid the wait before CI failure and the time it
takes to troubleshoot.
Differential Revision: https://phabricator.services.mozilla.com/D116773
Rewrite `doctor.py` from an object-oriented paradigm to a simpler
function-based architecture. Each check is a function that returns
or yields `DoctorCheck`s. `DoctorCheck`s contain a status that describes
the status of the check (success, warning, fatal error etc), a string
of text with useful information for display to the terminal, and may
optionally attach a callable which can be executed to fix issues found
by the checks.
`mach doctor` is updated to use the new `run_doctor` function instead
of creating a `Doctor` object. The default value of `verbose` in `run_doctor`
is `False`, meaning only issues with the users system are printed to the
screen.
Most of the existing check logic is left intact from the previous
implementation. Checks for `platform` are changed to use the
`sys.platform.startswith` pattern. The latest MozillaBuild version is
pulled by querying the `tip` commit on hg.mozilla.org instead of using
a hard-coded variable.
Differential Revision: https://phabricator.services.mozilla.com/D116539
HEAD is a symbolic ref that points to the default branch of the
repository, it will point to master, main, or anything else that the
repository is using as its default branch.
Differential Revision: https://phabricator.services.mozilla.com/D116678
This will both:
* Ensure that developers aren't accidentally using Mach with Python 2
in the year 2021.
* Confirm that CI tasks aren't still using Python 2 with Mach.
* Remove support for Python 3.5, which is EOL'd.
Differential Revision: https://phabricator.services.mozilla.com/D116484
I chose to do this at the level of the outer Python invocation because:
1. `python -m cProfile ...` handles writing the file and some other
details. It's possible to rebuild the functionality -- the tools
are there -- but the APIs are awkward.
2. this allows to profile `mach` internals, instead of just the
invoked command's implementation.
This uses the return code of the `get_command` subshell to transmit
the single bit of information "is the flag present".
The Python-level argument is required in order to have `--help` know
about the option and to avoid the `mach` shell script having to filter
arguments.
Differential Revision: https://phabricator.services.mozilla.com/D116151
It's not practically make a difference (the flags we end up setting it
to don't do anything when linking all the wasm files into one).
Differential Revision: https://phabricator.services.mozilla.com/D116437
This will both:
* Ensure that developers aren't accidentally using Mach with Python 2
in the year 2021.
* Confirm that CI tasks aren't still using Python 2 with Mach.
* Remove support for Python 3.5, which is EOL'd.
Differential Revision: https://phabricator.services.mozilla.com/D116484
This could be easily generalized to avoid the updating the virtualenv
more generally, but I think this functionality should be in the
`virtualenv_manager` -- something like `install_pip_binary(...)` -- so
I've just done the most impactful example here.
Differential Revision: https://phabricator.services.mozilla.com/D116148
Another step towards avoiding the need for classes in mach commands;
here we are removing constructors either by changing them into helpers
or by simple refactorings.
Differential Revision: https://phabricator.services.mozilla.com/D112434
Now, when running mach commands that invoke `pip`, it will no longer
inform the user that it needs an update.
We reach into `distutils` to determine the "site-packages" directory
pattern, then apply it to our virtualenv.
Differential Revision: https://phabricator.services.mozilla.com/D115940
At least in PyCharm, expanding a `VirtualenvManager` instance means
resolving all the properties and fields of the instance.
However, if that property is doing non-trivial work, the debugger
wouldn't run that subprocess while we're stopped at a breakpoint.
So, the instance would sit there with the "Collecting data..." text.
Differential Revision: https://phabricator.services.mozilla.com/D115935
This was necessary before because we first started the build by invoking
make with client.mk, which would then invoke configure. But that changed
in bug 1671424, and we now never need make before configure has run
(except in tests, for which we implement a fallback).
Differential Revision: https://phabricator.services.mozilla.com/D116064
Another step towards avoiding the need for classes in mach commands;
here we are removing constructors either by changing them into helpers
or by simple refactorings.
Differential Revision: https://phabricator.services.mozilla.com/D112434
A clobber must be able to run, even if `substs` (configure output)
is not available.
However, when this behaviour was implemented, it was set up to replace
all usages.
Since we're trying to make mach commands no longer objects, this
kind of overriding won't be possible in the future. Fortunately, it's
not needed: only `Clobberer` needs a fallible substs, the other
consumers don't!
Differential Revision: https://phabricator.services.mozilla.com/D116128
Back-to-back `./mach clobber gradle` invocations were failing
because `shutil.rmtree(..)` would complain that the target directory
didn't exist.
Differential Revision: https://phabricator.services.mozilla.com/D116129
No longer includes `python` in the `./mach clobber` defaults since
python cache files should no longer be affecting `./mach artifact`.
Removes `third_party/python` purging from `./mach clobber python`
since we don't build native Python modules there.
Moves virtualenv-purging from `./mach clobber objdir` to
`./mach clobber python`.
Differential Revision: https://phabricator.services.mozilla.com/D115728
Accidentally missing a `return` in a code path could mean that
`handle_package(...)` would accidentally do an action _and_
raise the "Unknown action" error.
This change resolves that, and it simplifies the code a bit.
Differential Revision: https://phabricator.services.mozilla.com/D115923
`MACH_VIRTUALENV` was never used, and `MOZBUILD_VIRTUALENV` was never
set (the virtualenv was always activated during the build, rather
than before).
Differential Revision: https://phabricator.services.mozilla.com/D115921
This was originally set up so that tests wouldn't "create a new
`virtualenv` for no reason." However, virtual environments now will have
different packages installed, and therefore the separation is necessary.
So, for the virtual environment used for builds (regular or for tests):
* We want it to be able to reuse the build venv, if it already exists.
* We don't want to pollute a `pytest` virtualenv with build-specific
packages.
Differential Revision: https://phabricator.services.mozilla.com/D115641
We had split up `init` from `init_py3` because `mach` had
traditionally been invoked by either Python 2 or Python 3, and
the two couldn't share the same virtualenv.
Now that the same context isn't shared by both Python 2 and 3
3
(developers always use Python 3, and the remaining Python 2
usages are CI jobs that never reuse the objdir with Python 3),
We can centralize on a single default virtualenv.
I've called this "common" instead of "init" to clarify its
existing position as the virtualenv that's used by many different
commands. As we associate virtualenvs with requirement definitions,
it'll also make the file less confusing: it's a "common" requirement
definition as opposed to an "init" one.
Differential Revision: https://phabricator.services.mozilla.com/D115635
Now, when running mach commands that invoke `pip`, it will no longer
inform the user that it needs an update.
We reach into `distutils` to determine the "site-packages" directory
pattern, then apply it to our virtualenv.
Differential Revision: https://phabricator.services.mozilla.com/D115940
At least in PyCharm, expanding a `VirtualenvManager` instance means
resolving all the properties and fields of the instance.
However, if that property is doing non-trivial work, the debugger
wouldn't run that subprocess while we're stopped at a breakpoint.
So, the instance would sit there with the "Collecting data..." text.
Differential Revision: https://phabricator.services.mozilla.com/D115935
We were raising an exception as a result of a python error
which was masked by a catch statement that thought we didn't
know how to remove the file. But we did.
Fix the python error, and add an exception at the end of the
function to be raised and then caught when we really don't know
how to remove a file.
Differential Revision: https://phabricator.services.mozilla.com/D115936
As a step towards moving mach commands outside of classes, this converts all
properties into methods so that they can later become top-level helper functions.
Differential Revision: https://phabricator.services.mozilla.com/D112196
Instead of using the pushlog, grab the hg log using a revset to avoid
being confused by branches: we're looking for revisions that are
ancestors of the new revision but not of the previous tag, i.e.
"current % previous".
Differential Revision: https://phabricator.services.mozilla.com/D115624
Instead of using the pushlog, grab the hg log using a revset to avoid
being confused by branches: we're looking for revisions that are
ancestors of the new revision but not of the previous tag, i.e.
"current % previous".
Differential Revision: https://phabricator.services.mozilla.com/D115624
Copy / migrate strings from onboarding/defaultBrowserNotification. Add updated ltr/rtl images and show them for pin and pin+default. Dynamically adjust steps and record telemetry to identify what was shown. Handle button actions based on string ids. Alias upgradeDialog to aboutwelcome for now. Avoid oddness with scrollbars by hiding horizontal scroll.
Differential Revision: https://phabricator.services.mozilla.com/D115142
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).
Differential Revision: https://phabricator.services.mozilla.com/D109650
Added two fields:
1. Collect if the current shell is opened via vscode.
2. Connect if it's a remote ssh connection n the current shell.
Differential Revision: https://phabricator.services.mozilla.com/D114323
A `.arcconfig` may exist in the `.git` directory. Within worktrees,
the `.git` folder isn't at `<topsrcdir>/.git`, so we have to resolve it.
Differential Revision: https://phabricator.services.mozilla.com/D113320
While here, remove the unused clippyProcess.config. And because we need
to manipulate `$PATH` before running `cargo`, use the same wrapping
function in `get_clippy_version`.
Differential Revision: https://phabricator.services.mozilla.com/D113902
The `target_dir` is always the objdir, since we try to share compile
output. Therefore, storing it in `RustLibrary` shouldn't be necessary.
Removes the associated `RUST_LIBRARY_TARGET_DIR` moz.build option
as well.
Differential Revision: https://phabricator.services.mozilla.com/D113565
Newer versions of Windows don't allow un-elevated processes to access
the list of Windows Defender exclusions.
Rather than incorrectly falling back to assuming that there's no
exclusions, we now instead skip the exclusion check.
Differential Revision: https://phabricator.services.mozilla.com/D112648
Since Python 3, attempting to fetch a registry key that doesn't exist
throws `FileNotFoundError`, not `WindowsError: [Error 2]`.
Differential Revision: https://phabricator.services.mozilla.com/D112647
As a step towards moving mach commands outside of classes, this converts all
properties into methods so that they can later become top-level helper functions.
Differential Revision: https://phabricator.services.mozilla.com/D112196
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).
Differential Revision: https://phabricator.services.mozilla.com/D109650
Check if a user is a Mozilla employee by checking their
Bugzilla groups, or checking if their VCS email ends
with "@mozilla.com".
When a user is setting up a new build environment, telemetry
will be automatically enabled for them if they are an
employee. If they're not an employee, they'll be asked
if they want to opt in.
Differential Revision: https://phabricator.services.mozilla.com/D106315
Migrates two strings to fluent and uses sentence casing.
Places identity security block into a toolbar button.
Fixes margin spacing.
Removes green color from secure connection.
Differential Revision: https://phabricator.services.mozilla.com/D111368
Nick Alexander
Mike Conley
Butkovits Atila
Mitchell Hentges
Paul Adenot
Emilio Cobos Álvarez
Greg Tatum
Florin Strugariu
Andrey Bienkowski
Connor Sheehan
Steve Fink
Agi Sferro
Scott
Sandor Molnar
Dorel Luca
Mike Hommey
Alexandru Michis
june wilde
Alex Lopez
Tom Ritter
Henrik Skupin
Julien Cristau
Iulian Moraru
Ed Lee
Gregory Mierzwinski
Andi-Bogdan Postelnicu
Marco Bonardo
Narcis Beleuzu
Kershaw Chang
championshuttler
Emma Malysz
Cosmin Sabou