Граф коммитов

4976 Коммитов

Автор SHA1 Сообщение Дата
Mitchell Hentges ec0e3536bb Bug 1717645: Fix up_to_date for Python 3.6 r=ahal
Python 3.6 doesn't support the `capture_output` option, so use
`stdout=subprocess.PIPE` instead.

Differential Revision: https://phabricator.services.mozilla.com/D120027
2021-07-29 22:26:22 +00:00
Mitchell Hentges fbb9bb362b Bug 1717645: Check virtualenv pths are up-to-date r=ahal
It's possible for a virtualenv to have an incorrect list of directories
to add to the sys.path, such as the following cases:
* Its creation got cancelled halfway through
* The list of pths changed in a new revision
* It got modified by an external tool

By validating the list of provided pths against the list of required
pths, we can ensure that the virtualenv is more dependably up-to-date.

Differential Revision: https://phabricator.services.mozilla.com/D119686
2021-07-29 22:26:22 +00:00
Mitchell Hentges a6f09bc76d Bug 1717645: Resolve nested virtualenv requirements up-front r=ahal
This simplifies consumer logic, since they get the parsed list of pypi
and pth requirements, as well as the list of input files that were
parsed.

One benefit of this simplification is that we no longer
recursively create VirtualenvManagers.

Note that mach_bootstrap cannot (yet) take advantage
of `ParseMachEnvRequirements` because of a dependency cycle:
* `mach_bootstrap` must set up the `sys.path` to import
  `ParseMachEnvRequirements`.
* `mach_bootstrap` would want `ParseMachEnvRequirements` to
  determine which paths to add to the `sys.path`.

Differential Revision: https://phabricator.services.mozilla.com/D119685
2021-07-29 22:26:21 +00:00
Benjamin Beurdouche 4582da2473 Bug 1709817 - Enable NSS documentation in firefox-src-tree. r=beurdouche
Differential Revision: https://phabricator.services.mozilla.com/D119913
2021-07-22 12:10:19 +00:00
Sandor Molnar 77256682a1 Backed out 7 changesets (bug 1720215, bug 1717645, bug 1717051) for causing win debug build bustages. CLOSED TREE
Backed out changeset fb9919b26f30 (bug 1717051)
Backed out changeset 605651561f2a (bug 1717051)
Backed out changeset d1333fe69cf4 (bug 1717051)
Backed out changeset b75dcb922521 (bug 1720215)
Backed out changeset e43fb9c6cd0f (bug 1717645)
Backed out changeset eab19a103b56 (bug 1717645)
Backed out changeset 7864e8241d77 (bug 1717645)
2021-07-21 07:47:18 +03:00
Mitchell Hentges 9d2879a48a Bug 1717051: Move global mach dependencies to requirements definition r=ahal
After removing `optional` in Bug 1712804, we need to add a variant back
here because there's fallible dependencies. However, I've tweaked the
re-introduction of the feature to require a specific repercussion
message as well. This seemed like a decent tradeoff - the developer
becomes aware that the failure is bad, it has repercussions, but it's
not a blocking issue. Additionally, since we're printing pip's output,
the developer will be able to see the underlying error causing the
warning.

I also added comment functionality to requirements definitions to allow
adjacent documentation of why some requirements are fallible. (Related:
I'm looking forward to `mach_bootstrap` not needing to parse
requirements definitions. Almost there!)

Note that we'll temporarily lose the "pinned" nature of the
three moved dependencies until dependency locking is implemented
for Mach requirements definitions. Also note that the pinned
`zstandard_requirements.txt` can't be removed like the other
files because it still has a dangling usage.

Finally, in preparation for review: I didn't make
`PypiOptionalSpecifier` extend `PypiSpecifier` because I figured that
the benefit of flexibility (easier to allow implementations to diverge
without needing to untangle an inheritance relationship) was larger than
the cost of needing to add properties to both specifiers.
If we wanted re-use, I'd probably have `PypiOptionalSpecifier` _contain_
 a `PypiSpecifier`, but then you have to reach deeper into the object to
 get data, so *shrug*.

Differential Revision: https://phabricator.services.mozilla.com/D119835
2021-07-20 22:22:53 +00:00
Mitchell Hentges f4c722d4cf Bug 1717051: Only use stdlib for installing pip packages r=ahal
`_install_pip_package()` may be run from `populate()`, which
is invoked from a child Python process that doesn't
have in-tree Python modules in its sys.path.

An alternate solution would be to add in-tree modules
to the sys.path, but that seemed more costly than
simply using `tempfile` and `shutil`.

Differential Revision: https://phabricator.services.mozilla.com/D119834
2021-07-20 22:22:53 +00:00
Mitchell Hentges b34427450f Bug 1720215: Remove MACOSX_DEPLOYMENT_TARGET virtualenv workaround r=firefox-build-system-reviewers,andi
Now that the upstream Python bug has been resolved since Python 3.4 (at
the latest), we can safely remove the environment variable workaround.

Differential Revision: https://phabricator.services.mozilla.com/D119687
2021-07-20 22:22:52 +00:00
Mitchell Hentges c958faf669 Bug 1717645: Fix up_to_date for Python 3.6 r=ahal
Python 3.6 doesn't support the `capture_output` option, so use
`stdout=subprocess.PIPE` instead.

Differential Revision: https://phabricator.services.mozilla.com/D120027
2021-07-20 22:22:52 +00:00
Mitchell Hentges 6ae508148e Bug 1717645: Check virtualenv pths are up-to-date r=ahal
It's possible for a virtualenv to have an incorrect list of directories
to add to the sys.path, such as the following cases:
* Its creation got cancelled halfway through
* The list of pths changed in a new revision
* It got modified by an external tool

By validating the list of provided pths against the list of required
pths, we can ensure that the virtualenv is more dependably up-to-date.

Differential Revision: https://phabricator.services.mozilla.com/D119686
2021-07-20 22:22:51 +00:00
Mitchell Hentges d11e0a94e8 Bug 1717645: Resolve nested virtualenv requirements up-front r=ahal
This simplifies consumer logic, since they get the parsed list of pypi
and pth requirements, as well as the list of input files that were
parsed.

One benefit of this simplification is that we no longer
recursively create VirtualenvManagers.

Note that mach_bootstrap cannot (yet) take advantage
of `ParseMachEnvRequirements` because of a dependency cycle:
* `mach_bootstrap` must set up the `sys.path` to import
  `ParseMachEnvRequirements`.
* `mach_bootstrap` would want `ParseMachEnvRequirements` to
  determine which paths to add to the `sys.path`.

Differential Revision: https://phabricator.services.mozilla.com/D119685
2021-07-20 22:22:51 +00:00
Cosmin Sabou 84b8cec8b2 Backed out 7 changesets (bug 1717645, bug 1717051, bug 1720215) for busting the gecko decision task. CLOSED TREE DONTBUILD
Backed out changeset dbdd100c60d0 (bug 1717051)
Backed out changeset 4f32438176a0 (bug 1717051)
Backed out changeset 6597db537fa2 (bug 1717051)
Backed out changeset cc686ee3fb6e (bug 1720215)
Backed out changeset 82dc0771b98b (bug 1717645)
Backed out changeset 16b0fb30aff2 (bug 1717645)
Backed out changeset 370e02f73923 (bug 1717645)
2021-07-21 00:59:41 +03:00
Mitchell Hentges ffaaa6b319 Bug 1717051: Move global mach dependencies to requirements definition r=ahal
After removing `optional` in Bug 1712804, we need to add a variant back
here because there's fallible dependencies. However, I've tweaked the
re-introduction of the feature to require a specific repercussion
message as well. This seemed like a decent tradeoff - the developer
becomes aware that the failure is bad, it has repercussions, but it's
not a blocking issue. Additionally, since we're printing pip's output,
the developer will be able to see the underlying error causing the
warning.

I also added comment functionality to requirements definitions to allow
adjacent documentation of why some requirements are fallible. (Related:
I'm looking forward to `mach_bootstrap` not needing to parse
requirements definitions. Almost there!)

Note that we'll temporarily lose the "pinned" nature of the
three moved dependencies until dependency locking is implemented
for Mach requirements definitions. Also note that the pinned
`zstandard_requirements.txt` can't be removed like the other
files because it still has a dangling usage.

Finally, in preparation for review: I didn't make
`PypiOptionalSpecifier` extend `PypiSpecifier` because I figured that
the benefit of flexibility (easier to allow implementations to diverge
without needing to untangle an inheritance relationship) was larger than
the cost of needing to add properties to both specifiers.
If we wanted re-use, I'd probably have `PypiOptionalSpecifier` _contain_
 a `PypiSpecifier`, but then you have to reach deeper into the object to
 get data, so *shrug*.

Differential Revision: https://phabricator.services.mozilla.com/D119835
2021-07-20 21:42:04 +00:00
Mitchell Hentges 16b857cec1 Bug 1717051: Only use stdlib for installing pip packages r=ahal
`_install_pip_package()` may be run from `populate()`, which
is invoked from a child Python process that doesn't
have in-tree Python modules in its sys.path.

An alternate solution would be to add in-tree modules
to the sys.path, but that seemed more costly than
simply using `tempfile` and `shutil`.

Differential Revision: https://phabricator.services.mozilla.com/D119834
2021-07-20 21:42:03 +00:00
Mitchell Hentges 82436a7373 Bug 1720215: Remove MACOSX_DEPLOYMENT_TARGET virtualenv workaround r=firefox-build-system-reviewers,andi
Now that the upstream Python bug has been resolved since Python 3.4 (at
the latest), we can safely remove the environment variable workaround.

Differential Revision: https://phabricator.services.mozilla.com/D119687
2021-07-20 21:42:03 +00:00
Mitchell Hentges 9b2d50e981 Bug 1717645: Fix up_to_date for Python 3.6 r=ahal
Python 3.6 doesn't support the `capture_output` option, so use
`stdout=subprocess.PIPE` instead.

Differential Revision: https://phabricator.services.mozilla.com/D120027
2021-07-20 21:42:02 +00:00
Mitchell Hentges c12ca0aff6 Bug 1717645: Check virtualenv pths are up-to-date r=ahal
It's possible for a virtualenv to have an incorrect list of directories
to add to the sys.path, such as the following cases:
* Its creation got cancelled halfway through
* The list of pths changed in a new revision
* It got modified by an external tool

By validating the list of provided pths against the list of required
pths, we can ensure that the virtualenv is more dependably up-to-date.

Differential Revision: https://phabricator.services.mozilla.com/D119686
2021-07-20 21:42:02 +00:00
Mitchell Hentges 6e4b9c1e3b Bug 1717645: Resolve nested virtualenv requirements up-front r=ahal
This simplifies consumer logic, since they get the parsed list of pypi
and pth requirements, as well as the list of input files that were
parsed.

One benefit of this simplification is that we no longer
recursively create VirtualenvManagers.

Note that mach_bootstrap cannot (yet) take advantage
of `ParseMachEnvRequirements` because of a dependency cycle:
* `mach_bootstrap` must set up the `sys.path` to import
  `ParseMachEnvRequirements`.
* `mach_bootstrap` would want `ParseMachEnvRequirements` to
  determine which paths to add to the `sys.path`.

Differential Revision: https://phabricator.services.mozilla.com/D119685
2021-07-20 21:42:01 +00:00
Barret Rennie b09836e3a7 Bug 1721472 - Correctly return whether or not we are in an SSH connection r=mhentges
Differential Revision: https://phabricator.services.mozilla.com/D120393
2021-07-20 21:13:17 +00:00
Brindusan Cristian 262e09aeea Backed out changeset 919c542f5b8b (bug 1720951) for causing build bustages and py3 failures in test_jarmaker.py.
CLOSED TREE
2021-07-20 19:02:16 +03:00
surajeet310 2f5fc1d2b8 Bug 1720951 - Removed instances of 'ensure_bytes()' from python/mozbuild/mozbuild r=mhentges
Inlined the usage of 'ensure_bytes()' and removed dependency of 'ensure_bytes()' from util.py .

Differential Revision: https://phabricator.services.mozilla.com/D120325
2021-07-20 15:15:18 +00:00
surajeet310 d50cbbff03 Bug 1717651 - Removed 'ensure_subprocess_env()' from mozbuild/util.py r=mhentges
'ensure_subprocess_env()' is obsolete as Python 2 is no longer used in python/mozbuild .

Differential Revision: https://phabricator.services.mozilla.com/D120102
2021-07-19 21:22:27 +00:00
Alex Lopez 77ce415604 Bug 1696251 - Replace self with command_context where possible in existing mach commands. r=mhentges,webdriver-reviewers,perftest-reviewers,whimboo
This step removes all the dependencies of mach commands to
having a MachCommandBase as the `self` by using the `command_context`
argument instead. This also removes any remaining statefulness from those
classes that implement mach commands, ultimately making it easier to move
existing commands out of classes in a follow-up.

Differential Revision: https://phabricator.services.mozilla.com/D118058
2021-07-19 16:04:25 +00:00
Bob Owen b2d767fbb9 Bug 1716024 p2: Flag all binaries apart from firefox, xpcshell and plugin-container as CET compatible. r=glandium
We will only run the processes in CET compatible modules only mode when not
using the JIT code. So marking xul.dll as compatible should be OK.

Differential Revision: https://phabricator.services.mozilla.com/D117551
2021-07-19 07:36:19 +00:00
Butkovits Atila a07f790e42 Backed out changeset e1921c5112d8 (bug 1696251) for causing bustages complaining about 'CommandContext'. CLOSED TREE 2021-07-16 20:35:55 +03:00
Mitchell Hentges 09c7c7c9dc Bug 1720591: Remove unused patch_main() for Python < 3.4 on Windows r=ahal
Now that we use Python 3.6+, this code is obsolete.

Differential Revision: https://phabricator.services.mozilla.com/D119929
2021-07-16 16:34:42 +00:00
Alex Lopez 190e03aaab Bug 1696251 - Replace self with command_context where possible in existing mach commands. r=mhentges,webdriver-reviewers,perftest-reviewers,whimboo
This step removes all the dependencies of mach commands to
having a MachCommandBase as the `self` by using the `command_context`
argument instead. This also removes any remaining statefulness from those
classes that implement mach commands, ultimately making it easier to move
existing commands out of classes in a follow-up.

Differential Revision: https://phabricator.services.mozilla.com/D118058
2021-07-16 15:51:29 +00:00
Cosmin Sabou 3cd7630738 Backed out changeset 2c1257a95130 (bug 1720591) for causing build bustages complaining about patch_main. CLOSED TREE 2021-07-16 02:49:45 +03:00
Mitchell Hentges b5d3b00394 Bug 1720591: Remove unused patch_main() for Python < 3.4 on Windows r=ahal
Now that we use Python 3.6+, this code is obsolete.

Differential Revision: https://phabricator.services.mozilla.com/D119929
2021-07-15 21:44:37 +00:00
surajeet310 88365fbe52 Bug 1717651 - Removed instances of 'ensure_subprocess_env' from python/mozbuild r=mhentges
Since Python 2 is no longer used in python/mozbuild, hence 'ensure_subprocess_env' is obsolete.

Differential Revision: https://phabricator.services.mozilla.com/D119791
2021-07-14 15:17:27 +00:00
Mike Hommey 487119e349 Bug 1719229 - Allow the build system to use a sysroot for the host part of the build. r=firefox-build-system-reviewers,andi
Differential Revision: https://phabricator.services.mozilla.com/D119847
2021-07-14 08:54:43 +00:00
Jonathan Kew 0132f4c4df Bug 1720308 - Support VSCode installed as a snap package on Linux. r=firefox-build-system-reviewers,nalexander
Differential Revision: https://phabricator.services.mozilla.com/D119769
2021-07-13 15:04:06 +00:00
surajeet310 f2e1419d77 Bug 1717651 - Removed instances of 'ensure_subprocess_env' from python/mach r=mhentges
Since Python 2 is no longer used in python/mach, hence 'ensure_subprocess_env' is obsolete .

Differential Revision: https://phabricator.services.mozilla.com/D119185
2021-07-12 21:38:13 +00:00
Mitchell Hentges 1256637095 Bug 1720113: Temporarily add "python" to default clobber set r=firefox-build-system-reviewers,nalexander
Until virtualenvs are more dependably consistent, we should purge them
by default when "./mach clobber" is run.

Differential Revision: https://phabricator.services.mozilla.com/D119651
2021-07-12 16:43:09 +00:00
Florin Strugariu 2b3375937c Bug 1715478 - Automated recording for desktop with a list sites r=sparky
Differential Revision: https://phabricator.services.mozilla.com/D117298
2021-07-09 07:12:27 +00:00
june wilde 7db7c6efaf Bug 1716398 - support tracking tagged releases in mach vendor; r=tjr
Differential Revision: https://phabricator.services.mozilla.com/D118422
2021-07-08 18:20:04 +00:00
Mitchell Hentges f20cdb0c12 Bug 1712804: Check if `comm` directory is empty in TB identification r=ahal
Some developers hold onto a `comm` directory as a mount point
that they conditionally populate depending on whether they're working on
Firefox or Thunderbird.
The "`comm` directory exists == is a thunderbird checkout"
assumption isn't compatible with this workflow.

The fix embraced by this patch is to check if the `comm` directory has
any contents.

Differential Revision: https://phabricator.services.mozilla.com/D119193
2021-07-06 18:33:58 +00:00
Emilio Cobos Álvarez 780f3a05d2 Bug 1719144 - Fix various imports for Python 3.10. r=firefox-build-system-reviewers,mhentges
These are enough for me to run bootstrap+configure+build.

Some touch third-party code (gyp), but per discussion in the earlier
versions of this revision that seems fine.

Differential Revision: https://phabricator.services.mozilla.com/D119080
2021-07-06 15:26:29 +00:00
Mike Hommey 544436ed03 Bug 1718156 - Avoid some configure tests creating the common virtualenv. r=firefox-build-system-reviewers,mhentges,andi
When running `mach python-test` on a clobbered tree, we create both the
python-test and common virtualenvs, but the former should be enough. The
latter is only created when running tests that use the BaseConfigureTest
class, when it indirectly includes init.configure, which makes the
configure sandbox run the virtualenv_python3 function, which ends up
initializing the common virtualenv. We only don't end up re-executing
that virtualenv python thanks to 6680ca0acc27 which was a workaround
that would allow any virtualenv in $objdir/_virtualenvs, while still
creating a virtualenv it won't use.

Tests now tell virtualenv_python3 to not use the normal virtualenv
manager, and remove the workaround.

Differential Revision: https://phabricator.services.mozilla.com/D118775
2021-07-05 22:33:56 +00:00
Mitchell Hentges c25aec1f74 Bug 1717104: Update "pth" docs for virtualenvs r=ahal
The "pth" action no longer has a customizable filename, and the docs
should be updated accordingly.

Depends on D118609

Differential Revision: https://phabricator.services.mozilla.com/D118610
2021-07-05 22:10:35 +00:00
Mitchell Hentges 86ae5fbc84 Bug 1717104: Remove obsolete "set-variable" notes from virtualenv docs r=ahal
Since https://phabricator.services.mozilla.com/D115921, "set-variable"
is no longer a valid action. Looks like that patch forgot to remove the
associated docs

Differential Revision: https://phabricator.services.mozilla.com/D118609
2021-07-05 22:10:35 +00:00
surajeet310 498013ddc6 Bug 1717651 - Removed instances of 'ensure_subprocess_env()' from virtualenv.py r=mhentges
Since Python 2 is no longer used in virtualenv.py ,hence 'ensure_subprocess_env()' is obsolete.

Differential Revision: https://phabricator.services.mozilla.com/D118883
2021-07-05 19:59:53 +00:00
Sandor Molnar fe63cb08df Backed out 2 changesets (bug 1719144) for causing build bustages. CLOSED TREE
Backed out changeset 8e4b00b9d57c (bug 1719144)
Backed out changeset d2a88c5423a0 (bug 1719144)
2021-07-05 21:15:50 +03:00
Emilio Cobos Álvarez 52a85aaca1 Bug 1719144 - Fix various imports for Python 3.10. r=firefox-build-system-reviewers,mhentges
These are enough for me to run bootstrap+configure+build.

Some touch third-party code (gyp), but per discussion in the earlier
versions of this revision that seems fine.

Differential Revision: https://phabricator.services.mozilla.com/D119080
2021-07-05 17:55:41 +00:00
june wilde 68adbeeae5 Bug 1703066 - add tracking key to mozyaml file schema; r=tjr
Differential Revision: https://phabricator.services.mozilla.com/D110829
2021-06-29 14:01:58 +00:00
Mike Hommey 81228f00cd Bug 1717947 - Remove packager support for brotli in Jars. r=firefox-build-system-reviewers,nalexander
This was added ing bug 1352595. It has only been used briefly, and we
since then have gone all the way to never compress omni.ja. Even if we
did go back to compressing, we'd probably go with zstd rather than
brotli.
The gecko-side support for this has only ever been nightly-only.

Differential Revision: https://phabricator.services.mozilla.com/D118655
2021-06-25 00:20:10 +00:00
Mitchell Hentges 43949e4a1c Bug 1717702: Resolve `print-checks` python unpacking typo r=andi
`_get_infer_config()` returns 3 items, but the existing code expects 2.
Add a third `_` to capture the other unwanted item.

Differential Revision: https://phabricator.services.mozilla.com/D118535
2021-06-24 13:21:57 +00:00
Marian-Vasile Laza 4728ef73b2 Bug 1700534 - black lint fix r=fix CLOSED TREE 2021-06-22 09:22:33 +03:00
Mike Hommey 744db845c6 Bug 1700534 - Coalesce RLBox wasmboxed libraries. r=firefox-build-system-reviewers,shravanrn,bholley,andi,mhentges
Differential Revision: https://phabricator.services.mozilla.com/D116440
2021-06-22 05:31:33 +00:00
Mitchell Hentges ee5a948268 Bug 1716600: Don't use node/npm that are too old or have no version r=firefox-build-system-reviewers,nalexander
In the existing behaviour, we would return a node/npm executable from
`find_executable()`, even if we couldn't extract a version from it or if
the extracted version was insufficient.

Now, we fail early if the detected node/npm is insufficient.

Differential Revision: https://phabricator.services.mozilla.com/D118377
2021-06-21 16:47:56 +00:00
Mitchell Hentges 00d96c3fed Bug 1716600: Remove looping from Node's find_executable() r=firefox-build-system-reviewers,nalexander
Since we're no longer checking for both "nodejs" and "node",
`find_executable()` can be simplified.

Differential Revision: https://phabricator.services.mozilla.com/D118376
2021-06-21 16:47:55 +00:00
Mitchell Hentges 73814a7f0f Bug 1716600: Always search for "node" instead of "nodejs" r=firefox-build-system-reviewers,nalexander
As :julienw beautifully described on Bugzilla,
we no longer need to search for "nodejs" since all
packages providing Node >= v12 will provide the `node`
binary.
See his comment here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1716600#c4

Base-toolchain tasks with Node v10 should still be
happy because, unlike Debian's Node v10, they provide
a `node` binary instead of `nodejs`.

Differential Revision: https://phabricator.services.mozilla.com/D118375
2021-06-21 16:47:55 +00:00
Mitchell Hentges 2438ea8889 Bug 1712133: Validate that pypi dependencies are installed r=ahal
Check that all expected pypi packages are installed
when checking if a virtualenv is up-to-date.

Differential Revision: https://phabricator.services.mozilla.com/D117712
2021-06-17 14:58:17 +00:00
Mitchell Hentges 2fa59acd4f Bug 1712133: Adds "pypi" action to virtualenv `handle_package()` r=ahal
The `pypi` action uses `pip` to fetch a package and its dependencies

Differential Revision: https://phabricator.services.mozilla.com/D115925
2021-06-17 14:58:17 +00:00
Mitchell Hentges 24f634d924 Bug 1712133: Remove "pth" name customization r=ahal
Having separate `<name>.pth` files in the virtual environments
isn't providing an advantage. We can simplify configuration
by putting all `pth` adjustments into a single file: `mach.pth`.

Differential Revision: https://phabricator.services.mozilla.com/D117710
2021-06-17 14:58:16 +00:00
Mitchell Hentges 31474e8925 Bug 1712133: Make virtualenv package parsing more specific r=ahal
This has two benefits:
1. `handle_package()` becomes more clear - rather than referring to
   `action` and `package` with array index numbers, we now give
   them real names. The benefit here is also shown in `up_to_date()`.
2. This makes the top-level parser for `packages()` less opinionated
   about sub-formats: if an action has a nested structure, it should
   have the flexibility to define what it looks like.

Differential Revision: https://phabricator.services.mozilla.com/D117708
2021-06-17 14:58:16 +00:00
Mitchell Hentges 722ac646db Bug 1713857: Don't set PYTHONEXECUTABLE environment variable r=glandium
After some testing in `try` and locally, the manual
`PYTHONEXECUTABLE` definitions shouldn't be needed
anymore.

There's been some work on Brew's
Python to improve its `sitecustomize` behaviour.
The most likely improvement is:
https://github.com/Homebrew/homebrew-core/pull/65297

However, I'm not fully confident in this change.
If it fails, it's more likely to affect developers
than CI. I think it's worth attempting a landing,
because if the variable is indeed obsolete, then
deleting it will avoid some spicy "action-at-a-
distance" behaviour.

Differential Revision: https://phabricator.services.mozilla.com/D117452
2021-06-17 13:28:29 +00:00
Marian-Vasile Laza 19f23a7e8b Bug 1716911 - black lint fix r=fix CLOSED TREE 2021-06-17 09:39:54 +03:00
Mike Hommey 83c4b6a318 Bug 1716911 - Switch --enable-bootstrap on by default on macos builds on central. r=firefox-build-system-reviewers,nalexander
Differential Revision: https://phabricator.services.mozilla.com/D118107
2021-06-17 04:59:42 +00:00
Mitchell Hentges 6d154c1ed3 Bug 1713377: Change vendoring to use wheels where possible r=ahal,glandium
Vendoring wheels has three benefits:
* There's far less files, so Firefox checkouts will be smaller.
* It works around `zipp` not allowing `pip install`
  from extracted source `tar.gz` files. Now, we should
  be able to use the pip resolver against vendored
  packages, which will be needed for future
  mach virtualenv work.
* `./mach vendor python` takes far less time to execute.

Since we need the raw Python to be available to add to the `sys.path`,
we extract the wheels before putting them in tree.
Due to the structure of some wheels being less nested
than of a source `tar.gz`, `common_virtualenv_packages`
needed to be adjusted accordingly.

`install_pip_package()` had to be tweaked as well since you can't
`pip install` an extracted wheel. So, we "re-bundle" the wheel
before installing from a vendored package.

Replace python packages with wheels where possible

This contains the vendoring changes caused by the
last patch.

For reviewing, there's a couple things to note:
* A bunch of files are deleted, since there's generally
  less files in a wheel than in a source archive.
* There's a new `.dist-info` directory for each
  extracted wheel, so expect roughly 5 or
  6 new files for each wheel'd package.
* There should be no source code changes other than
  moves from package names changing from having
  `-` to having `_`.

Differential Revision: https://phabricator.services.mozilla.com/D116512
2021-06-16 15:53:16 +00:00
surajeet310 c3511254e0 Bug 1714688 - Removed 'if PY2'/'if PY3' logic from python/mozbuild r=mhentges
Python2 is no longer used in Mozbuild.

Differential Revision: https://phabricator.services.mozilla.com/D117561
2021-06-15 19:31:21 +00:00
Mitchell Hentges 9f7c330b60 Bug 1712133: Remove "windows" action support from virtualenvs r=ahal
Last year, we stopped vendoring Python packages that have native
code. Since we have only had pure-python packages since, the
Windows-specific qualifier (or excluder in the case of `!windows`)
hasn't been needed.

I don't foresee us needing it again, but if anything we can peel it
back from `hg` history if this assumption is incorrect.

Differential Revision: https://phabricator.services.mozilla.com/D117468
2021-06-15 14:46:55 +00:00
Mitchell Hentges 1b74e5a575 Bug 1713201: Resolve test_telemetry failing locally r=ahal
On Windows, Python's temporary directories are placed in `%APPDATA%`.
Unfortunately, the failing tests want to assume that some "other"
path (as a subpath of a tempdir) won't be within any
`filter_args()` known top-level paths, which includes the user's home
dir.

The only reason that this assumption has passed in CI so far is due
to Windows case insensitivity: running locally had `~` evaluated
to `c:\Users\...`, while running in CI had it be `c:\users\...`.
Since the tempdir path started with `C:\Users`, `filter_args()`
would `<path omitted>` in CI but would `$HOME/...` locally.

By not basing the `other_path` on `tempdir`, we now consistently get the
same results across platforms.

Differential Revision: https://phabricator.services.mozilla.com/D117698
2021-06-15 14:45:53 +00:00
Alexandru Michis 542a65a629 Bug 1715282 - Fix black py lint in test_toolchain_configure.py a=lint-fix
CLOSED TREE
2021-06-15 18:04:43 +03:00
Mike Hommey 7f9a51a8f5 Bug 1715282 - Bump minimum Rust version to 1.51. r=firefox-build-system-reviewers,andi
Also update the estimated rust versions for future releases, based on
the updated Firefox release dates.

Differential Revision: https://phabricator.services.mozilla.com/D117737
2021-06-15 09:23:20 +00:00
Csoregi Natalia 6b0106d5b1 Backed out changeset 15c6a16f5828 (bug 1714688) for docker images bustage. CLOSED TREE 2021-06-14 20:51:29 +03:00
surajeet310 0ac7b98b51 Bug 1714688 - Removed 'if PY2'/'if PY3' logic from python/mozbuild r=mhentges
Python2 is no longer used in Mozbuild.

Differential Revision: https://phabricator.services.mozilla.com/D117561
2021-06-14 16:10:11 +00:00
Mitchell Hentges e389cc4cc4 Bug 1714684: Remove usages of vendored "mock" library r=perftest-reviewers,ahal,sparky
Python 3 has access to `unittest.mock` in the standard library.

Differential Revision: https://phabricator.services.mozilla.com/D117073
2021-06-14 15:34:46 +00:00
Mitchell Hentges dc2fa7819c Bug 1713610: Require PyPI-vendored packages be added to requirements.in r=ahal
`./mach vendor python <package>` was already adding its new package
to `requirements.txt`, so we were getting full resolver support, which
is good. However, it caused our `requirements.in` file to start
getting out-of-date, and therefore made it harder to identify the
top-level dependencies.

Arguably, we could have `./mach vendor python <package>` automatically
update `requirements.in`, too, but then we need to solve the edge cases,
such as "What if the package is already in `requirements.in`? What if
that existing item has a different version?"

The hardest part of updating `requirements.in` was finding it, so I've
also modified the `./mach vendor python` help text to make it more
identifiable.

Differential Revision: https://phabricator.services.mozilla.com/D116386
2021-06-10 19:19:06 +00:00
Nick Alexander bef7042147 Bug 1524662 - Add --enable-path-remapping producing compiled objects with generic paths. r=glandium
For "c" (i.e., gcc, clang, and clang-cl), this configures the
`-f{debug,macro}-path-prefix` flags.  We'd prefer to use
`-ffile-path-prefix`, but it seems that `clang-cl` does not recognize
that flag.

For "rust" (i.e., rustc/cargo), this configures `--remap-path-prefix`.

This is one step toward getting `sccache` hits across source and
object directories.

Differential Revision: https://phabricator.services.mozilla.com/D113065
2021-06-10 17:08:06 +00:00
Mitchell Hentges b9531f2856 Bug 1714641: Remove usages of vendored "pathlib2" library r=ahal
Based on the docs and the code within `pathlib2`, it seems to focus
only on backporting the main features of `pathlib` to be available for
Python 2. It does _not_ provide features in newer Python versions
of `pathlib` (such as `is_mount`, new in Python 3.7). Therefore, it
doesn't provide anything that the standard library of Python 3.6+
doesn't have.

Differential Revision: https://phabricator.services.mozilla.com/D117071
2021-06-09 15:48:55 +00:00
Mitchell Hentges 70806e17b5 Bug 1714641: Remove usages of vendored "wptserve_py2" library r=jgraham,ahal
We should only be using the Python3 "wptserve" library at this point.

Differential Revision: https://phabricator.services.mozilla.com/D117062
2021-06-09 15:48:51 +00:00
Mitchell Hentges eca9f5e421 Bug 1714641: Remove `pythonX` actions from virtualenv config r=ahal
The actions are no longer used, so we can delete the associated
logic.

Differential Revision: https://phabricator.services.mozilla.com/D117060
2021-06-09 15:48:51 +00:00
Paul Adenot 36dc6db808 Bug 1714577 - Part 1 - Include the Gecko Profiler include directory in WebRTC.org files. r=ng
Differential Revision: https://phabricator.services.mozilla.com/D116838
2021-06-09 15:43:37 +00:00
Emilio Cobos Álvarez 2c99123249 Bug 1715493 - Update callers of should_replace_task to pass a missing deadline argument. r=ahal a=bustage-fix
Differential Revision: https://phabricator.services.mozilla.com/D117280
2021-06-09 10:58:31 +00:00
Andrey Bienkowski 888618300d Bug 1714376 - Replace a number of "exception.message" usages. r=mhentges,jgraham
Differential Revision: https://phabricator.services.mozilla.com/D116723
2021-06-08 15:50:10 +00:00
Connor Sheehan 632d552e70 Bug 1702017: add a mach doctor check for `invalid locale: UTF-8` r=mhentges
Adds a `mach doctor` check to assert Python's locale is not set to
`UTF-8`, an invalid value due to a Python bug in most pre-3.8 versions.

Differential Revision: https://phabricator.services.mozilla.com/D117081
2021-06-08 15:24:00 +00:00
Connor Sheehan ff302e46aa Bug 1713988: add a check for ssh access to `hg.mozilla.org` to mach doctor r=mhentges
Adds a check that SSH access to hg.mozilla.org is properly configured.
Attempt to run `ssh hg.mozilla.org` which will authorize to hgmo over
SSH via `pash`, without running any commands on the service. In the case
the user has properly configured SSH we will see a success message and
information on the SCM levels granted to the user. If we don't see
a successful connection, parse the output to ensure the user is using
an email address as their username (which is required on hg.mozilla.org).
If the user couldn't connect yet does have an email address as their
username, notify them that they do not have permission to push to hgmo.

Differential Revision: https://phabricator.services.mozilla.com/D117080
2021-06-08 15:23:59 +00:00
Connor Sheehan dc08a2c897 Bug 1713988: add a check that `mozilla.org` is reachable over https r=mhentges
Adds a simple check that a Mozilla domain is reachable over HTTPS.

Differential Revision: https://phabricator.services.mozilla.com/D117079
2021-06-08 15:04:06 +00:00
Connor Sheehan 9f03fdf3e4 Bug 1713988: add a `mach doctor` check for a DNS query r=mhentges
Adds a simple DNS query to `mozilla.org` as a doctor check.
We use `socket.getaddrinfo` to resolve the hostname to IP.

Differential Revision: https://phabricator.services.mozilla.com/D117078
2021-06-08 15:04:06 +00:00
Steve Fink 084cf95676 Bug 1714680 - Implement `mach gtest --list-tests` r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D116904
2021-06-07 20:55:05 +00:00
Mitchell Hentges c2d548efc9 Bug 1712819: Remove redundant pip warning suppression r=ahal
Now that we configure pip within mach virtualenvs to turn off
its "outdated" warning, we no longer need to apply the suppression
as a CLI argument.

Differential Revision: https://phabricator.services.mozilla.com/D116891
2021-06-07 14:46:09 +00:00
Agi Sferro 65b64cb3e0 Bug 1709640 - Rename android-api-16 to android-arm. r=marionette-reviewers,firefox-build-system-reviewers,nalexander,whimboo,aklotz,releng-reviewers,mhentges,glandium,mtabara
We don't offer API splits any more, and with the separation of GeckoView with
the rest of the front-end it's increasingly unlikely that we will in the
future.

This change makes it so that the build name doesn't contain the API version so
that we can update it without breaking all the automation that relies on the
build name.

Differential Revision: https://phabricator.services.mozilla.com/D114369
2021-06-04 22:31:14 +00:00
Sandor Molnar 2a126fe036 Backed out changeset 453cb34e4a4a (bug 1709640) for causing linting failures in perfdocs/config. CLOSED TREE 2021-06-05 01:00:19 +03:00
Agi Sferro ecf4f5f0ad Bug 1709640 - Rename android-api-16 to android-arm. r=marionette-reviewers,firefox-build-system-reviewers,nalexander,whimboo,aklotz,releng-reviewers,mhentges,glandium,mtabara
We don't offer API splits any more, and with the separation of GeckoView with
the rest of the front-end it's increasingly unlikely that we will in the
future.

This change makes it so that the build name doesn't contain the API version so
that we can update it without breaking all the automation that relies on the
build name.

Differential Revision: https://phabricator.services.mozilla.com/D114369
2021-06-04 21:20:56 +00:00
Mitchell Hentges 6ac951e5aa Bug 1713613: Assert unmodified PyPI-vendored packages r=ahal
Adds test to ensure that we don't modify files that are vendored
with PyPI.

If a modification needs to happen to a vendored library, we should
instead:
* Modify the library upstream and vendor the new version, or
* Remove the library from requirements.in and requirements.txt so that
  it's now a "manually-vendored" library. Note that there many be
  issues if this occurs to a library depended on by 3rd-party
  libraries.

Differential Revision: https://phabricator.services.mozilla.com/D116428
2021-06-04 21:19:35 +00:00
Mitchell Hentges b8c530e2f1 Bug 1713613: Require Python 3.6 to update vendored packages r=ahal
Generating Python lockfiles is system-dependent, and this currently
mostly affects us just on the Python 3.6 <=> 3.7 boundary - 3.6 requires
a few additional packages like `importlib_metadata` and `iso8601`.

If a different Python version is used, the test in CI is guaranteed to
fail. By validating in advance, we help future vendoring developers
by helping them avoid the wait before CI failure and the time it
takes to troubleshoot.

Differential Revision: https://phabricator.services.mozilla.com/D116773
2021-06-04 21:19:35 +00:00
Dorel Luca f021feb29f Backed out changeset 68f73772f51e (bug 1714376) for Xpcom failures in builds/worker/checkouts/gecko/xpcom/idl-parser/xpidl/runtests.py. CLOSED TREE 2021-06-04 02:19:33 +03:00
Connor Sheehan 34cac10551 Bug 1713988: rewrite `mach doctor` to be function based, simplify some things r=mhentges
Rewrite `doctor.py` from an object-oriented paradigm to a simpler
function-based architecture. Each check is a function that returns
or yields `DoctorCheck`s. `DoctorCheck`s contain a status that describes
the status of the check (success, warning, fatal error etc), a string
of text with useful information for display to the terminal, and may
optionally attach a callable which can be executed to fix issues found
by the checks.

`mach doctor` is updated to use the new `run_doctor` function instead
of creating a `Doctor` object. The default value of `verbose` in `run_doctor`
is `False`, meaning only issues with the users system are printed to the
screen.

Most of the existing check logic is left intact from the previous
implementation. Checks for `platform` are changed to use the
`sys.platform.startswith` pattern. The latest MozillaBuild version is
pulled by querying the `tip` commit on hg.mozilla.org instead of using
a hard-coded variable.

Differential Revision: https://phabricator.services.mozilla.com/D116539
2021-06-03 22:11:10 +00:00
Mike Hommey d6e973ee86 Bug 1714244 - Use HEAD instead of master by default for `mach vendor yaml` r=releng-reviewers,bhearsum
HEAD is a symbolic ref that points to the default branch of the
repository, it will point to master, main, or anything else that the
repository is using as its default branch.

Differential Revision: https://phabricator.services.mozilla.com/D116678
2021-06-03 20:45:05 +00:00
Andrey Bienkowski a7a4542e20 Bug 1714376 - Replace a number of "exception.message" usages. r=mhentges,jgraham
Differential Revision: https://phabricator.services.mozilla.com/D116723
2021-06-03 20:08:51 +00:00
Alexandru Michis fd44fe5ee5 Bug 1714156 - Fix lint failures in test_manifest.py
CLOSED TREE
2021-06-02 21:25:07 +03:00
june wilde bece79b87e Bug 1714156 - Disallow 0 weeks in frequency in moz.yaml schema; r=tjr
Differential Revision: https://phabricator.services.mozilla.com/D116626
2021-06-02 18:05:49 +00:00
Mike Hommey 2733dbe0dc Bug 1713733 - Remove WASM_LDFLAGS. r=firefox-build-system-reviewers,mhentges
It's not practically make a difference (the flags we end up setting it
to don't do anything when linking all the wasm files into one).

Differential Revision: https://phabricator.services.mozilla.com/D116437
2021-06-02 04:34:20 +00:00
Mitchell Hentges 06a214573a Bug 1712819: Avoid pip's "outdated" warning in virtualenvs r=ahal
Now, when running mach commands that invoke `pip`, it will no longer
inform the user that it needs an update.

We reach into `distutils` to determine the "site-packages" directory
pattern, then apply it to our virtualenv.

Differential Revision: https://phabricator.services.mozilla.com/D115940
2021-05-27 23:04:06 +00:00
Mitchell Hentges 79c56399e1 Bug 1712819: Fix VirtualenvManager not being expandable in debugger r=ahal
At least in PyCharm, expanding a `VirtualenvManager` instance means
resolving all the properties and fields of the instance.
However, if that property is doing non-trivial work, the debugger
wouldn't run that subprocess while we're stopped at a breakpoint.
So, the instance would sit there with the "Collecting data..." text.

Differential Revision: https://phabricator.services.mozilla.com/D115935
2021-05-27 23:04:06 +00:00
Mike Hommey 95e6d3cd22 Bug 1712350 - Stop guessing where make is in python code. r=firefox-build-system-reviewers,mhentges
This was necessary before because we first started the build by invoking
make with client.mk, which would then invoke configure. But that changed
in bug 1671424, and we now never need make before configure has run
(except in tests, for which we implement a fallback).

Differential Revision: https://phabricator.services.mozilla.com/D116064
2021-05-27 22:27:25 +00:00
Mitchell Hentges c329fa104b Bug 1713060: Don't override "substs" to fetch it fallibly r=ahal
A clobber must be able to run, even if `substs` (configure output)
is not available.
However, when this behaviour was implemented, it was set up to replace
all usages.

Since we're trying to make mach commands no longer objects, this
kind of overriding won't be possible in the future. Fortunately, it's
not needed: only `Clobberer` needs a fallible substs, the other
consumers don't!

Differential Revision: https://phabricator.services.mozilla.com/D116128
2021-05-27 21:00:42 +00:00
Mitchell Hentges 78fcd81e02 Bug 1712382: `./mach clobber gradle` works if directory already gone r=nalexander
Back-to-back `./mach clobber gradle` invocations were failing
because `shutil.rmtree(..)` would complain that the target directory
didn't exist.

Differential Revision: https://phabricator.services.mozilla.com/D116129
2021-05-27 18:04:32 +00:00
Mitchell Hentges de10b8114b Bug 1712382: Includes virtualenvs when clobbering Python r=glandium
No longer includes `python` in the `./mach clobber` defaults since
python cache files should no longer be affecting `./mach artifact`.

Removes `third_party/python` purging from `./mach clobber python`
since we don't build native Python modules there.

Moves virtualenv-purging from `./mach clobber objdir` to
`./mach clobber python`.

Differential Revision: https://phabricator.services.mozilla.com/D115728
2021-05-27 18:04:32 +00:00
Dorel Luca 61aa3325c5 Backed out 2 changesets (bug 1712819) for Python failures. CLOSED TREE
Backed out changeset f51d72e5f0b8 (bug 1712819)
Backed out changeset 40c109bb5f9a (bug 1712819)
2021-05-27 19:05:15 +03:00
Mitchell Hentges 20d4059a6f Bug 1712133: Simplify virtualenv "sitecustomize" writing r=ahal
Child `handle_package(...)` invocations don't need to modify
`sitecustomize.py`, so don't pass it to them.

Differential Revision: https://phabricator.services.mozilla.com/D115924
2021-05-27 15:18:37 +00:00
Mitchell Hentges 17f68a38c4 Bug 1712133: Use if/else chain instead of early-return in handle_package r=ahal
Accidentally missing a `return` in a code path could mean that
`handle_package(...)` would accidentally do an action _and_
raise the "Unknown action" error.

This change resolves that, and it simplifies the code a bit.

Differential Revision: https://phabricator.services.mozilla.com/D115923
2021-05-27 15:18:37 +00:00
Mitchell Hentges 0d1d6ed813 Bug 1712804: Change comm action to be specific to Thunderbird r=rjl,ahal
Avoid the "Error processing command" warning when building virtualenvs
in a Firefox checkout

Differential Revision: https://phabricator.services.mozilla.com/D115922
2021-05-27 15:18:37 +00:00
Mitchell Hentges 9522e92323 Bug 1712382: Remove "set-variable" action support from virtualenvs r=ahal
`MACH_VIRTUALENV` was never used, and `MOZBUILD_VIRTUALENV` was never
set (the virtualenv was always activated during the build, rather
than before).

Differential Revision: https://phabricator.services.mozilla.com/D115921
2021-05-27 15:18:36 +00:00
Mitchell Hentges 04a6f9736c Bug 1712382: Remove "setup.py" action support from virtualenvs r=ahal
The `setup.py` action is no longer used, so we can safely remove it.

Depends on D115641

Differential Revision: https://phabricator.services.mozilla.com/D115913
2021-05-27 15:18:36 +00:00
Mitchell Hentges 5eb07cd570 Bug 1712133: Remove build VIRTUALENV_NAME customization r=glandium
This was originally set up so that tests wouldn't "create a new
`virtualenv` for no reason." However, virtual environments now will have
different packages installed, and therefore the separation is necessary.

So, for the virtual environment used for builds (regular or for tests):
* We want it to be able to reuse the build venv, if it already exists.
* We don't want to pollute a `pytest` virtualenv with build-specific
  packages.

Differential Revision: https://phabricator.services.mozilla.com/D115641
2021-05-27 15:18:35 +00:00
Mitchell Hentges 846e25c93b Bug 1712133: Rename default virtualenv to "common" r=ahal
We had split up `init` from `init_py3` because `mach` had
traditionally been invoked by either Python 2 or Python 3, and
the two couldn't share the same virtualenv.

Now that the same context isn't shared by both Python 2 and 3
3
(developers always use Python 3, and the remaining Python 2
usages are CI jobs that never reuse the objdir with Python 3),
We can centralize on a single default virtualenv.

I've called this "common" instead of "init" to clarify its
existing position as the virtualenv that's used by many different
commands. As we associate virtualenvs with requirement definitions,
it'll also make the file less confusing: it's a "common" requirement
definition as opposed to an "init" one.

Differential Revision: https://phabricator.services.mozilla.com/D115635
2021-05-27 15:18:35 +00:00
Mitchell Hentges d41a0b2774 Bug 1712819: Avoid pip's "outdated" warning in virtualenvs r=ahal
Now, when running mach commands that invoke `pip`, it will no longer
inform the user that it needs an update.

We reach into `distutils` to determine the "site-packages" directory
pattern, then apply it to our virtualenv.

Differential Revision: https://phabricator.services.mozilla.com/D115940
2021-05-27 15:16:16 +00:00
Mitchell Hentges f8ce33772c Bug 1712819: Fix VirtualenvManager not being expandable in debugger r=ahal
At least in PyCharm, expanding a `VirtualenvManager` instance means
resolving all the properties and fields of the instance.
However, if that property is doing non-trivial work, the debugger
wouldn't run that subprocess while we're stopped at a breakpoint.
So, the instance would sit there with the "Collecting data..." text.

Differential Revision: https://phabricator.services.mozilla.com/D115935
2021-05-27 15:16:16 +00:00
Tom Ritter 2b1afa9921 Bug 1712815 - Fix the 'remove a file from a moz.build file' logic r=jewilde
We were raising an exception as a result of a python error
which was masked by a catch statement that thought we didn't
know how to remove the file. But we did.

Fix the python error, and add an exception at the end of the
function to be raised and then caught when we really don't know
how to remove a file.

Differential Revision: https://phabricator.services.mozilla.com/D115936
2021-05-27 14:19:25 +00:00
Tom Ritter 8bcc518a54 Bug 1712953 - Add a frequency attribute to Updatebot tasks r=jewilde
Differential Revision: https://phabricator.services.mozilla.com/D116020
2021-05-26 20:36:13 +00:00
Alex Lopez 07cf832a8a Bug 1696251 - Turn all properties in MachCommandBase subclasses into methods. r=mhentges,perftest-reviewers,sparky
As a step towards moving mach commands outside of classes, this converts all
properties into methods so that they can later become top-level helper functions.

Differential Revision: https://phabricator.services.mozilla.com/D112196
2021-05-26 18:08:02 +00:00
Mike Hommey c3d194a44b Bug 1712023 - Avoid displaying truncated symbols in check_binary_compat. r=firefox-build-system-reviewers,mhentges
Differential Revision: https://phabricator.services.mozilla.com/D115561
2021-05-20 22:32:10 +00:00
Alex Lopez 455d9a088b Bug 1696251 - Pass MachCommandBase object as first argument for Mach Commands. r=mhentges,remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).

Differential Revision: https://phabricator.services.mozilla.com/D109650
2021-05-17 16:15:58 +00:00
Mike Hommey d9c58f174d Bug 1707829 - Go back to building http3server as a rust program. r=necko-reviewers,kershaw
This mostly revert bug 1616238.

Differential Revision: https://phabricator.services.mozilla.com/D113485
2021-05-07 20:22:56 +00:00
Sandor Molnar 93e6afe93a Backed out changeset bd2c9d3fa4b0 (bug 1707829) for causing xpc failures. CLOSED TREE 2021-05-07 01:58:43 +03:00
Mike Hommey eae232636b Bug 1707829 - Go back to building http3server as a rust program. r=necko-reviewers,kershaw
This mostly revert bug 1616238.

Differential Revision: https://phabricator.services.mozilla.com/D113485
2021-05-06 21:37:01 +00:00
Andi-Bogdan Postelnicu d6010d98f2 Bug 1702172 - collect if shell is opened via vscode and if it's a remote connection through ssh. r=firefox-build-system-reviewers,mhentges
Added two fields:

1. Collect if the current shell is opened via vscode.
2. Connect if it's a remote ssh connection n the current shell.

Differential Revision: https://phabricator.services.mozilla.com/D114323
2021-05-05 14:56:32 +00:00
Narcis Beleuzu 6371054f62 Bug 1709401 - Fix lint failure a=lint-fix . CLOSED TREE 2021-05-04 20:59:02 +03:00
Tom Ritter b5376d204b Bug 1709401 - Shim the python 3.8 API get_source_segment for ./mach vendor r=jewilde
Differential Revision: https://phabricator.services.mozilla.com/D114254
2021-05-04 17:33:17 +00:00
Mike Hommey f2dcdd5390 Bug 1708592 - Get tools from MOZ_FETCHES_DIR on automation. r=firefox-build-system-reviewers,mhentges
While here, remove the unused clippyProcess.config. And because we need
to manipulate `$PATH` before running `cargo`, use the same wrapping
function in `get_clippy_version`.

Differential Revision: https://phabricator.services.mozilla.com/D113902
2021-04-30 21:09:11 +00:00
Mitchell Hentges f6e819dc08 Bug 1708005: Remove `target_dir` from `RustLibrary` r=firefox-build-system-reviewers,glandium
The `target_dir` is always the objdir, since we try to share compile
output. Therefore, storing it in `RustLibrary` shouldn't be necessary.

Removes the associated `RUST_LIBRARY_TARGET_DIR` moz.build option
as well.

Differential Revision: https://phabricator.services.mozilla.com/D113565
2021-04-29 15:19:09 +00:00
Butkovits Atila 2e34e363b9 Backed out 2 changesets (bug 1696251) for causing js-bench-sm failures. CLOSED TREE
Backed out changeset 1c84c9a34575 (bug 1696251)
Backed out changeset e169193b7423 (bug 1696251)
2021-04-23 02:53:36 +03:00
Alex Lopez 41687360e8 Bug 1696251 - Turn all properties in MachCommandBase subclasses into methods. r=mhentges,perftest-reviewers,sparky
As a step towards moving mach commands outside of classes, this converts all
properties into methods so that they can later become top-level helper functions.

Differential Revision: https://phabricator.services.mozilla.com/D112196
2021-04-22 19:44:52 +00:00
Alex Lopez d1a82b8092 Bug 1696251 - Pass MachCommandBase object as first argument for Mach Commands. r=mhentges,remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).

Differential Revision: https://phabricator.services.mozilla.com/D109650
2021-04-22 18:56:15 +00:00
Steve Fink 768eac28f5 Bug 1706197 - Avoid Python version dependent line number issues r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D112760
2021-04-22 14:30:49 +00:00
Kershaw Chang f9a2c05f44 Bug 1616238 - Enable http3 tests on windows, r=necko-reviewers,valentin
Differential Revision: https://phabricator.services.mozilla.com/D112528
2021-04-22 11:53:32 +00:00
Mitchell Hentges cb92d64d4d Bug 1507272: Enable build telemetry for Mozilla devs by default r=firefox-build-system-reviewers,glandium
Check if a user is a Mozilla employee by checking their
Bugzilla groups, or checking if their VCS email ends
with "@mozilla.com".

When a user is setting up a new build environment, telemetry
will be automatically enabled for them if they are an
employee. If they're not an employee, they'll be asked
if they want to opt in.

Differential Revision: https://phabricator.services.mozilla.com/D106315
2021-04-20 13:58:51 +00:00
imoraru 44819cf83e Backed out changeset f2203cb8ec98 (bug 1616238) for build bustages on Android. CLOSED TREE 2021-04-20 09:51:19 +03:00
Kershaw Chang 9e4dc813a7 Bug 1616238 - Enable http3 tests on windows, r=necko-reviewers,valentin
Differential Revision: https://phabricator.services.mozilla.com/D112528
2021-04-19 18:34:05 +00:00
Cosmin Sabou 785f9b8a87 Backed out changeset d4a5d8567977 (bug 1696251) for non-unified build bustages. CLOSED TREE 2021-04-19 19:43:31 +03:00
Alex Lopez 75dfe35468 Bug 1696251 - Pass MachCommandBase object as first argument for Mach Commands. r=mhentges,remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).

Differential Revision: https://phabricator.services.mozilla.com/D109650
2021-04-19 16:15:11 +00:00
Steve Fink 06fa1e33cc Bug 1697929 - Make mach work in the spidermonkey release package r=firefox-build-system-reviewers,glandium
Differential Revision: https://phabricator.services.mozilla.com/D108129
2021-04-19 14:21:43 +00:00
Noemi Erli 40d417bd3c Bug 1704126 - Fix lint failure r=fix CLOSED TREE 2021-04-16 04:20:52 +03:00
Mike Hommey f2dafefef4 Bug 1704126 - Make mach create-mach-environment fail when running under Rosetta. r=firefox-build-system-reviewers,mhentges
Differential Revision: https://phabricator.services.mozilla.com/D112151
2021-04-16 01:03:56 +00:00
Mike Hommey 3b212b0272 Bug 1705296 - Don't create a python2 mach virtualenv. r=firefox-build-system-reviewers,mhentges
No mach command uses python2 anymore, so we don't need to create a
virtualenv for python2.

Differential Revision: https://phabricator.services.mozilla.com/D112148
2021-04-16 01:00:09 +00:00
Claudia 69e599d847 Bug 1651012 - Organize certerror code into its own directory in browser/base/content/ r=prathiksha,Gijs
Differential Revision: https://phabricator.services.mozilla.com/D110951
2021-04-14 10:24:38 +00:00
Alexandru Michis 9b58a1c6f5 Backed out changeset 8a21200047bb (bug 1651012) for causing bc failures in browser_parsable_script.js
CLOSED TREE
2021-04-14 01:52:26 +03:00
Claudia d6b7c6e76b Bug 1651012 - Organize certerror code into its own directory in browser/base/content/ r=prathiksha,Gijs
Differential Revision: https://phabricator.services.mozilla.com/D110951
2021-04-13 18:54:28 +00:00
Nick Alexander faad7e9113 Bug 1391106 - Support `mach watch` for non-artifact builds. r=firefox-build-system-reviewers,mhentges
I manually compared `$TOPOBJDIR/faster/unified_install_dist_bin` for
an artifact and a non-artifact build, and found only a few
pre-processing differences.  I can't recall why this wasn't supported
initially; I think it used to be that manifests overlapped and that
the interaction with the RecursiveMake backend was more complicated.
But I see no reason why this shouldn't work, and would like to get
some folks to test it in the wild.

Differential Revision: https://phabricator.services.mozilla.com/D110311
2021-04-13 16:02:21 +00:00
Mike Hommey 9bce63da4f Bug 1704685 - Remove leftover from bug 1686888. r=firefox-build-system-reviewers,nalexander
Differential Revision: https://phabricator.services.mozilla.com/D111765
2021-04-13 04:57:48 +00:00
Kagami Sascha Rosylight d9f6cded53 Bug 1704375 - Close tar file after use in vendor_manifest.py r=firefox-build-system-reviewers,nalexander
Differential Revision: https://phabricator.services.mozilla.com/D111566
2021-04-12 03:14:06 +00:00
Nick Alexander 29fc399d04 Bug 1703886 - Let `MOZ_BACKGROUNDTASKS` and `MOZ_UPDATE_AGENT` ride the trains. r=application-update-reviewers,firefox-build-system-reviewers,mhentges,bytesized
This also makes `MOZ_UPDATE_AGENT` depend on `MOZ_BACKGROUNDTASKS`.

There's no need to lint the `tools/update-programs` project, since
it's not a supported configuration.  The next consumer can
rehabilitate it or remove it.

Differential Revision: https://phabricator.services.mozilla.com/D111313
2021-04-11 20:50:15 +00:00
Chris Fallin e888f83b57 Bug 1701603 part 1 - Support WASI as a new target for compilation. r=glandium
Add new OS - WASI and new processor - wasm32 to the SM's build system.

Differential Revision: https://phabricator.services.mozilla.com/D110067
2021-04-08 08:02:15 +00:00
Cosmin Sabou f06d692475 Backed out changeset 23251cfb884e (bug 1616239) for causing windows aarch bustages. 2021-04-01 23:03:29 +03:00
Kershaw Chang 93423e9aa1 Bug 1616239 - Enable http3 tests for asan r=necko-reviewers,dragana
Differential Revision: https://phabricator.services.mozilla.com/D108635
2021-04-01 18:31:23 +00:00
june wilde 9a74eac849 Bug 1699448 - Allow any valid git reference in moz.yaml revision field; r=tjr
Differential Revision: https://phabricator.services.mozilla.com/D108953
2021-03-31 14:17:19 +00:00
Emilio Cobos Álvarez 22061e8965 Bug 1697053 - Let non-native-theme ride the trains in Firefox 89. r=cpeterson,mstange
Differential Revision: https://phabricator.services.mozilla.com/D109555
2021-03-24 19:11:05 +00:00
Tom Ritter 7d6a463f12 Bug 1700621: Add needinfo field to the moz.yaml schema r=jewilde
Differential Revision: https://phabricator.services.mozilla.com/D109615
2021-03-24 17:17:11 +00:00