The purpose of this is to remove as many docstrings from CommandProvider
classes to make the step of moving commands out of classes simpler.
Where possible, the docstring has been moved to or merged with the function.
Differential Revision: https://phabricator.services.mozilla.com/D123288
Chrome has removed 3DES completely[0], but we're still seeing some uses of it
in telemetry. Our assumption is that this is either due to old devices that
can't be upgraded, and hence probably use TLS 1.0, or servers that bafflingly
choose 3DES when there are other, better, ciphersuites in common.
This patch allows 3DES to only be enabled when deprecated versions of TLS are
enabled. This should protect users against the latter case (where 3DES is
unnecessary) while allowing them to use it in the former case (where it may be
necessary).
NB: The only 3DES ciphersuite gecko makes possible to enable is
TLS_RSA_WITH_3DES_EDE_CBC_SHA. This patch also changes the preference
corresponding to this ciphersuite from "security.ssl3.rsa_des_ede3_sha" to
"security.ssl3.deprecated.rsa_des_ede3_sha".
[0] https://www.chromestatus.com/feature/6678134168485888
Differential Revision: https://phabricator.services.mozilla.com/D121797
It's an unnecessary difference with our other clang builds. It looks
like it was cargo-culted from some external script. It makes the
android-cross clang different in the way it finds e.g. compiler
runtimes, some of which are in lib/, and others in lib64/, and only the
latter are actually found.
Differential Revision: https://phabricator.services.mozilla.com/D123017
What currently happens when Cargo.lock doesn't match is that the first
time we run cargo to generate the cbindgen metadata, we don't run it
with --frozen (unlike subsequent calls), and that locally fixes
Cargo.lock. Instead of that, we make the build fail, which will prevent
out-of-sync Cargo.lock from slipping in the tree in the first place.
Differential Revision: https://phabricator.services.mozilla.com/D123010
This patch updates ssltunnel to have a configuration where it only uses a 3DES
ciphersuite, rather than RC4 ciphersuites. This has to be done in stages
because otherwise android tests will fail, because their hostutils doesn't
actually use what's in-tree, and has to be updated separately.
Differential Revision: https://phabricator.services.mozilla.com/D122393
Local builds don't use it, and currently the only builds that happen on
macs on automation are not using the clang toolchain, so they're not
going through the first condition, and the remainder of the mozconfig
is essentially no-ops (plus, the only builds on macs on automation that
do use mozconfigs at all are rusttests, for which those settings wouldn't
matter anyways).
Meaning in practice, the mozconfig is not doing anything useful.
Differential Revision: https://phabricator.services.mozilla.com/D122823
The sysroot setup on macOS is different from Linux, and pkg-config
files, if there are, would be coming from e.g. homebrew rather than
the macOS SDK.
Differential Revision: https://phabricator.services.mozilla.com/D122793
The Glean SDK was traditionally represented in-tree as `glean_sdk`.
However, on PyPI and in `pip list`, it appears as `glean-sdk` (`pip` was
doing the translation implicitly and internally).
This is breaking our virtualenv state-checking code, since it thinks
that `glean_sdk` isn't installed.
Since `glean_sdk` was marked as `pypi-optional`, this hasn't affected us
yet, but it will when we bump the Glean SDK version, because developers'
virtualenvs won't be detected as out-of-date.
Differential Revision: https://phabricator.services.mozilla.com/D122500
The wasi-sysroot toolchain contains both a sysroot for wasi and a
compiler-rt for clang. That makes it impractical to use as a
bootstrapped sysroot for wasm32-wasi builds of Spidermonkey.
We thus split the toolchain in two, one for the compiler-rt and one
for the sysroot. Ideally, the compiler-rt one would avoid building
clang/llvm the same way the sysroot one does, but that leads to
a case of chicken-and-egg, because the compiler-rt is needed to build
the clang toolchain. Eventually, the clang build would be split from
the addition of the compiler-rt, but we're not there yet.
Differential Revision: https://phabricator.services.mozilla.com/D122402
While some custom builds of GCC work successfully, system GCC builds
usually don't, because of several factors:
- they don't use the C++ headers or libstdc++.so from the sysroot
- they don't necessarily know to use the multiarch directories in the
sysroot
Differential Revision: https://phabricator.services.mozilla.com/D121946
We're currently using -isysroot as a preprocessor flag and
-Wl,--syslibroot/-Wl,--sysroot as a linker flag, but --sysroot is
generalization of both, so we can use that instead.
Differential Revision: https://phabricator.services.mozilla.com/D121943
They are equivalent, except for the fact that MACOSX_DEPLOYMENT_TARGET
may apply to more processes in a normal build. In practice, all the
processes that matter are covered through compiler flags.
On the opposite end, MACOSX_DEPLOYMENT_TARGET isn't necessary passed in
all cases (like clangd, mach static-analysis, etc.), while compiler
flags are.
Differential Revision: https://phabricator.services.mozilla.com/D122145
As this means stdc++compat also would need to be built when
cross-building for Windows on Linux, we also recurse in the stdc++compat
directory in that case. It was already possible to use
--enable-stdcxx-compat in that configuration, and that wasn't working
(there are other problems that prevent it from working, but we're going
to fix them shortly).
Differential Revision: https://phabricator.services.mozilla.com/D122195
Since the minimum required version of gcc and libstdc++ is 7.1 as of bug
1536848, the _GLIBCXX_RELEASE macro is available on all supported
versions. Which means instead of grabbing the largest version of the
libstdc++ symbols and tweaking for that, we can just tweak for
_GLIBCXX_RELEASE. We also remove the conditions that are always true due
to the versions involved.
Differential Revision: https://phabricator.services.mozilla.com/D122194
Chrome has removed 3DES completely[0], but we're still seeing some uses of it
in telemetry. Our assumption is that this is either due to old devices that
can't be upgraded, and hence probably use TLS 1.0, or servers that bafflingly
choose 3DES when there are other, better, ciphersuites in common.
This patch allows 3DES to only be enabled when deprecated versions of TLS are
enabled. This should protect users against the latter case (where 3DES is
unnecessary) while allowing them to use it in the former case (where it may be
necessary).
NB: The only 3DES ciphersuite gecko makes possible to enable is
TLS_RSA_WITH_3DES_EDE_CBC_SHA. This patch also changes the preference
corresponding to this ciphersuite from "security.ssl3.rsa_des_ede3_sha" to
"security.ssl3.deprecated.rsa_des_ede3_sha".
[0] https://www.chromestatus.com/feature/6678134168485888
Differential Revision: https://phabricator.services.mozilla.com/D121797
We're currently using -isysroot as a preprocessor flag and
-Wl,--syslibroot/-Wl,--sysroot as a linker flag, but --sysroot is
generalization of both, so we can use that instead.
Differential Revision: https://phabricator.services.mozilla.com/D121943
A lot of work happens on the wayland backend, and it regularly breaks tier-3
systems where wayland is not supported. Setting up X11-only builds will help
catch those breakages earlier.
Differential Revision: https://phabricator.services.mozilla.com/D121691
After removing `optional` in Bug 1712804, we need to add a variant back
here because there's fallible dependencies. However, I've tweaked the
re-introduction of the feature to require a specific repercussion
message as well. This seemed like a decent tradeoff - the developer
becomes aware that the failure is bad, it has repercussions, but it's
not a blocking issue. Additionally, since we're printing pip's output,
the developer will be able to see the underlying error causing the
warning.
I also added comment functionality to requirements definitions to allow
adjacent documentation of why some requirements are fallible. (Related:
I'm looking forward to `mach_bootstrap` not needing to parse
requirements definitions. Almost there!)
Note that we'll temporarily lose the "pinned" nature of the
three moved dependencies until dependency locking is implemented
for Mach requirements definitions. Also note that the pinned
`zstandard_requirements.txt` can't be removed like the other
files because it still has a dangling usage.
Finally, in preparation for review: I didn't make
`PypiOptionalSpecifier` extend `PypiSpecifier` because I figured that
the benefit of flexibility (easier to allow implementations to diverge
without needing to untangle an inheritance relationship) was larger than
the cost of needing to add properties to both specifiers.
If we wanted re-use, I'd probably have `PypiOptionalSpecifier` _contain_
a `PypiSpecifier`, but then you have to reach deeper into the object to
get data, so *shrug*.
Differential Revision: https://phabricator.services.mozilla.com/D119835
The three removed paths don't exist in-repo, and after a cursory glance
they don't appear to be populated dynamically.
Note that the removal of the `six` path for WPT is different: it's
technically just incorrect, and should amended to point to
`$WPT/tools/third_party/six`. However, Python only allows a single
instance of a library to exist in import scope, and we're already
consuming `six` from the Firefox-wide vendored 3rd-party libs.
Differential Revision: https://phabricator.services.mozilla.com/D119825
There are two different codepaths in old-configure for AS, and one of
them uses ac_cv_prog_AS and the other ac_cv_path_AS. We thus need to set
both.
Differential Revision: https://phabricator.services.mozilla.com/D121828
We do so as a side effect of using system_lib_option for all *-system-*
options for old-configure, all but one of which are for system
libraries.
Differential Revision: https://phabricator.services.mozilla.com/D121690
After removing `optional` in Bug 1712804, we need to add a variant back
here because there's fallible dependencies. However, I've tweaked the
re-introduction of the feature to require a specific repercussion
message as well. This seemed like a decent tradeoff - the developer
becomes aware that the failure is bad, it has repercussions, but it's
not a blocking issue. Additionally, since we're printing pip's output,
the developer will be able to see the underlying error causing the
warning.
I also added comment functionality to requirements definitions to allow
adjacent documentation of why some requirements are fallible. (Related:
I'm looking forward to `mach_bootstrap` not needing to parse
requirements definitions. Almost there!)
Note that we'll temporarily lose the "pinned" nature of the
three moved dependencies until dependency locking is implemented
for Mach requirements definitions. Also note that the pinned
`zstandard_requirements.txt` can't be removed like the other
files because it still has a dangling usage.
Finally, in preparation for review: I didn't make
`PypiOptionalSpecifier` extend `PypiSpecifier` because I figured that
the benefit of flexibility (easier to allow implementations to diverge
without needing to untangle an inheritance relationship) was larger than
the cost of needing to add properties to both specifiers.
If we wanted re-use, I'd probably have `PypiOptionalSpecifier` _contain_
a `PypiSpecifier`, but then you have to reach deeper into the object to
get data, so *shrug*.
Differential Revision: https://phabricator.services.mozilla.com/D119835
The three removed paths don't exist in-repo, and after a cursory glance
they don't appear to be populated dynamically.
Note that the removal of the `six` path for WPT is different: it's
technically just incorrect, and should amended to point to
`$WPT/tools/third_party/six`. However, Python only allows a single
instance of a library to exist in import scope, and we're already
consuming `six` from the Firefox-wide vendored 3rd-party libs.
Differential Revision: https://phabricator.services.mozilla.com/D119825
After removing `optional` in Bug 1712804, we need to add a variant back
here because there's fallible dependencies. However, I've tweaked the
re-introduction of the feature to require a specific repercussion
message as well. This seemed like a decent tradeoff - the developer
becomes aware that the failure is bad, it has repercussions, but it's
not a blocking issue. Additionally, since we're printing pip's output,
the developer will be able to see the underlying error causing the
warning.
I also added comment functionality to requirements definitions to allow
adjacent documentation of why some requirements are fallible. (Related:
I'm looking forward to `mach_bootstrap` not needing to parse
requirements definitions. Almost there!)
Note that we'll temporarily lose the "pinned" nature of the
three moved dependencies until dependency locking is implemented
for Mach requirements definitions. Also note that the pinned
`zstandard_requirements.txt` can't be removed like the other
files because it still has a dangling usage.
Finally, in preparation for review: I didn't make
`PypiOptionalSpecifier` extend `PypiSpecifier` because I figured that
the benefit of flexibility (easier to allow implementations to diverge
without needing to untangle an inheritance relationship) was larger than
the cost of needing to add properties to both specifiers.
If we wanted re-use, I'd probably have `PypiOptionalSpecifier` _contain_
a `PypiSpecifier`, but then you have to reach deeper into the object to
get data, so *shrug*.
Differential Revision: https://phabricator.services.mozilla.com/D119835
The three removed paths don't exist in-repo, and after a cursory glance
they don't appear to be populated dynamically.
Note that the removal of the `six` path for WPT is different: it's
technically just incorrect, and should amended to point to
`$WPT/tools/third_party/six`. However, Python only allows a single
instance of a library to exist in import scope, and we're already
consuming `six` from the Firefox-wide vendored 3rd-party libs.
Differential Revision: https://phabricator.services.mozilla.com/D119825
Originally, we only were using bindgen for stylo, and we also went a bit
overkill by including a lot of extra CFLAGS in there, which end up
being unnecessary. So we now only pass a more minimal set of flags, and
add some extra CFLAGS in cases where they are necessary.
Differential Revision: https://phabricator.services.mozilla.com/D121043
While the use of toml allows the flags to be separated, the split is
done via some shell shenanigans anyways, and servo's build.rs can
handle the same just fine.
Differential Revision: https://phabricator.services.mozilla.com/D121042
We do that with everything else (i.e. clang, cbindgen, etc.). We weren't
doing it for sysroots because the support was experimental and the path
was independent of the target, so there was no guarantee that the
sysroot would actually work for the target. As that's not the case
anymore, we can go with more consistency with the bootstrapped
toolchains.
Differential Revision: https://phabricator.services.mozilla.com/D120417
After removing `optional` in Bug 1712804, we need to add a variant back
here because there's fallible dependencies. However, I've tweaked the
re-introduction of the feature to require a specific repercussion
message as well. This seemed like a decent tradeoff - the developer
becomes aware that the failure is bad, it has repercussions, but it's
not a blocking issue. Additionally, since we're printing pip's output,
the developer will be able to see the underlying error causing the
warning.
I also added comment functionality to requirements definitions to allow
adjacent documentation of why some requirements are fallible. (Related:
I'm looking forward to `mach_bootstrap` not needing to parse
requirements definitions. Almost there!)
Note that we'll temporarily lose the "pinned" nature of the
three moved dependencies until dependency locking is implemented
for Mach requirements definitions. Also note that the pinned
`zstandard_requirements.txt` can't be removed like the other
files because it still has a dangling usage.
Finally, in preparation for review: I didn't make
`PypiOptionalSpecifier` extend `PypiSpecifier` because I figured that
the benefit of flexibility (easier to allow implementations to diverge
without needing to untangle an inheritance relationship) was larger than
the cost of needing to add properties to both specifiers.
If we wanted re-use, I'd probably have `PypiOptionalSpecifier` _contain_
a `PypiSpecifier`, but then you have to reach deeper into the object to
get data, so *shrug*.
Differential Revision: https://phabricator.services.mozilla.com/D119835
The three removed paths don't exist in-repo, and after a cursory glance
they don't appear to be populated dynamically.
Note that the removal of the `six` path for WPT is different: it's
technically just incorrect, and should amended to point to
`$WPT/tools/third_party/six`. However, Python only allows a single
instance of a library to exist in import scope, and we're already
consuming `six` from the Firefox-wide vendored 3rd-party libs.
Differential Revision: https://phabricator.services.mozilla.com/D119825
After removing `optional` in Bug 1712804, we need to add a variant back
here because there's fallible dependencies. However, I've tweaked the
re-introduction of the feature to require a specific repercussion
message as well. This seemed like a decent tradeoff - the developer
becomes aware that the failure is bad, it has repercussions, but it's
not a blocking issue. Additionally, since we're printing pip's output,
the developer will be able to see the underlying error causing the
warning.
I also added comment functionality to requirements definitions to allow
adjacent documentation of why some requirements are fallible. (Related:
I'm looking forward to `mach_bootstrap` not needing to parse
requirements definitions. Almost there!)
Note that we'll temporarily lose the "pinned" nature of the
three moved dependencies until dependency locking is implemented
for Mach requirements definitions. Also note that the pinned
`zstandard_requirements.txt` can't be removed like the other
files because it still has a dangling usage.
Finally, in preparation for review: I didn't make
`PypiOptionalSpecifier` extend `PypiSpecifier` because I figured that
the benefit of flexibility (easier to allow implementations to diverge
without needing to untangle an inheritance relationship) was larger than
the cost of needing to add properties to both specifiers.
If we wanted re-use, I'd probably have `PypiOptionalSpecifier` _contain_
a `PypiSpecifier`, but then you have to reach deeper into the object to
get data, so *shrug*.
Differential Revision: https://phabricator.services.mozilla.com/D119835
The three removed paths don't exist in-repo, and after a cursory glance
they don't appear to be populated dynamically.
Note that the removal of the `six` path for WPT is different: it's
technically just incorrect, and should amended to point to
`$WPT/tools/third_party/six`. However, Python only allows a single
instance of a library to exist in import scope, and we're already
consuming `six` from the Firefox-wide vendored 3rd-party libs.
Differential Revision: https://phabricator.services.mozilla.com/D119825
The background hang monitor uses the same timing thresholds as the one used by
the compositor thread, for similar reasons.
Differential Revision: https://phabricator.services.mozilla.com/D120117
This step removes all the dependencies of mach commands to
having a MachCommandBase as the `self` by using the `command_context`
argument instead. This also removes any remaining statefulness from those
classes that implement mach commands, ultimately making it easier to move
existing commands out of classes in a follow-up.
Differential Revision: https://phabricator.services.mozilla.com/D118058
We will only run the processes in CET compatible modules only mode when not
using the JIT code. So marking xul.dll as compatible should be OK.
Differential Revision: https://phabricator.services.mozilla.com/D117551
Clang >= 11 ignores PYTHON_EXECUTABLE entirely (and uses python3, which
is not even what we pass, but that's actually fine), and all the build
tasks we have on older versions find the python executable they need on
their own.
Differential Revision: https://phabricator.services.mozilla.com/D120049
I'm guessing that the code moved to the bootstrap command but the
associated comment was forgotten and left abandoned, which is very sad.
Differential Revision: https://phabricator.services.mozilla.com/D120098
There's a "minimum Python check" that happens when Mach
bootstraps that makes the bootstrap command's check obsolete.
Since we want to use modern Python features throughout Mach, we need to
move Python-out-of-date logic up to the front of the process.
We're duplicating the OS-detection logic a bit, but now we regain
per-platform Python instructions.
Differential Revision: https://phabricator.services.mozilla.com/D120093
Bug 1692137 added code that would end up calling
toolchain_task_definitions twice, once for each of host and target.
With the recent changes from bug 1719229, that is not useful anymore.
We also don't need the toolchain prefixes for toolchains on unsupported
hosts (which were useful before bug 1719229 for the target sysroots).
Differential Revision: https://phabricator.services.mozilla.com/D120044
This step removes all the dependencies of mach commands to
having a MachCommandBase as the `self` by using the `command_context`
argument instead. This also removes any remaining statefulness from those
classes that implement mach commands, ultimately making it easier to move
existing commands out of classes in a follow-up.
Differential Revision: https://phabricator.services.mozilla.com/D118058
It turns out calling gdk_display_close gets us a rematch of bug 1626536,
so remove the call that was added in bug 1718131, and adjust valgrind
suppressions accordingly.
Differential Revision: https://phabricator.services.mozilla.com/D120037
After changeset 5e4b33fe9218703f0b29e2446159bcf4202d15fa the Attributes.cpp file was
changed so our patched version was not sticking on top of it. This patch does the proper
rebase.
Differential Revision: https://phabricator.services.mozilla.com/D119951
In cross-compilation setups (x86_64 host, i686 or aarch64 target), we're
going to need two sysroots. Obviously, we need the sysroot paths to be
different in that case, so the sysroot path themselves need to contain
some distinctive name, and we'll use the `target.toolchain` name for
that (the target triplet with the vendor/machine stripped out).
Because the path name needs to be reflected in the artifact name as well
as the toolchain name, we also change them.
And because the current prefix in the toolchain name is now redundant
with the suffix, we remove the prefix, and allow the bootstrapping
mechanism to try toolchains without the prefix.
Differential Revision: https://phabricator.services.mozilla.com/D119846
It was enabled in bug 1712633 by mistake.
Because there wasn't a way to distinguish esr yet, we add one, and while
at it, expose it to the build system.
Differential Revision: https://phabricator.services.mozilla.com/D119682
As written in 00c93ac66282, relying on the 2.27-3ubuntu1.2 package being
available is a timebomb. We can however get the older sources from
launchpad, and build them ourselves.
Differential Revision: https://phabricator.services.mozilla.com/D119258
This allows to use the same toolchain docker images as other toolchains,
based on Debian buster.
While here, use the default max-run-time, which is more than enough for
this toolchain.
Differential Revision: https://phabricator.services.mozilla.com/D119137
Because GCC is built in stages, the final stage is built with
intermediate stages's GCC, which handles the sysroot correctly, so we
end up with headers and libraries with the expected compatibility.
This allows to use the same toolchain docker images as other toolchains,
based on Debian buster.
Differential Revision: https://phabricator.services.mozilla.com/D119136
- we needed -gcc-toolchain to pick C/C++ standard headers from the right
version of GCC, but we now have them in the toolchain sysroot (bug
1719207), so we can use that instead.
- we needed LD_LIBRARY_PATH when clang was built on an older version of
Debian, but that was changed in bug 1694775.
Differential Revision: https://phabricator.services.mozilla.com/D119135
We have a separate binutils toolchain already, and the only remaining
use of the binutils part of the the GCC toolchain is for the gold plugin
headers for clang, which we can add to the toolchain sysroot.
Differential Revision: https://phabricator.services.mozilla.com/D119133
When running `mach python-test` on a clobbered tree, we create both the
python-test and common virtualenvs, but the former should be enough. The
latter is only created when running tests that use the BaseConfigureTest
class, when it indirectly includes init.configure, which makes the
configure sandbox run the virtualenv_python3 function, which ends up
initializing the common virtualenv. We only don't end up re-executing
that virtualenv python thanks to 6680ca0acc27 which was a workaround
that would allow any virtualenv in $objdir/_virtualenvs, while still
creating a virtualenv it won't use.
Tests now tell virtualenv_python3 to not use the normal virtualenv
manager, and remove the workaround.
Differential Revision: https://phabricator.services.mozilla.com/D118775
I removed the -Wno-range-loop-analysis warning flag in bug 1718408 because I didn't see any build failures on Try without it, but apparently I missed some.
Differential Revision: https://phabricator.services.mozilla.com/D119039
This requires a backport of valgrind to buster instead of jessie (which
we don't need a backport for anymore). Somehow the buster-based
android-build was using the backport for jessie. It now can use the
backport for buster.
We now also need a few extra packages in the valgrind docker image that
used to be installed as a side effect of installing other packages, but
aren't installed automatically anymore, while necessary.
This allows to remove the debian8-amd64-build docker image, now unused.
We add a patch to workaround an issue with rust that was fixed in older
version of valgrind but that resurfaced in a slightly different manner.
Filed upstream as https://bugs.kde.org/show_bug.cgi?id=433641.
Finally, we update the suppressions to account for system changes.
Differential Revision: https://phabricator.services.mozilla.com/D106402
This requires a backport of valgrind to buster instead of jessie (which
we don't need a backport for anymore). Somehow the buster-based
android-build was using the backport for jessie. It now can use the
backport for buster.
We now also need a few extra packages in the valgrind docker image that
used to be installed as a side effect of installing other packages, but
aren't installed automatically anymore, while necessary.
This allows to remove the debian8-amd64-build docker image, now unused.
We add a patch to workaround an issue with rust that was fixed in older
version of valgrind but that resurfaced in a slightly different manner.
Filed upstream as https://bugs.kde.org/show_bug.cgi?id=433641.
Finally, we update the suppressions to account for system changes.
Differential Revision: https://phabricator.services.mozilla.com/D106402
As there are some things to solve first for the sysroot, we don't enable
the sysroot bootstrap unless --enable-bootstrap is given explicitly. The
default will still bootstrap everything else when building on central.
Differential Revision: https://phabricator.services.mozilla.com/D118450
The `mozbase` modules were being unconditionally added to the
`sys.path` regardless of the Mach command being run, so there isn't
much value keeping them in a separate file. Besides, all other
source module paths are described in `common_virtualenv_packages`,
why is `mozbase` special?
In the future, we're going to want to make improvements here (such as:
there's a difference between informing mach of first-party code
versus defining which third_party vendored packages should be in scope,
and that workflow difference should be represented in-code).
It's useful to peel out the existing, less useful abstraction before
we can build a stronger one.
Differential Revision: https://phabricator.services.mozilla.com/D117711
Having separate `<name>.pth` files in the virtual environments
isn't providing an advantage. We can simplify configuration
by putting all `pth` adjustments into a single file: `mach.pth`.
Differential Revision: https://phabricator.services.mozilla.com/D117710
This has two benefits:
1. `handle_package()` becomes more clear - rather than referring to
`action` and `package` with array index numbers, we now give
them real names. The benefit here is also shown in `up_to_date()`.
2. This makes the top-level parser for `packages()` less opinionated
about sub-formats: if an action has a nested structure, it should
have the flexibility to define what it looks like.
Differential Revision: https://phabricator.services.mozilla.com/D117708
After some testing in `try` and locally, the manual
`PYTHONEXECUTABLE` definitions shouldn't be needed
anymore.
There's been some work on Brew's
Python to improve its `sitecustomize` behaviour.
The most likely improvement is:
https://github.com/Homebrew/homebrew-core/pull/65297
However, I'm not fully confident in this change.
If it fails, it's more likely to affect developers
than CI. I think it's worth attempting a landing,
because if the variable is indeed obsolete, then
deleting it will avoid some spicy "action-at-a-
distance" behaviour.
Differential Revision: https://phabricator.services.mozilla.com/D117452
Vendoring wheels has three benefits:
* There's far less files, so Firefox checkouts will be smaller.
* It works around `zipp` not allowing `pip install`
from extracted source `tar.gz` files. Now, we should
be able to use the pip resolver against vendored
packages, which will be needed for future
mach virtualenv work.
* `./mach vendor python` takes far less time to execute.
Since we need the raw Python to be available to add to the `sys.path`,
we extract the wheels before putting them in tree.
Due to the structure of some wheels being less nested
than of a source `tar.gz`, `common_virtualenv_packages`
needed to be adjusted accordingly.
`install_pip_package()` had to be tweaked as well since you can't
`pip install` an extracted wheel. So, we "re-bundle" the wheel
before installing from a vendored package.
Replace python packages with wheels where possible
This contains the vendoring changes caused by the
last patch.
For reviewing, there's a couple things to note:
* A bunch of files are deleted, since there's generally
less files in a wheel than in a source archive.
* There's a new `.dist-info` directory for each
extracted wheel, so expect roughly 5 or
6 new files for each wheel'd package.
* There should be no source code changes other than
moves from package names changing from having
`-` to having `_`.
Differential Revision: https://phabricator.services.mozilla.com/D116512
Last year, we stopped vendoring Python packages that have native
code. Since we have only had pure-python packages since, the
Windows-specific qualifier (or excluder in the case of `!windows`)
hasn't been needed.
I don't foresee us needing it again, but if anything we can peel it
back from `hg` history if this assumption is incorrect.
Differential Revision: https://phabricator.services.mozilla.com/D117468
The `mach_bootstrap:search_path()` implementation is out of
date compared to `virtualenv.py`.
Since `python2:`, `python3:` and `optional:` packages are no
longer valid virtualenv requirement actions, they can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D117707
For "c" (i.e., gcc, clang, and clang-cl), this configures the
`-f{debug,macro}-path-prefix` flags. We'd prefer to use
`-ffile-path-prefix`, but it seems that `clang-cl` does not recognize
that flag.
For "rust" (i.e., rustc/cargo), this configures `--remap-path-prefix`.
This is one step toward getting `sccache` hits across source and
object directories.
Differential Revision: https://phabricator.services.mozilla.com/D113065
Based on the docs and the code within `pathlib2`, it seems to focus
only on backporting the main features of `pathlib` to be available for
Python 2. It does _not_ provide features in newer Python versions
of `pathlib` (such as `is_mount`, new in Python 3.7). Therefore, it
doesn't provide anything that the standard library of Python 3.6+
doesn't have.
Differential Revision: https://phabricator.services.mozilla.com/D117071
We still need the blocklist to control rollout on Android, but on
desktop, we should be shipping Software WebRender to all users, except
those in safe mode or whom have explicitly disabled WebRender.
Differential Revision: https://phabricator.services.mozilla.com/D117088
This will both:
* Ensure that developers aren't accidentally using Mach with Python 2
in the year 2021.
* Confirm that CI tasks aren't still using Python 2 with Mach.
* Remove support for Python 3.5, which is EOL'd.
Differential Revision: https://phabricator.services.mozilla.com/D116484
This will both:
* Ensure that developers aren't accidentally using Mach with Python 2
in the year 2021.
* Confirm that CI tasks aren't still using Python 2 with Mach.
* Remove support for Python 3.5, which is EOL'd.
Differential Revision: https://phabricator.services.mozilla.com/D116484
With D100581 LLVM added support for -Wunused-but-set-parameter and -Wunused-but-set-variable.
1. we shouldn't treat these warning as errors
2. for harfbuzz we apply a patch that has also been upstreamed, #2995, since pragma directives are somehow borken in clang vs gcc.
Differential Revision: https://phabricator.services.mozilla.com/D116604
There is a valgrind suppression on a generic signature containing the hash of rust types. The hash changes when crate dependencies are updated.
Differential Revision: https://phabricator.services.mozilla.com/D116588
To compile SM to .wasm we need to use clang from wasi-sdk, this patch
allows us to use default clang-12 from mozilla's repo.
We remove usage of rpath-link flag because wasm-ld doesn't support it,
enable single thread model because wasi is single threaded
and finally add <unistd.h> becase it is forbidden to use read/write/lseek
without this header.
Differential Revision: https://phabricator.services.mozilla.com/D116207
This will both:
* Ensure that developers aren't accidentally using Mach with Python 2
in the year 2021.
* Confirm that CI tasks aren't still using Python 2 with Mach.
* Remove support for Python 3.5, which is EOL'd.
Differential Revision: https://phabricator.services.mozilla.com/D116484
`MACH_VIRTUALENV` was never used, and `MOZBUILD_VIRTUALENV` was never
set (the virtualenv was always activated during the build, rather
than before).
Differential Revision: https://phabricator.services.mozilla.com/D115921
This was originally set up so that tests wouldn't "create a new
`virtualenv` for no reason." However, virtual environments now will have
different packages installed, and therefore the separation is necessary.
So, for the virtual environment used for builds (regular or for tests):
* We want it to be able to reuse the build venv, if it already exists.
* We don't want to pollute a `pytest` virtualenv with build-specific
packages.
Differential Revision: https://phabricator.services.mozilla.com/D115641
We had split up `init` from `init_py3` because `mach` had
traditionally been invoked by either Python 2 or Python 3, and
the two couldn't share the same virtualenv.
Now that the same context isn't shared by both Python 2 and 3
3
(developers always use Python 3, and the remaining Python 2
usages are CI jobs that never reuse the objdir with Python 3),
We can centralize on a single default virtualenv.
I've called this "common" instead of "init" to clarify its
existing position as the virtualenv that's used by many different
commands. As we associate virtualenvs with requirement definitions,
it'll also make the file less confusing: it's a "common" requirement
definition as opposed to an "init" one.
Differential Revision: https://phabricator.services.mozilla.com/D115635
At the same time, move the option to python configure, which changes how
to disable it (setting to an empty value rather than 0).
Differential Revision: https://phabricator.services.mozilla.com/D115844
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).
Differential Revision: https://phabricator.services.mozilla.com/D109650
As far as I can see, all this does is protect the user from
running some useless code if they manually enable the priority
manager using a pref on an OS that doesn't support it. The
upside of allowing this is that it makes it possible to debug
the priority manager on OSX and Linux with just a pref flip.
Differential Revision: https://phabricator.services.mozilla.com/D114767
Added two fields:
1. Collect if the current shell is opened via vscode.
2. Connect if it's a remote ssh connection n the current shell.
Differential Revision: https://phabricator.services.mozilla.com/D114323
Did you know that our version of requests was from 2015? Phwoar!
This bumps `jinja2`, `py`, `pyyaml`, `requests` and `urllib3`.
There's significant risk for regressions due to breaking changes,
though due to the dynamic nature of Python, they're tricky to track
down.
The potential breaking changes I'm expecting to potentially affect
us are:
* `requests@2.11.0`: No longer accepts non-strings as header values.
* `requests@2.16.0`: `requests.packages` namespace was removed due to
packages no longer being vendored. The namespace has been
incrementally restored over future releases, but it's unclear to
what degree.
* `requests@2.24.0`: Redirect resolution now only happens when
`allow_redirects` is `True`.
* `requests` version of `urllib3` was bumped from `1.13.1` to `1.26`,
unsure what repercussions that will have.
This also added dependencies to `flake8_requirements`. This is because
the vendored `importlib-metadata` update (and the new
`typing-extensions` dependency) is found by `pip`, but the
vendored `typing-extensions` is _not_ discovered (perhaps due to
details around "eggs" and the way we add vendored code to the sys.path).
So, `pip` tries to install it manually and fails when it finds no hash.
By re-compiling `flake8_requirements`, the necessary hashes are made
available.
Differential Revision: https://phabricator.services.mozilla.com/D108526
The maximum version check has caused more unnecessary failed
builds (for freshly up-to-date devs) than helpful guards against
tough-to-diagnose issues.
Differential Revision: https://phabricator.services.mozilla.com/D113675
irrespective of the optimization level. The code was previously using
the optimization level as a proxy for whether --enable-release is
enabled or not, which was invalidated by bug 1689284.
Differential Revision: https://phabricator.services.mozilla.com/D113730
It's a relatively minor one (two keyframes are leaked). I built latest
GTK3 from source and it wasn't reproducible locally. I think our GTK
version on automation might be too old and not have this commit:
10fa786dfa
Or something of that sort.
Differential Revision: https://phabricator.services.mozilla.com/D113948
Not all in-tree Rust libraries were using workspace-hack.
Additionally, some needed winapi features were missing from
workspace-hack's configuration.
Now, winapi is re-compiled less frequently on a full build.
Differential Revision: https://phabricator.services.mozilla.com/D113564
Sometimes there is a deadlock between user-requested ccov counters flush (through a signal) and flush happening when a process is dying.
So in order to avoid it, just remove the latter which is out of our control (user-requested flushes are protected using a cross mutex).
Differential Revision: https://phabricator.services.mozilla.com/D112632
Mike Hommey
Alex Lopez
Andi-Bogdan Postelnicu
Dana Keeler
Mitchell Hentges
Narcis Beleuzu
Iulian Moraru
Brindusan Cristian
Noemi Erli
Csoregi Natalia
Mike Kaply
Butkovits Atila
Sandor Molnar
Cosmin Sabou
Gerald Squelart
Brad Werth
Bob Owen
Andi-Bogdan Postelnicu
Benjamin Beurdouche
lyavor
Chris Peterson
Alexandru Michis
Kai Engert
Alexis Beingessner
surajeet310
Nick Alexander
Julien Cristau
Deian Stefan
Andrew Osmond
Alexandre Lissy
Dorel Luca
Nicolas Silva
Dmitry Bezhetskov
Steve Fink
Makoto Kato
byron jones
Andrew McCreight
Chris H-C
Adam Gashlin
Kagami Sascha Rosylight
Emilio Cobos Álvarez
Calixte Denizet