This makes the bootstrap behavior wrt. Mercurial consistent on all
platforms, making Windows bootstrap only upgrade Mercurial if the
version is older than MODERN_MERCURIAL_VERSION.
As a side effect, this avoids upgrading to version 5.0.1, which doesn't
come with wheels at the moment.
Differential Revision: https://phabricator.services.mozilla.com/D35754
--HG--
extra : moz-landing-system : lando
The `mach resource-usage` page supports a list of build resource data,
but only ever displays the first one. Nothing actually creates a list
with multiple items automatically, but one might want to do that
manually to explore data from multiple builds more conveniently.
So if such a list exists, we expose a dropdown list of all the data
available, and switch the graph when a different item is chosen from the
list.
Differential Revision: https://phabricator.services.mozilla.com/D35757
--HG--
extra : moz-landing-system : lando
The git version shipped in some versions of OSX is patched by apple in a
way such that doing `git push hg::ssh://...` fails with an error message
like `Invalid remote name "hg::ssh://...`.
So instead, we define a named remote via inline configuration, and use
that remote's name for the push.
Differential Revision: https://phabricator.services.mozilla.com/D35351
--HG--
extra : moz-landing-system : lando
The git version shipped in some versions of OSX is patched by apple in a
way such that doing `git push hg::ssh://...` fails with an error message
like `Invalid remote name "hg::ssh://...`.
So instead, we define a named remote via inline configuration, and use
that remote's name for the push.
Differential Revision: https://phabricator.services.mozilla.com/D35351
--HG--
extra : moz-landing-system : lando
This makes running without mach more consistent. e.g. running
`make -C $objdir/toolkit/library/rust target` makes the cargo log
verbose, and adding `-s` makes it less verbose.
Differential Revision: https://phabricator.services.mozilla.com/D35521
--HG--
extra : moz-landing-system : lando
When the `MOZ_RUST_TIER` environment variable is set, we enable a separate
tier that builds rust code only. This is useful to measure build times for
rust code separately from other compilations.
Differential Revision: https://phabricator.services.mozilla.com/D35501
--HG--
extra : moz-landing-system : lando
NSS_ALLOW_SSLKEYLOGFILE no longer has issues upstream, we can allow it again.
Differential Revision: https://phabricator.services.mozilla.com/D34915
--HG--
extra : moz-landing-system : lando
Bug 1394825 bumped the minimum version of clang we use on automation,
for the base toolchain jobs, and there's a libclang test for bindgen,
but we should reject the compiler in the first place.
The check to do that works in both C and C++, contrary to the original
3.6 test that was checking a C++ feature ; the test was later changed
for 3.9, using a check that would have worked in both C and C++, but
the C exception was left around. We remove it now.
Differential Revision: https://phabricator.services.mozilla.com/D34083
--HG--
extra : moz-landing-system : lando
This works around the following situation: the Ngv/fat AAR task is
deterministic, but it depends on N tasks for multiple architectures,
adn the exact l10n changeset pulled by each N task depends on the tip
of the l10n HG repository at the time the N task executes.
For now, just ignore the obvious localized resources: *.ftl, *.dtd,
*.properties.
Differential Revision: https://phabricator.services.mozilla.com/D33413
--HG--
extra : moz-landing-system : lando
Not sure why this wasn't done before, but it prevented me from running `./mach rust vendor`.
Differential Revision: https://phabricator.services.mozilla.com/D34127
--HG--
extra : moz-landing-system : lando
Skips over the incremental cache when performing a clobber. The incremental compilation cache is located at:
`$(objdir)/$(rust_target)/$(rust_build_kind)/incremental`
When cross compiling there can be two caches, one for the host and one for the target so we handle both.
Differential Revision: https://phabricator.services.mozilla.com/D31310
--HG--
extra : moz-landing-system : lando
Pass in the substs dictionary to `Clobberer` so that we can use it to query `cargo` paths.
Differential Revision: https://phabricator.services.mozilla.com/D31309
--HG--
extra : moz-landing-system : lando
Pull out the logic for filtering subdirectories and deleting them into
reusable functions.
Differential Revision: https://phabricator.services.mozilla.com/D31308
--HG--
extra : moz-landing-system : lando
Use the same logic as for --with-android-sdk to use the NDK downloaded
by mach bootstrap by default.
Differential Revision: https://phabricator.services.mozilla.com/D34251
--HG--
extra : moz-landing-system : lando
After bug 1519954, the build system will pick the compiler in
~/.mozbuild on its own, so there's no need to suggest people to manually
put that in their mozconfig.
Differential Revision: https://phabricator.services.mozilla.com/D33903
--HG--
extra : moz-landing-system : lando
As established previously, generated source files that are not included
don't need to be built during the export tier.
Differential Revision: https://phabricator.services.mozilla.com/D33771
--HG--
extra : moz-landing-system : lando
Pull out the logic for filtering subdirectories and deleting them into
reusable functions.
Differential Revision: https://phabricator.services.mozilla.com/D31308
--HG--
extra : moz-landing-system : lando
In a branch for `isinstance(obj, (HostSources, HostGeneratedSources))`,
`isinstance(obj, GeneratedSources)` is clearly not going to match
anything. `isinstance(obj, HostGeneratedSources)` is what was intended.
Differential Revision: https://phabricator.services.mozilla.com/D33603
--HG--
extra : moz-landing-system : lando
webidlsrcs.mk is treated specially by the build system, and this entire
process should be supported better by moz.build, but in the meantime this
fixes the clobber bug caused by not re-building these targets when defines
are updated.
Differential Revision: https://phabricator.services.mozilla.com/D33415
--HG--
extra : moz-landing-system : lando
Invoking MozbuildObject.substs causes an exception to be raised if there is no
BuildEnvironment. This means that we hit an error on |mach help| if there is no
build, which is not ideal for people trying to learn how to build :).
Differential Revision: https://phabricator.services.mozilla.com/D33217
--HG--
extra : moz-landing-system : lando
This follows the model set down for EME artifacts:
- a new tier is added that uses a new Python build action to fetch and
artifacts
- the action unpacks the fetched artifacts and moves specific inputs
into places expected by the build and packager
- in automation, MOZ_ARTIFACT_TASK* is used to ensure the artifacts
come from the correct tasks
In this case, the artifact fetching is done entirely in a new Python
build action that internally uses `mach artifact install --job ...`.
The action also verifies that the fetched artifacts are compatible and
that we're not assembling a fat AAR that is nonsensical. The specific
inputs are not used in the Fennec APK that is produced; they're only
used in the GeckoView AAR that is produced.
The artifact fetching itself required tweaking to fetch only
`target.maven.zip` artifacts and to not unpack them.
The specific inputs used are the native libraries (libs/$ARCH/*.so)
and the architecture-specific preference files ($ARCH/greprefs.js and
defaults/pref/$ARCH/geckoview-prefs.js). None of these inputs are
impacted by l10n.
Differential Revision: https://phabricator.services.mozilla.com/D31572
--HG--
extra : moz-landing-system : lando
This allows to use the existing artifacts VCS-based crawling to
download the "raw" target.maven.zip from Android jobs and not process
it further. It's just put in a specific directory, ready for use.
This isn't a big deal in automation, where all URLs are known, but
it's very useful when building locally and the VCS and the pushlog
must be consulted to determine task URLs.
Differential Revision: https://phabricator.services.mozilla.com/D31571
--HG--
extra : moz-landing-system : lando
(adds __build_class__ to the builtin sandbox and allow CombinedDependsFunction to be hashed)
Differential Revision: https://phabricator.services.mozilla.com/D28114
--HG--
extra : moz-landing-system : lando
shellutil.py needed special handling since it is invoked earlier than the
vendor directory adding we do, so it does not have six available.
Differential Revision: https://phabricator.services.mozilla.com/D28096
--HG--
extra : moz-landing-system : lando
Lint python/mozbuild/{mozbuild,mozpack}.
This makes sure we use byte strings (type(b'')) in many places.
This also has a few places where we know we want unicode, by enforcing it.
This code also has a few places where we call into windows API's which expect either all args to be unicode,
or all args to be bytestrings (not-unicode) so make sure those calls align.
This is the most risky of the stack, since there are some mach commands and codepaths that are neither excercised in automation
nor excercised in tests.
Differential Revision: https://phabricator.services.mozilla.com/D26645
--HG--
extra : moz-landing-system : lando
Lint python/mozbuild/{mozbuild,mozpack}. r=#build
Add 'from __future__ import absolute_import' to all missing files in python/mozbuild/{mozbuild,mozpack}/**/*.py
Differential Revision: https://phabricator.services.mozilla.com/D26643
--HG--
extra : moz-landing-system : lando
Leaving one fix for an explicit review.
Lint python/mozbuild/{mozbuild,mozpack}. r=#build
Differential Revision: https://phabricator.services.mozilla.com/D26641
--HG--
extra : moz-landing-system : lando
Lint python/mozbuild/{mozbuild,mozpack}.
ran './mach lint -l py2 -l flake8 -l shellcheck -l codespell -l yaml python/mozbuild/{mozbuild,mozpack}/ --fix' in order
to undo some black changes and get closer to making this folder able to be validated on every lint run
Differential Revision: https://phabricator.services.mozilla.com/D26640
--HG--
extra : moz-landing-system : lando
We're moving to IR-level PGO instrumentation for clang-cl. We've also
moved to using static linker ordering files, which was the primary
application of the previous style of PGO instrumentation. We therefore
we no longer need this code.
Differential Revision: https://phabricator.services.mozilla.com/D31134
--HG--
extra : moz-landing-system : lando
We used to support specifying single Rust files for compilation,
emphasis on "used to".
Differential Revision: https://phabricator.services.mozilla.com/D32314
--HG--
extra : moz-landing-system : lando
Bug 1533425 makes Gecko try to load from $ARCH/greprefs.js, etc on
Android. This patch teaches the packager to put certain preferences
into those architecture-specific locations for that code to find.
Differential Revision: https://phabricator.services.mozilla.com/D24984
--HG--
extra : moz-landing-system : lando
This way compilation backends don't all have to figure out the right way
to handle symbol files.
Differential Revision: https://phabricator.services.mozilla.com/D29673
--HG--
extra : moz-landing-system : lando
Our current OS X builds use `--target=x86_64-darwin11` (which
corresponds to OS X 10.7). This target is problematic for two reasons:
* We're actually targeting for OS X 10.9 (`MACOSX_DEPLOYMENT_TARGET`);
* It's slightly different from the default Rust target.
Let's address these problems in reverse order: differences from the Rust
target are bad, because the `--target` we provide to `clang` and the
Rust target find their way into LLVM bitcode files and the linker will
refuse to link together bitcode files that have incompatible targets.
Why are the two incompatible? The current `--target` doesn't have a
"vendor" in triple-speak, whereas the Rust one has "apple" as the
vendor (`x86_64-apple-darwin`) We therefore need to change the
`--target` we pass to `clang` to have a vendor of "apple".
This need is behind the {init,toolchain}.configure changes,
but it has ramifications elsewhere, because `clang` looks for
`--target`-prefixed build tools. So we have to change the `--target`
for cctools to get the right tool prefixes and we have to change the
`--target` for building clang ourselves so that *those* builds can find
the newly renamed cctools.
Once we've done, that's really enough; we don't *need to address the
first problem: While the `--target` might be `x86_64-apple-darwin11`,
both `clang` and `rustc` will dynamically choose the target triple that
eventually lands in LLVM bitcode files based on
`MACOSX_DEPLOYMENT_TARGET`, which we set in all builds. But the current
target is slightly misleading, and the cctools don't need to be prefixed
with a particular Darwin version, since they work for all Darwin
targets. Let's just drop the "11" from the `--target` and eliminate a
little bit of confusion.
Differential Revision: https://phabricator.services.mozilla.com/D31128
--HG--
extra : moz-landing-system : lando
Our current OS X builds use `--target=x86_64-darwin11` (which
corresponds to OS X 10.7). This target is problematic for two reasons:
* We're actually targeting for OS X 10.9 (`MACOSX_DEPLOYMENT_TARGET`);
* It's slightly different from the default Rust target.
Let's address these problems in reverse order: differences from the Rust
target are bad, because the `--target` we provide to `clang` and the
Rust target find their way into LLVM bitcode files and the linker will
refuse to link together bitcode files that have incompatible targets.
Why are the two incompatible? The current `--target` doesn't have a
"vendor" in triple-speak, whereas the Rust one has "apple" as the
vendor (`x86_64-apple-darwin`) We therefore need to change the
`--target` we pass to `clang` to have a vendor of "apple".
This need is behind the {init,toolchain}.configure changes,
but it has ramifications elsewhere, because `clang` looks for
`--target`-prefixed build tools. So we have to change the `--target`
for cctools to get the right tool prefixes and we have to change the
`--target` for building clang ourselves so that *those* builds can find
the newly renamed cctools.
Once we've done, that's really enough; we don't *need to address the
first problem: While the `--target` might be `x86_64-apple-darwin11`,
both `clang` and `rustc` will dynamically choose the target triple that
eventually lands in LLVM bitcode files based on
`MACOSX_DEPLOYMENT_TARGET`, which we set in all builds. But the current
target is slightly misleading, and the cctools don't need to be prefixed
with a particular Darwin version, since they work for all Darwin
targets. Let's just drop the "11" from the `--target` and eliminate a
little bit of confusion.
Differential Revision: https://phabricator.services.mozilla.com/D31128
--HG--
extra : moz-landing-system : lando
The zipfile.is_zipfile() function is overly lenient, in that non-zip
files that contain the four bytes "PK\005\006" near the end of the file
are reported as zip files even if the zip structure is not valid.
Occasionally, our target.tar.bz2 files randomly contain those 4 bytes at
the expected location, which causes the mar repackaging code to try to
process the package as a zip file instead of a tar file.
The tarfile.is_tarfile() logic looks a little more robust in that it
actually tries to open the file, so if we try tar first before falling
back to zip we should be a little more resilient.
Differential Revision: https://phabricator.services.mozilla.com/D31908
--HG--
extra : moz-landing-system : lando
Our current OS X builds use `--target=x86_64-darwin11` (which
corresponds to OS X 10.7). This target is problematic for two reasons:
* We're actually targeting for OS X 10.9 (`MACOSX_DEPLOYMENT_TARGET`);
* It's slightly different from the default Rust target.
Let's address these problems in reverse order: differences from the Rust
target are bad, because the `--target` we provide to `clang` and the
Rust target find their way into LLVM bitcode files and the linker will
refuse to link together bitcode files that have incompatible targets.
Why are the two incompatible? The current `--target` doesn't have a
"vendor" in triple-speak, whereas the Rust one has "apple" as the
vendor (`x86_64-apple-darwin`) We therefore need to change the
`--target` we pass to `clang` to have a vendor of "apple".
This need is behind the {init,toolchain}.configure changes,
but it has ramifications elsewhere, because `clang` looks for
`--target`-prefixed build tools. So we have to change the `--target`
for cctools to get the right tool prefixes and we have to change the
`--target` for building clang ourselves so that *those* builds can find
the newly renamed cctools.
Once we've done, that's really enough; we don't *need to address the
first problem: While the `--target` might be `x86_64-apple-darwin11`,
both `clang` and `rustc` will dynamically choose the target triple that
eventually lands in LLVM bitcode files based on
`MACOSX_DEPLOYMENT_TARGET`, which we set in all builds. But the current
target is slightly misleading, and the cctools don't need to be prefixed
with a particular Darwin version, since they work for all Darwin
targets. Let's just drop the "11" from the `--target` and eliminate a
little bit of confusion.
Differential Revision: https://phabricator.services.mozilla.com/D31128
--HG--
extra : moz-landing-system : lando
This just separates out the Android definitions into
mobile/android/mach_commands.py. There was vestigial support for
running on Android with debuggers, but it was for wiring up JimDB,
which is no longer supported and in fact hasn't worked on actual
devices for a very long time. (The new flow for running on Android
under a debugger goes through the Android Studio hybrid debugger.)
Differential Revision: https://phabricator.services.mozilla.com/D18291
--HG--
extra : moz-landing-system : lando
We want `mach run` for Android to be wildly different than `mach run`
for Desktop. But right now, mach really doesn't support two different
implementations of the same underlying named command. The avenues
that might support different implementations, mostly run through
`conditions`.
`conditions` were added to mach commands in Bug 901972, and never
really anticipated this use case: commands are keyed by name condition
evaluation is delayed until dispatch-time. In order to have different
commands with the same name, and have full support for `--help`,
command matching, suggestions, etc, we really need conditions to
evaluate at parse-time. Indeed, since Bug 901972 landed, we've moved
context creation earlier in the dispatch flow and hacked in things
that look like parse-time conditions (see Bug 1291335 and Bug
1305695).
This approach is not the prettiest, but it handles this narrow
use-case -- making `mach run` and `mach install` different on Android
-- without much code churn.
Differential Revision: https://phabricator.services.mozilla.com/D18290
--HG--
extra : moz-landing-system : lando
Mike Hommey
L. David Baron
Dorel Luca
Kartikaya Gupta
prathiksha
Eric Rahm
Chris Manchester
Bob Clary
Andi-Bogdan Postelnicu
Justin Wood
Nick Alexander
David Teller
Henri Sivonen
shindli
Jorg K
Cosmin Sabou
Samuel Thibault
Jeremy Lempereur
Andrew Halberstadt
Tom Prince
Rok Garbas
Nathan Froyd
Myk Melez
Dzmitry Malyshau
Sylvestre Ledru
Robert Bartlensky
Ian Neal
Andreea Pavel
Coroiu Cristina
Mike Shal
b4hand
Bastien Orivel
Mihai Alexandru Michis
Narcis Beleuzu