Android artifacts (GeckoView AARs, GeckoViewExample (and Fennec) APKs)
require native libraries (`libxul.so`) and an omnijar (`omni.ja`).
These are produced by `mach package` (really, the `stage-package`
target). Engineers essentially never want a build without a package
for mobile/android. This adds mobile/android-only tiers that run
`mach package` and then `mach android assemble-app`. The latter
consumes `libxul.so` and `omni.ja` to produce _all the things_
relevant to GeckoView engineers.
Differential Revision: https://phabricator.services.mozilla.com/D41450
--HG--
extra : moz-landing-system : lando
Bug 1563711 changed MOZ_FETCHES_DIR to make the MOZ_ANDROID_FAT_AAR_*
environment variables absolute paths. Unfortunately, the replacement
relies on non-osx/windows tasks to run in docker-worker, which is not
necessarily true, and that makes the MOZ_FETCHES_DIR wrong in
non-osx/windows generic-worker tasks. Apparently, that currently works,
but that's not guaranteed to stay this way.
The MOZ_ANDROID_FAT_AAR_* environment variables don't need to be
absolute paths, though. MOZ_FETCHES_DIR is normalized by run-task, and
MOZ_ANDROID_FAT_AAR_* can be set relative to that, which we do here.
Differential Revision: https://phabricator.services.mozilla.com/D40334
At the same time, make things a little more tangible in the tiers setup
in baseconfig.mk.
Differential Revision: https://phabricator.services.mozilla.com/D37942
--HG--
extra : moz-landing-system : lando
Windows 1-tier PGO builds only partially clobber between the
profile-generate and profile-use stages, so that exports/installed files
don't have to be reprocessed. Unfortunately we can't skip the install
manifests in 3-tier PGO because the profile-generate build happens on a
different machine, so we have to differentiate between 1-tier and 3-tier
PGO builds. A new variable, MOZ_1TIER_PGO, is used for this purpose.
Eventually this logic can be cleaned up in bug 1557788 once all PGO
builds use the 3-tier model.
Differential Revision: https://phabricator.services.mozilla.com/D34802
--HG--
extra : moz-landing-system : lando
This follows the model set down for EME artifacts:
- a new tier is added that uses a new Python build action to fetch and
artifacts
- the action unpacks the fetched artifacts and moves specific inputs
into places expected by the build and packager
- in automation, MOZ_ARTIFACT_TASK* is used to ensure the artifacts
come from the correct tasks
In this case, the artifact fetching is done entirely in a new Python
build action that internally uses `mach artifact install --job ...`.
The action also verifies that the fetched artifacts are compatible and
that we're not assembling a fat AAR that is nonsensical. The specific
inputs are not used in the Fennec APK that is produced; they're only
used in the GeckoView AAR that is produced.
The artifact fetching itself required tweaking to fetch only
`target.maven.zip` artifacts and to not unpack them.
The specific inputs used are the native libraries (libs/$ARCH/*.so)
and the architecture-specific preference files ($ARCH/greprefs.js and
defaults/pref/$ARCH/geckoview-prefs.js). None of these inputs are
impacted by l10n.
Differential Revision: https://phabricator.services.mozilla.com/D31572
--HG--
extra : moz-landing-system : lando
We're planning on switching to IR-based profiling, so we can't use the
frontend-based instrumentation to collect the order in which functions
are executed...at least not during the build itself. Performance tests
indicate that not having the order information decreases performance
significantly. So we're going to check in static files for Win32 and
Win64 and use those to perform the ordering. It's OK if these files are
slightly out of date; as of this writing, builds that generate and then
use these files complain that ~1/3 of the functions can't be found (!).
We're just trying to do something slightly smarter than whatever the
linker default is.
Differential Revision: https://phabricator.services.mozilla.com/D31132
--HG--
extra : moz-landing-system : lando
If mozbuild parsing fails due to a missing file (eg: a file not existing
in UNIFIED_SOURCES), then no Makefiles are written out, but
config.status exists. This would cause mozbuild to think that configure
doesn't need to run, and rely on make to perform the backend-out-of-date
check in rebuild-backend.mk. Unfortunately since no Makefiles were
written, the make command fails immediately and no attempt is made to
re-create the backend. Note that this is only a problem if the first
mozbuild parsing from a clobber build fails, otherwise there is
typically a top-level Makefile from a previous build to call into (at
which point make can determine it is out-of-date, and re-invoke itself).
The fix is to have the RecursiveMake backend re-use the same logic that
was introduced into mozbuild for alternate backends, and remove
rebuild-backend.mk. This way, mozbuild can always determine if the
backend needs to be regenerated, even if the initial parsing failed.
Test code was also relying on rebuild-backend.mk to generate the
TestBackend, but moving backend_out_of_date() into MozbuildObject allows
this code to be shared.
Differential Revision: https://phabricator.services.mozilla.com/D26262
--HG--
rename : build/gen_test_backend.py => python/mozbuild/mozbuild/gen_test_backend.py
extra : moz-landing-system : lando
We add to `GARBAGE_DIRS` in the toplevel `Makefile.in` because all of
our Rust libraries share a single `CARGO_TARGET_DIR`, located in
topobjdir.
We add to `GARBAGE_DIRS` for Rust programs because Rust programs
currently do not share compilation artifacts with Rust libraries (as our
libraries are built with `panic=abort` and our programs are not, sharing
compilation artifacts between the two is a non-starter).
Differential Revision: https://phabricator.services.mozilla.com/D26762
--HG--
extra : moz-landing-system : lando
If mozbuild parsing fails due to a missing file (eg: a file not existing
in UNIFIED_SOURCES), then no Makefiles are written out, but
config.status exists. This would cause mozbuild to think that configure
doesn't need to run, and rely on make to perform the backend-out-of-date
check in rebuild-backend.mk. Unfortunately since no Makefiles were
written, the make command fails immediately and no attempt is made to
re-create the backend. Note that this is only a problem if the first
mozbuild parsing from a clobber build fails, otherwise there is
typically a top-level Makefile from a previous build to call into (at
which point make can determine it is out-of-date, and re-invoke itself).
The fix is to have the RecursiveMake backend re-use the same logic that
was introduced into mozbuild for alternate backends, and remove
rebuild-backend.mk. This way, mozbuild can always determine if the
backend needs to be regenerated, even if the initial parsing failed.
Test code was also relying on rebuild-backend.mk to generate the
TestBackend, but moving backend_out_of_date() into MozbuildObject allows
this code to be shared.
Differential Revision: https://phabricator.services.mozilla.com/D26262
--HG--
rename : build/gen_test_backend.py => python/mozbuild/mozbuild/gen_test_backend.py
extra : moz-landing-system : lando
On one-go MOZ_PGO builds, it's generally not wanted to do LTO during the
profile-generate phase. And the build system doesn't really support
different build options between both phases in this case, so we relied
on MOZ_PROFILE_GENERATE to disable the LTO flags.
However, in standalone profile-generate builds, if --enable-lto is
passed explicitly, the build should respect that choice.
So instead of checking MOZ_PROFILE_GENERATE to disable the LTO flags,
we disable them when MOZ_LTO is not set, and we force it to be disabled
during the profile-generate phase of one-go MOZ_PGO builds.
Differential Revision: https://phabricator.services.mozilla.com/D21659
--HG--
extra : moz-landing-system : lando
Consequently, this removes:
- MOZ_LIBPRIO, which is now always enabled.
- non_msvc_compiler, which is now always true.
- The cl.py wrapper, since it's not used anymore.
- CL_INCLUDES_PREFIX, which was only used for the cl.py wrapper.
- NONASCII, which was only there to ensure CL_INCLUDES_PREFIX still
worked in non-ASCII cases.
This however keeps a large part of detecting and configuring for MSVC,
because we still do need it for at least headers, libraries, and midl.
Depends on D19614
Differential Revision: https://phabricator.services.mozilla.com/D19615
--HG--
extra : moz-landing-system : lando
Support for running profileserver.py multiple times was originally added
in bug 663251 and subsequently removed in bug 672563. Since then we've
been copying around the argument to profileserver.py from mozconfigs in
buildbot-configs, to mozconfigs in mozilla-central, to client.mk, and
ultimately to Makefile.in. It no longer does anything and can be
removed.
Differential Revision: https://phabricator.services.mozilla.com/D15748
--HG--
extra : moz-landing-system : lando
The test is actually redundant with the TEXTREL check in
check_binary.py (R_386_PC32 relocations will only happen as text
relocations)
Differential Revision: https://phabricator.services.mozilla.com/D16002
--HG--
extra : moz-landing-system : lando
This addresses a related issue along the way: a build that results in running
configure would not update the value of self.config_environment (and therefore
self.substs) as seen from the build driver, so out of date values would have
been used.
The changes to Makefile.in and client.mk made exploit the assumption that by
he time anything in the Make build is running, config.status is up to date.
Users running "make" without the benefit of "mach" will need to manually run
configure when necessary in order to take this into account.
Differential Revision: https://phabricator.services.mozilla.com/D8473
--HG--
extra : moz-landing-system : lando
This patch makes it so we write the sccache log directly to the artifact
directory, so that it will be uploaded even if the build fails. It also makes
the log slightly more verbose. Both of these should help with diagnosing
sccache failures in CI.
The sccache log will no longer be explicitly gzip compressed, but some
Taskcluster client implementations will store it as gzip compressed.
Differential Revision: https://phabricator.services.mozilla.com/D6187
--HG--
extra : moz-landing-system : lando
This patch moves the logic for include/exclude/compress from the makefile into
symbols_archive.py, and adds an additional check to the full-archive case to
allow the inclusion of test symbols in try builds.
--HG--
extra : amend_source : 306419aae90a1935ced49fec89112df05142501c
Now that we've generated an order file of the first N functions invoked
during startup, let's tell the linker about said functions so it can
cluster them appropriately.
test_build.py fails on local builds with messages like:
make[4]: Entering directory '/tmp/tmp65BjCH'
make[4]: *** No rule to make target 'buildid.h'. Stop.
make[4]: Leaving directory '/tmp/tmp65BjCH'
/home/froydnj/src/gecko-dev.git/config/faster/rules.mk:76: recipe for target '/tmp/tmp65BjCH/buildid.h' failed
make[3]: *** [/tmp/tmp65BjCH/buildid.h] Error 2
make[3]: *** Waiting for unfinished jobs....
make[4]: Entering directory '/tmp/tmp65BjCH'
make[4]: *** No rule to make target 'source-repo.h'. Stop.
make[4]: Leaving directory '/tmp/tmp65BjCH'
/home/froydnj/src/gecko-dev.git/config/faster/rules.mk:76: recipe for target '/tmp/tmp65BjCH/source-repo.h' failed
make[3]: *** [/tmp/tmp65BjCH/source-repo.h] Error 2
make[3]: Leaving directory '/tmp/tmp65BjCH/faster'
Makefile:155: recipe for target 'faster' failed
The tests pass in automation, however, because automation always defines
the rules for buildid.h and source-repo.h in the toplevel Makefile.in.
For local builds, however, those rules are not defined to avoid build
churn. Let's ensure that the necessary rules are defined during testing
as well.
Tup can have a lot of unnecessary log lines, particularly on a fresh
build when it is scanning directories for the first time. Turning on
--quiet when MOZ_AUTOMATION is set will help trim down the log size
significantly.
MozReview-Commit-ID: AaRppurkXsA
--HG--
extra : rebase_source : dca18a0e3657853eacfc6cdbb935998cadb339ce
For automation builds, force buildid.h to be regenerated, and always
#include buildid.h in nsAppRunner.cpp. This will eliminate the mismatch at the
cost of re-linking libxul on every build.
For developer builds, always #include buildid.h in nsAppRunner.cpp, but do not
force buildid.h to be re-generated for incremental builds. This means we will
never have a mismatch between platform and application buildids, although
subsequent builds are no longer guaranteed to have a new buildid.
MozReview-Commit-ID: 19hvu8AQXgN
--HG--
extra : rebase_source : 9f4fd063f89b9b2165d5889fdaa8f83104c4411c
Chris AtLee
Nick Alexander
Mike Hommey
Johan Lorenzo
Mike Shal
Nathan Froyd
Mike Shal
Andreea Pavel
Chris Manchester
Ted Mielczarek
Gabriele Svelto
arthur.iakab
David Major
Aaron Klotz
Nathan Froyd
Ciure Andrei
Gurzau Raul