This is strictly a quality of life improvement when cross-compiling to
Windows targets. A common scenario is mounting an object directory
into a Windows VM, which requires additional VM configuration to
handle (absolute) symlinks. With this patch, `export NSDISTMODE=copy`
in a mozconfig sidesteps any such symlink issues.
Differential Revision: https://phabricator.services.mozilla.com/D77118
We were using `config.status*` as a catchall in `DIST_GARBAGE` for
`config.status` and `config.statusd`. Unfortunately, the latter was a
directory, so `rm config.status*` error'd out. We can fix this easily
by putting `config.statusd` in `GARBAGE_DIRS`, which will use `rm -r`,
as intended.
Differential Revision: https://phabricator.services.mozilla.com/D64422
--HG--
extra : moz-landing-system : lando
One of these appears to be mistakenly leftover from bug 861178
which was intended to avoid purging dist/, but the code inside the ifndef
no longer does that.
The other is from bug 1246881 to avoid re-writing the buildid.h file
during the profile-use build, but now that there are two separate
builds, they will each have their own buildid.h / source-repo.h files.
Differential Revision: https://phabricator.services.mozilla.com/D56116
--HG--
extra : moz-landing-system : lando
MOZ_1TIER_PGO was a temporary hack to support 1-tier PGO builds while
they were being ported to 3-tier. Now that all builds are 3-tier, it can
be removed.
Differential Revision: https://phabricator.services.mozilla.com/D56112
--HG--
extra : moz-landing-system : lando
Local PGO builds now use 3-tier machinery under the hood. Instead of a
single object directory that gets cleaned in between the instrumented
and profile-use builds, now the instrumented build uses a separate
'${objdir}/instrumented' directory. This makes it easier to handle
within mach since we can drive the two builds with environment variables
and keep all build artifacts separate, without needing to do manual
cleanup in between.
Differential Revision: https://phabricator.services.mozilla.com/D50098
--HG--
extra : moz-landing-system : lando
Android artifacts (GeckoView AARs, GeckoViewExample (and Fennec) APKs)
require native libraries (`libxul.so`) and an omnijar (`omni.ja`).
These are produced by `mach package` (really, the `stage-package`
target). Engineers essentially never want a build without a package
for mobile/android. This adds mobile/android-only tiers that run
`mach package` and then `mach android assemble-app`. The latter
consumes `libxul.so` and `omni.ja` to produce _all the things_
relevant to GeckoView engineers.
Differential Revision: https://phabricator.services.mozilla.com/D41450
--HG--
extra : moz-landing-system : lando
Bug 1563711 changed MOZ_FETCHES_DIR to make the MOZ_ANDROID_FAT_AAR_*
environment variables absolute paths. Unfortunately, the replacement
relies on non-osx/windows tasks to run in docker-worker, which is not
necessarily true, and that makes the MOZ_FETCHES_DIR wrong in
non-osx/windows generic-worker tasks. Apparently, that currently works,
but that's not guaranteed to stay this way.
The MOZ_ANDROID_FAT_AAR_* environment variables don't need to be
absolute paths, though. MOZ_FETCHES_DIR is normalized by run-task, and
MOZ_ANDROID_FAT_AAR_* can be set relative to that, which we do here.
Differential Revision: https://phabricator.services.mozilla.com/D40334
At the same time, make things a little more tangible in the tiers setup
in baseconfig.mk.
Differential Revision: https://phabricator.services.mozilla.com/D37942
--HG--
extra : moz-landing-system : lando
Windows 1-tier PGO builds only partially clobber between the
profile-generate and profile-use stages, so that exports/installed files
don't have to be reprocessed. Unfortunately we can't skip the install
manifests in 3-tier PGO because the profile-generate build happens on a
different machine, so we have to differentiate between 1-tier and 3-tier
PGO builds. A new variable, MOZ_1TIER_PGO, is used for this purpose.
Eventually this logic can be cleaned up in bug 1557788 once all PGO
builds use the 3-tier model.
Differential Revision: https://phabricator.services.mozilla.com/D34802
--HG--
extra : moz-landing-system : lando
This follows the model set down for EME artifacts:
- a new tier is added that uses a new Python build action to fetch and
artifacts
- the action unpacks the fetched artifacts and moves specific inputs
into places expected by the build and packager
- in automation, MOZ_ARTIFACT_TASK* is used to ensure the artifacts
come from the correct tasks
In this case, the artifact fetching is done entirely in a new Python
build action that internally uses `mach artifact install --job ...`.
The action also verifies that the fetched artifacts are compatible and
that we're not assembling a fat AAR that is nonsensical. The specific
inputs are not used in the Fennec APK that is produced; they're only
used in the GeckoView AAR that is produced.
The artifact fetching itself required tweaking to fetch only
`target.maven.zip` artifacts and to not unpack them.
The specific inputs used are the native libraries (libs/$ARCH/*.so)
and the architecture-specific preference files ($ARCH/greprefs.js and
defaults/pref/$ARCH/geckoview-prefs.js). None of these inputs are
impacted by l10n.
Differential Revision: https://phabricator.services.mozilla.com/D31572
--HG--
extra : moz-landing-system : lando
We're planning on switching to IR-based profiling, so we can't use the
frontend-based instrumentation to collect the order in which functions
are executed...at least not during the build itself. Performance tests
indicate that not having the order information decreases performance
significantly. So we're going to check in static files for Win32 and
Win64 and use those to perform the ordering. It's OK if these files are
slightly out of date; as of this writing, builds that generate and then
use these files complain that ~1/3 of the functions can't be found (!).
We're just trying to do something slightly smarter than whatever the
linker default is.
Differential Revision: https://phabricator.services.mozilla.com/D31132
--HG--
extra : moz-landing-system : lando
If mozbuild parsing fails due to a missing file (eg: a file not existing
in UNIFIED_SOURCES), then no Makefiles are written out, but
config.status exists. This would cause mozbuild to think that configure
doesn't need to run, and rely on make to perform the backend-out-of-date
check in rebuild-backend.mk. Unfortunately since no Makefiles were
written, the make command fails immediately and no attempt is made to
re-create the backend. Note that this is only a problem if the first
mozbuild parsing from a clobber build fails, otherwise there is
typically a top-level Makefile from a previous build to call into (at
which point make can determine it is out-of-date, and re-invoke itself).
The fix is to have the RecursiveMake backend re-use the same logic that
was introduced into mozbuild for alternate backends, and remove
rebuild-backend.mk. This way, mozbuild can always determine if the
backend needs to be regenerated, even if the initial parsing failed.
Test code was also relying on rebuild-backend.mk to generate the
TestBackend, but moving backend_out_of_date() into MozbuildObject allows
this code to be shared.
Differential Revision: https://phabricator.services.mozilla.com/D26262
--HG--
rename : build/gen_test_backend.py => python/mozbuild/mozbuild/gen_test_backend.py
extra : moz-landing-system : lando
We add to `GARBAGE_DIRS` in the toplevel `Makefile.in` because all of
our Rust libraries share a single `CARGO_TARGET_DIR`, located in
topobjdir.
We add to `GARBAGE_DIRS` for Rust programs because Rust programs
currently do not share compilation artifacts with Rust libraries (as our
libraries are built with `panic=abort` and our programs are not, sharing
compilation artifacts between the two is a non-starter).
Differential Revision: https://phabricator.services.mozilla.com/D26762
--HG--
extra : moz-landing-system : lando
If mozbuild parsing fails due to a missing file (eg: a file not existing
in UNIFIED_SOURCES), then no Makefiles are written out, but
config.status exists. This would cause mozbuild to think that configure
doesn't need to run, and rely on make to perform the backend-out-of-date
check in rebuild-backend.mk. Unfortunately since no Makefiles were
written, the make command fails immediately and no attempt is made to
re-create the backend. Note that this is only a problem if the first
mozbuild parsing from a clobber build fails, otherwise there is
typically a top-level Makefile from a previous build to call into (at
which point make can determine it is out-of-date, and re-invoke itself).
The fix is to have the RecursiveMake backend re-use the same logic that
was introduced into mozbuild for alternate backends, and remove
rebuild-backend.mk. This way, mozbuild can always determine if the
backend needs to be regenerated, even if the initial parsing failed.
Test code was also relying on rebuild-backend.mk to generate the
TestBackend, but moving backend_out_of_date() into MozbuildObject allows
this code to be shared.
Differential Revision: https://phabricator.services.mozilla.com/D26262
--HG--
rename : build/gen_test_backend.py => python/mozbuild/mozbuild/gen_test_backend.py
extra : moz-landing-system : lando
On one-go MOZ_PGO builds, it's generally not wanted to do LTO during the
profile-generate phase. And the build system doesn't really support
different build options between both phases in this case, so we relied
on MOZ_PROFILE_GENERATE to disable the LTO flags.
However, in standalone profile-generate builds, if --enable-lto is
passed explicitly, the build should respect that choice.
So instead of checking MOZ_PROFILE_GENERATE to disable the LTO flags,
we disable them when MOZ_LTO is not set, and we force it to be disabled
during the profile-generate phase of one-go MOZ_PGO builds.
Differential Revision: https://phabricator.services.mozilla.com/D21659
--HG--
extra : moz-landing-system : lando
Consequently, this removes:
- MOZ_LIBPRIO, which is now always enabled.
- non_msvc_compiler, which is now always true.
- The cl.py wrapper, since it's not used anymore.
- CL_INCLUDES_PREFIX, which was only used for the cl.py wrapper.
- NONASCII, which was only there to ensure CL_INCLUDES_PREFIX still
worked in non-ASCII cases.
This however keeps a large part of detecting and configuring for MSVC,
because we still do need it for at least headers, libraries, and midl.
Depends on D19614
Differential Revision: https://phabricator.services.mozilla.com/D19615
--HG--
extra : moz-landing-system : lando
Support for running profileserver.py multiple times was originally added
in bug 663251 and subsequently removed in bug 672563. Since then we've
been copying around the argument to profileserver.py from mozconfigs in
buildbot-configs, to mozconfigs in mozilla-central, to client.mk, and
ultimately to Makefile.in. It no longer does anything and can be
removed.
Differential Revision: https://phabricator.services.mozilla.com/D15748
--HG--
extra : moz-landing-system : lando
The test is actually redundant with the TEXTREL check in
check_binary.py (R_386_PC32 relocations will only happen as text
relocations)
Differential Revision: https://phabricator.services.mozilla.com/D16002
--HG--
extra : moz-landing-system : lando
This addresses a related issue along the way: a build that results in running
configure would not update the value of self.config_environment (and therefore
self.substs) as seen from the build driver, so out of date values would have
been used.
The changes to Makefile.in and client.mk made exploit the assumption that by
he time anything in the Make build is running, config.status is up to date.
Users running "make" without the benefit of "mach" will need to manually run
configure when necessary in order to take this into account.
Differential Revision: https://phabricator.services.mozilla.com/D8473
--HG--
extra : moz-landing-system : lando
This patch makes it so we write the sccache log directly to the artifact
directory, so that it will be uploaded even if the build fails. It also makes
the log slightly more verbose. Both of these should help with diagnosing
sccache failures in CI.
The sccache log will no longer be explicitly gzip compressed, but some
Taskcluster client implementations will store it as gzip compressed.
Differential Revision: https://phabricator.services.mozilla.com/D6187
--HG--
extra : moz-landing-system : lando
Coroiu Cristina
Nick Alexander
Ricky Stewart
Bogdan Tara
Mike Hommey
Andrea Marchesini
Mike Shal
Daniel Varga
Nathan Froyd
Chris AtLee
Johan Lorenzo
Mike Shal
Andreea Pavel
Chris Manchester
Ted Mielczarek
Gabriele Svelto
arthur.iakab
David Major