Listing the tests in all-tests.json is prerequisite to adding support
for Marionette tests to `./mach test FILE`.
Marionette tests will be run from the source directory, as they do not
currently need packaging.
--HG--
extra : rebase_source : 41b80531b671cb2f4cca4337c41ba94ab189f8c0
The current rule is only for "backend.RecursiveMakeBackend", but, with
the current default of generating both the RecursiveMake and FasterMake
backends, the command creates/refreshes both backends. This is, in fact,
how the FasterMake backend is refreshed in most cases.
Moreover, with an hybrid backends, the generated file is not
"backend.RecursiveMakeBackend" anymore, so we need a more generic way to
handle this.
Furthermore, it's not necessarily desirable for all backends to have a
dependency file to handle the dependencies to refresh the backend, so
generate a plain list instead. This has the side effect of making `mach
build-backend --diff` more readable for changes to that file.
Finally, make the backend.* files created like any other backend file,
such that its diff appears in the `mach build-backend --diff` output.
The FasterMake build system is meant to be invoked through `mach build
faster`, which does it already, or, in the near future, as part of an
hybrid build system, which will deal with it as well. People doing
`make -C objdir/faster` won't have the backend automatically refreshed,
but that's not a supported way to use it anyways.
Install manifests are not empty in normal conditions, apart a few
exceptions where they are only used for a "magic" `rm -rf`.
However, we're going to introduce changes that will empty some of
the install manifests and make their work happen from a different
backend, in which case we don't want them to correspond to a `rm -rf`.
When doing build system changes affecting backend-generated files, I
often use `mach build-backend --diff`. But most of the time I end up
wanting to look at the full diff again when doing further changes, which
leads me to stash my changes away, run `mach build-backend` to get the
initial state again, unstash and rerun `mach build-backend --diff`.
This has been a time drain for long enough :)
The code doing this was added before we had install manifests, back when
they were purge manifests, and before we tracked all files created by
the backend. Nowadays, if an install manifest is removed, it will be
removed in BuildBackend.consume.
In fact, purging the install manifests in the backend itself breaks the
deleted files count displayed after reticunating splines.
XPI_NAME affects no tier on its own, as it merely changes paths where
things end up that are defined by other variables.
FILES_PER_UNIFIED_FILE had an erroneous value.
Those are the worst offenders of affected_tiers. The rules to handle
them are all in directories that is not necessarily related to where
the variables are defined, each of which has a Makefile.in for those
rules, which forces export to go through them.
They are respectively using PP_TARGETS and INSTALL_TARGETS. Both affect
the misc tier since bug 1240671, per the *_TARGET value they set in the
backend.
This has the nice side effect of now skipping directories where test
harness files are handled by install manifests.
JAR_MANIFESTS affects the libs tiers through config/rules.mk rules.
While we could move the rules in the backend, they are too complex to
just do that now.
GENERATED_FILES impacts the export tier through the config/rules.mk
definitions, now moved to the backend itself, so that everything is
close to each other.
Those are non-passthru variables with the same property as
ANDROID_GENERATED_RESFILES, ANDROID_APK_NAME and ANDROID_APK_PACKAGE, in
that they don't affect tiers through the backend code itself, but from
the Makefile.in along the moz.build they are defined in.
The are passthru variables that don't actually affect any tier per the
backend itself. They do affect the `export` tier by way of the Makefile.in
along the moz.build defining them, and the existence of those
Makefile.in already guarantees those directories not to be skipped for
`export`.
This initiates a move off affected_tiers in VARIABLES definition to
explicit in-backend handling, which will hopfully make things clearer.
HAS_MISC_RULE is currently used to opt-in to the misc tier in a few
directories with a misc:: rule. It is, in fact, mostly used for custom
xpi creation, which will be separately addressed in bug 1240676, so it
will eventually go away entirely, but in the meantime, we send it as a
throwaway passthru.
Historically, all tiers were handled as opt-out (may_skip), until we
added the first opt-in tier (no_skip). It doesn't make much sense to
differentiate them anymore, so handle them all as opt-in.
When adding a backend, we currently have to add them in three different
places so that they appear in the right places.
Instead, keep the list in a single place.
Currently |./mach artifact| installs an old version of the taskcluster client,
which installs an old version of requests that's incompatible with commonly
installed python versions. This bumps to a version of taskcluster client that
accepts and requests version < 3, so we pick up the in-tree version instead of
installing 2.4.3.
--HG--
extra : rebase_source : 7b4f450c0492fd3c840e4f0a0cce8b42d120df9f
generate_symbols_file only supports the global defines, and completely
ignores DEFINES from the same moz.build the SYMBOLS_FILE is defined.
This fixes this misfeature.
The result of running this backend is a json file containing an array with 3 elements:
a map from resource:// and chrome:// url prefixes to corresponding objdir directories,
a map of overrides, and a map from objdir files to srcdir files (with a flag for whether
the file was preprocessed) for js file in FinalTargetFiles. A subsequent commit implements
the procedure to find an replace based on the url mapping, subsistute based on overrides,
and find sourcedir files based on objdir files.
--HG--
extra : commitid : Di6Ro5c9sTK
A recent change regressed this behavior -- while an artifact build runs, it
doesn't load certain "about:" pages due to missing libraries in subdirectories
of dist/bin.
--HG--
extra : commitid : HOPgt9kMKoV
AFAICT, we don't actually access web-platform test files from the
objdir for anything except test packaging. And we already have
a mechanism for creating test archives from files directly in the
source directory. So, let's stop copying them to the objdir and
package them directly from the source directory!
The _tests install manifest reports the following change:
before: 41,977 files installed
after: 24,537 files installed
delta: -17,440 files
We still copy some WPT files to the objdir. We might be able to
eliminate these as well. However, since there are only ~200 files,
I'm not too concerned.
I manually compared the resulting web-platform zip archives from before
and after. No files were removed from the archive. However, the new
archive does gain several hundred empty directories with .gitkeep
files. This feels weird, but it shouldn't break anything (I would
think). I'm inclined to leave them for now. I'll file a follow-up
bug to deal with them (preferably by removing them from version
control).
--HG--
extra : rebase_source : d350f8fc223982c8a11a9bf542411e5ec5e44244
The faster make backend cannot support such things, so it's better to
avoid unsupported things to slip in because it happens that doesn't
break what automation runs.
Right now, each incremental build produces a new set of GeckoView
library files, but the previous files don't get cleaned up, and you end
up with a bunch of old libraries in your objdir. This patch cleans up
the old files before producing new ones.
Collecting the list of object files compiled, while not ideal, will give us
some indication of how much work was involved in the build. This will help
with analyzing the data.
--HG--
extra : rebase_source : e9861ed5c0766e3ee8038dbec0b9267022c523eb
This adds metadata about the local system to resource_usage.json that will
be useful for analysis when we start collecting these results. Some of it
is redundant with data collected for individual tiers, but having it
available at a top level should make analysis easier.
--HG--
extra : rebase_source : 43b2f6335a4de9c7843f75a3461d2871a97b7752
This moves monitor start/end recording so it also occurs for "what"
builds rather than only for full builds.
--HG--
extra : rebase_source : 833c7311ca5ecfe6935e0a160da695acaecbdeeb
Pushing on a CLOSED TREE after network errors.
--HG--
extra : commitid : 40hmCLC8zZJ
extra : rebase_source : 44a623fdf7cb97bfaca76617062d57c6b50fd970
extra : amend_source : 710db8302b5bf88d6bf293d71f3cc001187b6015
This mounts the downloaded DMG and copies a subset of the libraries
into the correct places in the processed archive. They'll be
installed, with paths, into dist/bin from there.
--HG--
extra : commitid : 9BXzXzIlh3I
extra : rebase_source : 29ddbe8348f27fff5205da6c40839a04d950fe0a
For Android, this just copies .so files from the APK assets/ and libs/
directories into the root, which ends up in dist/bin.
I include .ini files as a partial replacement for having the build ID
in the downloaded file name, and then don't put the .ini files in
dist/bin.
--HG--
extra : commitid : 9uLXEzBNBdQ
extra : rebase_source : 53ba0e8a7740c2a6eb48e2772f7e12ff9157d21a
Sadly, it's slow to extract the build ID from Mac OS X DMG artifacts.
It's better to sacrifice human-legible names in order to know the
final name for an artifact quickly.
--HG--
extra : commitid : I99dtzv1O2N
extra : rebase_source : c164be1314b3132fba2e1fc86cda65b12b4baa1f
It turns out to be much easier to hook |mach artifact install| into
config.status and |mach build| than to hook into client.mk.
The additional virtualenv package avoids an import error when running
|mach artifact install|.
--HG--
extra : commitid : EnfWU0uyRfQ
extra : rebase_source : f7d11fc4c542f9798712c013c4319d92d40c28e5
Limit ourselves to include paths for now, because there are tricky things
involved in making this globally.
While here, use shell_quote instead of manual quoting for those paths.
With all include flags now using absolute paths, there is no need to try
to post-process them when getting them for CompileDB and codecomplete.
As a matter of fact, doing so fixes the flags in media/gmp-clearkey/0.1,
since they use literal "-include stdio.h", which was wrongly transformed
into "-include $objdir/media/gmp-clearkey/0.1/stdio.h".
MOZ_DEBUG_DEFINES are essentially defines used everywhere. So treat them as
feeding the initial value for DEFINES in each moz.build sandbox. This allows
the kind overrides that was done in the past by resetting MOZ_DEBUG_DEFINES
in Makefiles.
Currently, one needs to define DEFFILE or LD_VERSION_SCRIPT appropriately,
and somehow deal with the fact that their input format is different, which
currently relies on manual invocations of the convert_def_file script, with
awkward aggregations.
This simplifies the problem by using a simple list of symbols, with
preprocessing, allowing #includes.
We want to move it to CommonBackend, so it's better to make it more
independent, which the Defines instances now attached to ContextDerived
instances allow.
Syd Polk
Andreas Tolfsen
Gregory Szorc
Mike Hommey
Jacek Caban
Chris Manchester
Gijs Kruitbosch
Gregory Szorc
Carsten "Tomcat" Book
Jim Chen
Nick Alexander
Dan Minor
Bryce Van Dyk
Richard Newman
Henrik Skupin
Ryan VanderMeulen
Nigel Babu
Shane Caraveo
