I considered three ways to do this:
* one, as a Python script executed with $(shell);
* two, as a Python script that writes an include file for the preprocessor;
* three, as a function exposed to the moz.build sandbox.
I rejected two because it's both tied to the preprocessor, and awkward
to make handle the dependency on the buildid (in a file) and
additional build defines (in config.status).
I rejected three because I know of no precedent for this approach, and
it hides the dependency on the buildid.
One doesn't handle failures in the script gracefully, but neither did
the existing approach. This patch is at least testable.
--HG--
extra : commitid : 8dfw1ri7qjr
extra : rebase_source : da0e5ec705e0ac4c795bd2d7892f73857a1699ac
DONTBUILD NPOTB
This was just an oversight during the initial landing, leading to two
copies of artifact libraries being appended to the same destination
file.
--HG--
extra : commitid : JvvZDrjUOQZ
extra : rebase_source : f48c1a9d4b506b6ed931043aeeaca437418ea9c3
extra : histedit_source : 3dbfd049039c0adc595c1abb0df3dca3af9db207
This implements a new "scan" mode for DMD that records the address
and contents of every live unsampled block in the DMD log. This
enables the low-level analysis of references from one block to
another, which can help leak investigations.
bf34d16b6ab2 added absolute_import to this file. When changed, "import
mozinfo" stopped picking up mozbuild.mozinfo and started importing
mozinfo instead.
Use relative imports to force mozbuild.mozinfo to be picked up.
--HG--
extra : commitid : 4GC5lJWrBFq
extra : rebase_source : c91674b16f29dadf8dcda460787ea94654f2864a
This import brought in significant parts of the mozbuild package. Moving
it to a deferred import reduces the total number of Python modules
imported during mach dispatch by 43.
--HG--
extra : commitid : GdXsF7agvCT
extra : rebase_source : 586f1960c8e7eb400f61467045b064167784f68b
This removes ambiguity as to which modules are being imported, making
import slightly faster as Python doesn't need to test so many
directories for file presence.
All files should already be using absolute imports because mach command
modules aren't imported to the package they belong to: they instead
belong to the "mach" package. So relative imports shouldn't have been
used.
--HG--
extra : commitid : 6tFME1KKfTD
extra : rebase_source : 78728f82f5487281620e00c2a8004cd5e1968087
DONTBUILD ON A CLOSED TREE: Android-only and the build changes are cosmetic.
Very much a first cut, but I'd like to get some Fennec early adopters testing.
This adds:
* |mach artifact install| to fetch and install Fennec binaries;
* |mach artifact last| to print details about what was last installed;
* |mach artifact {print,clear}-caches|, for debugging.
This code is exposed as a new mozbuild.artifacts Python but it's not
particularly general. My intention was to get things out of the mach command
more than produce a general artifact fetching API. We can leave that bike
shed to Bug 1124378.
I've been testing this with --disable-compile-environment and it works well
locally, although there's no reason a knowledgeable developer couldn't use
this in conjunction with a fully-built tree. (I don't know when such a
situation would arise, but I know of no technical impediment.)
--HG--
extra : commitid : 1T28aVfArqF
extra : rebase_source : b8c11244de8be0a14d605853f30cd47312d0a4ba
extra : histedit_source : 78a224501cd3cf0f86707c9c9549b61b4b248ba7
We want this logic to live next to where metadata types are defined so
downstream consumers of Files-based metadata don't have to reinvent the
wheel.
The work in this commit will be used to enable auto filing bugs during
pushes to MozReview.
--HG--
extra : commitid : F55RzhDCVmR
extra : rebase_source : 1fe1a03f8d592c60d2ed8d760fd62c7bc60421a5
This file hasn't been updated in ages and the current configuration
doesn't produce working packages. Change that.
--HG--
extra : commitid : AfiwohniLPq
extra : rebase_source : 70786c7bd9e37c9acc9b38f54c956fec424cfcf5
extra : amend_source : e4e3752dc97b934456b973736ef55633366c8ee6
`mach help <command>` currently only displays a brief description of the
command along with its arguments. Sometimes more detailed help text is
needed.
With this commit, the docstrings of mach command handlers will appear in
the output of `mach help <command>` if they are defined.
I've implemented basic docstrings for the three flavors of mach commands
(normal command, main subcommand, subcommand) to demonstate things work.
My hope is others will start to fill in docstrings once this feature
lands so the output for `mach help` can serve as a better learning guide
for new contributors.
--HG--
extra : commitid : Hx6ZkHDxbCK
extra : rebase_source : 01ced5a044442e370a45cd3fb245ac6283316925
extra : amend_source : fceb771e0e1ffa4e6f3f1b7c22eae6e25cf82034
The hglib Mercurial finder was nice. But it is somewhat slow, as it
involves a separate Mercurial process.
This commit introduces a native Mercurial finder that speaks directly
to a Mercurial repository instance. It is significantly faster.
The new code is isolated to its own file because it imports Mercurial
code, which is GPL.
--HG--
extra : commitid : 8CzDFt3lQmx
extra : rebase_source : 1540ad20d795086926b316612062e2a1f10c4958
This commit adds support for specifying a Mercurial revision with `mach
file-info`. Aside from being a potentially useful feature, it proves
that MercurialRevisionFinder works with BuildReader.
--HG--
extra : commitid : 7143XT9ENqb
extra : rebase_source : d8c0c98d536d07db76653b648fd4b7d74d8f43ae
The moz.build reader uses absolute paths when referencing moz.build
files. *Finder classes expect paths to be relative to a base. When we
switched the reader to use *Finder instances for I/O, we simply provided
a default FileFinder based at the filesystem root. This was quick and
easy. Things worked.
Unfortunately, that solution isn't ideal because not all *Finder
instances can accept absolute paths like that. The
MercurialRevisionFinder is one of them.
Changing the moz.build reader to talk in terms of relative paths is a
lot of work. While this would be ideal, it is significantly easier to
defer the problem until later and hack up MercurialRevisionFinder to
accept absolute paths. This commit does exactly that.
Bug 1171069 has been filed to track converting BuildReader to relative
paths.
--HG--
extra : commitid : 2JmzOBldBLy
extra : rebase_source : a8af6ce88dd9e2b98f131c92b45c3ece852e13d2
Now that moz.build files use finders for I/O, we can start reading
moz.build data from other sources.
Version control is essentially a filesystem. We implement a finder
that speaks to Mercurial to obtain file data. It is able to obtain
file data from a specific revision in the repository.
We use the hglib package (which uses the Mercurial command server) for
speaking with Mercurial. This adds overhead compared to consuming the
raw Mercurial APIs. However, it also avoids GPL side-effects of
importing Mercurial's Python modules.
Testing shows that performance is good but not great. A follow-up
commit will introduce a GPL licensed Mercurial finder. For now, get
the base functionality in place.
--HG--
extra : commitid : BkwaQOW9MiR
extra : rebase_source : 915d6015317ccc79c228a76eed861d9f43e2fd17
Sometimes moz.build data may not come from the local filesystem. To
support defining moz.build data in other backing stores, we switch
moz.build reading to use mozpack's *Finder classes for I/O.
The use of a FileFinder bound to / is sub-optimal. We should ideally
be creating a finder bound to topsrcdir. However, doing this would
require refactoring a lot of path handling in the reader, as that code
makes many assumptions that paths are absolute. This would be excellent
follow-up work.
While I was here, I cleaned up some linter warnings for unused imports.
On my machine, this commit results in a slight slowdown of moz.build
reading. Before, `mach build-backend` was consistently reporting 1.99s
for reading 2,572 moz.build files. After, it is consistently reporting
2.07s, an increase of 4%. This is likely due to a) function call
overhead b) the cost of instantiating a few thousand File instances
c) FileFinder performing an os.path.exists() before returning File
instances. I believe the regression is tolerable.
--HG--
extra : commitid : 1WDcrSU3SQD
extra : rebase_source : a23aa5a4063c6f7080ee00b6f0fe0ee532da3ce7
Passing raw file handles around is a bit dangerous because it could lead
to leaking file descriptors. Add a read() method that handles the simple
case of obtaining the full contents of a File instance.
This should ideally be a method on BaseFile. But this would require
extra work and isn't needed. So we've deferred it until bug 1170329.
--HG--
extra : commitid : 82qw76XmpjC
extra : rebase_source : 422b16c5a3b1577f080097925aeaeb560aa3e798
Today, the *Finder classes are optimized for doing matching and
returning multiple results. However, sometimes only looking for a
single file is wanted.
This commit implements the "get" method on all the *Finder classes.
It returns a BaseFile or None.
FileFinder._find_file has been refactored into FileFinder.get
to avoid code duplication.
--HG--
extra : commitid : K9It2ZJ3Rbo
extra : rebase_source : a56f8f70aa1902d26373a7196eae2847cce653d3
The current mode of execution of templates doesn't allow them to more advanced
control flow, like returning early, returning or yielding values, because that
mode of execution is equivalent to running the code at the top level of a .py
file.
Making the templates executed through a function call, although trickier,
allows those control flows, which will be useful for template as context
managers.
inspect.getsourcelines() and inspect.getfile() involve I/O out of our control.
Our use of those functions, however, doesn't require all their smarts. In fact,
we only use them on function objects, for which we can just do the work
ourselves without involving inspect functions that trigger I/O.
The old way of writing scripts for generated files would invoke the script thusly:
python script.py arg1...
Invoking the script this way means that the script's directory is
automatically added to sys.path, and importing modules from that
directory is easy. Let's make it equally easy in the new world for
GENERATED_FILES, too.
Add generic support for different forms of paths in moz.build:
'/topsrcdir/relative/paths'
'srcdir/relative/paths'
'!/topobjdir/relative/paths'
'!objdir/relative/paths'
This drops the use of UserString for performance reasons, which required
going around with a meta class.
Jonathan Griffin
Ehsan Akhgari
Ryan VanderMeulen
Peter Elmers
Carsten "Tomcat" Book
Nick Alexander
Mike Hommey
Julien Pagès
Nicholas Hurley
Edmund Wong
Ted Mielczarek
Wes Kocher
Andrew McCreight
Gregory Szorc
Birunthan Mohanathas
Gijs Kruitbosch
Nathan Froyd
James Graham
Andrew Halberstadt