Since MozbuildObject.from_environment() reads from mozinfo.json, tup
picks up that file as a dependency for anything that imports buildconfig
(eg: all generated files). Using FileAvoidWrite when creating
mozinfo.json will help avoid unnecessary work after re-running configure
in the tup backend.
MozReview-Commit-ID: EEOPQYJA1MV
--HG--
extra : rebase_source : 136a0579090776dc55ea5cee870574f13cc27c58
The build system knows at build-backend time where to find each IDL
file; making xpidl-process.py rediscover this by requiring
xpidl-process.py to search through directories to find input IDL files
is silly. To rememdy this, we're going to modify things so full paths
are passed into the script. Those paths can then be used directly, with
no searching.
The previous patch required us to pass a single -I argument pointing at
$(DIST)/idl so IDL include statements would work correctly. This patch
lifts that limitation and explicitly points xpidl-process.py at the
locations of all the IDL source directories to search for included IDL
files. Invocations of xpidl-process.py no longer depend on IDL files
being copied to the objdir.
Building on the last patch, we can change the build process to pass in
the directories where the input IDL files can be found. It is
convenient to pass in just the relative source directory paths, to
encourage people to not look in the object directory and to make the
command lines slightly shorter.
xpidl-process.py still assumes that included IDL files can be found by
looking in a single directory. We add a single -I argument to the
invocation of xpidl-process.py to accommodate this short-sightedness.
The current IDL build setup assumes that all IDL files can be found in a
single directory. This setup requires that all IDL files be copied to a
single directory, which is suboptimal in terms of disk I/O and also
complicates things like generating IDL files at build time.
As a first step in moving away from this state of affairs,
xpidl-process.py needs to be taught that the input IDL files could
potentially be found in multiple directories. The current setup can
just specify $(DIST)/idl as the lone directory to examine. Future
patches will change this to examine multiple directories.
This method is only called in one place, and it doesn't pass
allow_existing. Whatever ugly thing this keyword was working around
doesn't exist anymore, so let's get rid of it.
We no longer want to update mtimes of FileAvoidWrites so that downstream
rules aren't triggered if the files aren't changed. Since the .stub file
target of GENERATED_FILES are always touched, make won't continually
rebuild them.
MozReview-Commit-ID: GxrFgCJTYk
--HG--
extra : rebase_source : f4412af1dc29142b76f7695627ba3354baf84edd
The make backend was treating the first output of a GENERATED_FILES rule
specially, since it was the target of the rule containing the script
invocation. We want the outputs of GENERATED_FILES rules to be
FileAvoidWrite so that we avoid triggering downstream rules if the
outputs are unchanged, but if the target of the script invocation is
FileAvoidWrite, then make may continually re-run the script during a
no-op build.
The solution here is to use a stub file as the target of the script
invocation which will always be touched when the script runs. Since
nothing else in the build depends on the stub, we don't need to
FileAvoidWrite it. All actual outputs of the script can be
FileAvoidWrite, and make can properly avoid work for files that haven't
changed.
MozReview-Commit-ID: 3GejZw2tpqu
--HG--
extra : rebase_source : 2b9be82f893e89a4c2f254f05b1e8b9a0f9c631b
Some GENERATED_FILES entries don't have .scripts associated with them
(notably midl on Windows builds). In this case, we don't want to
generate dependencies automatically since they will be handled by the
Makefiles.
MozReview-Commit-ID: AXmN2Unk9AY
--HG--
extra : rebase_source : 1f06672add87c46ae199189fcae27b721e008f9e
Some commands produce a large number of output files, such as
make-system-wrappers.py, which has over 1000 outputs. The GeneratedFile
handler in the tup backend displayed all the outputs, which makes the
build output unreadable, and breaks 'tup graph'. This patch displays
only the first 3 outputs and truncates the rest.
MozReview-Commit-ID: 5AnrmMe0Nyx
--HG--
extra : rebase_source : 1a6766be36aef4603c1e5333cfc13af006369966
Files returned from version control (i.e via --outgoing or --workdir), are currently joined to
cwd. This will cause failures if |mach lint| is run from anywhere other than topsrcdir.
However we *do* want to join manually specified paths to cwd so things like:
cd devtools && mach lint client
continue to work. This patch makes sure we join the proper kind of path to the proper place.
MozReview-Commit-ID: EQmRhAr3Oog
--HG--
extra : rebase_source : 2629cc27f79059e44369d46d4f8278f83923582c
This gets raptor to use the newly created "perf" profile that talos
also uses. There is a single pref that raptor sets that we can't set
in talos. To that end, this also creates a "raptor" specific profile.
This means to set a pref in talos and raptor, edit:
testing/profiles/perf/user.js
To set a pref in raptor only, edit:
testing/profiles/raptor/user.js
The performance of extensions can now be tested by dropping the
extension into:
testing/profiles/perf/extensions
MozReview-Commit-ID: LEJeytmmiFF
--HG--
extra : rebase_source : 0d2a6b18868f8cc6ff198ef868ad0324b57b1dc2
This moves all of the global prefs that were previously defined
in testing/talos/talos/config.py, into a new "perf" profile under
testing/profiles/perf/user.js.
This perf profile will be shared with raptor, so changes to one
framework will result in changes to the other.
MozReview-Commit-ID: JRxZEDlPu6b
--HG--
extra : rebase_source : 38f61eb6f9dd3e8dd9e0425ffe32dbdf845fcf65
The current mechanism for reading SPHINX variables assumes we always want to
read metadata for the entire tree. Now that we have the ability to rebuild
specific subtrees, this assumption is false.
This patch allows us to specify a path that find_sphinx_variables can use to
filter down the set of moz.build variables it will traverse, yielding only
moz.builds that could potentially impact the specified path.
MozReview-Commit-ID: ALrCFLFgMLH
--HG--
extra : source : 22f2dc60e6d859d3ca411826c77002d87c1a49bd
In the mozbuild.sphinx extension, we create a new SphinxManager instance each
time. However this isn't ideal now that we can rebuild the docs within the same
interpreter using the livereload server.
This makes use of a singleton so that we can share state not only between
multiple invocations of sphinx-build, but also with the mach command. This will
be taken advantage of more heavily in future commits in this series.
MozReview-Commit-ID: 7ERYeN5BPeI
--HG--
extra : source : 8309212d820bcca29aa95b7892d39940437f2aa8
The current mechanism for reading SPHINX variables assumes we always want to
read metadata for the entire tree. Now that we have the ability to rebuild
specific subtrees, this assumption is false.
This patch allows us to specify a path that find_sphinx_variables can use to
filter down the set of moz.build variables it will traverse, yielding only
moz.builds that could potentially impact the specified path.
MozReview-Commit-ID: ALrCFLFgMLH
--HG--
extra : rebase_source : d0c26a006bb4dbc429be5eedad7825d4412dc2a4
In the mozbuild.sphinx extension, we create a new SphinxManager instance each
time. However this isn't ideal now that we can rebuild the docs within the same
interpreter using the livereload server.
This makes use of a singleton so that we can share state not only between
multiple invocations of sphinx-build, but also with the mach command. This will
be taken advantage of more heavily in future commits in this series.
MozReview-Commit-ID: 7ERYeN5BPeI
--HG--
extra : rebase_source : 44aee637ea9b828b43b82e8639ddc3cc7f68c797
Unfortunately, FileAvoidWrite causes us to re-generate .xpt files every time we
build if a dependency of the .xpt files doesn't affect their output.
The easiest solution is to stop performing this option until we get a better
build backend like `tup` which can handle problems like this.
This works around a situation observed with old hg versions (hg 4.2?)
with mozilla-unified. I don't know why we haven't witnessed it more
generally, since the sort order was textual and should have caused
issues.
MozReview-Commit-ID: DBtfRJ3NJGR
--HG--
extra : rebase_source : b8605e34341e2c3a40f424688ecef1dbac4dc58e
Enabling this flag on rules that produce many small outputs (especially
header files that may trigger many other rules) can help reduce
incremental build time. It probably doesn't make sense to have this
enabled for .o files and linker rules, because those are less likely to
be unchanged when an input file changes, and are also larger so the time
to diff them can be more significant.
MozReview-Commit-ID: BbJaMCqPU6z
--HG--
extra : rebase_source : 31c8be252c26d3aa85e4390cf27f3ec027e88c00
We bump the Mercurial version after a new Mercurial release. 4.6 was
just released. So...
MozReview-Commit-ID: LQ49eVCDuGG
--HG--
extra : rebase_source : 6b213a62216d1b8a9ec4f303d05d01e0609734a1
Andreea Pavel
Chris AtLee
Michael Kaply
Nathan Froyd
Mike Shal
Dorel Luca
Andrew Halberstadt
Myk Melez
Andi-Bogdan Postelnicu
Francesco Lodolo (:flod)
Sylvestre Ledru
Coroiu Cristina
Csoregi Natalia
Cosmin Sabou
Margareta Eliza Balazs
Nika Layzell
Kris Maglione
Jan Henning
Nick Alexander
Gregory Szorc
Rob Wood
Zibi Braniecki