Python API documentation requires the ability to import modules. So, we
set up a virtualenv in our Sphinx environment so module loading works.
This solution isn't perfect: a number of modules fail to import when run
under sphinx-build.
--HG--
extra : rebase_source : fce732e0b8aefe0e9a2ee594b8a08ac02e27579a
extra : histedit_source : bef27c947b95c3182fbc6cd656ae8c96acaaa6f4
Previously, code for staging the Sphinx documentation from moz.build
metadata lived in a mach command and in the moztreedocs module. This
patch moves the invocation to the Sphinx extension.
When the code is part of the Sphinx extension, it will run when executed
with sphinx-build. This is a prerequisite to getting RTD working, since
sphinx-build is the only supported entrypoint for generating
documentation there.
With this patch, we can now invoke sphinx-build to build the
documentation. The `mach build-docs` command is no longer needed.
--HG--
extra : rebase_source : 86e76c7d598ffa23dae858254eecedbdd12706a4
extra : histedit_source : 1826aa5ddfafdff62847cc293d1f0950b236caac
The recursivemake backend knows how to do several things with the IPDL
sources:
1) Determine the C++ sources that will be generated from given IPDL
sources.
2) Write out all the makefile rules and variables for said sources.
The first part isn't unique to the recursivemake backend; other backends
would eventually like to know what C++ sources come from IPDL source
files for easier cross-referencing purposes, etc. Let's take a first
cut at moving things into CommonBackend. (This may not be the best
interface, since it relies on consume_finished being invoked, and not
all backends call CommonBackend.consume_finished. Still, it's a start.)
Various bits of the test harnesses key off of mozinfo.info.get('asan');
we will need a similar switch for finding out whether this build
supports tsan.
Now that the mozbuild backend knows about FINAL_TARGET, we are able to
install generated xpt files into their final location. This saves us
from copying xpt files into their final location on every build.
Original patch by gps, rebased and comments addressed by Ms2ger
--HG--
extra : transplant_source : %E2%DC%0F%E0%AD%C2%25%A1%B8%A9%FE%B0%8C%60%FF%CB%02G%25%E5
mopack.BaseFile.copy() performs a generic read/write file copy. Windows
has an explicit CopyFile() call that tests have shown to be
significantly faster. Let's use that instead via the magic of ctypes.
Having SOURCES and its close relatives go through VariablePassthru
objects clutters the handling of VariablePassthru in build backends and
makes it less obvious how to handle things that actually get compiled.
Therefore, this patch introduces four new moz.build objects
corresponding to the major variants of SOURCES. It looks like a large
patch, but there's an ample amount of new tests included, which accounts
for about half of the changes.
Now that defining $DMD is no longer necessary to run DMD, this patch does the
following.
- Removes all the places where we set DMD=1 (test harnesses, etc.)
- Still handles DMD=1, for backwards compatibility.
- Prints "$DMD is undefined" at DMD start-up if appropriate.
- Writes a |null| value for |dmdEnvVar| in the JSON if $DMD is undefined. Bumps
the DMD output version number accordingly.
- Changes a bunch of the test files accordingly, including changing the mode of
script-ignore-alloc-fns.json in order to test a case where $DMD is undefined.
--HG--
extra : rebase_source : eb1ef5722410734ce6d7658465ff6f442ee4ed49
Various os.path attributes are being used in tight loops. Having local
variables prevents extra dictionary lookups.
This appears to shave 10-20ms off of the tests install manifest
processing time.
--HG--
extra : rebase_source : de941f2978cf0b1fd7c4f7401c848b61d406a2c8
extra : amend_source : e33c896856fa559197496b8227e10ab8149d146e
FileCopier.copy() was performing a lot of os.path.normpath() operations.
Profiling revealed that os.path.normpath() was the function with the
most wall time CPU usage when processing the tests manifests. Upon
subsequent examination of the code in question, all the paths being used
were already normalized. So, os.path.normpath() wasn't accomplishing
anything.
This patch results in ~300ms reduction in wall time to process the tests
install manifest on a fully populated page cache. Execution time drops
from ~2.8s to ~2.5s.
Profiling reveals that after this patch os.stat() is the #1 wall time
consumer. However, os.path.{join,dirname,normpath} still account for
~1.5x the wall time of os.stat(). There is still room to optimize
this function.
--HG--
extra : rebase_source : b6f0862baa5168c609499fd95eb3517854bc8cce
extra : amend_source : 7e04c1eb74132bbbe86e721f0f209b19309a7a51
This patch moves profiling mode selection from post-processing (in dmd.py) to
DMD start-up. This will make it easier to add new kinds of profiling, such as
cumulative heap profiling.
Specifically:
- There's a new --mode option. |LiveWithReports| is the default, as it is
currently.
- dmd.py's --ignore-reports option is gone.
- There's a new |mode| field in the JSON output.
- Reports-related operations are now no-ops if DMD isn't in LiveWithReports
mode.
- Diffs are only allowed for output files that have the same mode.
- A new function ResetEverything() replaces the SetSampleBelowSize() and
ClearBlocks(), which were used by the test to change DMD options.
- The tests in SmokeDMD.cpp are split up so they can be run multiple times, in
different modes. The exact combinations of tests and modes has been changed a
bit.
--HG--
rename : memory/replace/dmd/test/full-reports-empty-expected.txt => memory/replace/dmd/test/full-empty-dark-matter-expected.txt
rename : memory/replace/dmd/test/full-heap-empty-expected.txt => memory/replace/dmd/test/full-empty-live-expected.txt
rename : memory/replace/dmd/test/full-heap-sampled-expected.txt => memory/replace/dmd/test/full-sampled-live-expected.txt
rename : memory/replace/dmd/test/full-reports-unsampled1-expected.txt => memory/replace/dmd/test/full-unsampled1-dark-matter-expected.txt
rename : memory/replace/dmd/test/full-heap-unsampled1-expected.txt => memory/replace/dmd/test/full-unsampled1-live-expected.txt
rename : memory/replace/dmd/test/full-reports-unsampled2-expected.txt => memory/replace/dmd/test/full-unsampled2-dark-matter-expected.txt
rename : memory/replace/dmd/test/script-diff-basic-expected.txt => memory/replace/dmd/test/script-diff-live-expected.txt
rename : memory/replace/dmd/test/script-diff1.json => memory/replace/dmd/test/script-diff-live1.json
rename : memory/replace/dmd/test/script-diff2.json => memory/replace/dmd/test/script-diff-live2.json
extra : rebase_source : bf32cc4e0d82aa1a20ceb55e8ea259850b49cc06
This is a straight copy from
a878bf0ba0
paired with a tiny change to use the new quote_chars option.
--HG--
extra : rebase_source : 75d604ffafc7062c663bca4242af35546d2c1e3a
buildlist invocations are slow and can occur in parallel since the
underlying program obtains a lock on the modified file.
Moving the XPT-related buildlist invocation from the serial libs tier to
the parallel misc tier decreased my no-op build time on OS X from 43.5s
to 37.0s.
--HG--
extra : rebase_source : 7d274024c401b1ecfbc771424a69eb487808fcbf
When the misc tier was added, only directories with misc-associated
variables from moz.build were traversed. This patch adds a dummy
variable to moz.build whose presence will add the directory to the misc
tier.
This will enable us to aggressively convert existing libs:: rules
to the misc tier.
--HG--
extra : rebase_source : c5477010c7d87fafa512f4fbbf1d2b995e07ffbd
extra : amend_source : a54eee6254ef2d63ef6b247d45e38eda2ab88fef
extra : histedit_source : c0f3352e84ec18ca7b88571cd5f9c2a33ab8eda9
JS module installation performs simple file copying or preprocessing.
There is no reason it can't occur in parallel. Move it to the misc tier.
As part of this, I recognized that TESTING_JS_MODULES was assigned to a
tier. Since these files are managed by an install manifest, they don't
belong to any tier. So the tier is now listed as None.
--HG--
extra : rebase_source : 963f0813e43802c017837ce9d55f8e666decd76a
The build system being what it currently is, there are various cases where one
wants something explicit, rather than the current autodetection.
For instance, one may want to run
make -C $objdir chrome
instead of
make -C $objdir/chrome
that mach build chrome currently invokes.
There are several such usecases that mach's autodetection makes harder, and
it's sometimes awkward when telling people, to debug their issues, to run
make -C objdir something
and hear back that objdir doesn't exist or something along those lines,
because they took "objdir" literally.
There are, sadly, many combinations of linkage in use throughout the tree.
The main differentiator, though, is between program/libraries related to
Gecko or not. Kind of. Some need mozglue, some don't. Some need dependent
linkage, some standalone.
Anyways, these new templates remove the need to manually define the
right dependencies against xpcomglue, nspr, mozalloc and mozglue
in most cases.
Places that build programs and were resetting MOZ_GLUE_PROGRAM_LDFLAGS
or that build libraries and were resetting MOZ_GLUE_LDFLAGS can now
just not use those Gecko-specific templates.
Wes Kocher
Gregory Szorc
Wes Johnston
Nathan Froyd
Pavel Roskin
Brian O'Keefe
Xidorn Quan
Carsten "Tomcat" Book
Ankit Goyal
Nicholas Nethercote
Phil Ringnalda
Ting-Yu Lin
Nick Alexander
Kartikaya Gupta
Vaibhav Agrawal
Joshua Cranmer
Mike Hommey
Richard Newman
Benoit Girard