Граф коммитов

1541 Коммитов

Автор SHA1 Сообщение Дата
Gregory Szorc f15e40e75d Bug 1172645 - Make mozbuild's setup.py work; r=glandium
This file hasn't been updated in ages and the current configuration
doesn't produce working packages. Change that.

--HG--
extra : commitid : AfiwohniLPq
extra : rebase_source : 70786c7bd9e37c9acc9b38f54c956fec424cfcf5
extra : amend_source : e4e3752dc97b934456b973736ef55633366c8ee6
2015-06-12 09:37:54 -07:00
Wes Kocher 2aa142fe5e Merge inbound to central, a=merge 2015-06-11 17:02:47 -07:00
Gregory Szorc cb10ca03ae Bug 1168466 - Bump minimum Mercurial version in bootstrapper; r=me
This should have been done in 4c757e339f81.

DONTBUILD (NPOTB)

--HG--
extra : commitid : 41TCyM9iqVQ
extra : rebase_source : 8defc6b23d507e1580db77869ff2c2e99ee5db5a
extra : amend_source : 5f330ea7c2775675fcc8051e5c46d2439c4e5823
2015-06-10 15:38:46 -07:00
Timur Timirkhanov 507e72a004 Bug 1169089 - Detect CentOS 7 in bootstrapper; r=gps
CentOS has apparently changed its disto name. Detect the new version.

DONTBUILD (NPOTB)

--HG--
extra : commitid : JrYp4WJ5Fs3
extra : amend_source : c2ac16f7006c2e44450de54edb732fa722f150d0
2015-06-10 11:27:15 -07:00
Benjamin Smedberg 89ca26a1e4 Bug 1168861 - Make mach bootstrap install GConf2-devel on Fedora, r=glandium
--HG--
extra : rebase_source : 306ede940567b5c7922d9248651c8091677cc5b8
extra : histedit_source : 0d1bff410be6fcddfd74c85b9d4313d15c82a7cf
2015-05-27 11:01:22 -04:00
Gregory Szorc 013f437636 Bug 1168607 - Add a native Mercurial finder; r=glandium
The hglib Mercurial finder was nice. But it is somewhat slow, as it
involves a separate Mercurial process.

This commit introduces a native Mercurial finder that speaks directly
to a Mercurial repository instance. It is significantly faster.

The new code is isolated to its own file because it imports Mercurial
code, which is GPL.

--HG--
extra : commitid : 8CzDFt3lQmx
extra : rebase_source : 1540ad20d795086926b316612062e2a1f10c4958
2015-06-09 13:49:45 -07:00
Gregory Szorc d5d6c72dd3 Bug 1168607 - Make `mach file-info` work with Mercurial repos; r=glandium
This commit adds support for specifying a Mercurial revision with `mach
file-info`. Aside from being a potentially useful feature, it proves
that MercurialRevisionFinder works with BuildReader.

--HG--
extra : commitid : 7143XT9ENqb
extra : rebase_source : d8c0c98d536d07db76653b648fd4b7d74d8f43ae
2015-06-09 13:43:22 -07:00
Gregory Szorc ba5d71304d Bug 1168607 - Add mode to MercurialFileFinder to support non-relative paths; r=glandium
The moz.build reader uses absolute paths when referencing moz.build
files. *Finder classes expect paths to be relative to a base. When we
switched the reader to use *Finder instances for I/O, we simply provided
a default FileFinder based at the filesystem root. This was quick and
easy. Things worked.

Unfortunately, that solution isn't ideal because not all *Finder
instances can accept absolute paths like that. The
MercurialRevisionFinder is one of them.

Changing the moz.build reader to talk in terms of relative paths is a
lot of work. While this would be ideal, it is significantly easier to
defer the problem until later and hack up MercurialRevisionFinder to
accept absolute paths. This commit does exactly that.

Bug 1171069 has been filed to track converting BuildReader to relative
paths.

--HG--
extra : commitid : 2JmzOBldBLy
extra : rebase_source : a8af6ce88dd9e2b98f131c92b45c3ece852e13d2
2015-06-08 09:33:46 -07:00
Gregory Szorc 4f13520461 Bug 1168607 - Implement a finder that reads from a Mercurial repo; r=glandium
Now that moz.build files use finders for I/O, we can start reading
moz.build data from other sources.

Version control is essentially a filesystem. We implement a finder
that speaks to Mercurial to obtain file data. It is able to obtain
file data from a specific revision in the repository.

We use the hglib package (which uses the Mercurial command server) for
speaking with Mercurial. This adds overhead compared to consuming the
raw Mercurial APIs. However, it also avoids GPL side-effects of
importing Mercurial's Python modules.

Testing shows that performance is good but not great. A follow-up
commit will introduce a GPL licensed Mercurial finder. For now, get
the base functionality in place.

--HG--
extra : commitid : BkwaQOW9MiR
extra : rebase_source : 915d6015317ccc79c228a76eed861d9f43e2fd17
2015-06-09 13:39:01 -07:00
Gregory Szorc 8ce366a5f7 Bug 1168607 - Use mozpack Finders for reading moz.build files; r=glandium
Sometimes moz.build data may not come from the local filesystem. To
support defining moz.build data in other backing stores, we switch
moz.build reading to use mozpack's *Finder classes for I/O.

The use of a FileFinder bound to / is sub-optimal. We should ideally
be creating a finder bound to topsrcdir. However, doing this would
require refactoring a lot of path handling in the reader, as that code
makes many assumptions that paths are absolute. This would be excellent
follow-up work.

While I was here, I cleaned up some linter warnings for unused imports.

On my machine, this commit results in a slight slowdown of moz.build
reading. Before, `mach build-backend` was consistently reporting 1.99s
for reading 2,572 moz.build files. After, it is consistently reporting
2.07s, an increase of 4%. This is likely due to a) function call
overhead b) the cost of instantiating a few thousand File instances
c) FileFinder performing an os.path.exists() before returning File
instances. I believe the regression is tolerable.

--HG--
extra : commitid : 1WDcrSU3SQD
extra : rebase_source : a23aa5a4063c6f7080ee00b6f0fe0ee532da3ce7
2015-06-09 18:16:35 -07:00
Gregory Szorc 4bee702035 Bug 1168607 - Add a read() method to File; r=glandium
Passing raw file handles around is a bit dangerous because it could lead
to leaking file descriptors. Add a read() method that handles the simple
case of obtaining the full contents of a File instance.

This should ideally be a method on BaseFile. But this would require
extra work and isn't needed. So we've deferred it until bug 1170329.

--HG--
extra : commitid : 82qw76XmpjC
extra : rebase_source : 422b16c5a3b1577f080097925aeaeb560aa3e798
2015-06-02 09:37:45 -07:00
Gregory Szorc ab11dfb209 Bug 1168607 - Add get method to finders; r=glandium
Today, the *Finder classes are optimized for doing matching and
returning multiple results. However, sometimes only looking for a
single file is wanted.

This commit implements the "get" method on all the *Finder classes.
It returns a BaseFile or None.

FileFinder._find_file has been refactored into FileFinder.get
to avoid code duplication.

--HG--
extra : commitid : K9It2ZJ3Rbo
extra : rebase_source : a56f8f70aa1902d26373a7196eae2847cce653d3
2015-06-04 11:24:03 -07:00
Mike Hommey 0a4831d945 Bug 1172800 - Create actual functions to execute moz.build templates. r=gps
The current mode of execution of templates doesn't allow them to more advanced
control flow, like returning early, returning or yielding values, because that
mode of execution is equivalent to running the code at the top level of a .py
file.

Making the templates executed through a function call, although trickier,
allows those control flows, which will be useful for template as context
managers.
2015-06-10 09:33:22 +09:00
Mike Hommey e7ceb2bd70 Bug 1172800 - Avoid using inspect.getsourcelines() and inspect.getfile(). r=gps
inspect.getsourcelines() and inspect.getfile() involve I/O out of our control.
Our use of those functions, however, doesn't require all their smarts. In fact,
we only use them on function objects, for which we can just do the work
ourselves without involving inspect functions that trigger I/O.
2015-06-10 09:33:21 +09:00
Mike Hommey 17363f4d4e Bug 1172800 - Move moz.build template handling in a separate class. r=gps
This will make subsequent refactorings a bit more intelligible.
2015-06-10 09:33:20 +09:00
Mike Hommey 0d786e2057 Bug 1172800 - Fixup after bug 991983. r=gps 2015-06-10 09:33:19 +09:00
Mike Hommey ad90bd664a Bug 985857 - Automatically log mach output and add a command to display it. r=gps 2015-06-10 09:33:19 +09:00
Gregory Szorc 67e6a15a08 Bug 1173633 - Print docstrings of mach command handlers in help output; r=ahal
`mach help <command>` currently only displays a brief description of the
command along with its arguments. Sometimes more detailed help text is
needed.

With this commit, the docstrings of mach command handlers will appear in
the output of `mach help <command>` if they are defined.

I've implemented basic docstrings for the three flavors of mach commands
(normal command, main subcommand, subcommand) to demonstate things work.

My hope is others will start to fill in docstrings once this feature
lands so the output for `mach help` can serve as a better learning guide
for new contributors.

--HG--
extra : commitid : Hx6ZkHDxbCK
extra : rebase_source : 01ced5a044442e370a45cd3fb245ac6283316925
extra : amend_source : fceb771e0e1ffa4e6f3f1b7c22eae6e25cf82034
2015-06-11 08:32:02 -07:00
Jonathan Griffin eaaad653b6 Bug 1170632 - Fix platform_guess for osx; add pgo runtimes for mbc, r=ahal 2015-06-05 16:29:18 -07:00
Wes Kocher 079a117eae Merge m-c to fx-team, a=merge 2015-06-05 16:05:33 -07:00
Gijs Kruitbosch 015c8609b5 Bug 1150417 - use theme manifest for overrides, r=glandium,dao
--HG--
extra : commitid : 7Ip6kOB2BdY
extra : histedit_source : ada1cc46a63d0b7b2b26cd55f6085bd65be5d70d
2015-06-05 12:48:34 +01:00
Michael Comella 627fb4620a Bug 1165422 - Bump mach bootstrap versions for SDK 22. r?nalexander
--HG--
extra : commitid : BNLZwuY4k7V
extra : rebase_source : d36b0269bb69febd2201ef9d8a9f928e4bddb7ec
2015-06-04 10:53:24 -07:00
Nathan Froyd fc6a141ee0 Bug 1170691 - part 1 - add the generating script's directory to sys.path in file_generate.py; r=glandium
The old way of writing scripts for generated files would invoke the script thusly:

    python script.py arg1...

Invoking the script this way means that the script's directory is
automatically added to sys.path, and importing modules from that
directory is easy.  Let's make it equally easy in the new world for
GENERATED_FILES, too.
2015-06-02 16:00:48 -04:00
Axel Hecht fc69cd94d8 Bug 1165906 - Add docs for l10n to the tree. r=gps
Document the role of l10n.ini, filter.py, file paths and checks.

Also add a glossary for the jargon used in the doc.

--HG--
extra : rebase_source : ec71f9b7123ba76370d34d2349b2f078f14dd1de
2015-06-01 17:13:44 +02:00
James Graham d27ca37fce Bug 1169410 Add support for web-platform-tests to |mach test| r=gps
--HG--
extra : rebase_source : 9a1fb715e0844a61c47a1de53f8a4a54ff2e05f0
extra : amend_source : 8fb1e54b76ae543fa2b01002c2d300334dc051be
2015-05-28 20:27:23 +01:00
Mike Hommey 79ea9f2368 Bug 1170431 part 0 - Use the *Path classes for GENERATED_FILES scripts and inputs. r=gps 2015-06-03 07:10:12 +09:00
Andrew Halberstadt 7aa80fb529 Bug 1164597 - Consolidate all mochitest mach commands into single |mach mochitest|, r=chmanchester
--HG--
extra : rebase_source : 0a0d8a454df10ae9df9678c98e11b5eaf8249f51
2015-05-05 16:41:59 -04:00
David Keeler f23dffbae1 bug 1166976 - add Python-RSA python library r=ted,gerv 2015-05-20 16:34:03 -07:00
David Keeler 932ce8516b bug 1166976 - add pyasn1-modules python library r=ted,gerv 2015-05-20 16:33:23 -07:00
Mike Hommey 5b9df4e5b5 Bug 991983 - Add a deprecation hint for GENERATED_SOURCES. r=gps 2015-05-28 07:39:05 +09:00
Mike Hommey 9b83baa130 Bug 991983 - Make TEST_HARNESS_FILES use the *Path classes instead of a separate set of methods to resolve paths. r=gps 2015-05-28 07:34:16 +09:00
Mike Hommey 806e0220a2 Bug 991983 - Use objdir-relative SOURCES instead of GENERATED_SOURCES. r=gps 2015-05-28 07:34:16 +09:00
Mike Hommey ea7750bcb1 Bug 991983 - Define SOURCES as SourcePath. r=gps 2015-05-28 07:34:15 +09:00
Mike Hommey f8b674d9a9 Bug 991983 - Refactor SourcePath handling for moz.build, add new classes and extensive tests. r=gps
Add generic support for different forms of paths in moz.build:
  '/topsrcdir/relative/paths'
  'srcdir/relative/paths'
  '!/topobjdir/relative/paths'
  '!objdir/relative/paths'

This drops the use of UserString for performance reasons, which required
going around with a meta class.
2015-05-28 07:34:14 +09:00
Mike Hommey 138454bcb9 Bug 991983 - Remove commented code in gyp_reader.py. r=gps
This code has been commented since it landed, and enabling it requires
adding the proper conditionals around all non_unified_sources in gyp
configurations, which is a daunting task. OTOH, the commented code is
already outdated (it would need updates to work) and the related code
that is not commented gets in the way of upcoming changes, so remove
it.
2015-05-28 07:34:14 +09:00
Mike Hommey fc325a2d7e Bug 991983 - Don't pass template paths to contexts. r=gps
When doing a template call, what's relevant for relative paths is the source
path of the caller moz.build, not where the template is defined.
2015-05-28 07:34:13 +09:00
Mike Hommey 65120e7e62 Bug 991983 - Add a ContextDerivedTypedListWithItems type. r=gps 2015-05-28 07:34:12 +09:00
Mike Hommey bc23a0ddc5 Bug 991983 - Emit absolute paths for other sources. r=gps 2015-05-28 07:34:11 +09:00
Mike Hommey 188ce081ec Bug 991983 - Emit absolute paths for UnifiedSources. r=gps
This helps upcoming changes, and relieves backends from path resolution.
This has the side effect of chaning the order of some unified sources,
which consequently breaks nsTextFormatter because it declares snprintf
methods after nsStringAPI #defines it.
2015-05-28 07:34:11 +09:00
Mike Hommey fc645ba14f Bug 991983 - Set GARBAGE for GeneratedSources in the recursivemake backend. r=gps
GARBAGE is an implementation detail of the recursivemake backend, and
doesn't need to spill in what the emitter emits.
2015-05-28 07:34:10 +09:00
Mike Hommey 2818f32729 Bug 1168231 - Normalize file mode in jars. r=gps 2015-05-27 11:33:25 +09:00
Kartikaya Gupta 077ad1ddfb Bug 1117958 - Allow any debugging options to the run or gtest mach subcommands to automatically enable debugging. r=gps 2015-05-25 15:32:09 -04:00
Mike Hommey 4a4cc01c62 Bug 1166538 - Make it more straightforward, path-wise, to change $(ZIP) uses to the zip py_action. r=gps 2015-05-21 15:17:01 +09:00
Mike Hommey aa8f7a51f7 Bug 1166538 - Make dozip.py a py_action. r=gps
--HG--
rename : toolkit/mozapps/installer/dozip.py => python/mozbuild/mozbuild/action/zip.py
2015-05-21 15:17:00 +09:00
Brian O'Keefe c6ba641724 Bug 924187 - Deal with interfaces.manifest from the backend. r=mshal
--HG--
extra : rebase_source : 2d2cbc1ebc97496c54aef63de73b00af8126e750
2015-02-13 08:11:57 -05:00
Joe Steele 1afe160293 Bug 1139125 - Add Windows 64-bit support and certificate chain support to Adobe EME voucher tool. r=ted 2015-05-15 15:26:52 +12:00
Wes Kocher 60d01c38b7 Merge inbound to m-c a=merge 2015-05-12 16:24:41 -07:00
Andrew Halberstadt 77b43440dc Bug 1163112 - [mach core] Consolidate functionality between Main._run and Registrar.dispatch, r=gps
--HG--
extra : rebase_source : 37fb8698c388b63b212c4b02b387dd8814eeecde
2015-05-08 17:03:15 -04:00
Chris Manchester 82719046b5 Bug 1161270 - Add source manifest to reftest manifest representation in moz.build.;r=gps,roc 2015-05-11 16:35:39 -07:00
Nick Alexander c688992859 Bug 1163077 - Never elfhack if --disable-compile-environment is set. r=glandium
A host elfhack binary is only built when there is a compile environment.

--HG--
extra : rebase_source : 4f1da429c581dfd81cbe3d5164c7586066cf6e79
extra : amend_source : f26fe7f3b44291f38459a81b9ff31bd6dbd220aa
extra : histedit_source : 93c628a500c66c46d522bfe678500bf5b5bf0de9
2015-05-08 11:40:06 -07:00
Mike Hommey c917606f95 Bug 1043692 - Move DIST_INSTALL to moz.build. r=gps 2015-05-12 07:55:22 +09:00
Mike Hommey fe90037862 Bug 1043692 - Add a DIST_INSTALL variable to moz.build, and replace NO_DIST_INSTALL with it. r=gps 2015-05-12 07:55:21 +09:00
Mike Hommey 23f754db72 Bug 1162852 - Remove the VARIABLES method of GypContext now it is doing nothing. r=gps 2015-05-12 07:55:20 +09:00
Mike Hommey cb805db780 Bug 1162852 - Remove EXTRA_ASSEMBLER_FLAGS. r=gps 2015-05-12 07:55:19 +09:00
Mike Hommey 141732a8ea Bug 1162852 - Remove EXTRA_COMPILE_FLAGS. r=gps
We now have corresponding variables in moz.build contexts that can be used
instead.
2015-05-12 07:55:15 +09:00
Mike Hommey 82a36b3780 Bug 1162852 - Don't set IS_GYP_DIR from the gyp_reader. r=gps
With TemplateContexts keeping the name of the associated template, and the
Gyp context being declared as a TemplateContext, it is now possible to know
the equivalent of IS_GYP_DIR just by looking at the template name.
2015-05-12 07:54:10 +09:00
Mike Hommey 8a0ffdf337 Bug 1162851 - Keep the template name in TemplateContext instances. r=gps 2015-05-12 07:54:09 +09:00
Mike Hommey 7c306bdc19 Bug 1162845 - Move ASFLAGS to moz.build. r=mshal 2015-05-12 07:54:06 +09:00
Mike Hommey 0218c16ec8 Bug 1162826 - Display a proper error when a moz.build tries to set a "reserved keyword". r=mshal 2015-05-12 07:27:31 +09:00
Mike Hommey 28d5a92945 Bug 1162826 - Properly display full moz.build processing errors when empty KeyError or ValueError are thrown. r=mshal 2015-05-12 07:26:49 +09:00
Mike Hommey 8a1bd09c71 Bug 1162779 - Move NO_EXPAND_LIBS to moz.build. r=mshal 2015-05-12 07:26:42 +09:00
Ralph Giles 2ea918802d Bug 1161339 - Add rust support to moz.build. r=ted
Add .rs as a recognized file extension in SOURCES.

Propagate that through to the Makefile backend and add a dependency
generated and an explicit rule to call $(RUSTC) to compile them.

rustc builds static libraries, not obj files. At least, if one
asks it to output an obj file, I'm not clear how to get all the
compiler-specific runtime libraries the code will expect to link
to. Therefore we generate a static library for each rust source
file (which must be a complete crate for the time being) and link
that. Because of the extension it ends up on the LIBS line in the
the corresponding .desc file.

Note that the static library does still depend on some system
libraries, e.g. -ldl -lpthread -lm on linux. Gecko already
links to all of those, so we don't keep track of it here.

Should we need to add explicit linkage for other targets,
rustc does print a list to stderr which can be parsed.

--HG--
extra : rebase_source : 9f66101fd15a649a952d5481cf9561416204272a
2015-05-08 15:18:00 -07:00
Mike Hommey bfe476e5f9 Bug 1128037 - Minimalist support for wildcards in jar manifests. r=gps 2015-05-05 09:25:24 +09:00
Brian O'Keefe 2a6275c310 Bug 978591 - Remove per-directory uses of MOZ_CHROME_FILE_FORMAT. r=glandium
--HG--
extra : rebase_source : 15031044bb2be11e69848ddebc3e40315dda3cb1
2015-04-23 18:43:06 -04:00
Brian O'Keefe bb0f2688e6 Bug 786520 - Install things to $(DIST)/branding from moz.build instead of manual rules in Makefile.ins. r=mshal
--HG--
extra : rebase_source : 5c9900d28263466014ff391ae4142e98866e4032
2015-02-26 16:53:31 -05:00
Randell Jesup 49c3b667fa Bug 1159300: Add a clone of gmp-fake that doesn't do decryption r=glandium,cpearce
--HG--
rename : dom/media/gmp-plugin/Makefile.in => dom/media/gmp-plugin-openh264/Makefile.in
rename : dom/media/gmp-plugin/fake.info => dom/media/gmp-plugin-openh264/fakeopenh264.info
rename : dom/media/gmp-plugin/fake.voucher => dom/media/gmp-plugin-openh264/fakeopenh264.voucher
rename : dom/media/gmp-plugin/moz.build => dom/media/gmp-plugin-openh264/moz.build
2015-04-30 00:52:26 -04:00
Chris Manchester aaa0c48f52 Bug 1158019 - Tests exercising the proposed behavior. r=gps
--HG--
extra : rebase_source : c4eb68099397dba0bb3f9b191406fb569613f9bc
2015-04-24 17:14:03 -07:00
Chris Manchester 827c49f69d Bug 1158019 - Only resolve to tests under a subdirectory if an input to mozbuild's test resolver is the prefix of a directory containing tests. r=gps
Formerly, running |./mach test image/| would result in running a number of devtools tests
in addition to running tests under the image/ subdirectory. With this change, only tests
under the image/ directory would be run. Note that ./mach test animations or similar will
still run a variety of tests across the tree, because this input does not match a directory.

--HG--
extra : rebase_source : 8a0482796ccb51cec8cd873cf8cd9933e30d46d1
2015-04-24 17:14:03 -07:00
Gregory Szorc 82b77a20c5 Bug 1158898 - Cache config.guess output; r=mshal
mach dispatch makes separate, independent calls to construct build
system state. Part of this resolution is determining the object
directory. For environments without an object directory defined, we must
execute config.guess to determine the object directory. This redundant
execution of config.guess can result in significant execution overhead.

Before this patch, `mach help` with no mozconfig took ~1.5s on my OS X
machine. After this patch, it goes down to ~0.750s. On Windows, the
difference is even more pronounced, with execution time dropping from
8.5s to 0.930s.

--HG--
extra : rebase_source : e8c0f49ceb71e1371e174e31f6e1f354d1e51b65
extra : amend_source : 133b013d5dd8f3817f92555aed1717534769585d
2015-04-27 17:07:49 -07:00
David Major ebde6b9f4f Bug 1157835: Remove the MSVC_ENABLE_PGO flag from the build system. r=glandium
--HG--
extra : rebase_source : 0c47c99bb8b92f8361a51fd81b20a2cc8647a986
2015-04-27 19:59:27 -04:00
Philipp Kewisch 2a29a9fb64 Bug 1153790 - mac unification doesn't work with packaged addons. r=glandium 2015-04-27 00:46:20 +02:00
Gregory Szorc 243db8e774 Bug 1128586 - Properly look for Mercurial version; r=RyanVM
`mercurial version` doesn't always print the version number. `mercurial
--version` does. Use the latter.

--HG--
extra : rebase_source : 6f3c305ce8b38599a60c01d9cee410b724bfc37f
extra : amend_source : 430a3cf66094a62fabee59dbf4a486d4392a8c47
2015-04-27 10:43:25 -07:00
Ehsan Akhgari 3e88d83106 Bug 892973 - Add support for the YouCompleteMe vim plugin; r=gps 2015-04-27 13:08:43 -04:00
Nathan Froyd 5bda08035d Bug 1155816 - followup - delete trailing whitespace, DONTBUILD for whitespace-only change; r=tidybot 2015-04-24 15:46:45 -04:00
Nathan Froyd 83a343a076 Bug 1155816 - part 2 - move EXTRA_*COMPONENTS manifest check to build-backend time; r=mshal
Now that moz.build can see EXTRA_*COMPONENTS and NO_JS_MANIFEST, we can
move some logic from rules.mk (executed every build) to moz.build's
emitter.py (executed only at build-backend time).
2015-04-17 15:02:19 -04:00
Nathan Froyd 7e539337cc Bug 1155816 - part 1 - move NO_JS_MANIFEST to moz.build; r=mshal 2015-04-17 14:50:12 -04:00
Nathan Froyd d264a57b76 Bug 1155776 - move USE_EXTENSION_MANIFEST to moz.build; r=mshal 2015-04-17 13:51:42 -04:00
Morgan Phillips bdaeab13cb Bug 1151834 - Allow users to skip all interactive prompts; r=gps
Interactive prompts make automation more difficult for developers looking to stand up machines using bootstrap en masse.
Two new options have been added to allow users with such needs to bypass all prompts: one for selecing an application
(desktop/mobile) and another for assuming yes to all questions posed by package managers (apt-get/yum).
2015-04-22 16:37:19 +00:00
Brian O'Keefe 0fb2aa4bf1 Bug 1154687 - Restore EXTERNALLY_MANAGED_MAKE_FILE handling to the moz.build blacklist check. r=gps
--HG--
extra : rebase_source : 030e223536b2077cbfffce5b8a7cd43f4deef94e
2015-04-15 08:15:03 -04:00
ISHIKAWA, Chiaki 2807063b52 Bug 1153566 - Incorrect re in _check_blacklisted_variables did not ignore comment. r=gps
--HG--
extra : rebase_source : fe0ce8cb121befee16dccaa0215cb24c5bdf76f9
2015-04-11 09:41:00 +02:00
Andrew Halberstadt 063752e915 Bug 1154006 - [mach] Ability to lazy load parsers passed in via the @Command decorator, r=gps
--HG--
extra : rebase_source : d596a8c1c5a947de18c8c6bfe1e7642074871fcb
2015-04-13 15:36:56 -04:00
Nathan Froyd d876b3e545 Bug 870891 - move DIST_FILES to moz.build; r=mshal 2015-04-15 16:04:49 -04:00
travis e525795e3f Bug 1151005 - Show notifications when the 'install' and 'package' mach commands complete r=gps 2015-04-14 10:35:10 -05:00
travis afc2e8bd4d Bug 1151005 - Refactor terminal notification stuff from 'mach build' into MozbuildObject r=gps 2015-04-14 10:35:09 -05:00
Andrew Halberstadt 6c25784b26 Bug 1151370 - Only guess platform/buildtype in mozinfo.py if processor is known, r=ted
--HG--
extra : rebase_source : 3cbb0cb504bf17d91936e2b995dac6c8d9f8badc
2015-04-07 09:07:32 -04:00
Brian O'Keefe 48eccfb774 Bug 852814 - Move mozbuild variable blacklist logic into the backend. r=gps
--HG--
extra : rebase_source : 01acc12d0ac4d08f7f5367990037eb87db20a404
2015-03-27 15:39:34 -04:00
Andrew Halberstadt 046e79ee98 Bug 1046992 - Consolidate desktop/b2g/b2g desktop mochitest mach commands, r=gbrown
--HG--
extra : rebase_source : 9b9fc8c9fa50530a4bfc574bd0782c2594237474
2015-04-08 11:12:52 -04:00
Ryan VanderMeulen f478f66176 Backed out changeset 6bbe2e6958a9 (bug 1127801) for intermittent (?!?!) Mulet mochitest bustage.
CLOSED TREE

--HG--
extra : rebase_source : 09ea9b1c4036b609424cf1e50476fe5272a4ca9c
2015-04-07 13:47:54 -04:00
naru-san 17258381ee Bug 1127801 - Capture path to milestone.txt from the OS automatically using MozbuildObject from mozbuild.base. r=ted
CLOSED TREE
2015-04-07 12:43:40 -04:00
Ryan VanderMeulen cc0f05835b Backed out changeset a67a0be7325e (bug 1127801) for test bustage. 2015-04-07 11:50:10 -04:00
naru-san 3c8a0350e0 Bug 1127801 - Capture path to milestone.txt from the OS automatically using MozbuildObject from mozbuild.base. r=ted 2015-04-06 16:54:00 -04:00
Markus Stange 78ec2f4d18 Bug 1151349 - Make lldb ns(Int)Region summary handle the numRects==0 case. r=jrmuizel
--HG--
extra : rebase_source : dad2c2c5ef4c17fe1e296b31ee7f36a26254d1ff
2015-04-05 14:06:43 -04:00
Mike Hommey 15e312dd82 Bug 1147207 - Use SimplePackager code to find manifest entries and base directories during l10n repack. r=gps
This avoids duplicating the logic from SimplePackager to find base
directories, and fixes some cases where the l10n repack code wouldn't
find them properly.
2015-04-01 13:52:07 +09:00
Mike Hommey c60d8adb58 Backout changeset 685aa4dfe753 (bug 1147207) for build bustage 2015-03-31 13:18:39 +09:00
Mike Hommey 082f551b9a Bug 1147207 - Allow to give extra l10n directories to l10n-repack.py. r=gps
This allows to use separate l10n staging directories for e.g. addons.
2015-03-31 12:32:53 +09:00
Mike Hommey 566e06c758 Bug 1147207 - Improve SimplePackager manifest consistency check. r=gps
Bug 910660 added a consistency check that rejects cases where a manifest
inside a directory detected as being the base of an addon is included from
outside the addon. Unfortunately, this triggered false positives when
a manifest is included from within the addon, but just happens to be at
the top-level of that addon.
2015-03-31 12:32:52 +09:00
Mike Hommey 5bbf16d22b Bug 1147207 - Use SimplePackager code to find manifest entries and base directories during l10n repack. r=gps
This avoids duplicating the logic from SimplePackager to find base
directories, and fixes some cases where the l10n repack code wouldn't
find them properly.
2015-03-31 12:32:52 +09:00
Mike Hommey 087817e0f7 Bug 1147207 - Add a ComposedFinder class that acts like a FileFinder proxy over multiple FileFinders. r=gps 2015-03-31 12:32:51 +09:00
Mike Hommey f37d8718c7 Bug 1147760 - In mozpack.files.FileCopier.copy, remove files after copying. r=gps
Some file types, such as XPTFile, read their associated data during the
copy. In the case of XPTFiles, several original files are linked together
in one destination file. When a FileCopier is used in-place, those
original files are removed before XPTFile.copy is invoked, so XPTFile.copy
fails.

Generally speaking, there are many other reasons why FileCopier can fail
in-place, but the only active use of this mode is l10n repack code, which
actually doesn't do much of the dangerous uses. However, it can end up
linking XPTFiles for some reason, which fails for the reasons given above.
2015-03-31 12:32:51 +09:00
Mike Hommey 3b1b1b57c7 Bug 1147723 - Avoid non TEST_PASS/TEST_UNEXPECTED_FAIL output from test_files.py. r=gps
Instead, capture the output and check it's being printed.
2015-03-31 12:32:50 +09:00
Andrew Halberstadt 81a076f73c Bug 1142050 - Add buildprops platform and buildtype guess to mozinfo.py, r=ted
--HG--
extra : rebase_source : bfee88c705174218c5c6ad1d1ebd06b172409f53
2015-03-13 11:09:47 -04:00
Wes Kocher 5b6f65bcdb Merge m-c to inbound a=merge CLOSED TREE 2015-03-26 17:46:35 -07:00
Mike Hommey 525ae5030d Bug 1147283 - Replace mozpack.path with mozpath. r=mshal
Back when mozpack.path was added, it was used as:

  import mozpack.path
  mozpack.path.func()

Nowadays, the common idiom is:

  import mozpack.path as mozpath
  mozpath.func()

because it's shorter.

$ git grep mozpath\\. | wc -l
423
$ git grep mozpack.path\\. | wc -l
123

This change was done with:
$ git grep -l mozpack.path\\. | xargs sed -i 's/mozpack\.path\./mozpath./g'
$ git grep -l 'import mozpack.path$' | xargs sed -i 's/import mozpack.path$/\0 as mozpath/'
$ (pat='import mozpack.path as mozpath'; git grep -l "$pat" | xargs sed -i "1,/$pat/b;/$pat/d")
2015-03-27 08:13:16 +09:00
Mike Hommey 2648f538a1 Bug 1147217 - Improve l10n repack error message when locale doesn't contain necessary files. r=mshal 2015-03-27 08:13:04 +09:00
Mike Hommey cb71023412 Bug 1147183 - Avoid test_generate_browsersearch.py printing out unicode. r=mshal 2015-03-27 08:12:15 +09:00
Dirkjan Ochtman e5d4f2f9fa Bug 1143660 - Fix spacing around operators. r=gps
--HG--
extra : rebase_source : cf25d18617de1420380535693d72968cc797b170
2015-03-18 19:13:27 +01:00
Dirkjan Ochtman 3b7f7d0578 Bug 1143660 - Remove empty lines before EOF. r=gps
--HG--
extra : rebase_source : a3ee12af2af8245fb00535ff1204959785abd8b1
2015-03-18 19:11:50 +01:00
Dirkjan Ochtman ac9d6d0b7b Bug 1143660 - Make sure inline comments are preceded by two spaces. r=gps
--HG--
extra : rebase_source : de42739a7fe81e9745e3b289c05f1e95dc5d64c2
2015-03-18 19:10:41 +01:00
Dirkjan Ochtman 9ec9badb1c Bug 1143660 - Fix continuation line indentations for visual indent. r=gps
--HG--
extra : rebase_source : 574f097c33e7ca58e98af18f238d95676d69ebe5
2015-03-18 19:08:55 +01:00
Dirkjan Ochtman 8fbf584ca4 Bug 1143660 - Ascertain 2 blank lines between top-level elements. r=gps
--HG--
extra : rebase_source : d54666b22bfbc58a78f7a88397f97a7bb282cfa6
2015-03-18 18:58:35 +01:00
Dirkjan Ochtman 0aee125108 Bug 1143660 - Remove unused imports from mozboot. r=gps
--HG--
extra : rebase_source : 531fd58980003eb2b30c38eb0bf9f9abb792b55e
2015-03-18 18:53:24 +01:00
Axel Hecht 1ce66ccec6 Bug 940103 - Add a mach command to call compare-locales. r=gps
We're using as many defaults from the configure step as we can. We're also opinionated upon the defaults, but obviously allow most compare-locales options to be specified.

There are two exceptions:
Reference language is specified to be en-US, without optional argument. This is our in-tree command, and the reference language is known.
We always clobber the merge dir, and don't give an option not to. We default to a merge dir in the objdir, so we don't need to be that paranoid as in the standalone version.

Also, compare-locales clobbers merge-dir/browser etc, so you're not going to get / removed.

--HG--
extra : rebase_source : c0f63e566779e83201708d05966f3583ae82e4ee
2015-03-18 18:47:36 +01:00
Axel Hecht d23c97f60c Bug 940103 - Import compare-locales in the tree. r=gps
This is http://hg.mozilla.org/l10n/compare-locales/file/48445f53a274 in the upstream repo.

To review future updates, using the github mirror will be easier, you can check
https://github.com/Pike/compare-locales/compare/873e557...master
for future commits.

--HG--
extra : rebase_source : 92f71ab35810aa62278007e193df5c8ec40c8771
2015-03-18 18:34:15 +01:00
Mike Hommey 3bf6773d2f Bug 910660 - Make the SimplePackager emit a separate base for addons. r=gps
Addons are detected by the presence of an install.rdf file in their base.
2015-03-24 09:10:33 +09:00
Mike Hommey 3a5298d3ca Bug 910660 - Make package formatters handle addons. r=gps
Addons are always formatted with jars for omnijar and jar packaging, and
flat for flat packaging.
2015-03-24 09:10:33 +09:00
Mike Hommey 0ca5f49005 Bug 910660 - Add a test for the unpacker code. r=gps 2015-03-24 09:10:32 +09:00
Mike Hommey dc2c448768 Bug 910660 - Refactor test_packager_formats.py so that it's easier to follow. r=gps 2015-03-24 09:10:31 +09:00
Ms2ger cb507a3e93 Bug 1143932 - Factor out test harness files processing code from emit_from_context; r=gps 2015-03-17 10:39:08 +01:00
Ms2ger f9967bbd8f Bug 1143932 - Factor out XPIDL processing code from emit_from_context; r=gps 2015-03-17 10:37:57 +01:00
Ms2ger a054fefd58 Bug 1143932 - Factor out programs processing code from emit_from_context; r=gps 2015-03-17 10:36:55 +01:00
Ms2ger d6f0cd0e86 Bug 1143932 - Factor out generated filed processing code from emit_from_context; r=gps 2015-03-17 10:33:45 +01:00
Ms2ger a42f1f680c Bug 1143932 - Factor out jar manifest processing code from emit_from_context; r=gps 2015-03-17 10:32:10 +01:00
Ms2ger b7facac379 Bug 1143932 - Factor out sources processing code from emit_from_context; r=gps 2015-03-17 10:32:10 +01:00
Ms2ger 64cfd6dd91 Bug 1143932 - Factor out test manifest processing code from emit_from_context; r=gps 2015-03-17 10:32:10 +01:00
Ms2ger a2615a13c5 Bug 1143932 - Factor out library handling code from emit_from_context; r=gps 2015-03-17 10:32:09 +01:00
Sylvestre Ledru 61bd88c38b Bug 1143412 - Make sure that we run "apt-get update". Otherwise, the install of packages will fail r=nalexander 2015-03-16 14:29:53 +01:00
Nick Alexander 39dd9eab84 Bug 1142897 - Include archlinux module in bin/bootstrap.py download list. r=gps
--HG--
extra : rebase_source : 061722c7064e188112fab661ccb0465d5b4adec7
2015-03-13 13:24:47 -07:00
Brian O'Keefe b27fb40678 Bug 1140161 - Sensibly handle patterns with absolute paths in TEST_HARNESS_FILES. r=gps 2015-03-05 09:22:43 -05:00
Ryan VanderMeulen ee67ad0d8d Merge inbound to m-c. a=merge 2015-03-12 18:05:45 -04:00
Anders Holbøll 1b962fa7af Bug 1138250 - add actual source files from UNIFIED_SOURCES to VS projects always; r=ted.mielczarek; a=RyanVM
DONTBUILD because NPOTB
2015-03-12 14:31:05 -04:00
Jonathan Griffin d1c61bc9b6 Bug 1116187 - Disable failing mochitest-chrome tests for B2G, r=gbrown 2015-02-06 16:30:37 -08:00
Geoffroy Planquart 66a3df3799 Bug 942475 - Add ArchlinuxBootstrapper; r=gps 2015-03-12 10:27:44 +01:00
Philipp Kewisch 183c66a101 Bug 1142127 - Fix typo in exception for "IDL already registered"; r=gps 2015-03-11 17:47:02 +01:00
Mike Hommey 713aaa7b44 Bug 1063880 - Do not emit Linkable-derived objects when COMPILE_ENVIRONMENT is not set. r=mshal 2015-03-11 16:03:55 +09:00
Ahmed Khalil 4d1013a6c2 Bug 927388 - Produce and upload GeckoView AAR during packaging. r=nalexander 2015-03-06 10:17:00 -08:00
Andrew Halberstadt 0594572840 Bug 1140441 - Add substring matching option to |mach mach-debug-commands|, r=gps
--HG--
extra : rebase_source : 266cfc67fdfbed559619a07dabc8a8d14fafa232
2015-03-06 11:25:55 -05:00
Wes Kocher 22674da129 Merge inbound to m-c a=merge CLOSED TREE 2015-03-05 15:17:41 -08:00
Wes Kocher 591b801503 Merge fx-team to m-c a=merge CLOSED TREE 2015-03-05 14:51:48 -08:00
Mitchell Field 6b5b381d25 Bug 987902 - Add a "doctor" mach command; r=gps
--HG--
extra : rebase_source : a69a0d2bfc243ad5c00974ff0ffdec4fa8e99c3e
2015-03-05 10:14:08 -08:00
vidit23 d60e6c9d5a Bug 1121317 - Install Maven repositories during |mach bootstrap| for mobile/android. r=nalexander
DONTBUILD NPOTB
2015-03-04 11:22:00 -08:00
Gregory Szorc 790d7c78f8 Bug 1139086 - Write entire docstring for SubContexts in Sphinx; r=glandium
--HG--
extra : rebase_source : 73ba63142af431d611cf15cca342a46ce121b156
extra : amend_source : a69e85eebf1c42a1acd6ed633d4e514130fd9e16
2015-03-04 00:03:05 -08:00
Gregory Szorc 151ecf016e Bug 1138581 - Support wildcard expansion in mach file-info; r=glandium
This change includes a few fixes to other issues in mach file-info, such
as error handling for invalid paths not working properly.

--HG--
extra : rebase_source : fca43650678d944730273353a5a9154a63247c58
extra : histedit_source : 314e7702ca061005525bd4e054360f79db68935b
2015-03-02 12:58:17 -08:00
Gregory Szorc 92f0b9eb58 Bug 1138283 - Fix bad documentation around wildcard patterns; r=glandium
--HG--
extra : rebase_source : 0df22844fa50658da86f6c111868ac67fda3e21a
extra : amend_source : 0f4451ce80e327382c76a49bef9e7bdb1466f00f
extra : histedit_source : 7c76ee5dca5610704342741e7cb608c2bbb6d297
2015-03-03 09:21:23 -08:00
Andrew Halberstadt c6d32a2950 Bug 1134395 - mozbuild should pass in rootdir to manifestparser to properly calculate test relpaths, r=gps
--HG--
extra : rebase_source : 0d77784a6e7eea31be009f3f475861cf57caeeb3
2015-02-18 17:07:55 -05:00
Ahmed Khalil 95fd7727b1 Bug 1119365 - Implement |mach ide| for mobile/android and IntelliJ/Android Studio. r=nalexander 2015-02-27 05:46:00 -08:00
Gregory Szorc 370a324bca Bug 1132771 - Implement file-info mach command; r=glandium
Now that we have a mechanism for defining file-based metadata, let's add
a mach command to interface with it.

Currently, we limit ourselves to simple Bugzilla data dumping. Features
will be added over time.

--HG--
extra : rebase_source : 20fa72cac6e6d5aff7973b7dd200c03c8f238639
2015-02-27 13:58:01 -08:00
Gregory Szorc eaf5064744 Bug 1132771 - Define some bug components; r=glandium
This patch defines bug components for code that I have historically
touched.

--HG--
extra : rebase_source : 9e670da52247a8feb149e6419e8b488e84f93333
2015-02-26 11:43:45 -08:00
Gregory Szorc 85fc3da5dc Bug 1132771 - Add Files to moz.build with ability to define Bugzilla component; r=glandium
The Files sub-context allows us to attach metadata to files based on
pattern matching rules.

Patterns are matched against files in a last-write-wins fashion.

The sub-context defines the BUG_COMPONENT variable, which is a 2-tuple
(actually a named tuple) defining the Bugzilla product and component for
files. There are no consumers yet. But an eventual use case will be to
suggest a bug component for a patch/commit. Another will be to
automatically suggest a bug component for a failing test.

--HG--
extra : rebase_source : 9489738136d929a53db7f54bbe6acf3186e0a47c
2015-03-01 22:15:07 -08:00
Gregory Szorc 6d01f694ab Bug 1132771 - Support and test for reading without a config object; r=glandium
We want the ability to read data from any moz.build file without needing
a full build configuration (running configure). This will enable tools
to consume metadata by merely having a copy of the source code and
nothing more.

This commit creates the EmptyConfig object. It is a config object that -
as its name implies - is empty. It will be used for reading moz.build
files in "no config" mode.

Many moz.build files make assumptions that variables in CONFIG are
defined and that they are strings. We create the EmptyValue type that
behaves like an empty unicode string. Since moz.build files also do some
type checking, we carve an exemption for EmptyValue, just like we do for
None.

We add a test to verify that reading moz.build files in "no config" mode
works. This required some minor changes to existing moz.build files to
make them work in the new execution mode.

--HG--
extra : rebase_source : f701417f83dfa4e196e39182f8d0a6fea46c6fbb
extra : source : af07351bf2d6e85293ae3edf0fe4ae6cbc0ce246
2015-02-26 10:21:52 -08:00
Gregory Szorc 14155bd553 Bug 1132771 - Support reading relevant moz.build files; r=glandium
Building on top of the API to retrieve relevant moz.build files for a
given path, we introduce a moz.build reading API that reads all
moz.build files relevant to a given set of paths. We plan to use this
new API to read metadata from moz.build files relevant to a set of
files.

This patch changes the generator behavior of read_mozbuild to emit the
main context before any processing occurs. This allows downstream
consumers to manipulate state of the context before things like
directory processing occurs. We utilize this capability in the new
reading API to forcefully declare the directory traversal order for
processed moz.build files, overriding DIRS and similar variables.

Since variable exporting doesn't work reliably in this new traversal
mode, variable exporting no-ops when this mode is activated.

--HG--
extra : rebase_source : aa5c67f275e8eb83516dc97648cc8fea62b52815
extra : source : 6c44edc8208a54a9d5d830266cded7b409a776e1
2015-02-26 09:44:55 -08:00
Gregory Szorc 7b44b57d42 Bug 1132771 - Pass special types down to sandboxes via metadata; r=glandium
Currently, MozSandbox assumes that the FUNCTIONS, SPECIAL_VARIABLES, and
SUBCONTEXTS data structures are the instances that should be associated
with the sandbox. As we introduce new moz.build processing modes that
wish to change processing behavior, it is necessary for them to have
control over these special symbols.

This patch moves the declaration of these types to the special metadata
dictionary which is inherited during recursion. The "read_topsrcdir" API
now explicitly passes the initial metadata into "read_mozbuild".

--HG--
extra : rebase_source : 59ad9ac7c288a5b610d7e3b513aaa0f5b2ec2009
extra : source : ed135df395751194bf379584a4d210f14ac849b4
2015-02-26 09:43:55 -08:00
Gregory Szorc d4f1a46062 Bug 1132771 - Implement strongly typed named tuples; r=glandium
An upcoming patch introduces a use case for a strongly typed named
tuple. So, we introduce a generic factory function that can produce these
types.

--HG--
extra : rebase_source : 7f4d17ff28925fbe8d850c036605aa03a38f0ef2
extra : source : acdd5491f10ecf8ea4e1a14150f9a2e282e2cf5d
2015-02-26 09:38:43 -08:00
Gregory Szorc 0950ccbf67 Bug 1132771 - API to return moz.build files relevant for a set of paths; r=glandium
We have an eventual goal to store file-level metadata in moz.build files
and to have this metadata "cascade" down directory hierarchies. e.g.
metadata in the root directory will apply to all children directories.

A prerequisite for this feature is a way to query which moz.build files
are relevant to a given file. In this patch, we implement an API that
returns this information.

--HG--
extra : rebase_source : 70726926aeddf89df27c46dc2f503dc8a3194633
extra : source : 4013d256b5910404cc04bb3caaf696b8ee551fc5
2015-02-25 14:10:10 -08:00
Ted Mielczarek be5d687375 bug 1135700 - Import the Python requests module. r=gps
--HG--
extra : rebase_source : 006da9e865e18df3d9ded6edbc00a96fec672dad
2015-02-23 11:27:50 -05:00
Nathan Froyd bed3a06f2a Bug 1136383 - enable specifying method names for GENERATED_FILES scripts; r=gps
The inputs to scripts for GENERATED_FILES are restricted to filenames
only.  We have several examples in the tree, however, where a script
takes non-filename arguments.  For converting those cases to use
GENERATED_FILES, we first need to provide some way of "injecting"
non-filename arguments into the script.

This commit adds a method for doing that, by extending the .script flag
on GENERATED_FILES to include an optional method name:

  f = GENERATED_FILES['foo']
  f.script = 'script.py:make_foo'

will invoke the make_foo function found in script.py instead of the
function named main.
2015-02-24 16:36:39 -05:00
Carsten "Tomcat" Book c2d3f30227 Backed out changeset fab243c8d0c5 (bug 1136383) for bustage on a CLOSED TREE 2015-02-25 14:23:10 +01:00
Nathan Froyd 2724dd55e4 Bug 1136383 - enable specifying method names for GENERATED_FILES scripts; r=gps
The inputs to scripts for GENERATED_FILES are restricted to filenames
only.  We have several examples in the tree, however, where a script
takes non-filename arguments.  For converting those cases to use
GENERATED_FILES, we first need to provide some way of "injecting"
non-filename arguments into the script.

This commit adds a method for doing that, by extending the .script flag
on GENERATED_FILES to include an optional method name:

  f = GENERATED_FILES['foo']
  f.script = 'script.py:make_foo'

will invoke the make_foo function found in script.py instead of the
function named main.
2015-02-24 16:36:39 -05:00
Gregory Szorc 0cb191aded Bug 1134072 - Support for sub-contexts; r=glandium
As content in moz.build files has grown, it has become clear that
storing everything in one global namespace (the "context") per moz.build
file will not scale. This approach (which is carried over from
Makefile.in patterns) limits our ability to do things like declare
multiple instances of things (like libraries) per file.

A few months ago, templates were introduced to moz.build files. These
started the process of introducing separate contexts / containers in
each moz.build file. But it stopped short of actually emitting multiple
contexts per container. Instead, results were merged with the main
context.

This patch takes sub-contexts to the next level.

Introduced is the "SubContext" class. It is a Context derived from
another context. SubContexts are special in that they are context
managers. With the context manager is entered, the SubContext becomes
the main context associated with the executing sandbox, temporarily
masking the existence of the main context. This means that UPPERCASE
variable accesses and writes will be handled by the active SubContext.
This allows SubContext instances to define different sets of variables.

When a SubContext is spawned, it is attached to the sandbox executing
it. The moz.build reader will now emit not only the main context, but
also every SubContext that was derived from it.

To aid with the creation and declaration of sub-contexts, we introduce
the SUBCONTEXTS variable. This variable holds a list of classes that
define sub-contexts.

Sub-contexts behave a lot like templates. Their class names becomes the
symbol name in the sandbox.

--HG--
extra : rebase_source : 5df4bcf073ce46605b972021f1e918ce4affa6f3
2015-02-24 13:05:59 -08:00
Mike Hommey e58e62aa83 Bug 1136456 - Remove leftovers from TIERS support in mozbuild.frontend.reader. r=gps 2015-02-25 13:12:14 +09:00
Gregory Szorc 53e3dc9ef2 Bug 1134800 - Properly cache file matching patterns; r=glandium
The regular expression cache for mozpack.path.match was keyed off the
original pattern. However, that variable was mutated as part of the
function and the mutated result was subsequently stored as the cache
key. This effectively resulted in a 0% cache hit rate.

On some tests being written for bug 1132111 which involve a full
filesystem traversal for moz.build files and subsequent execution of
those files, the following timings are indicative of the impact of this
patch.

Before:
real 16.082s
user 14.760s
sys   1.318s

After:
real 6.345s
user 5.085s
sys  1.257s

--HG--
extra : rebase_source : caf4a9f37fda9b43b444059f647535e1b085d422
2015-02-24 10:19:41 -08:00
Mike Hommey b5c1c72f7a Bug 1135984 - Avoid setting an empty GYP_DIRS in contexts when reading it. r=gps 2015-02-24 11:28:50 +09:00
Mike Hommey cd59da5d48 Bug 1135984 - Fix typo which made Context.__init__ set the unused executed_time attribute. r=gps 2015-02-24 11:28:36 +09:00
Gregory Szorc b183534ab4 Bug 1134072 - Remove support for post-eval sandbox callback; r=glandium
Support for a callback to be executed post sandbox evaluation was added
in 24b43ecb4cad (bug 949906) to unbust Sphinx as a result of some GYP
processing changes. e93c40d4344f and bug 1071012 subsequently changed
how Sphinx variables are extracted from moz.build, removing the only
consumer of this feature.

Since there are no consumers of this feature left, remove it and make
the code simpler.

--HG--
extra : rebase_source : 95bb619aba0bb00719e060a4cf346b9dd51eb541
extra : amend_source : ef195e478b122a18f89ad0b2b2b1aab4a4090e42
extra : histedit_source : 36a464ace2fe261b1315544c9ee6a23fc0ab71a2
2015-02-23 17:30:45 -08:00
Ting-Yu Lin 654880f9be Bug 1054892 - Fix ccache status not shown after shrinking cache size. r=gps
The value of cache files or cache size might decrease under these
circumstances.

* The user manually changes the max cache size to a value smaller than
  the current cache size.
* The cache size is reaching the cache max size.

We should not assume both values will be increased after the build is
finished.
2015-02-20 03:42:00 +01:00
Joe Steele 67b453a628 Bug 1131798 - Fix handling of CPU sub-type and rebasing WITHOUT requiring Python 3.3. r=ted 2015-02-21 16:43:36 +13:00
Nick Alexander a4579c9966 Bug 1093242 - Follow-up: define geckolibs-revision and log files written.
--HG--
extra : rebase_source : 8a95ad976f6b57d3ab756692887a623a11256e42
2015-02-20 11:42:28 -08:00
Nick Alexander f3aaef13fa Bug 1093242 - Produce and upload geckolibs artifacts for Android API v11+ opt builds. r=gps
The Android ARchive contains the compiled Gecko libraries that Firefox
for Android interfaces to.  It does not contain the Gecko resources
(the omnijar, omni.ja) nor the compiled Java code (classes.dex).

This also uploads metadata and sha1 hashes for future consumption by
Maven and/or Ivy dependency managers.  In some brave future world,
we'll work out exactly what that looks like; for now, this solves a
storage problem (each .aar file is ~20MB) and it's possible to point
Gradle directly at the uploaded Ivy metadata and artifacts.

--HG--
extra : rebase_source : 0c12b44f587d4a027ca5258bae8fcbb6f6028c24
2015-02-17 17:23:23 -08:00
Alessio Placitelli 0a6d929ae7 Bug 1075055 - Inform mozinfo about the MOZ_TELEMETRY_REPORTING build configuration. r=ted
--HG--
extra : rebase_source : 69f39a94d1284aaa087993cd65aa030f40e9ff92
2015-02-19 22:20:00 -08:00
Ryan VanderMeulen 35dc26890e Backed out changeset e4962045c59c (bug 1131798) for Win8 build bustage.
--HG--
extra : rebase_source : 0f196b6774783dd804bb39a19a850297a650c121
2015-02-18 14:46:42 -05:00
Joe Steele b6225617b0 Bug 1131798 - Fix handling of CPU sub-type and rebasing WITHOUT requiring Python 3.3. r=ted 2015-02-19 07:10:26 +13:00
Gregory Szorc db67b0d1f3 Bug 1139100 - Apply proper checking for command existence; r=ahal
Previously, we raised errors attempting to register a mach sub-command
if the parent command was not defined on the Registrar. This broke B2G
because its mach command Registrar imposes restrictions on which
commands it exposes. Commands there were not getting registered on the
Registrar, leading mach to give a false positive that the parent command
was never defined.

We change the verification logic to take present but unregistered
commands into consideration and to skip registering sub-commands if the
parent isn't present in the Registrar.

--HG--
extra : rebase_source : 150eea0ca85d9f2c9cd9e98f5bf905695da53bf4
2015-03-04 11:03:07 -08:00
Nathan Froyd 3ff1820ae5 Bug 883954 - part 2 - add support for defining generating scripts for GENERATED_FILES; r=gps
Now that we have proper moz.build objects for GENERATED_FILES, we can
add 'script' flags and 'args' flags in moz.build for select
GENERATED_FILES.  We restrict 'args' to being filenames for ease of
implementing checks for file existence, and many (all?) of the examples
of file generation throughout the tree don't need arbitrary strings or
Python data.
2014-12-16 15:13:27 -05:00
Nathan Froyd 52bf0b72d2 Bug 883954 - part 1 - make GENERATED_FILES emit proper moz.build objects; r=gps
This patch is mostly useful for being able to see these changes
independently of the major changes to GENERATED_FILES.  We are going to
need proper moz.build objects for GENERATED_FILES when we add the
ability to define scripts and arguments for them, so we might as well do
that first.
2014-12-16 12:55:02 -05:00
Nathan Froyd a87ff1ab76 Bug 1122812 - part 4 - update the Visual Studio backend for UnifiedSources changes; r=mshal
With the previous changes, we can now tell Visual Studio about the
actual unified files, rather than the files that are #include'd in them.
I believe this is much closer to what Visual Studio wants to see, and
enables things like Intellisense to work properly.
2015-02-05 14:55:54 -05:00
Nathan Froyd 4f29a0e3a7 Bug 1122812 - part 3 - move unified files writing for UnifiedSources to CommonBackend; r=mshal
CommonBackend is where the writing of the unified files belong, since
that's an operation common to all backends.  A special hook into
subclasses is used to enable subclass-specific processing of
UnifiedSources.
2015-02-05 14:54:28 -05:00
Nathan Froyd 2e2e8602a3 Bug 1122812 - part 2 - factor out the grabbing of an object's backend.mk; r=mshal
UnifiedSources will be processed outside of consume_object, so we need
some way of accessing the backend file for an object outside of
consume_object as well.
2015-02-05 14:25:14 -05:00
Nathan Froyd 3fc9689f75 Bug 1122812 - part 1 - move unified file grouping to UnifiedSources's constructor; r=mshal
UnifiedSources, not the recursivemake backend, should be responsible for
figuring out what unified files to generate, and what those unified
files should contain.
2015-02-05 14:05:36 -05:00
Mike Hommey 2dc29d7492 Bug 1114669 - Use a generated list of idl deps for xpt generation. r=gps
Generating the list of idl deps to generate an xpt from its dependency list
makes us give all _previous_ dependencies, inherited from the .deps makefiles.
This leads to removed files being listed on xpidl-process.py command line, and
the command subsequently failing.

Instead, use generated lists of idl dependencies. At the same time, lighten the
generated Makefile further by not emitting xpt dependencies on their containing
directory, and instead generating it from the $xpt_files list.

This brings down the Makefile size from 100k to 38k.
2015-02-13 10:29:19 +09:00
Mike Hommey cfddf46217 Bug 1063874 - Move the parts of XPIDL_SOURCES handling in misc to export. r=gps 2015-02-11 17:36:45 +09:00
Joshua Cranmer 5386f3a04e Bug 1127882: Sort the unified files list before splitting into multiple files, r=nfroyd
--HG--
extra : rebase_source : da8309728d29ed3a51d11536f962cf9255fb6f4f
2015-02-09 08:45:15 -06:00
Gregory Szorc 3d2bba6b22 Bug 1129798 - Use pdb.runcall to debug mach commands; r=ahal
The previous debugger was setting a breakpoint in the mach dispatcher.
This required users to step into the main function to be called into.
Using pdb.runcall(), the debugger starts at the first line in the
executed command, which is far more useful.

--HG--
extra : rebase_source : 10734805ad40ec85eedbb089b11d618dc32398e7
extra : amend_source : da89da1af1f137db2f5902bcb5a69312f4636a4b
2015-02-04 22:49:06 -08:00
Nathan Froyd b657260fda Bug 1129394 - produce a useful error message for |mach build-backend| without having run configure; r=ted.mielczarek
Checking for config.status ourselves avoids the unpleasant situation of
the subprocess module checking for us and producing a cryptic error
message.
2015-02-04 08:40:56 -05:00
Phil Ringnalda cbb7db17d6 Merge f-t to m-c, a=merge 2015-01-31 09:06:14 -08:00
Nick Alexander 413230539d Bug 1123824 - Include platforms/android-VERSION in suggested mozconfig. r=me,f=psd
DONTBUILD NPOTB

--HG--
extra : rebase_source : a273c9939b937ddbb0be387041eb645ec726bd26
2015-01-26 11:13:49 -08:00
Brian O'Keefe c6eb58c878 Bug 870366 - Part 1: Move PREF_JS_EXPORTS to moz.build (mozbuild logic). r=gps 2015-01-20 13:07:02 -05:00
Joshua Cranmer 4f74dc4614 Bug 934170: Make mach xpcshell-test work for comm-central, r=ted
--HG--
extra : rebase_source : d5ce19f18108c598aaa0a519e73a268f1dba91b5
2015-01-30 09:54:00 -06:00
Nathan Froyd 86ae9a9569 Bug 1108750 - part 8 - move WebIDL unification logic to CommonBackend; r=mshal
Similar to the changes made for IPDL files, this commit moves all of the
non-makefile related logic for WebIDL files out of the recursive make
backend and into the common build backend.  Derivative backends that
would like to do interesting things with WebIDL files now need to
implement _handle_webidl_build, which takes more parameters, but should
ideally require less duplication of logic.
2015-01-19 15:38:26 -05:00
Nathan Froyd bda1cacb6f Bug 1108750 - part 7 - move IPDL unification logic to CommonBackend; r=mshal
After a bunch of tiny changes, we're finally ready to make real
progress.  We can now move the grouping of the generated IDPL C++ files
and the actual writing of the unified files for them into the common
build backend.  Derivative backends now only have to concern themselves
with adding the particular logic that compiling those files requires.
2015-01-19 15:34:00 -05:00
Nathan Froyd 92b8b931ec Bug 1108750 - part 6 - move unified source writing to the callers of _add_unified_build_rules; r=mshal
_add_unified_build_rules shouldn't be in the business of writing out
unified files.  Some outside, higher-up logic should be doing that.
2015-01-19 15:28:46 -05:00
Nathan Froyd 76d7cf025c Bug 1108750 - part 5 - move logic for writing all unified files into CommonBackend; r=mshal
We'll need to write out unified files for multiple backends, not just
the recursive make one.  Put that logic someplace where all build
backends can access it.
2015-01-19 15:11:39 -05:00
Nathan Froyd 0f0ef25e9a Bug 1108750 - part 4 - remove handling of extra_dependencies from _add_unified_build_rules; r=mshal
No callers use it, and the makefile rules it generates don't really do
anything that the compiler won't already infer.  Let's get rid of it.
2015-01-19 15:07:43 -05:00
Nathan Froyd 1627061e64 Bug 1108750 - part 3 - move unified source mapping to the callers of _add_unified_build_rules; r=mshal
_add_unified_build_rules shouldn't be in the business of determining how
to group files into their unified files.  That logic belongs in the
caller of _add_unified_build_rules.  Once that's done, the logic for
determining how to group files can migrate out of the recursive make
bakend.
2015-01-19 14:58:21 -05:00
Nathan Froyd ea56566031 Bug 1108750 - part 2 - move _write_unified_file to CommonBackend; r=mshal
Nothing about writing unified files is specific to the recursive make
backend, and if we want to write the unified files for IPDL and WebIDL
files, we'll need this functionality available in the common build
backend.
2015-01-19 14:45:32 -05:00
Nathan Froyd b23c0da465 Bug 1108750 - part 1 - move group_unified_files to mozbuild.util; r=mshal
RecursiveMakeBackend._group_unified_files doesn't contain any
functionality specific to the recursive make backend.  We would also
like to move the unification of generated IPDL and WebIDL source files
into the common build backend.  Moving _group_unified_files into the
common build backend would be the logical place for it, but the frontend
should also be able to handle unifying files so that backends don't have
to duplicate logic for UNIFIED_FILES.  Therefore, we choose to move it
to mozbuild.util as its final resting place.
2015-01-19 14:32:44 -05:00
Ryan VanderMeulen 9aa5f90ca7 Merge inbound to m-c. a=merge 2015-01-26 16:28:17 -05:00
Ted Mielczarek 3cab8effef bug 1124192 - Fix PATH winding up as unicode when using activate_virtualenv. r=gps
--HG--
extra : rebase_source : 11bca3bcacfd27234100ceaf64166c0bfcb96b45
2015-01-26 13:43:01 -05:00
James Lal 51503a1a67 Bug 1068653 - Part 1 Add python dependencies for taskcluster mach commands r=gps
--HG--
extra : rebase_source : 7a91182ca85dde748a14b03fa93ae85769691042
extra : source : b91e85b02d796db5de9a0e726a7c3360ea67b400
2014-11-26 10:11:28 -08:00
Nick Alexander d4b22a3eed Bug 1123980 - Part 2: Write region-specific search settings into res/raw/browsersearch.json. r=mfinkle
Pushing on a CLOSED TREE since Android build only.

--HG--
extra : rebase_source : cc99efa734d1f4738d3f026c930a4d1955723783
extra : amend_source : 52c07807a77f263d2eb2593826dc0285928d9be4
2015-01-20 16:27:27 -08:00
Nick Alexander 79f6d1bfb3 Bug 1123980 - Part 1: Handle common prefixes in .properties lists and dicts. r=mshal
This handles:

list.0=A
list.1=B
list.sublist.0=C

so that

list=>[A, B]
list.sublist=>[C]

and

dict=default
dict.key1=A
dict.key2=B

so that

dict=>{key1:A, key2:B}
dict=>default

--HG--
extra : rebase_source : 2c8f5941d2fca9c56b3858d3e98078a43fa69090
2015-01-21 14:31:22 -08:00
Markus Stange 68273bef8c Bug 1118228 - Add an lldb helper that allows calling functions without debug info. r=jrmuizel, r=ehsan 2015-01-14 19:04:24 -05:00