We want annotationProcessors to be compiled and archived into a JAR at
build time, ready to generate JNI wrappers. (That is, until we turn
the whole thing into a real annotation processor.) But even if we do
use a real annotation processor, we still need to generate SDK
bindings, which is less clearly expressed as an annotation processor.
(It's more of a build step.)
Gradle provides a huge number of ways to organize build logic to
achieve this: see
https://docs.gradle.org/current/userguide/organizing_build_logic.html.
Unfortunately, the best such way -- putting the code into
$topsrcdir/buildSrc -- has key disadvantages:
1) it pollutes the top-level $topsrcdir, and there's no way to change the
location of buildSrc (https://github.com/gradle/gradle/issues/2472);
2) it's complicated to have a dependent project
(mobile/android/annotations) expose its code via a buildSrc project;
3) using buildSrc at all appears to conflict with the Android-Gradle
plugin version that we are using.
Therefore, this commit does something much simpler: it adds a
Java-only project and uses the resulting Gradle "Jar" task and archive
output as input to the existing Gradle "generate JNI wrappers" task.
MozReview-Commit-ID: 2OyYLPneE1M
--HG--
extra : rebase_source : d99b74a0a1e0bb3e8f4d4540978328388e5c2e42
Some content in Makefile.in is removed because after this change, the
scripts no longer invoke the preprocessor and thus don't have unknown
dependencies anymore outside what is provided in their inputs array.
The order of exports.PREFERENCES in properties-db changes because the
data file has shorthands placed after longhands. The only usage of it
is in test_css-properties-db.js which doesn't care about the order.
MozReview-Commit-ID: AMjzTRf2HYN
--HG--
extra : rebase_source : 7976e48e7c7bba467d77a34ab0d7709cde1ecdf4
We add a minimal Python script to run a process and prefix all its
output with a string. We change the automation tiers to evaluate all
make targets using this script.
MozReview-Commit-ID: 79g5KUd5ked
--HG--
extra : rebase_source : 63388a71b51e5abc05ca8bd48e180af72bf799e6
Some content in Makefile.in is removed because after this change, the
scripts no longer invoke the preprocessor and thus don't have unknown
dependencies anymore outside what is provided in their inputs array.
The order of exports.PREFERENCES in properties-db changes because the
data file has shorthands placed after longhands. The only usage of it
is in test_css-properties-db.js which doesn't care about the order.
MozReview-Commit-ID: AMjzTRf2HYN
--HG--
extra : rebase_source : f9db0659a81bea28b335806ac70e23dc0d36e493
This patch contains the meat of the changes here. The following summarize the changes:
1. xptinfo.h is rewritten to expose the new interface for reading the XPT data,
The nsXPTInterfaceInfo object exposes methods with the same signatures as
the methods on nsIInterfaceInfo, to make converting code which used
nsIInterfaceInfo as easy as possible, even when those methods don't have
signatures which make a ton of sense anymore. There are also a few methods
which are unnecessary (they return `true` or similar), which should be
removed over time.
Members of the data structures are made private in order to prevent reading
them directly. Code should instead call the getter methods. This should make
it easier to change their memory representation in the future. Constructing
these structs is made possible by making the structs `friend class` with the
XPTConstruct class, which is implemented by the code generator, and is able
to access the private fields.
In addition, rather than using integers with flag constants, I opted for
using C++ bitfields to store individual flags, as I found it made it easier
to both write the code generator, and reason about the layouts of the types.
I was able to shave a byte off of each nsXPTParamInfo (4 bytes -> 3 bytes)
by shoving the flags into spare bits in the nsXPTType. Unfortunately there
was not enough room for the retval flag. Fortunately, we already depend in
our code on the retval parameter being the last parameter, so I worked
around this by removing the retval flag and instead having a `hasretval`
flag on the method itself.
2. An xptinfo.cpp file is added for out-of-line definitions of more complex
methods, and the internal implementation details of the perfect hash.
Notable is the handling of xptshim interfaces. As the type is uniform, a
flag is checked when trying to read constant information, and a different
table with pointers into webidl data structures is checked when the type is
determined to be a shim.
Ideally we could remove this once we remove the remaining consumers of the
existing shim interfaces.
3. A python code generator which takes in the json XPT files generated in the
previous part, and emits a xptdata.cpp file with the data structures. I did
my best to heavily comment the code.
This code uses the friend class trick to construct the private fields of the
structs, and avoid a dependency on the ordering of fields in xptinfo.h.
The sInterfaces array's order is determined by a generated perfect hash
which is also written into the binary. This should allow for fast lookups by
IID or name of interfaces in memory. The hash function used for the perfect
hash is a simple FNV hash, as they're pretty fast.
For perfect hashing of names, another table is created which contains
indexes into the sInterfaces table. Lookup by name is less common, and this
form of lookup should still be very fast.
4. The necessary Makefiles are updated to use the new code generator, and
generate the file correctly.
The uninstaller was being built as a side-effect of building `setup.exe`. In
Bug 1385227, that was moved from "somewhere" to part of the windows installer
packaging, which happens after the zip and mar are generated. Since the
installer we ship is actually repackaged from the zip[1], we stopped shipping
translated uninstallers.
This changes things around so that the uninstaller gets translated:
- Explicitly build the uninstaller as part of the L10n repack step.
- Use the same logic to build the installer locally as we do to create the ones
we ship.
[1] Except on Thunderbird
Differential Revision: https://phabricator.services.mozilla.com/D672
--HG--
extra : rebase_source : 05fe935c1d2a9fbfeef786819bfe5913ed8ef862
extra : source : d6bf22099e2195dcb64c3c3d7700d3edd0850a3a
The uninstaller was being built as a side-effect of building `setup.exe`. In
Bug 1385227, that was moved from "somewhere" to part of the windows installer
packaging, which happens after the zip and mar are generated. Since the
installer we ship is actually repackaged from the zip[1], we stopped shipping
translated uninstallers.
This changes things around so that the uninstaller gets translated:
- Explicitly build the uninstaller as part of the L10n repack step.
- Use the same logic to build the installer locally as we do to create the ones
we ship.
[1] Except on Thunderbird
Differential Revision: https://phabricator.services.mozilla.com/D672
--HG--
extra : rebase_source : 2b28b9ff7196d12f4a188c8dddf750b9a5efac5b
extra : histedit_source : 9bc28891950ae8c226cfdefef6f8121ce0b51f58
This patch is a nearly complete reimplementation of BinASTReader, with the following changes:
- Files BinToken.h, BinSource-auto.h (new), BinSource-auto.cpp (new) are now autogenerated by the generator in js/src/frontend/binsouce from the webidl specifications of BinAST and a small
configuration file.
- Optional fields have been removed. Rather, some specific fields may, if so marked in the specifications, contain a Null constant.
- `hasDirectEval` is now checked for consistency (NOT completeness).
- `varDeclaredNames` is now checked for consistency (NOT completeness).
- `lexicallyDeclaredNames` is now checked for consistency (NOT completeness).
- `parameterNames` is now checked for consistency (NOT completeness).
- `capturedNames` is NOT checked.
- Atoms read are now properly expected to be UTF8.
This patch does not implement the entire specifications, but should implement most of ES5. In particular, it is sufficient to parse the source code of:
- Facebook;
- jQuery;
- mootools;
- Underscore;
- Backbone;
- Angular.
MozReview-Commit-ID: HwkVB5dliZv
--HG--
extra : rebase_source : fd7e068343e2af8926c5185e7199ea110a5149bc
This patch handles the actual generation of the static data structures
used to represent XPT information. XPT files are generated in the same
way as they are now, but they are used only as an intermediate
representation to speed up incremental compilation rather than
something used by Firefox itself. Instead of linking XPTs into a
single big XPT file at packaging time, they are linked into a single
big C++ file at build time, that defines the various static consts in
XPTHeader.
In xpt.py, every data structure that can get written to disk gets an
additional code_gen() method that returns a representation of that
data structure as C++ source code. CodeGenData aggregates this
information together, handling deduplication and the final source code
generation.
The ctors are needed for XPTConstValue to statically initialize the
different union cases without resorting to designated initializers,
which are part of C99, not C++. Designated initializers appear to be
supported in C++ code by Clang and GCC, but not MSVC. The ctors must
be constexpr to ensure they are actually statically initialized so
they can be shared between Firefox processes.
I also removed an unnecessary "union" in XPTConstDescriptor.
Together, these patches reduce the amount of memory reported by
xpti-working-set from about 860,000 bytes to about 200,000 bytes. The
remaining memory is used for xptiInterface and xptiTypelibGuts (which
are thin wrappers around the XPT interfaces and header) and hash
tables to speed up looking up interfaces by name or IID. That could
potentially be eliminated from dynamic allocations in follow up
work. These patches did not affect memory reporting because XPT arenas
are still used by the remaining XPTI data structures.
MozReview-Commit-ID: Jvi9ByCPa6H
--HG--
extra : rebase_source : a9e48e7026aab4ad1b7f97e50424adf4e3f4142f
Now that XPT files are not loaded from files at runtime, code for
packaging XPT files can be removed.
This means that a couple of test XPIDL interfaces will get shipped in
builds to users that weren't before, but I don't think that matters
much.
This also puts XPT files into the local objdir for the XPIDL makefile,
instead of dist/bin, because they are no longer part of the
distribution.
MozReview-Commit-ID: 7gWj8KWUun3
--HG--
extra : rebase_source : 65bac47c2cd1a20b3c675a01b44a25a1d2d3ab7a
This patch handles the actual generation of the static data structures
used to represent XPT information. XPT files are generated in the same
way as they are now, but they are used only as an intermediate
representation to speed up incremental compilation rather than
something used by Firefox itself. Instead of linking XPTs into a
single big XPT file at packaging time, they are linked into a single
big C++ file at build time, that defines the various static consts in
XPTHeader.
In xpt.py, every data structure that can get written to disk gets an
additional code_gen() method that returns a representation of that
data structure as C++ source code. CodeGenData aggregates this
information together, handling deduplication and the final source code
generation.
The ctors are needed for XPTConstValue to statically initialize the
different union cases without resorting to designated initializers,
which are part of C99, not C++. Designated initializers appear to be
supported in C++ code by Clang and GCC, but not MSVC. The ctors must
be constexpr to ensure they are actually statically initialized so
they can be shared between Firefox processes.
I also removed an unnecessary "union" in XPTConstDescriptor.
Together, these patches reduce the amount of memory reported by
xpti-working-set from about 860,000 bytes to about 200,000 bytes. The
remaining memory is used for xptiInterface and xptiTypelibGuts (which
are thin wrappers around the XPT interfaces and header) and hash
tables to speed up looking up interfaces by name or IID. That could
potentially be eliminated from dynamic allocations in follow up
work. These patches did not affect memory reporting because XPT arenas
are still used by the remaining XPTI data structures.
MozReview-Commit-ID: Jvi9ByCPa6H
--HG--
extra : rebase_source : 719dfbcb9f83235c0f1f0766270b7f127f9ab04e
Now that XPT files are not loaded from files at runtime, code for
packaging XPT files can be removed.
This means that a couple of test XPIDL interfaces will get shipped in
builds to users that weren't before, but I don't think that matters
much.
This also puts XPT files into the local objdir for the XPIDL makefile,
instead of dist/bin, because they are no longer part of the
distribution.
MozReview-Commit-ID: 7gWj8KWUun3
--HG--
extra : rebase_source : 6f7d4fd1d6cdea2c14866705a2dc972eb5f43382
This adds a basic test for the mach formatter. This will ensure that changes to
this format are intentional. It will also make it easier for reviewers of these
changes to see a diff of the old vs new format.
MozReview-Commit-ID: LBSfdyvOPVV
--HG--
extra : rebase_source : 5529ad1f03306dcf867d88af579b69d6005091c0
Now that we're no longer shipping the SDK we no longer need real libraries for
the libraries that were created by this rule.
MozReview-Commit-ID: ALATVGBayHu
--HG--
extra : rebase_source : a20905125e5ff1846ef29de12323ba7b0a58928b
The MSVC linker winds up generating import libraries when linking some of
our executables, presumably because they contain functions that are
__declspec(dllexport). By default the import libraries get written
alongside the exe, so we force them to be written to the objdir so they don't
clutter up dist/bin.
MozReview-Commit-ID: 7DTfCo3OdDQ
Historically we built all our binaries in directories in the objdir, then
symlinked them into dist/bin. Some binaries needed to be copied instead
so that certain relative path lookups work properly, so we resorted to
sprinkling `NSDISTMODE=copy` around Makefiles.
This change makes it so we build PROGRAMs (not any other sort of targets)
directly in dist/bin instead. We could do the same for our other targets
with a little more work.
There were several places in the tree that were copying built binaries to
some other place and needed fixup to match the new location of binaries.
On Windows pdb files are left in the objdir where the program was
originally linked. symbolstore.py needs to locate the pdb file both to
determine whether it should dump symbols for a binary and also to copy
the pdb file into the symbol package. We fix this by simply looking for
the pdb file in the current working directory if it isn't present next
to the binary, which matches how we invoke symbolstore.py.
MozReview-Commit-ID: 8TOD1uTXD5e
We use a wrapper script when compiling with MSVC to parse the
/showIncludes output and thereby generate a Makefile dependency
fragment. This fragment enables us to do correct and faster incremental
builds. But the cost of invoking the wrapper script can be significant;
it's an extra process or two to launch for every single compilation.
Instead, let's have clang-cl generate the dependencies directly, which
should be somewhat faster.
Nick Alexander
Xidorn Quan
Jan de Mooij
Cosmin Sabou
Chris Manchester
Gregory Szorc
Andreea Pavel
Nika Layzell
Tom Prince
Brindusan Cristian
Emilio Cobos Álvarez
David Teller
Andrew McCreight
Dorel Luca
André Bargull
Ciure Andrei
Jeff Walden
shindli
David Major
Dave Hunt
Andrew Halberstadt
Ted Mielczarek
Sebastian Hengst
Nathan Froyd
Margareta Eliza Balazs