This patch is a nearly complete reimplementation of BinASTReader, with the following changes:
- Files BinToken.h, BinSource-auto.h (new), BinSource-auto.cpp (new) are now autogenerated by the generator in js/src/frontend/binsouce from the webidl specifications of BinAST and a small
configuration file.
- Optional fields have been removed. Rather, some specific fields may, if so marked in the specifications, contain a Null constant.
- `hasDirectEval` is now checked for consistency (NOT completeness).
- `varDeclaredNames` is now checked for consistency (NOT completeness).
- `lexicallyDeclaredNames` is now checked for consistency (NOT completeness).
- `parameterNames` is now checked for consistency (NOT completeness).
- `capturedNames` is NOT checked.
- Atoms read are now properly expected to be UTF8.
This patch does not implement the entire specifications, but should implement most of ES5. In particular, it is sufficient to parse the source code of:
- Facebook;
- jQuery;
- mootools;
- Underscore;
- Backbone;
- Angular.
MozReview-Commit-ID: HwkVB5dliZv
--HG--
extra : rebase_source : fd7e068343e2af8926c5185e7199ea110a5149bc
This patch handles the actual generation of the static data structures
used to represent XPT information. XPT files are generated in the same
way as they are now, but they are used only as an intermediate
representation to speed up incremental compilation rather than
something used by Firefox itself. Instead of linking XPTs into a
single big XPT file at packaging time, they are linked into a single
big C++ file at build time, that defines the various static consts in
XPTHeader.
In xpt.py, every data structure that can get written to disk gets an
additional code_gen() method that returns a representation of that
data structure as C++ source code. CodeGenData aggregates this
information together, handling deduplication and the final source code
generation.
The ctors are needed for XPTConstValue to statically initialize the
different union cases without resorting to designated initializers,
which are part of C99, not C++. Designated initializers appear to be
supported in C++ code by Clang and GCC, but not MSVC. The ctors must
be constexpr to ensure they are actually statically initialized so
they can be shared between Firefox processes.
I also removed an unnecessary "union" in XPTConstDescriptor.
Together, these patches reduce the amount of memory reported by
xpti-working-set from about 860,000 bytes to about 200,000 bytes. The
remaining memory is used for xptiInterface and xptiTypelibGuts (which
are thin wrappers around the XPT interfaces and header) and hash
tables to speed up looking up interfaces by name or IID. That could
potentially be eliminated from dynamic allocations in follow up
work. These patches did not affect memory reporting because XPT arenas
are still used by the remaining XPTI data structures.
MozReview-Commit-ID: Jvi9ByCPa6H
--HG--
extra : rebase_source : a9e48e7026aab4ad1b7f97e50424adf4e3f4142f
Now that XPT files are not loaded from files at runtime, code for
packaging XPT files can be removed.
This means that a couple of test XPIDL interfaces will get shipped in
builds to users that weren't before, but I don't think that matters
much.
This also puts XPT files into the local objdir for the XPIDL makefile,
instead of dist/bin, because they are no longer part of the
distribution.
MozReview-Commit-ID: 7gWj8KWUun3
--HG--
extra : rebase_source : 65bac47c2cd1a20b3c675a01b44a25a1d2d3ab7a
This patch handles the actual generation of the static data structures
used to represent XPT information. XPT files are generated in the same
way as they are now, but they are used only as an intermediate
representation to speed up incremental compilation rather than
something used by Firefox itself. Instead of linking XPTs into a
single big XPT file at packaging time, they are linked into a single
big C++ file at build time, that defines the various static consts in
XPTHeader.
In xpt.py, every data structure that can get written to disk gets an
additional code_gen() method that returns a representation of that
data structure as C++ source code. CodeGenData aggregates this
information together, handling deduplication and the final source code
generation.
The ctors are needed for XPTConstValue to statically initialize the
different union cases without resorting to designated initializers,
which are part of C99, not C++. Designated initializers appear to be
supported in C++ code by Clang and GCC, but not MSVC. The ctors must
be constexpr to ensure they are actually statically initialized so
they can be shared between Firefox processes.
I also removed an unnecessary "union" in XPTConstDescriptor.
Together, these patches reduce the amount of memory reported by
xpti-working-set from about 860,000 bytes to about 200,000 bytes. The
remaining memory is used for xptiInterface and xptiTypelibGuts (which
are thin wrappers around the XPT interfaces and header) and hash
tables to speed up looking up interfaces by name or IID. That could
potentially be eliminated from dynamic allocations in follow up
work. These patches did not affect memory reporting because XPT arenas
are still used by the remaining XPTI data structures.
MozReview-Commit-ID: Jvi9ByCPa6H
--HG--
extra : rebase_source : 719dfbcb9f83235c0f1f0766270b7f127f9ab04e
Now that XPT files are not loaded from files at runtime, code for
packaging XPT files can be removed.
This means that a couple of test XPIDL interfaces will get shipped in
builds to users that weren't before, but I don't think that matters
much.
This also puts XPT files into the local objdir for the XPIDL makefile,
instead of dist/bin, because they are no longer part of the
distribution.
MozReview-Commit-ID: 7gWj8KWUun3
--HG--
extra : rebase_source : 6f7d4fd1d6cdea2c14866705a2dc972eb5f43382
This adds a basic test for the mach formatter. This will ensure that changes to
this format are intentional. It will also make it easier for reviewers of these
changes to see a diff of the old vs new format.
MozReview-Commit-ID: LBSfdyvOPVV
--HG--
extra : rebase_source : 5529ad1f03306dcf867d88af579b69d6005091c0
Now that we're no longer shipping the SDK we no longer need real libraries for
the libraries that were created by this rule.
MozReview-Commit-ID: ALATVGBayHu
--HG--
extra : rebase_source : a20905125e5ff1846ef29de12323ba7b0a58928b
The MSVC linker winds up generating import libraries when linking some of
our executables, presumably because they contain functions that are
__declspec(dllexport). By default the import libraries get written
alongside the exe, so we force them to be written to the objdir so they don't
clutter up dist/bin.
MozReview-Commit-ID: 7DTfCo3OdDQ
Historically we built all our binaries in directories in the objdir, then
symlinked them into dist/bin. Some binaries needed to be copied instead
so that certain relative path lookups work properly, so we resorted to
sprinkling `NSDISTMODE=copy` around Makefiles.
This change makes it so we build PROGRAMs (not any other sort of targets)
directly in dist/bin instead. We could do the same for our other targets
with a little more work.
There were several places in the tree that were copying built binaries to
some other place and needed fixup to match the new location of binaries.
On Windows pdb files are left in the objdir where the program was
originally linked. symbolstore.py needs to locate the pdb file both to
determine whether it should dump symbols for a binary and also to copy
the pdb file into the symbol package. We fix this by simply looking for
the pdb file in the current working directory if it isn't present next
to the binary, which matches how we invoke symbolstore.py.
MozReview-Commit-ID: 8TOD1uTXD5e
We use a wrapper script when compiling with MSVC to parse the
/showIncludes output and thereby generate a Makefile dependency
fragment. This fragment enables us to do correct and faster incremental
builds. But the cost of invoking the wrapper script can be significant;
it's an extra process or two to launch for every single compilation.
Instead, let's have clang-cl generate the dependencies directly, which
should be somewhat faster.
There are a lot of choices and moving pieces in this commit. I elected
to include the mechanics and the target use case in the same commit so
that readers can compare and contrast the implementation and final
expression in one review window.
- Initially, I wanted to make the {AB_CD} substitutions in
LOCALIZED_FILES and not in LOCALIZED_GENERATED_FILES. However, I ran
into conceptual blockers doing this. Fundamentally, LOCALIZED_FILES
is FINAL_TARGET_FILES, and my use case should _not_ be putting files
anywhere near dist/bin. In addition, LOCALIZED_FILES
(FINAL_TARGET_FILES) is handled using manifests, which would need to
grow locale-aware functionality to handle this. That's not desirable.
In addition, if we use manifests, then we lose the powerful locality
of |mach build mobile/android{/base}| re-generating changed
locale-dependent resources. This is similar to how the build system
plumbs dist/idl manifest processing throughout the build: we're
repairing local workflows after moving work into a global process.
For these reasons, this doesn't support {AB_CD} in LOCALIZED_FILES.
- There is even another layer of complexity! There are two axes
involved with these files: AB_CD controls localization and the Make
target controls destination. For the record, it is:
regular builds - AB_CD unset
multi-locale builds - AB_CD set
single-locale repacks - AB_CD set
For the record, the existing logic (before any changes) is:
regular builds - Make target is `libs` in mobile/android/base/locales
multi-locale builds - Make target is `chrome-%` in mobile/android/base/locales
single-locale repacks - Make target is `libs` in mobile/android/base/locales
This commit adds targets for both destinations, and uses Make
chrome-%:: and libs:: magic to control what is invoked in the various
situations. Tricky!
- I added MERGE_RELATIVE_FILES in order to be able to follow-up this
patch with more patches that will get rid of
m/a/base/locales/{moz.build,Makefile.in} altogether, and fold this work
into m/a/base. As it stands, we're already reaching from
m/a/base/locales all the way out to
mobile/locales/.../region.properties, so the existing code doesn't
follow the layout expected between mozilla-central and
l10n-central/$(AB_CD). But that'll impedance will get worse as we
improve the build system dependencies, not better, so we should grow
support for localized resources that aren't exactly as expected.
- I chose to follow Python's syntax for string substitutions. I
would have preferred to mark files that should be localized with a
leading '%'... but I took that for filesystem absolute paths in
moz.build files already. I also considered @AB_CD@ to echo the
preprocessor, but didn't want to open the door to an expecation that
_all_ preprocessor DEFINEs will work in the way {AB_CD} does.
- The generate_*py script changes required a bit of a hack to "turn
off" locale dependent resources. This would have been nicer if we had
marked localized resources with '%'... but we didn't. See the
--fallback flag. The real reason this is needed is that we're doing
work which is more like the work of compare-locales (merging
locale-dependent resources) at build-time rather than repack time. I
don't know why that's the case -- probably when we (I) implemented it,
compare-locales and the whole l10n process was entirely opaque. It's
not worth changing it now, so we use this --fallback flag approach.
- I didn't get to tup support. This should gently fail without
breaking tup builds: any {AB_CD} substitutions just won't be
expanded. I haven't a clue how this should work in tup in the future
(or, more generally, how to make any sense of repacks without
declaring the full set of expected locales at configure time.)
- strings.xml can't be a LOCALIZED_PP_FILES, since we need to
customize the output location based on AB_rCD, and since we need a
little more flexibility than PP_FILES gives for our inputs.
MozReview-Commit-ID: MyfIkNSEzt
--HG--
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/foo-data => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/foo-data
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/generate-foo.py => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/generate-foo.py
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/inner/locales/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/moz.build => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/moz.build
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/non-localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/non-localized-input
extra : rebase_source : 816b6f220758f2bb3bdd3ec81a2cb02269c6de5b
I wanted to lift this next to the definition of AB_CD, but that
doesn't allow to use it in a backend.mk file, due to the order in
which Makefile, config.mk, rules.mk, and backend.mk are processed.
Therefore, I've put it in a tiny include file, so that it can be used
by a Makefile and a backend.mk file.
This allows the `RecursiveMake` backend to owning defining AB_rCD in
backend.mk files, while not requiring consumers to arrange for AB_rCD
in a sibling Makefile.in file.
Other build backends will need to arrange for AB_rCD themselves: see
following commits.
MozReview-Commit-ID: I7GIzRbCCtf
--HG--
extra : rebase_source : 3277fedb43bc3d8007287c223554a085dae2f198
extra : source : 854c0f43a1f74b4e22aa7638b407580240c90dd5
The last APK produced using the ANDROID_APK_* moz.build/Makefile.in
mechanism was Robocop, so we can get rid of these now.
MozReview-Commit-ID: 9b08ZvvOAoC
--HG--
extra : rebase_source : ac4fea057bf6e731b0f26a1b6902f17a7362076d
Far more files #include "util/Windows.h" than <psapi.h>, so this makes the
style-checker check more things (and makes --fixup mode fix more things).
--HG--
extra : rebase_source : a0c4edf10ad7dd4b5e0265c49035acf242197e6c
This is ancient and the team that used it (gfx) is no longer using it.
MozReview-Commit-ID: HrDgmAU9QeW
--HG--
extra : rebase_source : c4a64965c4ae1a50888893e881a6e8a9688a58b6
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
David Teller
Andrew McCreight
Dorel Luca
André Bargull
Ciure Andrei
Jeff Walden
shindli
David Major
Dave Hunt
Andrew Halberstadt
Chris Manchester
Ted Mielczarek
Sebastian Hengst
Jan de Mooij
Nathan Froyd
Margareta Eliza Balazs
ffxbld
Philipp Kewisch
Nick Alexander
arthur.iakab
Kim Moir
Jason Orendorff
Gurzau Raul