This setup seems to work well enough to enable me to link
HOST_SIMPLE_PROGRAMS with an AArch64-cross setup. Necessary library
paths are passed to the linker via -LIBPATH and HOST_LDFLAGS rather than
letting MSVC fish them out of the environment. The change to
HOST_SIMPLE_PROGRAMS to pass HOST_LDFLAGS was necessary for this to
work, in addition to the HOST_LINKER changes.
yasm doesn't support aarch64, and trying to use GNU as with an MSVC
build seems like sadness waiting to happen. Instead, we'll generate our
own assembly file that armasm64 will accept.
For "real" Windows-to-Windows cross compiles, the setting of
HOST_OUTOPTION is incorrect: it assumes that if we are cross-compiling,
we'll be using `-o ` (GNU-style) rather than `-Fo` (MSVC-style). Our
normal x86 Windows automation builds are technically not
cross-compiles (host and target are both x86 Windows), so this case has
never bothered us before. But when compiling for AArch64 Windows, we
are really doing a cross-compile, and so we need to be more careful
about how we set this option; otherwise, host compilations will
mysteriously fail because they won't produce any output.
Summary:
This patch ports xptcodegen.py over to the new perfecthash.py system, removing
some special-case code generators, and taking advantage of the easier-to-use
interface.
In addition, the code was changed to take advantage of the endianness
information from Part 2, allowing us to avoid having to perform endianness swaps
at runtime when hashing nsIDs.
Depends On D2616
Reviewers: froydnj!
Tags: #secure-revision
Bug #: 1479484
Differential Revision: https://phabricator.services.mozilla.com/D2618
The file is added both unconditionally and conditionally to no effect.
MozReview-Commit-ID: 4t57o0bTF9P
--HG--
extra : rebase_source : c5f7bbb20e094cd4f7a9fcd1e143da90069b73b6
The file is added both unconditionally and conditionally to no effect.
MozReview-Commit-ID: JBZKN2qRf73
--HG--
extra : rebase_source : 74b601ae1d64f8ed7a47b02f79cbef66ac606abc
This implements an API in `nsIOSKeyStore.idl` and `OSKeyStore.cpp` to encrypt and decrypt bytes with a key that is stored in the OS key store.
There are two OS adapters in this patch.
Libsecret is used on Linux if available.
The NSS key store is used as fallback if no OS specific key store is implemented.
Differential Revision: https://phabricator.services.mozilla.com/D1858
--HG--
extra : rebase_source : 99d7d646968a46a13ffa61885bb246f6d3e443e4
The deletions in xptcall are when we don't even have support for the CPU
in moz.configure, so I assume that people haven't been compiling on
those architectures for quite some time.
Use wl_keyboard_listener and keymap event to get key mapping on Wayland. Weston simple-im.c example
is used as a reference implementation and actual key modifiers are derived from Wayland/GDK code from
gdkkeys-wayland.c.
MozReview-Commit-ID: 9fMwCvxkYy0
--HG--
extra : rebase_source : 21212cadfa7b1e8bacec2d6fb6970d2aaba7b3f6
For clang-cl, we want to add code to libxul that only exists during the
PGO generation phase, so we can collect data. The most expedient way to
do that is to enable certain files in SOURCES to be marked as to only be
compiled during the PGO generation step.
This adds just enough host shared library support for this one use case,
but also takes shortcuts, because fully supporting host shared library
is a deep rabbit hole I'm not ready to take just to fix --enable-lto
--enable-clang-plugin on mac builds.
One downside is that one my machine the plugin now takes > 80s to build,
instead of 15s before, thanks to the lack of unified sources.
--HG--
extra : rebase_source : bf52a72a01d4e3eb77cf52b646b19734b9273075
Many Rust build scripts compile C/C++ source files with the cc crate, but
cargo doesn't print the output of build scripts unless you pass `-vv`, so
pass that instead of just `--verbose` to get that output in automation and
builds where verbose output was requested.
MozReview-Commit-ID: EUazlKWFsDw
--HG--
extra : rebase_source : 137882c4e970bb0e7b28cc36d7f8e436b59a8011
This will make sure that when running |mach python-test --python 3| locally,
we only run the tests that also run in CI with python 3 (and therefore pass
presumably).
MozReview-Commit-ID: 3OBr9yLSlSq
--HG--
extra : rebase_source : 456340d0ecdddf1078f2b5b4ebb1eddf3813b26a
We perform, on the binaries we build, a series of check, that are
implemented as half-baked make commands, invoked after linking them.
- check libstdc++ symbol versions to ensure binary compatibility with
a baseline.
- check glibc symbol versions to ensure binary compatibility with a
baseline.
- check that target binaries don't contain text relocations.
- check that libmozglue is linked before libc on android.
- on libxul, check that NSModules are laid out correctly.
- on libxul, check that there is more than one PT_LOAD segment.
Those checks happen to work where they matter, but their setup is
unreliable. For example, the checks for symbol versions are supposed to
work for libclang-plugin on cross osx builds, but in fact, don't,
because the readelf path doesn't exist, and the command doesn't fail in
that case.
So move them all to a standalone script, performing the checks more
thoroughly (especially the NSModules one, where we now also check that
they are all adjacent), and more verbosely.
--HG--
extra : rebase_source : 7072e622e95f363d4a6c3a8e272d3445d998b592
This changes two config options:
pytest_classes = PyTest # only classes that start with 'PyTest' will be considered tests (previously this was Test)
xfail_strict = true # tests marked as xfail will cause pytest to return non-zero if they unexpectedly pass
MozReview-Commit-ID: DCWoDFbe6Mk
--HG--
extra : rebase_source : 9aa806e035d62d51bb338708396851c40f55ee00
The crash reporter symbol files are the easiest cross-platform way to
find static initializers. While some types of static initializers (e.g.
__attribute__(constructor) functions) don't appear there in a notable
way, the static initializers we do care the most about for tracking do
(static initializers from C++ globals). As a matter of fact, there is
only a difference of 2 compared to the currently reported count of 125
on a linux64 build, so this is a good enough approximation. And allows
us to easily track the count on Android, OSX and Windows builds, which
we currently don't do.
The tricky part is that the symbol files are in
dist/crashreporter-symbols/$lib/$fileid/$lib.sym, and $fileid is hard to
figure out. There is a `fileid` tool in testing/tools, but it is a
target binary, meaning it's not available on cross builds (OSX,
Android).
So the simplest is just to gather the data while creating the symbol
files, which unfortunately requires to go through some hoops to make it
happen for just the files we care about.
--HG--
extra : rebase_source : 458fed1ffd6f9294eefef61f10ff7a284af0d986
This one looks to be pretty straight-forward. It irritates me that
the jar.mn entry doesn't explicitly say that the result is coming from
the object directory, like
locale/browser/bookmarks.html (!bookmarks.html)
but that's for another day.
MozReview-Commit-ID: Cw8E0VJhSxv
--HG--
extra : rebase_source : a1045a5b564b0094b562729bc7234e69ec7a786d
Our bundled Hunspell now significantly differs from upstream Hunspell. Most
importantly, it supports loading dictionaries from jar: URIs, which is now a
requirement for loading bundled and extension dictionaries. This means that
system Hunspell libraries are no longer compatible with our spell checker
code. We should remove the option to use them so that users don't fall into
the trap of trying to use them.
MozReview-Commit-ID: 2ihJe6YOnGf
--HG--
extra : rebase_source : ceb091b9475a2b101156405a02a60015fc36da17
Original patch author is Takuro Ashie <ashie@clear-code.com>
Provide ability to create native EGL window and provide it under NS_NATIVE_EGL_WINDOW
to GL code. The native EGL window is owned/managed by mozcontainer.
MozReview-Commit-ID: 4d0Kk6DRSaD
--HG--
extra : rebase_source : e4677ce51fbf918eb1b0257c66ca4b7220174bbb
The build system knows at build-backend time where to find each IDL
file; making xpidl-process.py rediscover this by requiring
xpidl-process.py to search through directories to find input IDL files
is silly. To rememdy this, we're going to modify things so full paths
are passed into the script. Those paths can then be used directly, with
no searching.
The tail end of the xpidl Makefile.in contains a line, generated for
every xpt file:
$(1): $(addsuffix .idl,$(addprefix $(dist_idl_dir)/,$($(basename $(notdir $(1)))_deps)))
This line, in context, is saying that the xpt file depends on all of its
input IDL files. But xpidl-process.py already generates this
information when we pass it --depsdir, which we do. So this code is
redundant with what we already generate, and it can be removed.
The previous patch required us to pass a single -I argument pointing at
$(DIST)/idl so IDL include statements would work correctly. This patch
lifts that limitation and explicitly points xpidl-process.py at the
locations of all the IDL source directories to search for included IDL
files. Invocations of xpidl-process.py no longer depend on IDL files
being copied to the objdir.
Building on the last patch, we can change the build process to pass in
the directories where the input IDL files can be found. It is
convenient to pass in just the relative source directory paths, to
encourage people to not look in the object directory and to make the
command lines slightly shorter.
xpidl-process.py still assumes that included IDL files can be found by
looking in a single directory. We add a single -I argument to the
invocation of xpidl-process.py to accommodate this short-sightedness.
The current IDL build setup assumes that all IDL files can be found in a
single directory. This setup requires that all IDL files be copied to a
single directory, which is suboptimal in terms of disk I/O and also
complicates things like generating IDL files at build time.
As a first step in moving away from this state of affairs,
xpidl-process.py needs to be taught that the input IDL files could
potentially be found in multiple directories. The current setup can
just specify $(DIST)/idl as the lone directory to examine. Future
patches will change this to examine multiple directories.
The make backend was treating the first output of a GENERATED_FILES rule
specially, since it was the target of the rule containing the script
invocation. We want the outputs of GENERATED_FILES rules to be
FileAvoidWrite so that we avoid triggering downstream rules if the
outputs are unchanged, but if the target of the script invocation is
FileAvoidWrite, then make may continually re-run the script during a
no-op build.
The solution here is to use a stub file as the target of the script
invocation which will always be touched when the script runs. Since
nothing else in the build depends on the stub, we don't need to
FileAvoidWrite it. All actual outputs of the script can be
FileAvoidWrite, and make can properly avoid work for files that haven't
changed.
MozReview-Commit-ID: 3GejZw2tpqu
--HG--
extra : rebase_source : 2b9be82f893e89a4c2f254f05b1e8b9a0f9c631b
I don't understand how this will interact with the parts of the build
where we try to avoid installing the dist/bin manifest, but this makes
sense to me and it works locally for mobile/android and for browser/.
MozReview-Commit-ID: L7RtA4K3WrX
--HG--
extra : rebase_source : 3c08a5aab5398eb3b5685b18e5fe06e926db5f85
We want annotationProcessors to be compiled and archived into a JAR at
build time, ready to generate JNI wrappers. (That is, until we turn
the whole thing into a real annotation processor.) But even if we do
use a real annotation processor, we still need to generate SDK
bindings, which is less clearly expressed as an annotation processor.
(It's more of a build step.)
Gradle provides a huge number of ways to organize build logic to
achieve this: see
https://docs.gradle.org/current/userguide/organizing_build_logic.html.
Unfortunately, the best such way -- putting the code into
$topsrcdir/buildSrc -- has key disadvantages:
1) it pollutes the top-level $topsrcdir, and there's no way to change the
location of buildSrc (https://github.com/gradle/gradle/issues/2472);
2) it's complicated to have a dependent project
(mobile/android/annotations) expose its code via a buildSrc project;
3) using buildSrc at all appears to conflict with the Android-Gradle
plugin version that we are using.
Therefore, this commit does something much simpler: it adds a
Java-only project and uses the resulting Gradle "Jar" task and archive
output as input to the existing Gradle "generate JNI wrappers" task.
MozReview-Commit-ID: 2OyYLPneE1M
--HG--
extra : rebase_source : d99b74a0a1e0bb3e8f4d4540978328388e5c2e42
Some content in Makefile.in is removed because after this change, the
scripts no longer invoke the preprocessor and thus don't have unknown
dependencies anymore outside what is provided in their inputs array.
The order of exports.PREFERENCES in properties-db changes because the
data file has shorthands placed after longhands. The only usage of it
is in test_css-properties-db.js which doesn't care about the order.
MozReview-Commit-ID: AMjzTRf2HYN
--HG--
extra : rebase_source : 7976e48e7c7bba467d77a34ab0d7709cde1ecdf4
We add a minimal Python script to run a process and prefix all its
output with a string. We change the automation tiers to evaluate all
make targets using this script.
MozReview-Commit-ID: 79g5KUd5ked
--HG--
extra : rebase_source : 63388a71b51e5abc05ca8bd48e180af72bf799e6
Some content in Makefile.in is removed because after this change, the
scripts no longer invoke the preprocessor and thus don't have unknown
dependencies anymore outside what is provided in their inputs array.
The order of exports.PREFERENCES in properties-db changes because the
data file has shorthands placed after longhands. The only usage of it
is in test_css-properties-db.js which doesn't care about the order.
MozReview-Commit-ID: AMjzTRf2HYN
--HG--
extra : rebase_source : f9db0659a81bea28b335806ac70e23dc0d36e493
This patch contains the meat of the changes here. The following summarize the changes:
1. xptinfo.h is rewritten to expose the new interface for reading the XPT data,
The nsXPTInterfaceInfo object exposes methods with the same signatures as
the methods on nsIInterfaceInfo, to make converting code which used
nsIInterfaceInfo as easy as possible, even when those methods don't have
signatures which make a ton of sense anymore. There are also a few methods
which are unnecessary (they return `true` or similar), which should be
removed over time.
Members of the data structures are made private in order to prevent reading
them directly. Code should instead call the getter methods. This should make
it easier to change their memory representation in the future. Constructing
these structs is made possible by making the structs `friend class` with the
XPTConstruct class, which is implemented by the code generator, and is able
to access the private fields.
In addition, rather than using integers with flag constants, I opted for
using C++ bitfields to store individual flags, as I found it made it easier
to both write the code generator, and reason about the layouts of the types.
I was able to shave a byte off of each nsXPTParamInfo (4 bytes -> 3 bytes)
by shoving the flags into spare bits in the nsXPTType. Unfortunately there
was not enough room for the retval flag. Fortunately, we already depend in
our code on the retval parameter being the last parameter, so I worked
around this by removing the retval flag and instead having a `hasretval`
flag on the method itself.
2. An xptinfo.cpp file is added for out-of-line definitions of more complex
methods, and the internal implementation details of the perfect hash.
Notable is the handling of xptshim interfaces. As the type is uniform, a
flag is checked when trying to read constant information, and a different
table with pointers into webidl data structures is checked when the type is
determined to be a shim.
Ideally we could remove this once we remove the remaining consumers of the
existing shim interfaces.
3. A python code generator which takes in the json XPT files generated in the
previous part, and emits a xptdata.cpp file with the data structures. I did
my best to heavily comment the code.
This code uses the friend class trick to construct the private fields of the
structs, and avoid a dependency on the ordering of fields in xptinfo.h.
The sInterfaces array's order is determined by a generated perfect hash
which is also written into the binary. This should allow for fast lookups by
IID or name of interfaces in memory. The hash function used for the perfect
hash is a simple FNV hash, as they're pretty fast.
For perfect hashing of names, another table is created which contains
indexes into the sInterfaces table. Lookup by name is less common, and this
form of lookup should still be very fast.
4. The necessary Makefiles are updated to use the new code generator, and
generate the file correctly.
The uninstaller was being built as a side-effect of building `setup.exe`. In
Bug 1385227, that was moved from "somewhere" to part of the windows installer
packaging, which happens after the zip and mar are generated. Since the
installer we ship is actually repackaged from the zip[1], we stopped shipping
translated uninstallers.
This changes things around so that the uninstaller gets translated:
- Explicitly build the uninstaller as part of the L10n repack step.
- Use the same logic to build the installer locally as we do to create the ones
we ship.
[1] Except on Thunderbird
Differential Revision: https://phabricator.services.mozilla.com/D672
--HG--
extra : rebase_source : 05fe935c1d2a9fbfeef786819bfe5913ed8ef862
extra : source : d6bf22099e2195dcb64c3c3d7700d3edd0850a3a
The uninstaller was being built as a side-effect of building `setup.exe`. In
Bug 1385227, that was moved from "somewhere" to part of the windows installer
packaging, which happens after the zip and mar are generated. Since the
installer we ship is actually repackaged from the zip[1], we stopped shipping
translated uninstallers.
This changes things around so that the uninstaller gets translated:
- Explicitly build the uninstaller as part of the L10n repack step.
- Use the same logic to build the installer locally as we do to create the ones
we ship.
[1] Except on Thunderbird
Differential Revision: https://phabricator.services.mozilla.com/D672
--HG--
extra : rebase_source : 2b28b9ff7196d12f4a188c8dddf750b9a5efac5b
extra : histedit_source : 9bc28891950ae8c226cfdefef6f8121ce0b51f58
This patch is a nearly complete reimplementation of BinASTReader, with the following changes:
- Files BinToken.h, BinSource-auto.h (new), BinSource-auto.cpp (new) are now autogenerated by the generator in js/src/frontend/binsouce from the webidl specifications of BinAST and a small
configuration file.
- Optional fields have been removed. Rather, some specific fields may, if so marked in the specifications, contain a Null constant.
- `hasDirectEval` is now checked for consistency (NOT completeness).
- `varDeclaredNames` is now checked for consistency (NOT completeness).
- `lexicallyDeclaredNames` is now checked for consistency (NOT completeness).
- `parameterNames` is now checked for consistency (NOT completeness).
- `capturedNames` is NOT checked.
- Atoms read are now properly expected to be UTF8.
This patch does not implement the entire specifications, but should implement most of ES5. In particular, it is sufficient to parse the source code of:
- Facebook;
- jQuery;
- mootools;
- Underscore;
- Backbone;
- Angular.
MozReview-Commit-ID: HwkVB5dliZv
--HG--
extra : rebase_source : fd7e068343e2af8926c5185e7199ea110a5149bc
This patch handles the actual generation of the static data structures
used to represent XPT information. XPT files are generated in the same
way as they are now, but they are used only as an intermediate
representation to speed up incremental compilation rather than
something used by Firefox itself. Instead of linking XPTs into a
single big XPT file at packaging time, they are linked into a single
big C++ file at build time, that defines the various static consts in
XPTHeader.
In xpt.py, every data structure that can get written to disk gets an
additional code_gen() method that returns a representation of that
data structure as C++ source code. CodeGenData aggregates this
information together, handling deduplication and the final source code
generation.
The ctors are needed for XPTConstValue to statically initialize the
different union cases without resorting to designated initializers,
which are part of C99, not C++. Designated initializers appear to be
supported in C++ code by Clang and GCC, but not MSVC. The ctors must
be constexpr to ensure they are actually statically initialized so
they can be shared between Firefox processes.
I also removed an unnecessary "union" in XPTConstDescriptor.
Together, these patches reduce the amount of memory reported by
xpti-working-set from about 860,000 bytes to about 200,000 bytes. The
remaining memory is used for xptiInterface and xptiTypelibGuts (which
are thin wrappers around the XPT interfaces and header) and hash
tables to speed up looking up interfaces by name or IID. That could
potentially be eliminated from dynamic allocations in follow up
work. These patches did not affect memory reporting because XPT arenas
are still used by the remaining XPTI data structures.
MozReview-Commit-ID: Jvi9ByCPa6H
--HG--
extra : rebase_source : a9e48e7026aab4ad1b7f97e50424adf4e3f4142f
Now that XPT files are not loaded from files at runtime, code for
packaging XPT files can be removed.
This means that a couple of test XPIDL interfaces will get shipped in
builds to users that weren't before, but I don't think that matters
much.
This also puts XPT files into the local objdir for the XPIDL makefile,
instead of dist/bin, because they are no longer part of the
distribution.
MozReview-Commit-ID: 7gWj8KWUun3
--HG--
extra : rebase_source : 65bac47c2cd1a20b3c675a01b44a25a1d2d3ab7a
This patch handles the actual generation of the static data structures
used to represent XPT information. XPT files are generated in the same
way as they are now, but they are used only as an intermediate
representation to speed up incremental compilation rather than
something used by Firefox itself. Instead of linking XPTs into a
single big XPT file at packaging time, they are linked into a single
big C++ file at build time, that defines the various static consts in
XPTHeader.
In xpt.py, every data structure that can get written to disk gets an
additional code_gen() method that returns a representation of that
data structure as C++ source code. CodeGenData aggregates this
information together, handling deduplication and the final source code
generation.
The ctors are needed for XPTConstValue to statically initialize the
different union cases without resorting to designated initializers,
which are part of C99, not C++. Designated initializers appear to be
supported in C++ code by Clang and GCC, but not MSVC. The ctors must
be constexpr to ensure they are actually statically initialized so
they can be shared between Firefox processes.
I also removed an unnecessary "union" in XPTConstDescriptor.
Together, these patches reduce the amount of memory reported by
xpti-working-set from about 860,000 bytes to about 200,000 bytes. The
remaining memory is used for xptiInterface and xptiTypelibGuts (which
are thin wrappers around the XPT interfaces and header) and hash
tables to speed up looking up interfaces by name or IID. That could
potentially be eliminated from dynamic allocations in follow up
work. These patches did not affect memory reporting because XPT arenas
are still used by the remaining XPTI data structures.
MozReview-Commit-ID: Jvi9ByCPa6H
--HG--
extra : rebase_source : 719dfbcb9f83235c0f1f0766270b7f127f9ab04e
Now that XPT files are not loaded from files at runtime, code for
packaging XPT files can be removed.
This means that a couple of test XPIDL interfaces will get shipped in
builds to users that weren't before, but I don't think that matters
much.
This also puts XPT files into the local objdir for the XPIDL makefile,
instead of dist/bin, because they are no longer part of the
distribution.
MozReview-Commit-ID: 7gWj8KWUun3
--HG--
extra : rebase_source : 6f7d4fd1d6cdea2c14866705a2dc972eb5f43382
This adds a basic test for the mach formatter. This will ensure that changes to
this format are intentional. It will also make it easier for reviewers of these
changes to see a diff of the old vs new format.
MozReview-Commit-ID: LBSfdyvOPVV
--HG--
extra : rebase_source : 5529ad1f03306dcf867d88af579b69d6005091c0
Now that we're no longer shipping the SDK we no longer need real libraries for
the libraries that were created by this rule.
MozReview-Commit-ID: ALATVGBayHu
--HG--
extra : rebase_source : a20905125e5ff1846ef29de12323ba7b0a58928b
The MSVC linker winds up generating import libraries when linking some of
our executables, presumably because they contain functions that are
__declspec(dllexport). By default the import libraries get written
alongside the exe, so we force them to be written to the objdir so they don't
clutter up dist/bin.
MozReview-Commit-ID: 7DTfCo3OdDQ
Historically we built all our binaries in directories in the objdir, then
symlinked them into dist/bin. Some binaries needed to be copied instead
so that certain relative path lookups work properly, so we resorted to
sprinkling `NSDISTMODE=copy` around Makefiles.
This change makes it so we build PROGRAMs (not any other sort of targets)
directly in dist/bin instead. We could do the same for our other targets
with a little more work.
There were several places in the tree that were copying built binaries to
some other place and needed fixup to match the new location of binaries.
On Windows pdb files are left in the objdir where the program was
originally linked. symbolstore.py needs to locate the pdb file both to
determine whether it should dump symbols for a binary and also to copy
the pdb file into the symbol package. We fix this by simply looking for
the pdb file in the current working directory if it isn't present next
to the binary, which matches how we invoke symbolstore.py.
MozReview-Commit-ID: 8TOD1uTXD5e
We use a wrapper script when compiling with MSVC to parse the
/showIncludes output and thereby generate a Makefile dependency
fragment. This fragment enables us to do correct and faster incremental
builds. But the cost of invoking the wrapper script can be significant;
it's an extra process or two to launch for every single compilation.
Instead, let's have clang-cl generate the dependencies directly, which
should be somewhat faster.
There are a lot of choices and moving pieces in this commit. I elected
to include the mechanics and the target use case in the same commit so
that readers can compare and contrast the implementation and final
expression in one review window.
- Initially, I wanted to make the {AB_CD} substitutions in
LOCALIZED_FILES and not in LOCALIZED_GENERATED_FILES. However, I ran
into conceptual blockers doing this. Fundamentally, LOCALIZED_FILES
is FINAL_TARGET_FILES, and my use case should _not_ be putting files
anywhere near dist/bin. In addition, LOCALIZED_FILES
(FINAL_TARGET_FILES) is handled using manifests, which would need to
grow locale-aware functionality to handle this. That's not desirable.
In addition, if we use manifests, then we lose the powerful locality
of |mach build mobile/android{/base}| re-generating changed
locale-dependent resources. This is similar to how the build system
plumbs dist/idl manifest processing throughout the build: we're
repairing local workflows after moving work into a global process.
For these reasons, this doesn't support {AB_CD} in LOCALIZED_FILES.
- There is even another layer of complexity! There are two axes
involved with these files: AB_CD controls localization and the Make
target controls destination. For the record, it is:
regular builds - AB_CD unset
multi-locale builds - AB_CD set
single-locale repacks - AB_CD set
For the record, the existing logic (before any changes) is:
regular builds - Make target is `libs` in mobile/android/base/locales
multi-locale builds - Make target is `chrome-%` in mobile/android/base/locales
single-locale repacks - Make target is `libs` in mobile/android/base/locales
This commit adds targets for both destinations, and uses Make
chrome-%:: and libs:: magic to control what is invoked in the various
situations. Tricky!
- I added MERGE_RELATIVE_FILES in order to be able to follow-up this
patch with more patches that will get rid of
m/a/base/locales/{moz.build,Makefile.in} altogether, and fold this work
into m/a/base. As it stands, we're already reaching from
m/a/base/locales all the way out to
mobile/locales/.../region.properties, so the existing code doesn't
follow the layout expected between mozilla-central and
l10n-central/$(AB_CD). But that'll impedance will get worse as we
improve the build system dependencies, not better, so we should grow
support for localized resources that aren't exactly as expected.
- I chose to follow Python's syntax for string substitutions. I
would have preferred to mark files that should be localized with a
leading '%'... but I took that for filesystem absolute paths in
moz.build files already. I also considered @AB_CD@ to echo the
preprocessor, but didn't want to open the door to an expecation that
_all_ preprocessor DEFINEs will work in the way {AB_CD} does.
- The generate_*py script changes required a bit of a hack to "turn
off" locale dependent resources. This would have been nicer if we had
marked localized resources with '%'... but we didn't. See the
--fallback flag. The real reason this is needed is that we're doing
work which is more like the work of compare-locales (merging
locale-dependent resources) at build-time rather than repack time. I
don't know why that's the case -- probably when we (I) implemented it,
compare-locales and the whole l10n process was entirely opaque. It's
not worth changing it now, so we use this --fallback flag approach.
- I didn't get to tup support. This should gently fail without
breaking tup builds: any {AB_CD} substitutions just won't be
expanded. I haven't a clue how this should work in tup in the future
(or, more generally, how to make any sense of repacks without
declaring the full set of expected locales at configure time.)
- strings.xml can't be a LOCALIZED_PP_FILES, since we need to
customize the output location based on AB_rCD, and since we need a
little more flexibility than PP_FILES gives for our inputs.
MozReview-Commit-ID: MyfIkNSEzt
--HG--
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/foo-data => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/foo-data
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/generate-foo.py => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/generate-foo.py
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/inner/locales/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/moz.build => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/moz.build
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/non-localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/non-localized-input
extra : rebase_source : 816b6f220758f2bb3bdd3ec81a2cb02269c6de5b
I wanted to lift this next to the definition of AB_CD, but that
doesn't allow to use it in a backend.mk file, due to the order in
which Makefile, config.mk, rules.mk, and backend.mk are processed.
Therefore, I've put it in a tiny include file, so that it can be used
by a Makefile and a backend.mk file.
This allows the `RecursiveMake` backend to owning defining AB_rCD in
backend.mk files, while not requiring consumers to arrange for AB_rCD
in a sibling Makefile.in file.
Other build backends will need to arrange for AB_rCD themselves: see
following commits.
MozReview-Commit-ID: I7GIzRbCCtf
--HG--
extra : rebase_source : 3277fedb43bc3d8007287c223554a085dae2f198
extra : source : 854c0f43a1f74b4e22aa7638b407580240c90dd5
The last APK produced using the ANDROID_APK_* moz.build/Makefile.in
mechanism was Robocop, so we can get rid of these now.
MozReview-Commit-ID: 9b08ZvvOAoC
--HG--
extra : rebase_source : ac4fea057bf6e731b0f26a1b6902f17a7362076d
Far more files #include "util/Windows.h" than <psapi.h>, so this makes the
style-checker check more things (and makes --fixup mode fix more things).
--HG--
extra : rebase_source : a0c4edf10ad7dd4b5e0265c49035acf242197e6c
This is ancient and the team that used it (gfx) is no longer using it.
MozReview-Commit-ID: HrDgmAU9QeW
--HG--
extra : rebase_source : c4a64965c4ae1a50888893e881a6e8a9688a58b6
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
Now that writing the .purgecaches sentinel is done by |mach build|, we
can remove it from Make. In addition, we can cull the now-unused
app-rules.mk \o/
MozReview-Commit-ID: 6CnAqLeZwzB
--HG--
extra : rebase_source : 2a5b67e723f8e1654627e6e6f26704f6a4e57706
I very much doubt these are used, but even if we are -- we shouldn't
support this type of local customization, since it doesn't extend to
non-Make-based backends.
With the customization point removed, there's no way to set ETAGS, so
we remove what little support there was for generating Emacs tags.
MozReview-Commit-ID: IEF2Q4tISEn
--HG--
extra : rebase_source : 3bc8e651c03517edb797032db6ce60ed8852d9fa
Now that writing the .purgecaches sentinel is done by |mach build|, we
can remove it from Make. In addition, we can cull the now-unused
app-rules.mk \o/
MozReview-Commit-ID: 6CnAqLeZwzB
--HG--
extra : rebase_source : 7427ca9aecc6619f48090c6a26c9a2e0e7d37bad
I very much doubt these are used, but even if we are -- we shouldn't
support this type of local customization, since it doesn't extend to
non-Make-based backends.
With the customization point removed, there's no way to set ETAGS, so
we remove what little support there was for generating Emacs tags.
MozReview-Commit-ID: IEF2Q4tISEn
--HG--
extra : rebase_source : 140e07f78dbd72391c70c48f25e2b85c924fe78d
Fix several problems when building Breakpad with new NDK unified
headers.
- Unified headers define its own tgkill wrapper, so rename our own
wrapper to __tgkill.
- Unified headers define user_fpxregs_struct for all API levels, so
don't redefine it.
- Only the target sources under google-breakpad/src/common/linux should
use custom Android headers, so change the includes line in moz.build to
use OS_INCLUDES.
MozReview-Commit-ID: HGnUMu5vDUM
--HG--
extra : rebase_source : f5c29c9949a48a8376f84dcc676e5f8df886f130
Fix several problems when building Breakpad with new NDK unified
headers.
- Unified headers define its own tgkill wrapper, so rename our own
wrapper to __tgkill.
- Unified headers define user_fpxregs_struct for all API levels, so
don't redefine it.
- Only the target sources under google-breakpad/src/common/linux should
use custom Android headers, so change the includes line in moz.build to
use OS_INCLUDES.
MozReview-Commit-ID: HGnUMu5vDUM
--HG--
extra : rebase_source : 07da575044ca805a3f7f8ba87be8a8c279f17379
This patch adds basic support for the fuzzing interface in the JS engine on top
of the last patch. This includes all the necessary code except for actual
targets (just an example target skeleton) and also makes sure that the fuzzing
code is packaged for the standalone release.
MozReview-Commit-ID: D6Tyebz3jZS
--HG--
extra : rebase_source : f07a5d2bf8e09fb6c93501e58fbb958004e2fa05
This patch adds basic support for the fuzzing interface in the JS engine on top
of the last patch. This includes all the necessary code except for actual
targets (just an example target skeleton) and also makes sure that the fuzzing
code is packaged for the standalone release.
MozReview-Commit-ID: D6Tyebz3jZS
--HG--
extra : rebase_source : 58e4d85e657347b061de0ed912365f2a955a86e3
This patch disables the stdcxx-compat check for the sm-fuzzing build which
requires patching autospider as well. Furthermore, it switches the build
to linux64-clang-6-pre because the older clang 3.9 does not support trace-pc
instrumentation. Finally, it excludes fuzzing parts from the vanilla allocation
check.
MozReview-Commit-ID: FdhCIFdUore
--HG--
extra : rebase_source : c41bda01cb42f2ef0cd5a1675d88bdb55d9dc8c9
The MSVC linker winds up generating import libraries when linking some of
our executables, presumably because they contain functions that are
__declspec(dllexport). By default the import libraries get written
alongside the exe, so we force them to be written to the objdir so they don't
clutter up dist/bin.
MozReview-Commit-ID: 7DTfCo3OdDQ
--HG--
extra : rebase_source : fea69e8f60633b824726269c2296af9fe812d3ed
Historically we built all our binaries in directories in the objdir, then
symlinked them into dist/bin. Some binaries needed to be copied instead
so that certain relative path lookups work properly, so we resorted to
sprinkling `NSDISTMODE=copy` around Makefiles.
This change makes it so we build PROGRAMs (not any other sort of targets)
directly in dist/bin instead. We could do the same for our other targets
with a little more work.
There were several places in the tree that were copying built binaries to
some other place and needed fixup to match the new location of binaries.
On Windows pdb files are left in the objdir where the program was
originally linked. symbolstore.py needs to locate the pdb file both to
determine whether it should dump symbols for a binary and also to copy
the pdb file into the symbol package. We fix this by simply looking for
the pdb file in the current working directory if it isn't present next
to the binary, which matches how we invoke symbolstore.py.
MozReview-Commit-ID: 8TOD1uTXD5e
--HG--
extra : rebase_source : 9140be949b206bb595d9188ce7e8357347ecd9a9
Work around excessive command-line length issues by
disabling incremental rust compilation, which is enabled
by default outside `cargo --release` starting with Rust 1.24.
Incremental rust builds shouldn't help much in automation,
where sccache provides the only continuity between build
environments. In the meantime, they add a lot of object
files to the link line.
See https://github.com/rust-lang/rust/pull/47507 about addressing
the underlying issue upstream.
MozReview-Commit-ID: LRwUj3fhiaO
--HG--
extra : rebase_source : 1739a7570b2e7fe40ead3b301ea20c2fe79f0431
The pytest cache plugin writes its cache in the srcdir, which means that it
shows up in `hg status`, which is annoying. Writing files to the srcdir is
generally bad practice anyway, so we disable this plugin to stop this
from happening.
MozReview-Commit-ID: HytLLMUtKlc
--HG--
extra : rebase_source : f6acbf3650881312cef051126387220a0f78597f
Back when I filed this bug, $appname-$version was already useless, but
there actually were multiple supported versions (e.g. 3.5/3.6). So it
made sense to have e.g. firefox-3.5 and firefox-3.6, but not
firefox-3.5.12 and firefox-3.5.13.
Fast forward 10 years, and we change "major" versions every 6 to 8
weeks, have multiple chemspills every other week, and installing to
firefox-57.0, firefox-57.0.1, firefox-57.0.2, firefox-57.0.3 doesn't
make any kind of sense. Even firefox-56, firefox-57 is pretty much
useless.
There /kind/ of was some usefulness to the version in the SDK
directories, but those are gone.
I'm pretty sure no downstream is actually using versioned directories
anyways.
At this point, it seems better to just use the application name, without
the version. A case could be made about ESR, but that would be better
handled with a separate application name (e.g. firefox-esr).
--HG--
extra : rebase_source : 333e10ea1316714bf0008ec772b35093edfc45ff
Some of the variants are multi-platforms, and having
--enable-stdcxx-compat in their definition will break when the option is
moved over to python configure.
While here, prepare for --enable-stdcxx-compat actually doing something
(it currently doesn't), by adding an exception for it in
check_vanilla_allocations.py.
--HG--
extra : rebase_source : ee1647421542209cf0137db703c4f7e7f06cbc91
This has the virtue of not executing python three times during configure
just to read the same value of milestone.txt and munge it. We can also
remove milestone.py as a happy side effect, so all the milestone
computations can be done in init.configure.
We only had this "for flexibility with other platforms", but given that
we set it to the same thing for all platforms, and nobody has tried to
change that, this flexibility isn't actually needed.
Bug 1256642 introduced magic at the emitter level to determine whether a
binary contains C++ sources and should be linked with the C compiler or
the C++ compiler.
Unfortunately, the Binary() moz.build template always adds C++ OS
libraries on Android (through STLPORT_LIBS), and C++ libraries on Linux
(stdc++compat).
The latter only ends up forcing every Binary() to be linked with the C++
linker, which is unfortunate, but doesn't cause much problems. The
former, however, involving OS libraries, the magic from bug 1256642
doesn't kick in, so we end up trying to link C++ OS libraries with the C
linker. Which ends up failing, because the libraries in STLPORT_LIBS
require -lm, which, while it's added by the C++ compiler when linking,
is not when the linkage is driven by the C compiler.
Because the fallible library, linked to all GeckoBinary()s is a C++
library, we still ended up linking with the C++ compiler on Android, so
this wasn't actually causing any problem... until I tried to remove that
fallible library in bug 1423803.
Anyways, the core problem is that moz.build evaluation is happening too
early to know whether any C++ sources are being linked together, so
there is no way the Binary() template can do the right thing. So this
change moves the logic to the emitter.
This also changes the type of STLPORT_LIBS to a list.
--HG--
extra : rebase_source : a70ddf7a132f94dc10e7e1db94ae80fb8d7a269f
Marco Castelluccio
Andreea Pavel
Nathan Froyd
Mike Hommey
Akshay Chiwhane
Kartikaya Gupta
André Bargull
Tom Ritter
Chris Manchester
Narcis Beleuzu
Aaron Klotz
Noemi Erli
Nika Layzell
Karl Tomlinson
Ted Mielczarek
Franziskus Kiefer
David Major
Dave Hunt
Martin Stransky
Jacek Caban
Nathan Froyd
Bogdan Tara
Henri Sivonen
Kris Maglione
Andrew Halberstadt
Xidorn Quan
ffxbld
Nick Alexander
Robin Templeton
Sylvestre Ledru
Dorel Luca
Mike Shal
André Bargull
Ciure Andrei
Gurzau Raul
Jan de Mooij
Cosmin Sabou
Gregory Szorc
Tom Prince
Brindusan Cristian
Emilio Cobos Álvarez
David Teller
Andrew McCreight
Jeff Walden
shindli
Sebastian Hengst
Margareta Eliza Balazs
Philipp Kewisch
arthur.iakab
Kim Moir
Jason Orendorff
Coroiu Cristina
Jim Chen
Matt Howell
Christian Holler
Simon Fraser
Ralph Giles
Csoregi Natalia
Dragana Damjanovic
Ting-Yu Chou