This adds a basic test for the mach formatter. This will ensure that changes to
this format are intentional. It will also make it easier for reviewers of these
changes to see a diff of the old vs new format.
MozReview-Commit-ID: LBSfdyvOPVV
--HG--
extra : rebase_source : 5529ad1f03306dcf867d88af579b69d6005091c0
Now that we're no longer shipping the SDK we no longer need real libraries for
the libraries that were created by this rule.
MozReview-Commit-ID: ALATVGBayHu
--HG--
extra : rebase_source : a20905125e5ff1846ef29de12323ba7b0a58928b
The MSVC linker winds up generating import libraries when linking some of
our executables, presumably because they contain functions that are
__declspec(dllexport). By default the import libraries get written
alongside the exe, so we force them to be written to the objdir so they don't
clutter up dist/bin.
MozReview-Commit-ID: 7DTfCo3OdDQ
Historically we built all our binaries in directories in the objdir, then
symlinked them into dist/bin. Some binaries needed to be copied instead
so that certain relative path lookups work properly, so we resorted to
sprinkling `NSDISTMODE=copy` around Makefiles.
This change makes it so we build PROGRAMs (not any other sort of targets)
directly in dist/bin instead. We could do the same for our other targets
with a little more work.
There were several places in the tree that were copying built binaries to
some other place and needed fixup to match the new location of binaries.
On Windows pdb files are left in the objdir where the program was
originally linked. symbolstore.py needs to locate the pdb file both to
determine whether it should dump symbols for a binary and also to copy
the pdb file into the symbol package. We fix this by simply looking for
the pdb file in the current working directory if it isn't present next
to the binary, which matches how we invoke symbolstore.py.
MozReview-Commit-ID: 8TOD1uTXD5e
We use a wrapper script when compiling with MSVC to parse the
/showIncludes output and thereby generate a Makefile dependency
fragment. This fragment enables us to do correct and faster incremental
builds. But the cost of invoking the wrapper script can be significant;
it's an extra process or two to launch for every single compilation.
Instead, let's have clang-cl generate the dependencies directly, which
should be somewhat faster.
There are a lot of choices and moving pieces in this commit. I elected
to include the mechanics and the target use case in the same commit so
that readers can compare and contrast the implementation and final
expression in one review window.
- Initially, I wanted to make the {AB_CD} substitutions in
LOCALIZED_FILES and not in LOCALIZED_GENERATED_FILES. However, I ran
into conceptual blockers doing this. Fundamentally, LOCALIZED_FILES
is FINAL_TARGET_FILES, and my use case should _not_ be putting files
anywhere near dist/bin. In addition, LOCALIZED_FILES
(FINAL_TARGET_FILES) is handled using manifests, which would need to
grow locale-aware functionality to handle this. That's not desirable.
In addition, if we use manifests, then we lose the powerful locality
of |mach build mobile/android{/base}| re-generating changed
locale-dependent resources. This is similar to how the build system
plumbs dist/idl manifest processing throughout the build: we're
repairing local workflows after moving work into a global process.
For these reasons, this doesn't support {AB_CD} in LOCALIZED_FILES.
- There is even another layer of complexity! There are two axes
involved with these files: AB_CD controls localization and the Make
target controls destination. For the record, it is:
regular builds - AB_CD unset
multi-locale builds - AB_CD set
single-locale repacks - AB_CD set
For the record, the existing logic (before any changes) is:
regular builds - Make target is `libs` in mobile/android/base/locales
multi-locale builds - Make target is `chrome-%` in mobile/android/base/locales
single-locale repacks - Make target is `libs` in mobile/android/base/locales
This commit adds targets for both destinations, and uses Make
chrome-%:: and libs:: magic to control what is invoked in the various
situations. Tricky!
- I added MERGE_RELATIVE_FILES in order to be able to follow-up this
patch with more patches that will get rid of
m/a/base/locales/{moz.build,Makefile.in} altogether, and fold this work
into m/a/base. As it stands, we're already reaching from
m/a/base/locales all the way out to
mobile/locales/.../region.properties, so the existing code doesn't
follow the layout expected between mozilla-central and
l10n-central/$(AB_CD). But that'll impedance will get worse as we
improve the build system dependencies, not better, so we should grow
support for localized resources that aren't exactly as expected.
- I chose to follow Python's syntax for string substitutions. I
would have preferred to mark files that should be localized with a
leading '%'... but I took that for filesystem absolute paths in
moz.build files already. I also considered @AB_CD@ to echo the
preprocessor, but didn't want to open the door to an expecation that
_all_ preprocessor DEFINEs will work in the way {AB_CD} does.
- The generate_*py script changes required a bit of a hack to "turn
off" locale dependent resources. This would have been nicer if we had
marked localized resources with '%'... but we didn't. See the
--fallback flag. The real reason this is needed is that we're doing
work which is more like the work of compare-locales (merging
locale-dependent resources) at build-time rather than repack time. I
don't know why that's the case -- probably when we (I) implemented it,
compare-locales and the whole l10n process was entirely opaque. It's
not worth changing it now, so we use this --fallback flag approach.
- I didn't get to tup support. This should gently fail without
breaking tup builds: any {AB_CD} substitutions just won't be
expanded. I haven't a clue how this should work in tup in the future
(or, more generally, how to make any sense of repacks without
declaring the full set of expected locales at configure time.)
- strings.xml can't be a LOCALIZED_PP_FILES, since we need to
customize the output location based on AB_rCD, and since we need a
little more flexibility than PP_FILES gives for our inputs.
MozReview-Commit-ID: MyfIkNSEzt
--HG--
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/foo-data => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/foo-data
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/generate-foo.py => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/generate-foo.py
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/en-US/localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/inner/locales/en-US/localized-input
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/moz.build => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/moz.build
rename : python/mozbuild/mozbuild/test/backend/data/localized-generated-files/non-localized-input => python/mozbuild/mozbuild/test/backend/data/localized-generated-files-AB_CD/non-localized-input
extra : rebase_source : 816b6f220758f2bb3bdd3ec81a2cb02269c6de5b
I wanted to lift this next to the definition of AB_CD, but that
doesn't allow to use it in a backend.mk file, due to the order in
which Makefile, config.mk, rules.mk, and backend.mk are processed.
Therefore, I've put it in a tiny include file, so that it can be used
by a Makefile and a backend.mk file.
This allows the `RecursiveMake` backend to owning defining AB_rCD in
backend.mk files, while not requiring consumers to arrange for AB_rCD
in a sibling Makefile.in file.
Other build backends will need to arrange for AB_rCD themselves: see
following commits.
MozReview-Commit-ID: I7GIzRbCCtf
--HG--
extra : rebase_source : 3277fedb43bc3d8007287c223554a085dae2f198
extra : source : 854c0f43a1f74b4e22aa7638b407580240c90dd5
The last APK produced using the ANDROID_APK_* moz.build/Makefile.in
mechanism was Robocop, so we can get rid of these now.
MozReview-Commit-ID: 9b08ZvvOAoC
--HG--
extra : rebase_source : ac4fea057bf6e731b0f26a1b6902f17a7362076d
Far more files #include "util/Windows.h" than <psapi.h>, so this makes the
style-checker check more things (and makes --fixup mode fix more things).
--HG--
extra : rebase_source : a0c4edf10ad7dd4b5e0265c49035acf242197e6c
This is ancient and the team that used it (gfx) is no longer using it.
MozReview-Commit-ID: HrDgmAU9QeW
--HG--
extra : rebase_source : c4a64965c4ae1a50888893e881a6e8a9688a58b6
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
Now that writing the .purgecaches sentinel is done by |mach build|, we
can remove it from Make. In addition, we can cull the now-unused
app-rules.mk \o/
MozReview-Commit-ID: 6CnAqLeZwzB
--HG--
extra : rebase_source : 2a5b67e723f8e1654627e6e6f26704f6a4e57706
I very much doubt these are used, but even if we are -- we shouldn't
support this type of local customization, since it doesn't extend to
non-Make-based backends.
With the customization point removed, there's no way to set ETAGS, so
we remove what little support there was for generating Emacs tags.
MozReview-Commit-ID: IEF2Q4tISEn
--HG--
extra : rebase_source : 3bc8e651c03517edb797032db6ce60ed8852d9fa
Now that writing the .purgecaches sentinel is done by |mach build|, we
can remove it from Make. In addition, we can cull the now-unused
app-rules.mk \o/
MozReview-Commit-ID: 6CnAqLeZwzB
--HG--
extra : rebase_source : 7427ca9aecc6619f48090c6a26c9a2e0e7d37bad
I very much doubt these are used, but even if we are -- we shouldn't
support this type of local customization, since it doesn't extend to
non-Make-based backends.
With the customization point removed, there's no way to set ETAGS, so
we remove what little support there was for generating Emacs tags.
MozReview-Commit-ID: IEF2Q4tISEn
--HG--
extra : rebase_source : 140e07f78dbd72391c70c48f25e2b85c924fe78d
Fix several problems when building Breakpad with new NDK unified
headers.
- Unified headers define its own tgkill wrapper, so rename our own
wrapper to __tgkill.
- Unified headers define user_fpxregs_struct for all API levels, so
don't redefine it.
- Only the target sources under google-breakpad/src/common/linux should
use custom Android headers, so change the includes line in moz.build to
use OS_INCLUDES.
MozReview-Commit-ID: HGnUMu5vDUM
--HG--
extra : rebase_source : f5c29c9949a48a8376f84dcc676e5f8df886f130
Fix several problems when building Breakpad with new NDK unified
headers.
- Unified headers define its own tgkill wrapper, so rename our own
wrapper to __tgkill.
- Unified headers define user_fpxregs_struct for all API levels, so
don't redefine it.
- Only the target sources under google-breakpad/src/common/linux should
use custom Android headers, so change the includes line in moz.build to
use OS_INCLUDES.
MozReview-Commit-ID: HGnUMu5vDUM
--HG--
extra : rebase_source : 07da575044ca805a3f7f8ba87be8a8c279f17379
This patch adds basic support for the fuzzing interface in the JS engine on top
of the last patch. This includes all the necessary code except for actual
targets (just an example target skeleton) and also makes sure that the fuzzing
code is packaged for the standalone release.
MozReview-Commit-ID: D6Tyebz3jZS
--HG--
extra : rebase_source : f07a5d2bf8e09fb6c93501e58fbb958004e2fa05
This patch adds basic support for the fuzzing interface in the JS engine on top
of the last patch. This includes all the necessary code except for actual
targets (just an example target skeleton) and also makes sure that the fuzzing
code is packaged for the standalone release.
MozReview-Commit-ID: D6Tyebz3jZS
--HG--
extra : rebase_source : 58e4d85e657347b061de0ed912365f2a955a86e3
This patch disables the stdcxx-compat check for the sm-fuzzing build which
requires patching autospider as well. Furthermore, it switches the build
to linux64-clang-6-pre because the older clang 3.9 does not support trace-pc
instrumentation. Finally, it excludes fuzzing parts from the vanilla allocation
check.
MozReview-Commit-ID: FdhCIFdUore
--HG--
extra : rebase_source : c41bda01cb42f2ef0cd5a1675d88bdb55d9dc8c9
The MSVC linker winds up generating import libraries when linking some of
our executables, presumably because they contain functions that are
__declspec(dllexport). By default the import libraries get written
alongside the exe, so we force them to be written to the objdir so they don't
clutter up dist/bin.
MozReview-Commit-ID: 7DTfCo3OdDQ
--HG--
extra : rebase_source : fea69e8f60633b824726269c2296af9fe812d3ed
Historically we built all our binaries in directories in the objdir, then
symlinked them into dist/bin. Some binaries needed to be copied instead
so that certain relative path lookups work properly, so we resorted to
sprinkling `NSDISTMODE=copy` around Makefiles.
This change makes it so we build PROGRAMs (not any other sort of targets)
directly in dist/bin instead. We could do the same for our other targets
with a little more work.
There were several places in the tree that were copying built binaries to
some other place and needed fixup to match the new location of binaries.
On Windows pdb files are left in the objdir where the program was
originally linked. symbolstore.py needs to locate the pdb file both to
determine whether it should dump symbols for a binary and also to copy
the pdb file into the symbol package. We fix this by simply looking for
the pdb file in the current working directory if it isn't present next
to the binary, which matches how we invoke symbolstore.py.
MozReview-Commit-ID: 8TOD1uTXD5e
--HG--
extra : rebase_source : 9140be949b206bb595d9188ce7e8357347ecd9a9
Work around excessive command-line length issues by
disabling incremental rust compilation, which is enabled
by default outside `cargo --release` starting with Rust 1.24.
Incremental rust builds shouldn't help much in automation,
where sccache provides the only continuity between build
environments. In the meantime, they add a lot of object
files to the link line.
See https://github.com/rust-lang/rust/pull/47507 about addressing
the underlying issue upstream.
MozReview-Commit-ID: LRwUj3fhiaO
--HG--
extra : rebase_source : 1739a7570b2e7fe40ead3b301ea20c2fe79f0431
The pytest cache plugin writes its cache in the srcdir, which means that it
shows up in `hg status`, which is annoying. Writing files to the srcdir is
generally bad practice anyway, so we disable this plugin to stop this
from happening.
MozReview-Commit-ID: HytLLMUtKlc
--HG--
extra : rebase_source : f6acbf3650881312cef051126387220a0f78597f
Back when I filed this bug, $appname-$version was already useless, but
there actually were multiple supported versions (e.g. 3.5/3.6). So it
made sense to have e.g. firefox-3.5 and firefox-3.6, but not
firefox-3.5.12 and firefox-3.5.13.
Fast forward 10 years, and we change "major" versions every 6 to 8
weeks, have multiple chemspills every other week, and installing to
firefox-57.0, firefox-57.0.1, firefox-57.0.2, firefox-57.0.3 doesn't
make any kind of sense. Even firefox-56, firefox-57 is pretty much
useless.
There /kind/ of was some usefulness to the version in the SDK
directories, but those are gone.
I'm pretty sure no downstream is actually using versioned directories
anyways.
At this point, it seems better to just use the application name, without
the version. A case could be made about ESR, but that would be better
handled with a separate application name (e.g. firefox-esr).
--HG--
extra : rebase_source : 333e10ea1316714bf0008ec772b35093edfc45ff
Some of the variants are multi-platforms, and having
--enable-stdcxx-compat in their definition will break when the option is
moved over to python configure.
While here, prepare for --enable-stdcxx-compat actually doing something
(it currently doesn't), by adding an exception for it in
check_vanilla_allocations.py.
--HG--
extra : rebase_source : ee1647421542209cf0137db703c4f7e7f06cbc91
This has the virtue of not executing python three times during configure
just to read the same value of milestone.txt and munge it. We can also
remove milestone.py as a happy side effect, so all the milestone
computations can be done in init.configure.
We only had this "for flexibility with other platforms", but given that
we set it to the same thing for all platforms, and nobody has tried to
change that, this flexibility isn't actually needed.
Bug 1256642 introduced magic at the emitter level to determine whether a
binary contains C++ sources and should be linked with the C compiler or
the C++ compiler.
Unfortunately, the Binary() moz.build template always adds C++ OS
libraries on Android (through STLPORT_LIBS), and C++ libraries on Linux
(stdc++compat).
The latter only ends up forcing every Binary() to be linked with the C++
linker, which is unfortunate, but doesn't cause much problems. The
former, however, involving OS libraries, the magic from bug 1256642
doesn't kick in, so we end up trying to link C++ OS libraries with the C
linker. Which ends up failing, because the libraries in STLPORT_LIBS
require -lm, which, while it's added by the C++ compiler when linking,
is not when the linkage is driven by the C compiler.
Because the fallible library, linked to all GeckoBinary()s is a C++
library, we still ended up linking with the C++ compiler on Android, so
this wasn't actually causing any problem... until I tried to remove that
fallible library in bug 1423803.
Anyways, the core problem is that moz.build evaluation is happening too
early to know whether any C++ sources are being linked together, so
there is no way the Binary() template can do the right thing. So this
change moves the logic to the emitter.
This also changes the type of STLPORT_LIBS to a list.
--HG--
extra : rebase_source : a70ddf7a132f94dc10e7e1db94ae80fb8d7a269f
Back when mozalloc was a separate library, the xpcom glue code could not
use the infallible allocator API. But since bug 868814, that's not the
case anymore, so we can safely include mozalloc.h when XPCOM_GLUE is
set.
--HG--
extra : rebase_source : a8fbf8dc7020765d7287e2eb7ceaf41c99be8b18
This accounts for LOCAL_INCLUDES in the moz.build files, as well as the
default INCLUDES specified in config.mk that are used for host
compilation. Since some of the HOST_CFLAGS were also used for linking,
those flags are split off into HOST_C{XX}_LDFLAGS so that
the linker-only flags can be placed in those variables.
MozReview-Commit-ID: J1LxIZVeFJ
--HG--
extra : rebase_source : ed7293604e5428e3124f1ecfb2b706e087436b72
These flags were added way back in bug 59454 to both the compiling and
linking rules for HOST_SIMPLE_PROGRAMS. However, the INCLUDES aren't
actually needed when linking so we can safely remove them.
MozReview-Commit-ID: 8QywO7tGPpU
--HG--
extra : rebase_source : 34870adcf73b459771413e52bc84c6c63669d0f9
Both SFLAGS and ASFLAGS are used to compile assembly, but SFLAGS include
DEFINES and LOCAL_INCLUDES whereas ASFLAGS do not. It seems easiest to
just separate them into two different ComputedFlags values so that the
backend can distinguish between the two types.
MozReview-Commit-ID: Bkm3621ImJG
--HG--
extra : rebase_source : 420204e37d591512f700d77b780939d20c2feeb0
These have been replaced by stl-headers.mozbuild and
system-headers.mozbuild
MozReview-Commit-ID: EpPasSzgiJk
--HG--
extra : source : 86302df0b38a05ce8f6f830e324631a42c1c643e
The make-system-wrappers.py invocation is largely identical to
make-stl-wrappers.py, though this script generates wrappers for both the
STL headers and every other system header that can be used.
Note that the nsprpub script didn't create multiple layers of
subdirectories properly, so for example the 'ia64/sys/inline.h' wrapper
is now generated properly. Additionally, MOZ_SYSTEM_ICU define was
incorrectly using '#ifdef' instead of '#if ... == 1', which causes those
unicode headers to have wrappers when they shouldn't. These will show up
as differences when comparing the Makefile output to the moz.build
output.
MozReview-Commit-ID: KvQAawfzXao
--HG--
extra : source : 5a967cc85e28e63c283a81e2c76444a76dfbd266
This is fairly straightforward to represent as a GENERATED_FILES, though
we have to take some care to construct the outputs tuple correctly. This
script needs to run during export, and unfortunately none of the STL
headers have proper file extensions, so the 'new' header is
special-cased in the recursive make backend to serve as a marker for
running it in the correct tier.
We can't remove the stl-headers file yet because it is still used for
the system header generation.
MozReview-Commit-ID: 3tQTOY0LAsQ
--HG--
extra : source : 828d43ec1b16edaac69c42f15561f26e209051f1
These don't actually install correctly, because 'private' is already a
system wrapper. Neither of these files are included anywhere, so there's
no harm in removing them.
MozReview-Commit-ID: 1IAFMZDvKDV
--HG--
extra : source : 8d770908a5b9a3421294c93f8a34a14265674e9b
This is no longer used in the tree, and the $ will get eaten somewhere
along the way on the path from moz.build to the python invocation by
make despite being shell_quoted.
MozReview-Commit-ID: 5mwWQYDn8Nf
--HG--
extra : source : ffe9aed0944f57e329ee279f5b14266d5c4069a2
These have been replaced by stl-headers.mozbuild and
system-headers.mozbuild
MozReview-Commit-ID: EpPasSzgiJk
--HG--
extra : rebase_source : 33bc7a5a8de794f882bd1438ac800da2150a2ce6
The make-system-wrappers.py invocation is largely identical to
make-stl-wrappers.py, though this script generates wrappers for both the
STL headers and every other system header that can be used.
Note that the nsprpub script didn't create multiple layers of
subdirectories properly, so for example the 'ia64/sys/inline.h' wrapper
is now generated properly. Additionally, MOZ_SYSTEM_ICU define was
incorrectly using '#ifdef' instead of '#if ... == 1', which causes those
unicode headers to have wrappers when they shouldn't. These will show up
as differences when comparing the Makefile output to the moz.build
output.
MozReview-Commit-ID: KvQAawfzXao
--HG--
extra : rebase_source : 758e325c6ec192d83fcfa10f8c878ba2629e45a3
This is fairly straightforward to represent as a GENERATED_FILES, though
we have to take some care to construct the outputs tuple correctly. This
script needs to run during export, and unfortunately none of the STL
headers have proper file extensions, so the 'new' header is
special-cased in the recursive make backend to serve as a marker for
running it in the correct tier.
We can't remove the stl-headers file yet because it is still used for
the system header generation.
MozReview-Commit-ID: 3tQTOY0LAsQ
--HG--
extra : rebase_source : bd9f00e45a7bce4daaa0e1c16e22b28536658e37
These don't actually install correctly, because 'private' is already a
system wrapper. Neither of these files are included anywhere, so there's
no harm in removing them.
MozReview-Commit-ID: 1IAFMZDvKDV
--HG--
extra : rebase_source : 6e153324d2add32a179d07020270db96e7ec6dd0
This is no longer used in the tree, and the $ will get eaten somewhere
along the way on the path from moz.build to the python invocation by
make despite being shell_quoted.
MozReview-Commit-ID: 5mwWQYDn8Nf
--HG--
extra : rebase_source : f2289f5f5e15529475cac6cba3fc675592c111f2
These are only set by nss' make build system, which we no longer use.
MozReview-Commit-ID: 9XRhpDmlekK
--HG--
extra : rebase_source : 5bff741863a463eca7d72f4e23c1c78e04a848de
The make backend is the only thing that is aware of PROGILE_GEN and
PROFILE_USE, so we move these ldflags to their own variable while
converting the remainder of ldflags to mozbuild.
MozReview-Commit-ID: GwbPD6Q4Oyn
--HG--
extra : rebase_source : 3c90def9ee8cd949dd135ba8fa9b192f114f6727
These flags are only relevant on OS X, and will not be necessary soon, so this
commit moves them to a separate variable while we move the remainder of the
ldflags to mozbuild.
MozReview-Commit-ID: 1NDgz3HIYpT
--HG--
extra : rebase_source : 6e9b5f5a5be5ff916db89a0b73896b9058eb040e
These targets no longer exist in client.mk. I'm not sure when they
were removed. But this is obviously dead code.
A target in rules.mk pointing back to this target in client.mk has
also been removed.
MozReview-Commit-ID: 5v1MOOTP3qq
--HG--
extra : rebase_source : b90f0788b074330b45005223f41c804589d13163
Currently mozconfig.cache overrides a few build options for sccache. This
patch moves them into toolchain.configure so that the build system will
set them properly when sccache is in use. Additionally, {CC,CXX}_WRAPPER
are set in config.mk, so just avoid setting them when sccache is in use.
MozReview-Commit-ID: FYlVKRI8OiN
--HG--
extra : rebase_source : 00715beb5fbd2c11311dec43809bd1febab56a11
extra : intermediate-source : 0f2b1b75b83737378d882a3c3e0d8dfb4efecd1f
extra : source : a8032ae9cb2ad1c4574c6ac6f5c2778863cd71e0
This also introduces C{XX}_LDFLAGS variables which contain cflags that
are meant to be passed to the linker, and adds them to various linker
command lines in place of CFLAGS.
MozReview-Commit-ID: GyKlD9nMqrt
This commit establishes a separate variable to add PROFILE_GEN and PROFILE_USE
CFLAGS to compile and link command lines. Currently the make backend
orchestrates the pgo build steps and is the only thing aware of whether
we're in the profile generate or profile use stage. The flags are separated
here to allow other flags to be moved to mozbuild, but this will not yet
sufficient to perform a PGO build independent of the make backend.
MozReview-Commit-ID: IX30l2MvvNc
Before this commit, RUSTFLAGS was derived in rules.mk by consulting
various variables set by configure. It isn't clear to me why things
are implemented this way. We don't appear to have moz.build level
overrides for Rust compiler flags. So there doesn't appear to be a
compelling reason why we can't derive these values in configure.
So, this commit ports the code for deriving default RUSTFLAGS from
rules.mk to toolchain.configure.
The port is pretty straightforward as far as the logic goes.
MozReview-Commit-ID: JhAE9Qlo8SK
--HG--
extra : rebase_source : 6186cb81cd37c516b3d645419b9461bf501d6ba2
libffi has assembly files on Windows that need to be preprocessed. Previously
this was handled by using a wrapper shell script as the assembler. This
patch handles them in GENERATED_FILES with a Python script that performs
an equivalent transformation.
MozReview-Commit-ID: 3w0HNpUb8TA
--HG--
extra : rebase_source : 284a3825cdaf2957507d60276a772f88f8c96488
extra : source : 3f422cbd4f222bc0b19f0a9e76878787652b3788
I *think* the modifications to MockedOpen are correct, but I'm not sure..
MozReview-Commit-ID: 6vTZBtdQ1dz
--HG--
extra : rebase_source : 2d2008f87640747ef831d000bf13a4b1b7fcd338
config/string-format.js was made obsolete in the removals in bug 1227388.
MozReview-Commit-ID: 1nQwlKhoC0e
--HG--
extra : rebase_source : 6b06e4d7ca2e8fc66fc2d93ade54c813d975acec
DEFFILE is currently just used as a passthrough variable. All but one of
the current uses of it use `SRCDIR + '/file.def'` to get a srcdir-relative
path anyway, and the other one wants an objdir-relative path, so using
Path makes everything clearer.
This makes it more straightforward to translate the paths for the WSL
build.
MozReview-Commit-ID: IRokABaZW2c
--HG--
extra : rebase_source : ae74c984bb2aab70211dc5974a8b052651e025dd
Incremental linking is automatically disabled when -OPT:REF or -OPT:ICF are used.
Seeing that we're disabling optimizations anyway, may as well not use those linker flags as they considerably slow down linking.
MozReview-Commit-ID: 9OXG19uxeI9
--HG--
extra : rebase_source : 4703bc4bb62af3f054ca621be7c1c51fe56cb2f3
And convert consumers to context managers because hglib requires that.
MozReview-Commit-ID: Ckf1yBYeUlm
--HG--
extra : rebase_source : 985220032bced1a7077fd9b04ca8ad6de822c887
This makes sure that:
* We don't define `MOZ_GLUE_IN_PROGRAM` so that everything in mozglue gets
defined.
* `MFBT_API`'s symbol export rules match `JS_PUBLIC_API` and `EXPORT_JS_API`.
* We add mozglue to SpiderMonkey's `USE_LIBS` when jemalloc is disabled.
This switches most tests over to use pytest as the runner instead of unittest (taking
advantage of the fact that pytest can run unittest based tests).
There were a couple tests that had failures when swithing to pytest:
config/tests/unit-expandlibs.py
xpcom/idl-parser/xpidl/runtests.py
For these tests, I added a runwith='unittest' argument so that they still run the
same way as before. Once we fix them to use pytest, the unittest logic in mozunit.py
can be deleted.
MozReview-Commit-ID: Gcsz6z8MeOi
--HG--
extra : rebase_source : 3c762422ce0af54cbbe7d9fc20085a2d1ebe7057
Similar to the existing LIB_IS_C_ONLY variable, these variables indicate
that the program in question has only C sources and so can be linked by
the C compiler rather than the C++ compiler. We need to add a little
more information to BaseProgram so we can avoid emitting periods into
Makefile variables.
As we move our compile command lines into mozbuild we want to stop using DEFINES
directly. This patch adds a variable that can be set from Makefiles to contribute
defines to a compile command line to help the transition.
MozReview-Commit-ID: 5xLB06pzpJY
--HG--
extra : rebase_source : 7ca0b9d8f5a77ec55eb751f75f58a69e64efb21d
libevent uses event.h header that is fuzzy name. Since our in-tree libevent is libevent2, we should use libevent2 headers instead of deprecated event.h
MozReview-Commit-ID: 6DjW9JEkNWL
--HG--
extra : rebase_source : b774e177b137bf7427122253a3e4c698689e08a4
To not merge the en-US language pack, the merge-% steps are in
a conditional function that disables that for en-US. Using a function
here as that's easier than a shell if in the merge rule, and
Makefile conditionals don't get evaluated late enough.
To liberate the l10n builds from settings in the automation,
we move the patch logic from LOCALE_MERGEDIR to REAL_LOCALE_MERGEDIR.
To determine strongly when we're in a repack or building a langpack,
the trick here is to
export IS_LANGUAGE_REPACK
in l10n.mk, and only set that to true in the entry-point rules.
Now, we can use that value in config.mk to define the l10n-specific
rules.
I did the same thing for langpack-%, which allows us to disable
the crashreporter files for language packs, for example.
With that,
make installers-de
just works, if you have localizations checked out.
For a while, we might run l10n-merge twice in automation, but it's really not
optional, so let's just make sure we run it.
MozReview-Commit-ID: 3nr33CKxkBQ
--HG--
extra : rebase_source : 0605a4adba018fa4b85d563cdafba80b0533bc91
cargo 1.19, and later cargo 1.20/rustc 1.19 gained support for the make
jobserver. But that doesn't work unless make passes the jobserver file
descriptors down to subprocesses, which it only does when there is a +
prefixing commands.
--HG--
extra : rebase_source : a00ffb7edb358243ac7b9469941ec6fb5ad84885
This is similar to bug 1301751, where something in rust seems to trigger errors
running dsymutil to generate debug symbols in OSX. We can set debuginfo=1 for
these builds as a temporary workaround for now, while we work on a more
permanent solution in rust and/or dsymutil. debuginfo=1 still gives us enough
info for stack traces, although without line info. debuginfo=2 would be useful
for debugging, but is irrelevant to crash reports.
MozReview-Commit-ID: DdA00GzVfWg
--HG--
extra : amend_source : 47d3573042098194a07f9b473e4a02c86a1eba7c
MOZ_REPLACE_MALLOC_LINKAGE was added back when there were problems with
getting weak references working properly for replace-malloc.
Versions of OSX < 10.6 needed flat namespace, but aren't supported
anymore.
Versions of Xcode < 4.5 required flat namespace + a dummy library in
order to produce proper weak references. There is virtually nobody still
building with such an ancient toolchain.
Keeping those around doesn't /really/ hurt, except recent versions of
Xcode don't expose dyldinfo in /usr/bin, used for the configure test.
Consequently, MOZ_REPLACE_MALLOC_LINKAGE ended up being set to use the
dummy library setup, which, by using flat namespace, now causes harm in
bug 1356701, causing bug 1378332.
--HG--
extra : rebase_source : e3edc1f2cf905943c33fafeb631f2f88fc87167e
Cargo recently introduced the `cargo check` command for shortening the
edit-compile cycle when working on large programs. Since we don't
really support invoking `cargo` directly, let's wire up this command to
`mach`. Gecko developers can then `mach cargo check` to ensure their
changes typecheck.
All the machinery we set up for `cargo build` is equally useful for
other commands, such as `cargo check`. Let's refactor things so that
it's easy to reuse said machinery for `cargo check`.
We currently only honor LIB_IS_C_ONLY for cases where we set a
LIBRARY (and, implicitly, REAL_LIBRARY) and FORCE_SHARED_LIB. For many
libraries, such as the libraries from NSS, we never set LIBRARY, which
leads to not setting REAL_LIBRARY, which leads to not honoring
LIB_IS_C_ONLY. This practice has not been harmful thus far (except
perhaps linking in more things than necessary to our NSS shared
libraries), but on some platforms, linking with the C++ compiler will
drag in more things than we would like.
Consulting LIBRARY first should not be necessary; checking
FORCE_SHARED_LIB should be enough to tell us if we're building a shared
library for the purposes of honoring LIB_IS_C_ONLY.
We can avoid the symbol visibility problem by putting
sanitizer/asan_interface.h in the config/system-headers.
--HG--
extra : rebase_source : bc81a81ef8970c3544febf06631740208583c7fa
Stable Rust currently has logic similar to the following for finding the
linker on Windows:
if VCINSTALLDIR in os.env:
search for things in PATH
if found, return, otherwise fallthrough
look for MSVC installations of various kinds
For our current MozillaBuild setup where we clear out PATH prior to
invoking Cargo, this works: even though VCINSTALLDIR is set and we
(unsuccessfully) search through an empty PATH, we'll still fallback to
looking for MSVC installations.
Beta Rust has altered this search strategy:
if VCINSTALLDIR in os.env:
search for things in PATH
return the result of the search, regardless of success
look for MSVC installations of various kinds
Note that in the above logic, if VCINSTALLDIR is set, we are only going
to try looking in PATH for an appropriate linker. Since PATH is empty
in our current MozillaBuild setup, beta Rust will therefore fail to find
the correct linker. This failure causes numerous problems.
Since Firefox is already trying to "clean" the environment that Cargo is
invoked in, we should work a little bit harder by unsetting VCINSTALLDIR
entirely, and simply force Rust to search for known MSVC installations.
Making this change makes stable and beta Rust behave identically.
(We don't have to worry about this distinction in automation, because
the "cleaning" code is only activated when we detect a MozillaBuild
shell.)
Since we set WINVER=601 after dropping Windows XP support, it is unnecessary to define LOCALE_SNAME. So, we should remove it to remove compiler warning.
MozReview-Commit-ID: EawbHAznXNA
--HG--
extra : histedit_source : d3157300315052048756c22ac2c999523d28679d
Final target Rust programs are currently not picked up and
symlinked/copied to ${objdir}/dist/bin because the RUST_PROGRAMS output
variable is missing from the if-condition.
This change causes Rust binaries from RUST_PROGRAMS to be placed alongside
the other final target programs.
MozReview-Commit-ID: 5OIH1UMmCq2
--HG--
extra : rebase_source : cfcff21e9371b920c895ad27df344008962d7471
These paths can contain spaces, particularly if they point into
${HOME}/.mozbuild on Windows. If people are bold enough to use the
Windows packages from llvm.org, the LLVM packages are installed by
default into C:\Program Files\LLVM, so paths from there would contain
spaces as well.
Procedural macros for rust are compiled as dynamic libraries.
The compiler subsequently loads those libraries at runtime
when it compiles crates which make use of them.
We pass the generic LDFLAGS to our linker wrapper which ensures
cargo links static libraries with the correct flags. However
this also means that any final system dependency libraries
end up linked into the procedural macro dynamic libraries
whether they use them or not.
When building on macOS 10.7, this causes a SIGTRAP when dlopen
calls CFInitialize for system libraries like Cocoa off the main
thread. We believe this is a macOS bug, reported fixed in macOS 10.9.
https://openradar.appspot.com/7209349
Work around the issue by stripping those libraries and Frameworks
from the linker flags we pass to cargo. We could do a more careful
job of parsing and removing all -framework and -lfoo arguments
inside the linker script, but since this is a temporary issue
while we're using older build machines, I thought just stripping
them once in the build system was simpler.
MozReview-Commit-ID: CGXD8ARvojj
--HG--
extra : rebase_source : 33f28e8fbafb43dbdd257d9162a6e792fca79e51
We're getting an intermittent failure running `hg manifest` in CI. I
have no clue why.
What I do know is that we now have the mozversioncontrol Python package
for containing utility code for interacting with version control. It is
slightly more robust and I'm willing to support it more than I am
check_utils.py.
This commit adds a new API to our abstract repository class to obtain the
files in the working directory by querying version control.
Since I suspect cwd was coming into play in automation, I've also
added a utility function to mozversioncontrol to attempt to find
a version control checkout from the current working directory. It
simply traces ancestor paths looking for a .hg or .git directory.
Finally, I've ported all callers of the now-deleted API to the new
one. The old code had some "../.." paths in it, meaning it only
worked when cwd was just right. Since we resolve the absolute path
to the checkout and store it on the repo object, I've updated the
code so it should work no matter what cwd is as long as a repo can
be found. I'm not 100% confident I found all consumers assuming cwd.
But it's a start.
I'm not 100% confident this will fix the intermittent issues in CI. But
at least we should get a better error message and at least we'll be
running less hacky code.
MozReview-Commit-ID: AmCraHXcTEX
--HG--
extra : rebase_source : 815ae369776577ad374333920fd645d412a55148
gfx/skia/skia/src/core/SkCpu.cpp might use sys/auxv.h to detect CPU feature. So we should add it to system-headers
MozReview-Commit-ID: SijyUYvLgn
--HG--
extra : rebase_source : 0b509c2f5b4f3272578ae511e9c69cf6b0999c42
We now have code that unconditionally requires the rust
compiler and are committed to adding more. Remove this
last vestige of conditional support.
MozReview-Commit-ID: EK6FBnAbR
--HG--
extra : rebase_source : 6efda10a74f9ca0482304c2b1ffe6941e42138f8
Build system switch for optional inclusion of libaom
for support of the Alliance for Open Media AV1 video
codec.
MozReview-Commit-ID: 2C4o1ogRS9v
--HG--
extra : rebase_source : d4a68f1fc4654895f62a905666f0b75726e20e7f
The Java Addons mechanism never got traction and is not Web Extensions
compatible. Removing it simplifies the product and the build system.
MozReview-Commit-ID: ABUxkqqMISa
--HG--
extra : rebase_source : 346f88882774f072316714cf637a54d771d81a9a
Several years ago, joey starting writing a handful of unit tests for
primitives in our make files. IIRC a lot of the impetus behind this
work was to flush out bugs between GNU make and pymake.
AFAICT the only survivor of these tests today is check_mkdir.py. The
test is a one-off and has been a bit fragile over the years,
contributing technical debt along the way. And as part of removing the
last references to pymake in automation, it is once again showing
itself and failing in a way that has to do with the way the test
is written and not an issue with the code it is testing. Enough is
enough. This commit removes the test and eliminates the technical
debt. I don't think it will be missed.
MozReview-Commit-ID: 3OzurtFbcyD
--HG--
extra : rebase_source : 6d3cf6121c90e9c47948862b8c4117bb1d3e5afd
For linking static libraries, rustc will use whatever `cc` it finds (or
the equivalent on Windows). For our purposes, however, `cc` is not what
we use to link and we may have additional options we would like to pass
to the linker.
To do this, we need to tell Cargo about our alternate linker (currently
only used for target compilations, on the theory that the host compiler
rustc finds is probably good enough) and we also need to pass our linker
options into the process. We do this with environment variables, which
is not a great solution, but works surprisingly well.
This alternate linker is disabled for ASan builds due to peculiar
crashes when running Rust build scripts and for Windows, because we
don't do any interesting cross-compiling there.
This change paves the way for injecting target- or host-specific
environment variables for a particular `cargo build` invocation. Doing
this is not strictly necessary: all of our current `cargo build`
invocations use mostly target-specific environment
variables (e.g. PKG_CONFIG_ALLOW_CROSS). But separating things out
makes the code notationally cleaner and also avoids weirdness when
target==host.
rustc generates .lib files for its libraries when compiling for Windows
(even using MinGW on Linux). But MinGW expects .a files. So we add in
rust-specific prefix and suffixes so MinGW builds can find the libs that
rustc generates. (And the RUST_LIB- variables default to the same vales
as the LIB_ variables otherwise.)
MozReview-Commit-ID: ClsA0YuJaxh
--HG--
extra : rebase_source : 7b46460c94ceb34b7a5a302ce91d3f1dca600041
By using $(REPORT_BUILD) instead of just echoing the filename, there is
no change during a regular build without REBUILD_CHECK, but specifying
REBUILD_CHECK in the environment will show which files triggered an .xpt
to rebuild. This is helpful in debugging why these files may be built
unnecessarily.
MozReview-Commit-ID: GGNaKAl02Ea
--HG--
extra : rebase_source : 098a81265deed9afc0b284d9863f18ebd74fda33
Instead of using the -i flag to clean out the entire set of environment
variables, we explicitly just clear out the variables that are affected by
32- vs 64-bit builds in vcvars.bat. This still allows cargo to use it's
internal autodetection to do host/target builds as it wants when cross-compiling,
but also allows other things that rely on include directories and so on to
work properly.
MozReview-Commit-ID: 99SECKv1zQa
--HG--
extra : rebase_source : 2c2a7d79201b66ac3262afcc5f060936ca2e8fa2
This change brings Cargo closer to what our C/C++ build already does,
and some forms of Cargo failure are only diagnosed when you pass
`--verbose`, so let's eliminate that extra step for people.
For people working on Rust code, compiling in debug mode (Cargo's "dev"
profile) is convenient: debug assertions are turned on, optimization is
turned off, and parallel compilation inside of rustc itself can be
used. These things make the build faster and the debugging experience
more pleasant.
To obtain that currently, one needs to --enable-debug at the Gecko
toplevel, which turns on debug assertions for the entire browser, which
makes things run unreasonably slowly. So it would be desirable to be
able to turn *off* debug mode for the entirety of the browser, but turn
on debug mode for the Rust code only.
Hence this added switch, --enable-rust-debug, which does what it
suggests and defaults to the value of --enable-debug. For our own
sanity and because we judge it a non-existent use case, we do not
support --enable-debug --disable-rust-debug.
This commit moves symbol dumping to the compile tier, to be run via "syms"
targets. Tracking files are used for the sake of incremental builds, because
dump_syms may genearate multiple outputs whose paths are not known ahead of
time.
Minimal changes to symbolstore.py are made here. More extensive
simplifications will be made in a future commit on the basis of symbolstore.py
handling one file at a time.
MozReview-Commit-ID: 3mOP8A6Y7iM
--HG--
extra : rebase_source : bfe97afcdfc05b9e79f01577701c83e8b00eb4e9
This creates "syms" targets that depend on the corresponding "target" for
directories containing shared libraries or programs. These targets are added
to the main compile graph in automation, and can be invoked through a special
"symbols" target. A future commit will use these targets to dump symbols for
shared libraries and programs during the compile tier.
MozReview-Commit-ID: KLuvmqsK4Zj
--HG--
extra : rebase_source : 8d76b999cb6fac8f11168ac6ebfb58774dfc2d3c
This prevents the variables set in vcvars.bat from causing build failures when
cargo tries to process build scripts in an environment where the default rust
toolchain doesn't match the MozillaBuild shell (in terms of 32-bit or 64-bit).
MozReview-Commit-ID: 5VRnQToYUYf
--HG--
extra : rebase_source : 4ac11f13e17ab04e70c7cb6f60ff46865027f506
VPATH applies to everything, but vpath applies to files matching the
pattern it's given.
--HG--
extra : rebase_source : 17339820e34f4d2f3d4ddde1a8a4325e92bfdb9e
The code doing that was already excluding OSX, so we obviously don't
need it on OSX. On FreeBSD, afaict, USDT (userspace dtrace) probes don't
require that extra ELF object. Solaris presumably does, but the 6 years
old bug 702179 says it's not been working for that long, and configure
doesn't support Solaris targets anyways.
Keeping this build system code working past the changes coming in bug
1262241 is not guaranteed and can't be tested, so I'd rather get rid of
effectively dead code rather than trying to maybe not break it while
moving it.
--HG--
extra : rebase_source : b41ab77804c17529e2580ced8e8b5fd302ff7831
This is a similar issue to bug 582910 which the stack isn't big enough
to run the tests that recurse deeply. Set it 3x larger because the
offical site states they've seen up to 3x stack memory increase for
ASan, also the number was confirmed by the measurements in bug
1259796.
MozReview-Commit-ID: FLASlkVrEUM
--HG--
extra : rebase_source : 3e82439e3df7b1e2fa1759c9693bd197431c9f71
sutagent is no longer built or used; devicemanagerSUT is completely
unused. After this change, devicemanagerADB is the only implementation of
devicemanager, and test harness options like --dm_trans are eliminated.
David Major
Dave Hunt
Andrew Halberstadt
Chris Manchester
Ted Mielczarek
Sebastian Hengst
Jan de Mooij
Nathan Froyd
Margareta Eliza Balazs
ffxbld
Philipp Kewisch
Nick Alexander
arthur.iakab
Kim Moir
Jason Orendorff
Gurzau Raul
Jeff Walden
Andrew McCreight
André Bargull
Coroiu Cristina
Jim Chen
Cosmin Sabou
Narcis Beleuzu
Matt Howell
Nika Layzell
Christian Holler
Simon Fraser
Ralph Giles
Csoregi Natalia
Sylvestre Ledru
Dragana Damjanovic
Tom Prince
Mike Hommey
Ting-Yu Chou
Andreea Pavel
Mike Shal
Alfredo.Yang
Brindusan Cristian
Ryan VanderMeulen
Phil Ringnalda
Jonathan Watt
Randall Barker
Jon Coppeard
Masatoshi Kimura
Gregory Szorc
Dragana Damjanovic dd.mozilla@gmail.com
Jan Beich
Dustin J. Mitchell
Gian-Carlo Pascutto
Zibi Braniecki
Robert Helmer
Mark Banner
Henri Sivonen
Wes Kocher
Jean-Yves Avenard
Nick Fitzgerald
Petr Sumbera
Eric Rahm
Matt Brubeck
Kartikaya Gupta
Makoto Kato
Axel Hecht
Robert Strong
Nicholas Nethercote
Nicolas B. Pierron
Jeff Gilbert
Marco Castelluccio
Andreas Tolfsen
Bruce Sun
Xidorn Quan
Iris Hsiao
Lee Salzman
Tom Ritter
Robert O'Callahan
Carsten "Tomcat" Book
Michael Layzell
Tooru Fujisawa
Geoff Brown