This adds some new optimization strategies. For tests, we use Either(SETA,
SkipUnlessSchedules), thereby giving both mechanisms a chance to skip tasks. On
try, SETA is omitted.
MozReview-Commit-ID: GL4tlwyeBa6
--HG--
extra : rebase_source : 0c1ce762afc7a691788379d4f4206df106f6df63
The HTTP server can dynamically map URLs to local filesystem paths.
We employ this so Speedometer's files are available to the server.
MozReview-Commit-ID: EpF1aD6meZH
--HG--
extra : rebase_source : 70f80e073c2bec92c9429dcc35cd12e27018f3cb
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: 554IPMRWgzK
--HG--
extra : rebase_source : 00827d3f6bd705419bc801eb05b543af1ddc274f
In some cases, they are false positives. In other cases, the callsite is #ifdef-ed
out, so it's a true positive, but not one we would correct.
MozReview-Commit-ID: 6ThZH3wEXTp
--HG--
extra : rebase_source : aabdb93cb924e74cf9592a93c86062c4435ceaad
``print()`` has no business being in library code like this. It was
a holdover from this code being copied from bootstrap. So remove it.
While we're here, replace the generic exception with a specific one.
We don't want to be swallowing bugs via ``except Exception``.
MozReview-Commit-ID: 49goUstfPBz
--HG--
extra : rebase_source : a821159bd12a449ed1a0edf21a1f9eb29711ad95
An inline comment wanted it. The change is trivial.
MozReview-Commit-ID: CqyOYqNzwzr
--HG--
extra : rebase_source : d779d6bbca4aa57d9e95ead3a8403e0163106bdf
The file was using 2 space indent. I ran `autopep8` on it. Most of the
changes are whitespace only.
MozReview-Commit-ID: 1C7yDUQCfKl
--HG--
extra : rebase_source : 9f2b52ef6d0887fbaa7f8410b32348d371e884d8
Mercurial's sparse checkout support allows "profile" files defining
what's in sparse checkouts to be defined in-tree. This convenient
feature means you simply need to point a client at a path inside the
repository and it dynamically resolves what files to include in the
checkout. As you update revisions, the "profile" pulls in updates
to the underlying file.
We introduce 2 sparse profiles: 1 for mach and another for taskgraph.
The goal of the "mach" profile is to provide enough files to run
`mach`. If you activate this profile and run `mach`, it runs without
error. But there are practically no commands available. So it isn't
terribly useful.
The "taskgraph" profile allows us to run `mach taskgraph`. This
profile demonstrates a sparse profile feature: including other
profiles. The "taskgraph" profile is thus a union of "mach" and
its own entries.
There is definitely some fat in these profiles. I didn't feel like
chasing the long tail and getting overly granular with the profiles.
If we want to optimize later, we can do that.
For reference:
Full checkout: ~234,000 files
mach: ~2,000 files
taskgraph: ~3,600 files
MozReview-Commit-ID: 7pALt0MwHfE
--HG--
extra : rebase_source : 1a4ba4b8a63c522dab2841e2c0019501476da2fe
python-hglib is a Python client for Mercurial's command server. It
facilitates querying Mercurial efficiently (using a single process)
and without having to parse output in the common case.
Let's vendor it so we can make use of it for more advanced Mercurial
scenarios.
Content vendored from changeset 820d7c1e470a without modifications
(other than deleting unwanted files).
As part of vendoring, we add the package to the virtualenv and make
it available to mach.
MozReview-Commit-ID: F4KLbW1lAvk
--HG--
extra : rebase_source : 39321a880a13a0b0323a7217f538978b729e2afe
Sparse checkouts may not have all mach_commands.py files.
mach raises an error when a mach_commands.py file is missing.
So, we teach the mach driver to ignore missing file errors when
a sparse checkout is present.
The added code is optimized to avoid an import of mozversioncontrol
and some I/O as part of resolving the repo and VCS binaries because
this file is in the critical path of all mach commands and avoiding
I/O is worthwhile. Since we aren't using sparse checkouts in the
common case, this effectively makes the new code 0 cost.
MozReview-Commit-ID: C6itJga31t5
--HG--
extra : rebase_source : 4b2c18d30ff8b923a940c80ac81372a4076b8fdc
"mach" in this scope is both the mach module and the driver instance.
Let's change the latter's variable name so we can access the mach
module.
MozReview-Commit-ID: Db6sxDFl2oo
--HG--
extra : rebase_source : 0a51c7514cfdba40483b3b9b361cd03b8000c2c9
The Repository interface gains a new method to determine if a sparse
checkout is present. Mercurial's implementation is somewhat crude,
but it should work (Mercurial's sparse support is still experimental
and I only intend to support sparse checkout in Firefox CI until it
is less experimental). Git's always returns False (for now).
To prove it works and to expose the information more widely, we hook
it up to moz.configure. We do this by first implementing a function
that returns a Repository instance. Then we simply call a function on
it to resolve the sparse checkout flag.
MozReview-Commit-ID: AlsT5LdSPdZ
--HG--
extra : rebase_source : f1e9aaa7d15f11c7c5e8d268d4ad82468732103b
This change adds an upload-generated-sources task kind that runs after nightly
builds, fetches their `target.generated-files.tar.gz` artifact, and uploads
all the contained files to an S3 bucket. For actual nightly and release builds
on SCM level 3 trees, the S3 bucket is configured to be publicly accessible,
so that tools like Socorro will be able to fetch generated source files that
appear in crash reports, and debuggers will be able to fetch generated sources
when they show up while debugging Nightly or Release builds.
There are also level-2 and level-1 S3 buckets configured for builds happening
on trees of other levels such as try. They are not configured as publicly
accessible, but they exist so that these tasks can be tested in try.
MozReview-Commit-ID: Js1HRftbtep
--HG--
extra : rebase_source : b1172c9cc8b8be437d3b94a6bf0ff6b2f7d3508b
extra : source : 73bf88110b3821d62a3d393e85b56896a12f2930
This change makes us upload an `$(PKG_BASENAME).generated-files.tar.gz` archive
alongside other build artifacts which contains all the generated source files
from the build. A change after this will introduce an `upload-generated-sources`
task to take this artifact and upload the individual files to an S3 bucket.
This will be used to provide links to generated source files when they appear
in stack traces in crash reports.
MozReview-Commit-ID: 6yQAdlZ5q3O
--HG--
extra : rebase_source : d92fb17ae737d1360e9724997f6688e29bedef12
extra : source : 14d18d7cf454c4c3d0f6d49d1d01660e06e4be4b
A followup change will be adding a new automation step that wants to be skipped
in artifact builds, and this will make that simpler.
MozReview-Commit-ID: 5xwRB9eCRQn
--HG--
extra : rebase_source : 2fccd9d128ab92c98515762a62a0a2e89bf9ca24
extra : source : a02695cbf5762eb0eb7087239319807eb447ca1e
shutil.copy2() will fail if the destination directory doesn't exist.
Switch to copy_tree() instead so we don't need to worry about the
error cases of copy2() and copytree().
MozReview-Commit-ID: 3kHfgL57KfX
--HG--
extra : rebase_source : c7335b0c2854d53699dda0f0d2bd9d17b57c4e5d
We define extra_toolchain_flags for passing extra flags to the target
compiler during configure. But the way things are currently set up, we
pass those flags to the host compiler during configure as well. This
behavior is incorrect, and we should only be passing the flags from
extra_toolchain_flags if we're compiling for the target.
This is similar to what we had until bug 1278456 removed them when we dropped
support for older libstdc++, but for a different symbol.
--HG--
extra : rebase_source : 641fc6c86c8f47e3dbd752bc20056f61646541a7
That the wrapper implementation works has been verified by creating a
dummy program such as:
$ cat test.cc
#include <thread>
int main() {
std::thread([]() {
printf("foo\n");
}).join();
return 0;
}
And compiling it with and without the hack:
$ g++ -fno-rtti -o test test.cc -lpthread
$ objdump -TC test | grep UND.*GLIBCXX_3.4.22
0000000000000000 DF *UND* 0000000000000000 GLIBCXX_3.4.22 std:🧵:_State::~_State()
0000000000000000 DF *UND* 0000000000000000 GLIBCXX_3.4.22 std:🧵:_M_start_thread(std::unique_ptr<std:🧵:_State, std::default_delete<std:🧵:_State> >, void (*)())
$ ./test
foo
$ g++ -fno-rtti -o test test.cc $objdir/build/unix/stdc++compat/stdc++compat.o -lpthread
$ objdump -TC test | grep UND.*GLIBCXX_3.4.22
$ ./test
foo
--HG--
extra : rebase_source : 53ca8e2d0424eaeb539d50510c441c8a3252c819
Various people want to start experimenting with Python 3 in the build
system and in related tools (like mach).
We want to make it easy to find and use an appropriate Python 3
binary.
This commit introduces a generic function for finding a Python 3
binary and resolving its version.
We use the new code in configure to set PYTHON3 and PYTHON3_VERSION
subst entries for later consultation.
We also expose a cached attribute on the base class used by many
mach and build system types to return a Python 3 executable's info.
By default, we only find Python 3.5+. From my experience, Python 3.5
was the first Python 3 where it was reasonable to write code that
supports both Python 2 and 3 (mainly due to the restoration of the
% operator on bytes types). We could probably support Python 3.4
in the build system. But for now I'd like to see if we can get
away with 3.5+.
MozReview-Commit-ID: BlwCJ3kkjY9
--HG--
extra : rebase_source : b00464972183ef1a97a0b5d888520be425717ae7
This patch:
* Adds a suppression for some leaks in libLLVM-3.6-mesa.so.
* Adds Valgrind flag --keep-debuginfo=yes so that the abovementioned leak
stacks can be symbolised and hence suppressed even after
libLLVM-3.6-mesa.so is unmapped from the process.
* Adds Valgrind flag --expensive-definedness-checks=yes as an attempt to
reduce Memcheck false positives from LLVM and rustc compiled code. This
change is aimed primarily at bug 1365915.
MozReview-Commit-ID: KiOZG2O8wzs
Bug 1338651 was backed out because when building a newer image, there
was a valgrind leak report that couldn't resolve symbols. Further
investigation showed the valgrind package installed had symbols stripped.
We upgrade valgrind version and build it from source with symbols.
We had to build inside the docker image because we need to run
"make install". Using "make dist" to generate a tar ball will also run
"make docs", and it is hard to make it work because of the outdated
texlive package present in CentOS 6.
We also apply a patch [1] to valgrind correctly generate symbols
for unloaded objects.
[1] https://bugs.kde.org/show_bug.cgi?id=79362#c62
MozReview-Commit-ID: 2IhuJY28Ke3
In bug 635961, elfhack was made to (ab)use the bss section as a
temporary space for a pointer. To find it, it scanned writable PT_LOAD
segments to find one that has a different file and memory size,
indicating the presence of .bss. This usually works fine, but when
the binary is linked with lld and relro is enabled, the end of the
file-backed part of the PT_LOAD segment containing the .bss section
ends up in the RELRO segment, making that location read-only and
subsequently making the elfhacked binary crash when it tries to restore
the .bss to a clean state, because it's not actually writing in the .bss
section: lld page aligns it after the RELRO segment.
So instead of scanning PT_LOAD segments, we scan for SHT_NOBITS
sections that are not SHF_TLS (i.e. not .tbss).
--HG--
extra : rebase_source : f18c43897fd0139aa8535f983e13eb785088cb18
Suppress leaks for allocations where any stack frame matches the string
style::gecko::global_style_data. This could be dangerous if someone were to
allocate non-global things with this on the stack, but hopefully that can be
avoided, given the name of the module.
MozReview-Commit-ID: 65HpUGsgPPM
--HG--
extra : rebase_source : 19ba5a5001de65628acdb0433341d14429e4af67
GYP of WebRTC should reference MOZ_SYSTEM_LIBEVENT values if available.
MozReview-Commit-ID: CshsPrRidM8
--HG--
extra : rebase_source : 9e619c2f49e7c2b3f680814b95b823996773fa6c
In some cases, we can end up linking some things with
--static-libstdc++. The notable (only?) example of that is for the
clang-plugin, and that happens because it gets some of its flags from
llvm-config, which contains --static-libstdc++ because clang itself is
built that way.
When that happens, the combination of --static-libstdc++ and
stdc++compat breaks the build because they have conflicting symbols,
which is very much by design.
There are two ways out of this:
- avoiding either -static-libstdc++ or stdc++compat
- work around the symbol conflicts
The former is not totally reliable ; we'd have to accurately determine
if we're in a potentially conflicting case, and remove one of the two in
that case, and while we can do that for the cases we explicitly know
about, that's not future-proof, and might fail just as much in the
future.
So we go with the latter. The way we do this is by defining all the
std++compat symbols weak, such that at link time, they're overridden by
any symbol with the same name. When building with -static-libstdc++,
libstdc++.a provides those symbols so the linker eliminates the weak
ones. When not building with -static-libstdc++, the linker keeps the
symbols from stdc++compat. That last assertion is validated by the
long-standing CHECK_STDCXX test that we run when linking shared
libraries and programs.
That still leaves the symbols weak in the final shared
libraries/programs, which is a change from the current setup, but
shouldn't cause problems because when using versions of libstdc++.so
that do provide those symbols, it's fine to use the libstdc++.so version
anyways.
Bump the minimum required version of the Rust toolchain to
the current stable release so we can take advantage of new
features.
Highlights of the 1.19.0 release:
* C-compatible `union` (untagged enums).
* Support for Visual Studio 2017.
* Non-capturing closures can be coerced to `fn` bindings.
* Numeric field names in tuple struct initializers.
* Higher macro recursion limit.
* `break` can return a value from `loop` expressions.
* Better error handling with mis-configured Visual Studio environments.
This change also enables 1.18.0 features. Some highlights:
* `pub(mod)` &c. for better control of symbol visibility.
* struct packing for better memory footprint in generated code.
* Faster build times.
MozReview-Commit-ID: 2OpUjAcytpE
--HG--
extra : rebase_source : 2ed0d7c4e7b78c26f7a7476e7b284bf1bdbe7c8b
Mostly removals, but also adding docs on how to create a
multilingual package right now.
I think I'd like to take another pass at those docs in a follow-up.
MozReview-Commit-ID: Dkw4MJ5DLyb
--HG--
extra : rebase_source : 4f79499e376cf6ddd23169a1c4525ed6b551a740
It becomes a library of some sort, so that multiple scripts can benefit
from it to build different versions of GCC.
The GPG key associated with GCC is also refreshed from keys.gnupg.net,
adding a new subkey, used to sign newer versions of GCC (and
postprocessed with pgpstrip to make it smaller).
We're soon going to build multiple versions of clang and gcc for linux,
and we need to differentiate them. Furthermore, there is a need for the
base-toolchains builds to use a fixed version of clang and gcc. So
rename the clang and gcc toolchain jobs to include their version, add
aliases to satisfy all existing jobs, and adjust the base-toolchains
jobs to use the explicit version.
--HG--
rename : build/build-clang/clang-linux64.json => build/build-clang/clang-3.9-linux64.json
rename : taskcluster/scripts/misc/build-gcc-linux.sh => taskcluster/scripts/misc/build-gcc-4.9-linux.sh
The lld linker creates separate segments for purely executable sections
(such as .text) and sections preceding those (such as .rel.dyn). Neither
gold nor bfd ld do that, and just put all those sections in the same
executable segment.
Since elfhack is putting its executable code between the two relocation
sections, it ends up in a non-executable segment, leading to a crash
when it's time to run that code.
We thus insert the elfhack code before the first executable section
instead of between the two relocation sections (which is where the
elfhack data lies, and stays).
--HG--
extra : rebase_source : ab18eb9ac518d69a8639ad0e785741395b662112
Since bug 635961, building with relro makes elfhack try to use the bss
data for a temporary function pointer. If there is not enough space for
a pointer in the bss, elfhack will complain it couldn't find the bss.
In normal circumstances, this is most likely fine. Libraries with a bss
so small that it can't fit a pointer are already too small to be
elfhacked anyways. In Firefox, the two libraries with the smallest bss
have enough space for two pointers, and aren't elfhacked (libmozgtk.so
and libplds4.so).
However, the testcase that is used during the build to validate that
elfhack works doesn't have a large enough bss on x86-64, making elfhack
bail out, and the build fail as a consequence.
This, in turn, is due to the only non-thread-local zeroed data being an
int, which is not enough to fit a pointer on x86-64. We thus make it a
size_t.
--HG--
extra : rebase_source : bca2ddbf9d4a5e8786881fc524d642c38d610227
Configure now detects VCS info. Configure now detects Watchman.
We can combine the two so configure can detect if Mercurial
is configured with Watchman enabled.
This commit does two things:
1) collects the Mercurial config so it is available to downstream checks
2) examines the config for presence and state of the fsmonitor
extension
We don't yet do anything with the fsmonitor state. But it should be
useful soon. Also, the return value is kinda wonky. This will almost
certainly be improved as soon as there is an actual consumer.
MozReview-Commit-ID: HyHZ2X8VI0h
--HG--
extra : rebase_source : e53928127470340275f0c0f07db72b536bba885b
extra : source : a8373914cbfd9b8595fc24f36c876cab0a26c02a
Often you only want to evaluate a function if all its dependencies
are true. Expressing this in a "when" can be difficult. So let's
add a convenience decorator for it.
The existing code for @depends_if() was refactored to take an
evaluation function as its first argument. This prevents some
duplicate code and turns @depends_if() and @depends_all() into
one-liners.
MozReview-Commit-ID: Jbugvf0lioM
--HG--
extra : rebase_source : 177741b80ac4fbfb547d6b36a6f5777fe514d91a
extra : source : 0c2bc12f4ebe44428385745266d2fd158e0c3382
Build Stylo (the styling system from servo) by default in all
builds for win32, win64, macOS and linux64 targets. It was
previously enabled for automation builds, so this just changes
the behaviour for local developer builds.
Note that this introduces a new dependency on libclang for the
binding generator. If you're developing on a tier-1 platform,
run `./mach boostrap` to install a working copy. Otherwise
llvm+libclang 4.0.1 is recommended.
Remove the explicit --enable-stylo=build in mozconfig.stylo
in favour of the configure default.
Add mozconfig.stylo to the hazard and debug-asan mozconfigs
so LLVM_CONFIG is defined properly for those builds.
Based on a patch by Bobby Holly in bug 1356991.
MozReview-Commit-ID: C2wRNl7JHpz
--HG--
extra : rebase_source : 1ed7c36a64e25b235a26864592cd7ea969a4cd25
Configure now detects VCS info. Configure now detects Watchman.
We can combine the two so configure can detect if Mercurial
is configured with Watchman enabled.
This commit does two things:
1) collects the Mercurial config so it is available to downstream checks
2) examines the config for presence and state of the fsmonitor
extension
We don't yet do anything with the fsmonitor state. But it should be
useful soon. Also, the return value is kinda wonky. This will almost
certainly be improved as soon as there is an actual consumer.
MozReview-Commit-ID: HyHZ2X8VI0h
--HG--
extra : rebase_source : d245d316cc8a27b2827b7824204549b91465bd34
Often you only want to evaluate a function if all its dependencies
are true. Expressing this in a "when" can be difficult. So let's
add a convenience decorator for it.
The existing code for @depends_if() was refactored to take an
evaluation function as its first argument. This prevents some
duplicate code and turns @depends_if() and @depends_all() into
one-liners.
MozReview-Commit-ID: Jbugvf0lioM
--HG--
extra : rebase_source : 38b4af7b668830589126e8a83f5d5ab73914d07c
In bug 1181040, we added ${var}_IS_SET variables for
mk_add_options-defined variables. In the two years since, that has never
been used for anything else than MOZ_PGO_IS_SET, and the only use for
that has now been removed, so remove those ${var}_IS_SET variables.
--HG--
extra : rebase_source : 2fc9abe0c3badbf06f3858fcf326237e67891fee
Note this will only enable it on try, autoland and mozilla-inbound,
which are the only branches where sccache are enabled at the moment.
Enabling on more builds (or not) is the subject of bug 1373460.
Also note that bug 1181040, that ensured PGO builds weren't using
sccache mentions that back then, link times went up when using sccache
(with -Z7) vs. without, but that was presumably with MSVC 2013. Try
suggests link times are the same using sccache now (still using -Z7,
pending bug 1318370).
--HG--
extra : rebase_source : 9f9d87714f3c841b790eb7d692ea0968b1755b70
We want most builds to be actually using sccache, so we include
mozconfig.cache from mozconfig.common. However, since the --with-ccache
configure option doesn't exist on non-compile jobs (e.g. artifact
builds), we move to using the CCACHE environment variable instead, which
allows us to unset it in mozconfig.no-compile.
And since mozconfig.no-compile is always included where no_sccache is
set, we can remove that variable.
--HG--
extra : rebase_source : a8c743de1fd7a3c0fbc53f7c233df36585897767
Some gaia-related code was also found and removed as part of the
cleanup.
MozReview-Commit-ID: DEjVSljzzu1
--HG--
extra : rebase_source : 58c4c34df44a258d90029853f29ea01338bd142b
Bug 1365859 introduced a dependency on the Visual Studio binary 'fxc'
to generate Shader bytecode. This was unavailable when compiling for
Windows on Linux as part of a MinGW build.
This commit adds a configure check for fxc, and also searches for
fxc2, which was written (https://github.com/tomrittervg/fxc2) to be
a tiny application that wraps D3DCompileFromFile and can produce
similar (but not exactly the same) output as fxc.
fxc2 is compiled using MinGW for Windows, and runs under wine, so
we need to check for wine also.
Finally, fxc outputs some include information fxc2 doesn't, so
we will just change that assert to not take effect.
MozReview-Commit-ID: 8LVxuODi6cV
--HG--
extra : rebase_source : 9116d266663284d6594e34aa53bd37eae01ba67f
The |mach try| implementation isn't really testing specific so figured while doing this refactor
anyway we could take the opportunity to move it to /tools. This renames the "autotry" module to
"syntax" as one of the 'selectors' under the newly created tryselect module.
MozReview-Commit-ID: DmnGU5sKuCk
--HG--
rename : testing/tools/autotry/autotry.py => tools/tryselect/selectors/syntax.py
extra : rebase_source : 52f2a61d97107b8d50e3894ea0f5cd9c24d4108d
This option was added ~decade ago; AFAICT from bug archaeology, the
option was added to prevent our servers from being overwhelmed.
Somewhere over the years, however, we obtained more capable servers and
the option disappeared from mozconfigs. It seems moderately unlikely
that we'll have a need for this option again, and we could reintroduce
this patch very easily in any event. Let's go ahead and remove it.
We define extra_toolchain_flags for passing extra flags to the target
compiler during configure. But the way things are currently set up, we
pass those flags to the host compiler during configure as well. This
behavior is incorrect, and we should only be passing the flags from
extra_toolchain_flags if we're compiling for the target.
This change adds an upload-generated-sources task kind that runs after nightly
builds, fetches their `target.generated-files.tar.gz` artifact, and uploads
all the contained files to an S3 bucket. For actual nightly and release builds
on SCM level 3 trees, the S3 bucket is configured to be publicly accessible,
so that tools like Socorro will be able to fetch generated source files that
appear in crash reports, and debuggers will be able to fetch generated sources
when they show up while debugging Nightly or Release builds.
There are also level-2 and level-1 S3 buckets configured for builds happening
on trees of other levels such as try. They are not configured as publicly
accessible, but they exist so that these tasks can be tested in try.
MozReview-Commit-ID: Js1HRftbtep
--HG--
extra : rebase_source : 13d6b71200cda11938af3048cbeddce4edfc9c07
This change makes us upload an `$(PKG_BASENAME).generated-files.tar.gz` archive
alongside other build artifacts which contains all the generated source files
from the build. A change after this will introduce an `upload-generated-sources`
task to take this artifact and upload the individual files to an S3 bucket.
This will be used to provide links to generated source files when they appear
in stack traces in crash reports.
MozReview-Commit-ID: 6yQAdlZ5q3O
--HG--
extra : rebase_source : 3f6ef734c062e0f5e9c2ca433ffad51fdf14b1ad
A followup change will be adding a new automation step that wants to be skipped
in artifact builds, and this will make that simpler.
MozReview-Commit-ID: 5xwRB9eCRQn
--HG--
extra : rebase_source : 0b5b5087eddbf030161482f054ddd4d7cc08ffd4
The values that we need to find in the registry can be inconsistent across
different installations, so we retrieve values from both views in our search
for a valid SDK. This also ensures this works for 32-bit and 64-bit python.
Mercurial installs on Windows often provide both a native hg.exe
and a hg shell script. This allows them to work from both Windows
and UNIX-like shells.
While the which Python package searches for .exe and other common
executable extensions on Windows automatically, it doesn't prefer
these results to an extension-less value. This meant that "hg"
was resolved to the shell script. When we went to run it, it
failed because it isn't a valid Win32 application.
We work around the problem by preferring "hg.exe" over "hg"
for the binary name.
Ideally this would be platform specific. But the harm should be
minimal.
--HG--
extra : rebase_source : 1354c739de1ff738812ad100176ad7e9d08fadde
This is now required as of 00ef8018730c.
Landing on a CLOSED TREE.
--HG--
extra : rebase_source : 410e4a677cdc1f1238f778a00078ac6e6de420bd
extra : amend_source : e81cf510948bedd76ac77790eb96aae1fe3cda1c
For reasons unknown to me, Windows CI is periodically failing to find
the Mercurial binary.
In addition, we've also reimplemented various VCS logic throughout
the build system. There is room to cut down on code complexity by
e.g. recording VCS info in configure instead of determining it
at run-time.
Also, for forensic purposes it is sometimes desirable to know which
VCS tool is in use by a build and which version of that tool is being
used.
This commit adds VCS type detection, binary searching, and version
resolution to configure.
substs now contains VCS_CHECKOUT_TYPE, HG, and GIT, which can be
consulted by downstream consumers.
If the Mercurial or Git versions could not be resolved, all variables
are not set. Otherwise, VCS_CHECKOUT_TYPE and one of HG or GIT is set.
If MOZ_AUTOMATION is set, we require that the VCS info be resolved.
This helps prevents weirdness in automation due to unexpected
environment configuration.
MozReview-Commit-ID: AMLy0Hfx5rD
--HG--
extra : rebase_source : edef9165d32dc47308a14b0fbabce3c1d3d28176
And include code to work around a bug on older Python versions.
MozReview-Commit-ID: 4pBnMQQJOGB
--HG--
extra : rebase_source : 6f7c5784230bd37b3496b9bb1781e8d342f741b4
developer_options (!--enable-release) implies to search for and prefer
the gold linker. As part of porting gold detection to moz.configure
in d0e782180741 (bug 1351109), the logic inadvertently changed to
require gold when either --enable-gold or !--enable-release were
present.
This commit relaxes the requirement to find gold when only --enable-gold
is true.
MozReview-Commit-ID: HTBicWNUkvy
--HG--
extra : rebase_source : dd3938a7914f5db6c315fb775e7cc5ea177bf600
check_prog, when used with a `when` argument, doesn't work the same way
as putting it under a `with only_when()` block, while it should. The
difference comes from the fact that `with only_when()` applies the
`when` to every option and depends used in the block (which check_prog
calls a bunch of).
So, we "manually" apply the `when` to all option and depends in
check_prog. An alternative solution would be to put the whole function
under a `with only_when()` block, but that would mean reindenting the
whole function.
Either way, as a consequence, this requires the `when` to have a
dependency on --help for "non-trivial" functions, which fortunately,
there's only one of.
--HG--
extra : rebase_source : d91eca9e303c7062394d92a526983714ef3e612f
LLVM_CONFIG, per the contents of toolkit/moz.configure, is tied to
--enable-stylo, but it currently is set on all types of builds. It
currently happens to work, but it's actually not meant to, and sure
enough, the fix for bug 1374727 exacerbates that.
So we create a new mozconfig.stylo file that enables stylo and sets
LLVM_CONFIG, such that only build types that do enable stylo have
LLVM_CONFIG set.
--HG--
extra : rebase_source : 01277a79951888046c0b8e29c61cfc3b049ee0f0
For reasons unknown to me, Windows CI is periodically failing to find
the Mercurial binary.
In addition, we've also reimplemented various VCS logic throughout
the build system. There is room to cut down on code complexity by
e.g. recording VCS info in configure instead of determining it
at run-time.
Also, for forensic purposes it is sometimes desirable to know which
VCS tool is in use by a build and which version of that tool is being
used.
This commit adds VCS type detection, binary searching, and version
resolution to configure.
substs now contains VCS_CHECKOUT_TYPE, HG, and GIT, which can be
consulted by downstream consumers.
If the Mercurial or Git versions could not be resolved, all variables
are not set. Otherwise, VCS_CHECKOUT_TYPE and one of HG or GIT is set.
If MOZ_AUTOMATION is set, we require that the VCS info be resolved.
This helps prevents weirdness in automation due to unexpected
environment configuration.
MozReview-Commit-ID: AMLy0Hfx5rD
--HG--
extra : rebase_source : edef9165d32dc47308a14b0fbabce3c1d3d28176
And include code to work around a bug on older Python versions.
MozReview-Commit-ID: 4pBnMQQJOGB
--HG--
extra : rebase_source : 6f7c5784230bd37b3496b9bb1781e8d342f741b4
This enables apm logging by setting the apm_debug_dump variable in gyp.mozbuild.
Prior to this change, some files were including apm_data_dumper.h with logging
enabled and some were not.
This also removes the AEC* C interface and calls into webrtc::Trace directly.
Whatever historical reasons for having a C interface into these calls no
longer seems to apply. In addition reserving a buffer for the base file name
and then ensuring it was null terminated caused an ASAN "stack-buffer-overflow"
while testing. This was because it was not handling an empty base file
name properly. This would not normally happen if AEC logging was enabled through
about:webrtc, but it still seems safer to use std::string.
MozReview-Commit-ID: Ikz5xO74syA
--HG--
extra : rebase_source : 8e0c59117135fadb75f4a7e6be5588af1404533d
When investigating why a particular build got a particularly high number
of cache misses, it is useful to know exactly *what* had cache misses,
and you can't get the missing information after the fact.
We used to have some stats stored in a log file uploaded as artifact
with the old python sccache per bug 1005495 (and before that inline in
the build log), and we actually still have remains of that in the form
of a EXTRA_UPLOAD_FILES in build/mozconfig.cache.
Let's reuse that, and get some useful logs from sccache. Ideally sccache
would give better logs, but those are sufficient for now. We'll figure
things out eventually. https://github.com/mozilla/sccache/issues/151
--HG--
extra : rebase_source : a2e7093697a5b720ab28f155560211d6c0a4347e
Rustc 1.20 gained support for make jobserver. But when rustc is invoked
through sccache, sccache needs to have inherited the make jobserver file
descriptor to pass it down to rustc. But what will be running sccache
first during the build is not really deterministic. It might as well end
up being a script invoking the preprocessor that ends up doing it. And
we can't add the necessary + prefix to all commands that potentially can
end up invoking sccache.
So we manually start the sccache server at the beginning of the build,
through preflight.
Note this works on automation, but not for local builds, but few people
are using sccache locally for rust (yet). Let's figure that out in a
followup.
--HG--
extra : rebase_source : eda5853d7247be247a1bec1a5d7e84b3846fd2c1
This patch does two things:
- add a Gradle-only ANDROID_COMPILE_SDK_VERSION substitution;
- uses it while uniformizing all of the Gradle Android SDK version
configurations.
The approach is fairly standard (and we were using it already); see,
for example
https://medium.com/@ali.muzaffar/gradle-configure-variables-for-all-android-project-modules-in-one-place-5a6e56cd384e
This will make bumping the Gradle configuration versions forward
easier.
MozReview-Commit-ID: 1j5siCvR5qt
--HG--
extra : rebase_source : 07afb00de0e4a72af4026eb19ff4f2530c119336
This is just a small piece in the larger project of minimizing the set
of global AC_DEFINEs. None of these are relevant to C/C++ code;
they're mobile/android specific, so we should get them out of
mozilla-config.h.
In addition, this replaces hard-coded values with values from the
build system throughout mobile/android.
MozReview-Commit-ID: 9IJlsm38LFK
--HG--
extra : rebase_source : 0e55c0967cbb7f0df0d297eb330fc615e5759346
Ryan VanderMeulen
Dustin J. Mitchell
Mike Hommey
Tom Ritter
Johan Lorenzo
Makoto Kato
David Major
Gregory Szorc
Wander Lairson Costa
Wes Kocher
Philip Chimento
Ting-Yu Chou
Tristan Bourvon
Sebastian Hengst
Phil Ringnalda
Chris Peterson
Eric Rahm
Ted Mielczarek
Dave Hunt
Nathan Froyd
Eugen Sawin
J. Ryan Stinnett
Matt Brubeck
Sylvestre Ledru
Julian Seward
Kris Maglione
Paul Adenot
Carsten "Tomcat" Book
Sebastian Kaspari
Ralph Giles
Axel Hecht
Cameron McCormack
Nick Alexander
Andrew Halberstadt
Bob Owen
Dan Minor
Jean-Yves Avenard
mozilla@mozilla-23773