Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: 554IPMRWgzK
--HG--
extra : rebase_source : 00827d3f6bd705419bc801eb05b543af1ddc274f
In some cases, they are false positives. In other cases, the callsite is #ifdef-ed
out, so it's a true positive, but not one we would correct.
MozReview-Commit-ID: 6ThZH3wEXTp
--HG--
extra : rebase_source : aabdb93cb924e74cf9592a93c86062c4435ceaad
``print()`` has no business being in library code like this. It was
a holdover from this code being copied from bootstrap. So remove it.
While we're here, replace the generic exception with a specific one.
We don't want to be swallowing bugs via ``except Exception``.
MozReview-Commit-ID: 49goUstfPBz
--HG--
extra : rebase_source : a821159bd12a449ed1a0edf21a1f9eb29711ad95
An inline comment wanted it. The change is trivial.
MozReview-Commit-ID: CqyOYqNzwzr
--HG--
extra : rebase_source : d779d6bbca4aa57d9e95ead3a8403e0163106bdf
The file was using 2 space indent. I ran `autopep8` on it. Most of the
changes are whitespace only.
MozReview-Commit-ID: 1C7yDUQCfKl
--HG--
extra : rebase_source : 9f2b52ef6d0887fbaa7f8410b32348d371e884d8
Mercurial's sparse checkout support allows "profile" files defining
what's in sparse checkouts to be defined in-tree. This convenient
feature means you simply need to point a client at a path inside the
repository and it dynamically resolves what files to include in the
checkout. As you update revisions, the "profile" pulls in updates
to the underlying file.
We introduce 2 sparse profiles: 1 for mach and another for taskgraph.
The goal of the "mach" profile is to provide enough files to run
`mach`. If you activate this profile and run `mach`, it runs without
error. But there are practically no commands available. So it isn't
terribly useful.
The "taskgraph" profile allows us to run `mach taskgraph`. This
profile demonstrates a sparse profile feature: including other
profiles. The "taskgraph" profile is thus a union of "mach" and
its own entries.
There is definitely some fat in these profiles. I didn't feel like
chasing the long tail and getting overly granular with the profiles.
If we want to optimize later, we can do that.
For reference:
Full checkout: ~234,000 files
mach: ~2,000 files
taskgraph: ~3,600 files
MozReview-Commit-ID: 7pALt0MwHfE
--HG--
extra : rebase_source : 1a4ba4b8a63c522dab2841e2c0019501476da2fe
python-hglib is a Python client for Mercurial's command server. It
facilitates querying Mercurial efficiently (using a single process)
and without having to parse output in the common case.
Let's vendor it so we can make use of it for more advanced Mercurial
scenarios.
Content vendored from changeset 820d7c1e470a without modifications
(other than deleting unwanted files).
As part of vendoring, we add the package to the virtualenv and make
it available to mach.
MozReview-Commit-ID: F4KLbW1lAvk
--HG--
extra : rebase_source : 39321a880a13a0b0323a7217f538978b729e2afe
Sparse checkouts may not have all mach_commands.py files.
mach raises an error when a mach_commands.py file is missing.
So, we teach the mach driver to ignore missing file errors when
a sparse checkout is present.
The added code is optimized to avoid an import of mozversioncontrol
and some I/O as part of resolving the repo and VCS binaries because
this file is in the critical path of all mach commands and avoiding
I/O is worthwhile. Since we aren't using sparse checkouts in the
common case, this effectively makes the new code 0 cost.
MozReview-Commit-ID: C6itJga31t5
--HG--
extra : rebase_source : 4b2c18d30ff8b923a940c80ac81372a4076b8fdc
"mach" in this scope is both the mach module and the driver instance.
Let's change the latter's variable name so we can access the mach
module.
MozReview-Commit-ID: Db6sxDFl2oo
--HG--
extra : rebase_source : 0a51c7514cfdba40483b3b9b361cd03b8000c2c9
The Repository interface gains a new method to determine if a sparse
checkout is present. Mercurial's implementation is somewhat crude,
but it should work (Mercurial's sparse support is still experimental
and I only intend to support sparse checkout in Firefox CI until it
is less experimental). Git's always returns False (for now).
To prove it works and to expose the information more widely, we hook
it up to moz.configure. We do this by first implementing a function
that returns a Repository instance. Then we simply call a function on
it to resolve the sparse checkout flag.
MozReview-Commit-ID: AlsT5LdSPdZ
--HG--
extra : rebase_source : f1e9aaa7d15f11c7c5e8d268d4ad82468732103b
This change adds an upload-generated-sources task kind that runs after nightly
builds, fetches their `target.generated-files.tar.gz` artifact, and uploads
all the contained files to an S3 bucket. For actual nightly and release builds
on SCM level 3 trees, the S3 bucket is configured to be publicly accessible,
so that tools like Socorro will be able to fetch generated source files that
appear in crash reports, and debuggers will be able to fetch generated sources
when they show up while debugging Nightly or Release builds.
There are also level-2 and level-1 S3 buckets configured for builds happening
on trees of other levels such as try. They are not configured as publicly
accessible, but they exist so that these tasks can be tested in try.
MozReview-Commit-ID: Js1HRftbtep
--HG--
extra : rebase_source : b1172c9cc8b8be437d3b94a6bf0ff6b2f7d3508b
extra : source : 73bf88110b3821d62a3d393e85b56896a12f2930
This change makes us upload an `$(PKG_BASENAME).generated-files.tar.gz` archive
alongside other build artifacts which contains all the generated source files
from the build. A change after this will introduce an `upload-generated-sources`
task to take this artifact and upload the individual files to an S3 bucket.
This will be used to provide links to generated source files when they appear
in stack traces in crash reports.
MozReview-Commit-ID: 6yQAdlZ5q3O
--HG--
extra : rebase_source : d92fb17ae737d1360e9724997f6688e29bedef12
extra : source : 14d18d7cf454c4c3d0f6d49d1d01660e06e4be4b
A followup change will be adding a new automation step that wants to be skipped
in artifact builds, and this will make that simpler.
MozReview-Commit-ID: 5xwRB9eCRQn
--HG--
extra : rebase_source : 2fccd9d128ab92c98515762a62a0a2e89bf9ca24
extra : source : a02695cbf5762eb0eb7087239319807eb447ca1e
shutil.copy2() will fail if the destination directory doesn't exist.
Switch to copy_tree() instead so we don't need to worry about the
error cases of copy2() and copytree().
MozReview-Commit-ID: 3kHfgL57KfX
--HG--
extra : rebase_source : c7335b0c2854d53699dda0f0d2bd9d17b57c4e5d
We define extra_toolchain_flags for passing extra flags to the target
compiler during configure. But the way things are currently set up, we
pass those flags to the host compiler during configure as well. This
behavior is incorrect, and we should only be passing the flags from
extra_toolchain_flags if we're compiling for the target.
This is similar to what we had until bug 1278456 removed them when we dropped
support for older libstdc++, but for a different symbol.
--HG--
extra : rebase_source : 641fc6c86c8f47e3dbd752bc20056f61646541a7
That the wrapper implementation works has been verified by creating a
dummy program such as:
$ cat test.cc
#include <thread>
int main() {
std::thread([]() {
printf("foo\n");
}).join();
return 0;
}
And compiling it with and without the hack:
$ g++ -fno-rtti -o test test.cc -lpthread
$ objdump -TC test | grep UND.*GLIBCXX_3.4.22
0000000000000000 DF *UND* 0000000000000000 GLIBCXX_3.4.22 std:🧵:_State::~_State()
0000000000000000 DF *UND* 0000000000000000 GLIBCXX_3.4.22 std:🧵:_M_start_thread(std::unique_ptr<std:🧵:_State, std::default_delete<std:🧵:_State> >, void (*)())
$ ./test
foo
$ g++ -fno-rtti -o test test.cc $objdir/build/unix/stdc++compat/stdc++compat.o -lpthread
$ objdump -TC test | grep UND.*GLIBCXX_3.4.22
$ ./test
foo
--HG--
extra : rebase_source : 53ca8e2d0424eaeb539d50510c441c8a3252c819
Various people want to start experimenting with Python 3 in the build
system and in related tools (like mach).
We want to make it easy to find and use an appropriate Python 3
binary.
This commit introduces a generic function for finding a Python 3
binary and resolving its version.
We use the new code in configure to set PYTHON3 and PYTHON3_VERSION
subst entries for later consultation.
We also expose a cached attribute on the base class used by many
mach and build system types to return a Python 3 executable's info.
By default, we only find Python 3.5+. From my experience, Python 3.5
was the first Python 3 where it was reasonable to write code that
supports both Python 2 and 3 (mainly due to the restoration of the
% operator on bytes types). We could probably support Python 3.4
in the build system. But for now I'd like to see if we can get
away with 3.5+.
MozReview-Commit-ID: BlwCJ3kkjY9
--HG--
extra : rebase_source : b00464972183ef1a97a0b5d888520be425717ae7
This patch:
* Adds a suppression for some leaks in libLLVM-3.6-mesa.so.
* Adds Valgrind flag --keep-debuginfo=yes so that the abovementioned leak
stacks can be symbolised and hence suppressed even after
libLLVM-3.6-mesa.so is unmapped from the process.
* Adds Valgrind flag --expensive-definedness-checks=yes as an attempt to
reduce Memcheck false positives from LLVM and rustc compiled code. This
change is aimed primarily at bug 1365915.
MozReview-Commit-ID: KiOZG2O8wzs
Wander Lairson Costa
Wes Kocher
Dustin J. Mitchell
Philip Chimento
Tom Ritter
Gregory Szorc
Ting-Yu Chou
Ryan VanderMeulen
Tristan Bourvon
David Major
Sebastian Hengst
Phil Ringnalda
Chris Peterson
Eric Rahm
Ted Mielczarek
Dave Hunt
Nathan Froyd
Mike Hommey
Eugen Sawin
J. Ryan Stinnett
Matt Brubeck
Sylvestre Ledru
Julian Seward