Bug 1546136 wrapped subprocess execution output to capture cmake's, but
at the detriment of other output, and hiding everything unless an error
occurs.
So instead, we only capture the output when the called process is cmake,
and even when it is cmake, we don't pipe stderr at all (since we only
care about cmake's stdout) and we print out stdout as it comes in rather
than later. We then later check the output for hints at the more useful
cmake logs and dump them.
While here, add verbosity to ninja output (which gives the command
lines, rather than generic "Building foo.o" output).
Differential Revision: https://phabricator.services.mozilla.com/D40142
This change gives us the ability to selectively turn on cross-language
PGO, just like we have the ability to selectively turn on cross-language LTO.
There is room for things to go wrong here: it's not guaranteed that
`--enable-profile-generate=cross` will always be used with
`--enable-profile-use=cross`. Nothing bad will happen in the sense that
the build will succeed, but it's possible that we miss out on
optimizations on the Rust side. Either we fail to generate profile data
for the Rust code, or the Rust compiler fails to use the profile data.
In the future, we may want to default to cross-language PGO to avoid
these kind of mismatches.
Differential Revision: https://phabricator.services.mozilla.com/D39727
--HG--
extra : moz-landing-system : lando
To do properly checks on LLVM version correspondence between `clang` and
`rustc`, we need information about both of those compilers to be
available. The current placement of the LTO/PGO checks is after we know
something about `clang`, but before we know something about `rustc`.
Therefore we need to move those checks after we've gathered information
about `rustc`.
The PGO bits come along for this bug because the LTO bits depend on
them, and we're going to need the Rust information for cross-language
PGO checks in a different bug. So we might as well move everything all
at once.
Differential Revision: https://phabricator.services.mozilla.com/D39390
--HG--
rename : build/moz.configure/toolchain.configure => build/moz.configure/lto-pgo.configure
extra : moz-landing-system : lando
This change will eventually enable us to cross-check `rustc`'s version
with `clang`'s version when doing cross-language LTO/PGO and avoid
people running into peculiar errors at link time.
Differential Revision: https://phabricator.services.mozilla.com/D39388
--HG--
extra : moz-landing-system : lando
Plenty of places use `nproc`, and only a couple use `getconf
_NPROCESSORS_ONLN`. Use the former instead of the latter.
Differential Revision: https://phabricator.services.mozilla.com/D39999
--HG--
extra : moz-landing-system : lando
On Windows in Python 2, the subprocess module requires the use of bytes with
the 'env' argument. For that reason, we would sometimes use byte strings with
'os.environ' like so:
os.environ[b"FOO"] = b"bar"
However, this is a failure with Python 3 as 'os.environ' must only be used with
the text type. This patch creates a new 'setenv' helper that ensures we create
new environment with 'bytes' on Python 2, and 'text' on Python 3.
Differential Revision: https://phabricator.services.mozilla.com/D38237
--HG--
extra : moz-landing-system : lando
These tests are based on `moz_global_class` and `moz_trivial_ctor_dtor` tests,
respectively, but adapted for the semantics of the new attributes.
Differential Revision: https://phabricator.services.mozilla.com/D39718
--HG--
rename : build/clang-plugin/tests/TestGlobalClass.cpp => build/clang-plugin/tests/TestStaticLocalClass.cpp
rename : build/clang-plugin/tests/TestTrivialCtorDtor.cpp => build/clang-plugin/tests/TestTrivialDtor.cpp
extra : moz-landing-system : lando
This patch is in support of adding a variant of Static{Auto,Ref}Ptr for use as
static locals, taking advantage of C++11 "magic statics" such that we can lazily
initialize those variables in a thread-safe way.
In support of those classes, this patch adds two new attributes:
* `moz_static_local_class` to ensure that any instantiations of that class only
occur as static local variables;
* `moz_trivial_dtor` to ensure that these classes do not implicitly call `atexit`
and add a whole bunch of shutdown crap.
`moz_static_local_class` works similarly to `moz_global_class`, except that its
object must only instantiate as static locals.
`TrivialDtorChecker` is based on `TrivialCtorDtorChecker`, with the ctor-specific
bits removed.
Differential Revision: https://phabricator.services.mozilla.com/D39717
--HG--
rename : build/clang-plugin/TrivialCtorDtorChecker.cpp => build/clang-plugin/TrivialDtorChecker.cpp
rename : build/clang-plugin/TrivialCtorDtorChecker.h => build/clang-plugin/TrivialDtorChecker.h
extra : moz-landing-system : lando
Toolchain path for Windows version is `<NDK ROOT>/toolchains/llvm/prebuilt/windows-x86_64` etc, so it isn't '`winnt`.
So we has to replace `host.kernel.lower()` with `windows`.
Differential Revision: https://phabricator.services.mozilla.com/D39474
--HG--
extra : moz-landing-system : lando
...rather than people running into peculiar crashes running their tests
because functions are pointing at the wrong thing.
It would be more robust to version-check `ld`, but I figure people
wanting to do local cross-language LTO builds is rare enough that
setting an environment variable and rerunning configure is not a huge
hardship.
Differential Revision: https://phabricator.services.mozilla.com/D36742
--HG--
extra : moz-landing-system : lando
...rather than people running into peculiar crashes running their tests
because functions are pointing at the wrong thing.
It would be more robust to version-check `ld`, but I figure people
wanting to do local cross-language LTO builds is rare enough that
setting an environment variable and rerunning configure is not a huge
hardship.
Differential Revision: https://phabricator.services.mozilla.com/D36742
--HG--
extra : moz-landing-system : lando
In browsertime.zip we should have:
browsertime/
package.json
package-lock.json
node_modules/
.bin/
browsertime -> ../browsertime/bin/browsertime.js
browsertime/
...
The idea is that we'll fetch browsertime.zip in a generic-worker
environment and be able to run Node.js from within the top level
browsertime/ directory.
Differential Revision: https://phabricator.services.mozilla.com/D38773
--HG--
extra : moz-landing-system : lando
This build re-uses the PGO profile from the win64 build in the
win64-aarch64-shippable-no-eme part of the aarch64 build. Even though
the profile isn't generated on the smae platform, we still get enough of
a performance win to make this worthwhile.
Note that the pgo_flags() in configure need to be tweaked slightly since
we don't supprt the -fprofile-generate flag for aarch64 (we don't build
the clang_rt.profile lib there). So we always want to return the flags
namespace to make sure we get the use_* versions of flags, which we do
need.
Differential Revision: https://phabricator.services.mozilla.com/D38928
--HG--
extra : moz-landing-system : lando
There is less incentive to keep things building with older versions of
clang for OSX builds, and we're going to require an objective-C feature
that was added in clang 5.
Differential Revision: https://phabricator.services.mozilla.com/D38581
--HG--
extra : moz-landing-system : lando
Actually we set _DEPEND_CFLAGS to both host and target compiler. But if host and target are different compiler type, we may pass invalid option.
Differential Revision: https://phabricator.services.mozilla.com/D38457
--HG--
extra : moz-landing-system : lando
This change is a little bit of a cheat, because of course MSVC doesn't
do cross-language LTO by default, but it seems consistent.
Depends on D33317
Differential Revision: https://phabricator.services.mozilla.com/D33318
--HG--
extra : moz-landing-system : lando
When we build mar, there is no reason not to build signmar as well. It
used to be optional because not all platforms were supported, but they
are now.
... except when building the newly added tools/update-packaging,
which builds the mar tool as a standalone thing, and building signmar
as well causes complications.
Differential Revision: https://phabricator.services.mozilla.com/D36992
--HG--
extra : moz-landing-system : lando
This was originally from bug 1528374 for Mac PGO, but that isn't able to
land yet and it should help Windows PGO runs in the meantime.
Differential Revision: https://phabricator.services.mozilla.com/D37807
--HG--
extra : moz-landing-system : lando
Credit: Callek for figuring out an issue in 'make check' making the binary
absolute in mozbuild.base.
Differential Revision: https://phabricator.services.mozilla.com/D37319
--HG--
extra : moz-landing-system : lando
I couldn't use 'mach vendor python' to get this module in-tree because that
puts a 'backports' module on the PYTHONPATH. This is a problem because many
(most?) compatibility backports also use a 'backports' library. So in the
likely event that a user happens to have one of these installed in their
system Python, we'll search that module before the vendored one and fail to
find it.
This gets around the issue by foregoing 'backports' and just putting
'shutil_which' on the path
Differential Revision: https://phabricator.services.mozilla.com/D36838
--HG--
extra : moz-landing-system : lando
When we build mar, there is no reason not to build signmar as well. It
used to be optional because not all platforms were supported, but they
are now.
Differential Revision: https://phabricator.services.mozilla.com/D36992
--HG--
extra : moz-landing-system : lando
Mike Hommey
Nathan Froyd
Razvan Maries
Sylvestre Ledru
Andrew Halberstadt
Aaron Klotz
Makoto Kato
Andreea Pavel
Nick Alexander
Mike Shal
Simon Sapin
Nicholas Nethercote
Agi Sferro
David Walsh
Csoregi Natalia
Andi-Bogdan Postelnicu
Kartikaya Gupta
Narcis Beleuzu
Brindusan Cristian
Bogdan Tara
Tom Prince