The expanded checks in clang 10 made arm64 builds hit CFG crashes on nsXPTCStubBase vtables on startup.
It's not clear why this doesn't happen on x86 builds. Given our current level of support for arm64, I can't really justify investigating this, although I suspect that fixing the underlying issue would be pretty much bug 1483885.
As a get-unblocked stopgap, `-guard:cf,nochecks` in clang 10 gives the same behavior as `-guard:cf` in clang 9.
Differential Revision: https://phabricator.services.mozilla.com/D76216
Configuration values defined above `mozconfig_options` in `init.configure` are those that can't be configured in `mozconfig`. As far as I can tell there is nothing wrong *in principle* with setting `PYTHON3` in `mozconfig`, so here we just bump `mozconfig_options` above `PYTHON3` configuration.
Side note, diagnosing this failure took a *long* time. The error message that this produced ("unknown option `PYTHON3`") is useless and the underlying algos being extremely mutable, I ended up having to spend a lot of time in the debugger and monitor all the underlying changes to a bunch of mutable data structures to determine where `PYTHON3` was getting lost. A better error message would be good here but I don't know how I would begin adding it.
Differential Revision: https://phabricator.services.mozilla.com/D75635
iOS support for Gecko has not been tested in years and is most probably
out of date. The build system part of it, specifically the checks in
build/autoconf/ios.m4, are not trivial to port to python configure, and
they prevent other things from moving to python configure (because some
of them change value when MOZ_IOS is set).
The code is left alone, although it could probably be stripped off as
well, but I'll leave that as an exercise for someone else.
Differential Revision: https://phabricator.services.mozilla.com/D75463
If we don't do this, configure will happily think the linker kind is
"unknown", which then causes problems downstream.
Differential Revision: https://phabricator.services.mozilla.com/D75300
ubsan's generated code trips an optimization corner case in clang 10. This is a short-term workaround. In the best case, llvm.org/pr45835 will land a fix soon, and we can merge that for our clang-10 update and not even land this patch. However, in case we're living in anything other than the best case world, I don't want to block on that happening.
Differential Revision: https://phabricator.services.mozilla.com/D74727
ubsan's generated code trips an optimization corner case in clang 10. This is a short-term workaround. In the best case, llvm.org/pr45835 will land a fix soon, and we can merge that for our clang-10 update and not even land this patch. However, in case we're living in anything other than the best case world, I don't want to block on that happening.
Differential Revision: https://phabricator.services.mozilla.com/D74727
This includes scripts that involve `tup`, jobs that build `tup` in automation, `tup.configure`, and related infrastructure and documentation.
Differential Revision: https://phabricator.services.mozilla.com/D73921
Now that there are only two uses of it, and there won't be more, remove
them and just use host_for_sub_configure and target_for_sub_configure
directly.
Differential Revision: https://phabricator.services.mozilla.com/D73411
As mentioned in previous change, js subconfigure used to be called from
old-configure, and some of that old setup remains to this day.
They don't really cause problems, except speed issues, especially on
Windows: arguments passed to old-configure is treated in a large shell
loop, and the more arguments are passed, the longer it takes. On my
machine, this cumulates to more than 10s! For arguments it doesn't
recognizes and doesn't use. And the environment variable js_options are
not used either, although they don't have a similar impact.
What we're doing here:
- Remove js_options from extra_old_configure_args, that leaves only
host_for_sub_configure and target_for_sub_configure there.
- Since none of the above are environment variables, we don't need to
separate out environment variables, nor to pass them down to
old-configure.
Differential Revision: https://phabricator.services.mozilla.com/D73410
js subconfigure used to be invoked from old-configure, and the
transition out of old-configure preserved the code sharing.
prepare_configure_options currently collects all the old configure
arguments, separating js_option arguments from js_option environment
variables. This distinction is necessary for old configure, but not
for python configure, so we can now handle all the js_options
similarly.
Differential Revision: https://phabricator.services.mozilla.com/D73409
Add the -X switch to ignore default include path, and tolerate proprocessor failures.
This enables the check to skip a directory that is missing headers, instead of
falling back on the default paths.
Differential Revision: https://phabricator.services.mozilla.com/D73371
Currently MozillaBuild with $PYTHON3 first creates virtualenv with the path from environment variable and then reexecutes with its own bundled executable. This patch forces it to run with virtualenv python when reexecuting.
Differential Revision: https://phabricator.services.mozilla.com/D72446
This was the case with my local rustc build today. We only use this directory for natvis files, it's fine to not have those.
Differential Revision: https://phabricator.services.mozilla.com/D72440
`os.path.realpath` in Python 3.8 now always uppercases Windows drive letter, while it was just an alias of `os.path.abspath` in Windows. This patch uses `.realpath()` consistently to get `topobjdir` to fix the incompatibility from the behavior change.
Differential Revision: https://phabricator.services.mozilla.com/D72188
When running e.g. check_symbols with extra flags like when checking
for vpx_codec_dec_init_ver when building against system libvpx, in some
configurations, the test can fail when the library flags (-l) appear
before the source file path.
The reason is that in some configurations, the compiler passes
--as-needed to the linker before both the flags and the object file
path, and the object file path is in the same position as the source
file path was. With --as-needed, -l flags are dropped if the library
wasn't needed for any of the linked code that appears *before* the flag.
So linking with `--as-needed -lfoo foo.o`, is equivalent to linking with
`foo.o` only in practice, while `--as-needed foo.o -lfoo` is equivalent
to `foo.o -lfoo`.
Differential Revision: https://phabricator.services.mozilla.com/D71456
This matches up to what we were doing before the Python 3 switch. Subprocess.Popen's default encoding doesn't necessarily match up to the system encoding as reported by mozbuild.util.system_encoding, and pre-Python 3.6 the constructor doesn't have `encoding` or `errors` parameters for no apparent reason. In the meantime, do the decoding manually.
Differential Revision: https://phabricator.services.mozilla.com/D71193
When running e.g. check_symbols with extra flags like when checking
for vpx_codec_dec_init_ver when building against system libvpx, in some
configurations, the test can fail when the library flags (-l) appear
before the source file path.
The reason is that in some configurations, the compiler passes
--as-needed to the linker before both the flags and the object file
path, and the object file path is in the same position as the source
file path was. With --as-needed, -l flags are dropped if the library
wasn't needed for any of the linked code that appears *before* the flag.
So linking with `--as-needed -lfoo foo.o`, is equivalent to linking with
`foo.o` only in practice, while `--as-needed foo.o -lfoo` is equivalent
to `foo.o -lfoo`.
Differential Revision: https://phabricator.services.mozilla.com/D71456
This also does a few remaining python 2 incompatible changes to
.configure files.
Differential Revision: https://phabricator.services.mozilla.com/D69538
--HG--
extra : moz-landing-system : lando
Using @checking doesn't go well with the virtualenv creation, so print
the version information after the fact. And do so uniformely for python
2 and python 3.
Differential Revision: https://phabricator.services.mozilla.com/D69529
--HG--
extra : moz-landing-system : lando
This is yet another cargo-cult from virtualenv_python2, which is not
necessary because by the time we reach virtualenv_python3, the mozconfig
configure *has* been injected already.
Differential Revision: https://phabricator.services.mozilla.com/D69528
--HG--
extra : moz-landing-system : lando
It was cargo-culted from virtualenv_python2, but is not necessary
because virtualenv_python3 is not going to re-execute python, which is
what the dependency is there to avoid.
Differential Revision: https://phabricator.services.mozilla.com/D69527
--HG--
extra : moz-landing-system : lando
This was cargo-culted from virtualenv_python2, but this is not
necessary: currently, the only way configure code will execute with
python 3 is if something uses the configure sandbox from python 3, not
configure. Only configure itself is meant to re-execute itself with
python, so we don't want python 3 to re-execute.
We'll be swapping virtualenv_python2 and virtualenv_python3 later, when
switching to python 3.
Differential Revision: https://phabricator.services.mozilla.com/D69526
--HG--
extra : moz-landing-system : lando
* We change the SDK version to use build tools 29.0.3;
* We change `build_tools_version` from a list to a string;
* We convert `android_build_tools` to work correctly with the above change;
* We set the build tools version in configure, as gradle needs to access it
in a subsequent patch.
Differential Revision: https://phabricator.services.mozilla.com/D69631
--HG--
extra : moz-landing-system : lando
We only enabled hardening with an explicit --enable-hardening because we
needed a patch. That patch was applied to upstream clang 8.0.1, so we
can now enable automatically whenever using the right version.
The explicit --enable-hardening was also not applied to win64-aarch64
debug builds, and this indirectly enables it there too, matching other
debug builds. This also avoids breaking debug builds when enabling
winchecksec on cross builds.
Differential Revision: https://phabricator.services.mozilla.com/D68161
--HG--
extra : moz-landing-system : lando
Several developers have reported hangs on native Mac builds. I think we should disable the flag on Mac hosts. Even if we could fix the current hangs, without coverage in CI for this build configuration, we're asking for more trouble down the road.
Differential Revision: https://phabricator.services.mozilla.com/D67007
--HG--
extra : moz-landing-system : lando
It turns out clang-cl adds an explicit /DEFAULTLIB:/full/path/to/runtime
to all the .obj it creates, which makes the linker add it on its own.
Differential Revision: https://phabricator.services.mozilla.com/D66368
--HG--
extra : moz-landing-system : lando
LLVM's new pass manager has been in the works for several years and has better optimization (sometimes much better) than the legacy pass manager. I think it's in good enough shape for us to try at this point.
While we only really need the new pass manager for shippable builds, as a general principle I'd like to use it as much as possible, to help catch bugs for upstream. Therefore this patch enables the new pass manager by default for all clang builds, with the only exceptions being compilers older than version 9, and xcode clang where we can't trust the version number. There isn't a specific problem with older versions; I just don't want to sign up for the support cost of debugging people's local builds that may be fixed already.
I don't expect it to be necessary, but just in case, an opt-out is available via `ac_add_options --disable-new-pass-manager`.
Differential Revision: https://phabricator.services.mozilla.com/D66109
--HG--
extra : rebase_source : 91df800146700e4958b8e645ebbd3cf7b11a2f1e
extra : source : 2f5aba2e2c099a1df26e3444ccec2be0b4ff4613
LLVM's new pass manager has been in the works for several years and has better optimization (sometimes much better) than the legacy pass manager. I think it's in good enough shape for us to try at this point.
While we only really need the new pass manager for shippable builds, as a general principle I'd like to use it as much as possible, to help catch bugs for upstream. Therefore this patch enables the new pass manager by default for all clang builds, with the only exceptions being compilers older than version 9, and xcode clang where we can't trust the version number. There isn't a specific problem with older versions; I just don't want to sign up for the support cost of debugging people's local builds that may be fixed already.
I don't expect it to be necessary, but just in case, an opt-out is available via `ac_add_options --disable-new-pass-manager`.
Differential Revision: https://phabricator.services.mozilla.com/D66109
--HG--
extra : moz-landing-system : lando
With this and all the previous changes, the necessary mozconfig for
local cross-builds only contains DIA_SDK_PATH, WINDOWSSDKDIR and
--target.
Differential Revision: https://phabricator.services.mozilla.com/D65295
--HG--
extra : moz-landing-system : lando
This was cargo culted from the autoconf equivalent, and while it makes a
command that does "$(PROG) foo" work because it becomes ": foo", that
may or may not actually be a desirable outcome.
OTOH, we do have some places where there are some "ifdef PROG" that are
just plain wrong when PROG is always actually set.
One place I do know that does check if the value is not ":" is for
OBJCOPY, which is still set from autoconf.
All in all, looking at all the check_prog(allow_missing=True) we have in
python configure, it doesn't seem anything is checking for ":", and that
doesn't seem like the right status quo.
Differential Revision: https://phabricator.services.mozilla.com/D65276
--HG--
extra : moz-landing-system : lando
- Remove the separate option() for MT, because it dates back from when
we needed `MT` not being an absolute path, but that hasn't been true
since bug 1290040.
- Extend what was done in bug 1617794 to MT, although the long term move
is to not rely on MT at all.
- Patch leftovers from bug 1613799.
Differential Revision: https://phabricator.services.mozilla.com/D64712
--HG--
extra : moz-landing-system : lando
Depending on the target, the assembler for Windows builds is one of ml,
ml64, or armasm64. Only for the latter currently are we looking for the
assembler with the .exe extension. All of them should be looked for the
same way, and we pick including the extension.
Differential Revision: https://phabricator.services.mozilla.com/D64260
--HG--
extra : moz-landing-system : lando
We're feeding old-configure with compiler and assembler as found by
python configure via ac_cv_* variables (abusing the autoconf cache).
When doing so, and when the assembler was provided via an environment
variable, while python configure resolves the path, old-configure
doesn't get the resolved path, because we're currently passing it the
path that was passed via the environment.
Now pass the resolved path instead.
Differential Revision: https://phabricator.services.mozilla.com/D64258
--HG--
extra : moz-landing-system : lando
While this works on Windows, because os.pathsep is `;`, it actually
doesn't work on non-Windows, because clang-cl is still expecting `;`,
but os.pathsep is `:`.
Differential Revision: https://phabricator.services.mozilla.com/D62862
--HG--
extra : moz-landing-system : lando
After a C++ language version bump, It's possible to wind up in
situations where we are using a new enough compiler version, but the
libstdc++ version in use is not new enough to support new language
features: self-compiled clang with system libraries, clang from `mach
bootstrap` prior to C++ language version bump (and thus including a new
libstdc++ with the boostrapped clang), etc.
Previously, such a situation would mean that things would work fine, and
then start breaking as soon as new library features started to be used.
Let's try to catch the problem earlier, when the update happens, by
verifying that the libstdc++ version is at least as new as the GCC
version we're requiring.
Differential Revision: https://phabricator.services.mozilla.com/D57516
--HG--
extra : moz-landing-system : lando
I need this to support individual #[cfg] in enum variants, which is used for
Image::PaintWorklet.
Differential Revision: https://phabricator.services.mozilla.com/D62160
--HG--
extra : moz-landing-system : lando
fts.h is also missing on musl. Since the Solaris fallback implementation
does not use any non-standard functions, use that as the fallback when
the FTS interface is not available.
Also, fix an issue with the readdir implementation. Previously, the
absolute directory path was passed to the recursive add_dir_entries()
call, which would then prepend install path again.
Differential Revision: https://phabricator.services.mozilla.com/D61407
--HG--
extra : moz-landing-system : lando
v0.12.2 at least is needed to have proper v2 lockfile support.
But v0.13.0 contains extra fixes, and features that wgpu and the style system
plan to use, so let's just update to that.
Differential Revision: https://phabricator.services.mozilla.com/D61334
--HG--
extra : moz-landing-system : lando
It generates a lot of noise and we don't want to fix them. In particular, all
the cycle collector macros are like this.
Differential Revision: https://phabricator.services.mozilla.com/D59646
--HG--
extra : moz-landing-system : lando
This gets rid of the last use of the 'which' module left in the tree. So not
only will this help 'configure' become a little more Python 3 compatible, but
we can now remove 'third_party/python/which'.
Differential Revision: https://phabricator.services.mozilla.com/D37427
--HG--
extra : moz-landing-system : lando
This gets rid of the last use of the 'which' module left in the tree. So not
only will this help 'configure' become a little more Python 3 compatible, but
we can now remove 'third_party/python/which'.
Differential Revision: https://phabricator.services.mozilla.com/D37427
--HG--
extra : moz-landing-system : lando
The doubling of `when` is needed: the `try_run` when ensures that we
only execute the compiler on OS X, and the `@depends` when ensures that
we don't look at the `None` result (which would normally indicate
failure) on non-OS X hosts.
Differential Revision: https://phabricator.services.mozilla.com/D57636
--HG--
extra : moz-landing-system : lando
This is needed to support min() / max() / clamp(), as LengthPercentage will have
a potential heap allocation and the current copy-assignment operators just do
bitwise copies.
Differential Revision: https://phabricator.services.mozilla.com/D57300
--HG--
extra : moz-landing-system : lando
-Wempty-init-stmt is a new clang diagnostic that diagnoses empty C++17 init-statements of if, switch, and range-based for statements:
void test() {
if(; // <- warning: init-statement of 'if' is a null statement
true)
;
switch (; // <- warning: init-statement of 'switch' is a null statement
x) {
...
}
for (; // <- warning: init-statement of 'range-based for' is a null statement
int y : S())
;
}
Differential Revision: https://phabricator.services.mozilla.com/D54866
--HG--
extra : moz-landing-system : lando
Make the $PYTHON3 build var point to a full virtualenv bootstrapped with
the same libraries as the $PYTHON Python 2 build var. This allows us to
upgrade build tasks from $PYTHON to $PYTHON3.
This patch adds some debug logging and documentation to the Python
2 virtualenv so that it is easier to diagnose issues that may arise
from running two different Python interpreters in re-entrant
multiprocess routines.
Differential Revision: https://phabricator.services.mozilla.com/D50819
--HG--
extra : moz-landing-system : lando
Nothing looks at the value of this option. We seem to have settled on the `--enable-profile-{generate,use}=cross` approach instead.
Differential Revision: https://phabricator.services.mozilla.com/D56588
--HG--
extra : moz-landing-system : lando
This includes more bitflag operators, which means we can remove some slightly
ugly code.
Differential Revision: https://phabricator.services.mozilla.com/D56308
--HG--
extra : moz-landing-system : lando
Local PGO builds now use 3-tier machinery under the hood. Instead of a
single object directory that gets cleaned in between the instrumented
and profile-use builds, now the instrumented build uses a separate
'${objdir}/instrumented' directory. This makes it easier to handle
within mach since we can drive the two builds with environment variables
and keep all build artifacts separate, without needing to do manual
cleanup in between.
Differential Revision: https://phabricator.services.mozilla.com/D50098
--HG--
extra : moz-landing-system : lando
In automation this is normally handled by mozconfigs, but we should
disable it automatically during the profile-generate stage for users if
they have LTO enabled in their mozconfig while doing a 3-tier PGO build.
Differential Revision: https://phabricator.services.mozilla.com/D50097
--HG--
extra : moz-landing-system : lando
Mike Hommey
David Major
Razvan Maries
Ricky Stewart
Nathan Froyd
Adam Gashlin
Bogdan Tara
Kagami Sascha Rosylight
Stefan Hindli
Jesse Schwartzentruber
Dzmitry Malyshau
Mihai Alexandru Michis
Oana Pop Rus
Aaron Klotz
Stephen A Pohl
James Willcox
Christian Holler
Noemi Erli
Arthur Iakab
Andi-Bogdan Postelnicu
Andreea Pavel
Tyson Smith
Tooru Fujisawa
Chris Manchester
Brindusan Cristian
Marco Bonardo
Coroiu Cristina
Emilio Cobos Álvarez
Michael Forney
Sylvestre Ledru
Csoregi Natalia
Cosmin Sabou
Gabriele Svelto
Andrew Halberstadt
Nicholas Nethercote
Chris Peterson
Māris Fogels
Tom Ritter
Mike Shal