Artifact mozconfigs are not necessarily up-to-date wrt changes to the
nightly mozconfigs, and all in all, shouldn't be much different from
them.
It's just better to use the nightly mozconfigs (or beta on beta, etc.)
and make the mozconfigs themselves handle the few things that need to be
different when the USE_ARTIFACT environment is set (which is now
consistently set by taskcluster)
This does have the side effect of turning builds that actually don't
support artifact builds red when using --artifact on try, instead of
having them silently not be artifact builds as currently happens.
Depends on D21314
Differential Revision: https://phabricator.services.mozilla.com/D21315
--HG--
extra : moz-landing-system : lando
The artifact builds that are automatically derived using the artifact
template set the USE_ARTIFACT environment variable from taskcluster.
After the previous change, --artifact builds from try syntax do that
too.
That leaves us with only the artifact-build build not doing it, so for
consistency, do it there. That makes it not necessary to set
USE_ARTIFACT from mozconfig.artifact.automation anymore.
Depends on D22056
Differential Revision: https://phabricator.services.mozilla.com/D21313
--HG--
extra : moz-landing-system : lando
The existing ANR and tombstone checks in remoteautomation are used
by mochitests and reftests but are awkward for other harnesses like cppunit
that do not use remoteautomation...and easily missed. This patch moves that
code to the mozharness AndroidMixin, treating ANRs and tombstones like the
logcat: Make sure any old logs are deleted when the mozharness script starts,
then move any logs found at the end of the run to the upload directory.
https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=f3de1e9836da2b0c9232f5d92c751b979459e19b
demonstrates tombstone artifacts for the Android 7.0 cppunit tests.
Differential Revision: https://phabricator.services.mozilla.com/D21774
--HG--
extra : moz-landing-system : lando
Optimizing jars without preloading/reordering data only moves the
jar central directory to the beginning of the file, which, without
preloading information, is not very useful. Let's just stop doing it if
there's not going to be preloading/reordering information at all.
Differential Revision: https://phabricator.services.mozilla.com/D21170
--HG--
extra : moz-landing-system : lando
This patch adds detection for when icecream is in use to build telemetry.
icecream is commonly enabled in two ways: by either setting CC/CXX to point
to icecream's cc/c++ symlinks, or by setting adding
mk_add_options 'export CCACHE_PREFIX=icecc' to a mozconfig when also using
ccache. For the former, this patch adds a simple configure check to see
if CXX is a symlink to a file named 'icecc'. For the latter this patch adds
CCACHE_PREFIX as a configure subst to capture the value.
We don't currently have a facility for writing telemetry tests that depend on
configure values. Local manual testing shows that it does work as expected.
Differential Revision: https://phabricator.services.mozilla.com/D18138
--HG--
extra : moz-landing-system : lando
Android requires us to use -fprofile-generate / -fprofile-use instead of
-fprofile-instr-generate / -fprofile-instr-use. Additionally, we have
to pass in the path to write out the profile data as /data/local/tmp,
since the cwd from where the Fennec process is invoked is not writeable.
Differential Revision: https://phabricator.services.mozilla.com/D20697
--HG--
extra : moz-landing-system : lando
In bug 1522354, we changed host and target detection to not invoke
config.sub, assuming the output from config.guess would satisfy our
needs in split_target.
It turns out that on some plaforms, that doesn't work out, so, while we
still skip config.sub, we now catch errors from split_target when doing
so, and try again after running config.sub when split_target fails.
Differential Revision: https://phabricator.services.mozilla.com/D19937
Consequently, this removes:
- MOZ_LIBPRIO, which is now always enabled.
- non_msvc_compiler, which is now always true.
- The cl.py wrapper, since it's not used anymore.
- CL_INCLUDES_PREFIX, which was only used for the cl.py wrapper.
- NONASCII, which was only there to ensure CL_INCLUDES_PREFIX still
worked in non-ASCII cases.
This however keeps a large part of detecting and configuring for MSVC,
because we still do need it for at least headers, libraries, and midl.
Depends on D19614
Differential Revision: https://phabricator.services.mozilla.com/D19615
--HG--
extra : moz-landing-system : lando
When the build system compiler is clang, and bindgen autodetection
actually finds a different clang via llvm-config, or a different clang
was given with --with-clang-path, we do want the proper flags to be
used for that clang, so we always get the right flags for that clang
instead of trying to shortcut.
Bug 1526857 will take care of making things more proper, while this is a
quick fix to unbreak builds in some unfortunately common kind of local
setups.
Differential Revision: https://phabricator.services.mozilla.com/D19328
--HG--
extra : moz-landing-system : lando
libclang 3.9 has a bug that makes bindgen unable to distinguish some typedefs
from the underlying type, which matters for bug 1523071.
We have had quite a few workarounds for this bug and I don't really want to add
more, since in this case it is non-trivial. I think requiring libclang 4.0+ is
reasonable at this point.
Of the distros that can't build Firefox out of the box with clang, dropping support
for clang 3.9 would only break Ubuntu 14.04 LTS, which support ends April 2019,
right before we release 67.
Differential Revision: https://phabricator.services.mozilla.com/D18889
--HG--
rename : build/build-clang/clang-3.9-linux64.json => build/build-clang/clang-4.0-linux64.json
rename : taskcluster/scripts/misc/build-clang-3.9-linux.sh => taskcluster/scripts/misc/build-clang-4.0-linux.sh
extra : moz-landing-system : lando
These are copies of the corresponding gcc-based base toolchain tasks,
with FORCE_GCC and the gcc dependency removed.
We also tweak things a little for those builds to actually end up green.
Differential Revision: https://phabricator.services.mozilla.com/D18903
--HG--
extra : moz-landing-system : lando
This apparently was necessary back when Android support was added 8.5
years ago, but is not desirable as a flag set globally. But it turns out
everything builds just fine without it now.
Differential Revision: https://phabricator.services.mozilla.com/D19128
--HG--
extra : moz-landing-system : lando
These are copies of the corresponding gcc-based base toolchain tasks,
with FORCE_GCC and the gcc dependency removed.
We also tweak things a little for those builds to actually end up green.
Differential Revision: https://phabricator.services.mozilla.com/D18903
--HG--
extra : moz-landing-system : lando
basic_bindgen_cflags's function is to set some flags for use with
rust-bindgen through clang/libclang for C++ code. Part of the flags it
sets are for the C++ standard, and the target.
Unfortunately, some of the logic wrt target-specific flags is currently
broken. It wants to apply per-compiler flags on Windows, but fails to do
so. First, because the condition test is wrong, and second, because it
only cares about msvc and not clang-cl.
OTOH, we already have those flags when the compiler is clang or
clang-cl. And we already have code to get the right flags for a given
compiler. So when the compiler is not clang or clang-cl, we can use that
to get the right flags for the clang we're going to use for bindgen.
Depends on D18316
Differential Revision: https://phabricator.services.mozilla.com/D18317
--HG--
extra : moz-landing-system : lando
In upcoming changes, we're going to use the normal compiler flags for
bindgen instead of having a separate logic for essentially the same
flags (and there's not much reason not to use the same flags after all).
One hiccup, though is that for some reason, --target=i686-linux-gnu
doesn't work with bindgen while it works with clang in the same setup.
But -m32 does.
Considering -m32 and -m64 are standard flags and that we're using them
in many cases, it doesn't hurt to use them instead of --target with
clang.
While we're doing that, we might as well refactor a little to avoid the
multiple branches handling the use of -m32/-m64, and fix the theoretical
compile-x86_64-with-x86-clang-cl case, as well as the weird check for
aarch64.
And because only two cases actually require -Xclang, only one of which
requires a condition, and the control flow is not too complex to avoid
flag duplication, we just remove the append_flag function; it was too
confusing, with all the cases we skipped it when -Xclang was not wanted.
Differential Revision: https://phabricator.services.mozilla.com/D18316
--HG--
extra : moz-landing-system : lando
This removes all .flake8 files except for the one at the root of the repo.
Instead we use the new 'per-file-ignores' config introduced in 3.7. To ignore
specific errors in a subdirectory, add a line like this to the root .flake8:
[per-file-ignores]
path/to/subdir/*: E100, F200, ...
The reasons for this change are:
1. Unblock flake8 blacklist (bug 1367092).
2. Simplify configuration and code.
3. Encourage more consistent styling.
4. Improve performance.
5. Greater editor consistency.
Differential Revision: https://phabricator.services.mozilla.com/D18354
--HG--
extra : moz-landing-system : lando
Since ./mach bootstrap installs Android SDK into ~/.mozbuild, we should detect
this location as default SDK install path.
Also, --with-android-max-sdk and --with-android-min-sdk are still in android.m4
because confvars.sh sets MOZ_ANDROID_MIN_SDK_VERSION.
Differential Revision: https://phabricator.services.mozilla.com/D15463
--HG--
extra : moz-landing-system : lando
Sometimes we want to store state that only applies to a particular srcdir, but
there isn't a standard directory where this lives. Let's add an argument to
'get_state_dir()' to provide an "official" place.
The new API to get the local state dir is 'get_state_dir(srcdir=True)'. Like
the global state dir, this directory is not guaranteed to exist. A reference to
this value can also be obtained via 'self._mach_context.local_state_dir' from
within a mach command (in this case it will be created automatically if it
doesn't exist).
Note: we should probably just make sure both exist at mach startup, but it felt
outside the scope of this change.
Differential Revision: https://phabricator.services.mozilla.com/D15724
--HG--
extra : moz-landing-system : lando
mozboot.util.get_state_dir() returns a tuple of (<path>, <bool). The bool
denotes whether or not the state dir came from an environment variable.
But this value is only used in a single place, and is very easy to test for
anyway. It's not worth the added complexity it imposes on all other consumers
of this function. Let's just make this function return the path.
Differential Revision: https://phabricator.services.mozilla.com/D15723
--HG--
extra : moz-landing-system : lando
We default to -Zi for Windows debug flags, which is for generating debug info
into a PDB file, but clang-cl doesn't actually implement that so -Zi is
an alias for -Z7 in clang-cl:
http://clang.llvm.org/docs/UsersManual.html#id9
sccache has special handling for -Zi because multiple compiles writing
to the same PDB file is not cacheable, so we've always overridden -Zi with -Z7
in CI when using sccache.
Given that everyone should be using clang-cl nowadays and MSVC will no longer
be supported soon, this patch changes the default and removes some PDB
file name flag setting from rules.mk, as well as the no-longer-necessary
overrides from mozconfig.cache.
Differential Revision: https://phabricator.services.mozilla.com/D18264
--HG--
extra : moz-landing-system : lando
Also use armv7a as the default when no target is given at all.
Also change bootstrap to create a simpler mozconfig. The downside is
that the resulting mozconfig would not work when building older
revisions.
Add unit tests for this as well as the simplications added in bug
1523341.
Differential Revision: https://phabricator.services.mozilla.com/D17906
--HG--
extra : moz-landing-system : lando
While not related, it turns out this test was opted in by ourselves,
while it's practically useless. Recent autoconf documentation says it's
obsolescent, and even autoconf 2.13 documentation said with "recent
versions of the GNU C compiler", the issue it's detecting "is becoming
a less prevalent problem". Recent as of 20 years ago.
Differential Revision: https://phabricator.services.mozilla.com/D18054
--HG--
extra : moz-landing-system : lando
Now that everything is lined up nicely, we don't need those anymore.
We can fill cargo-linker.bat with HOST_LINKER and HOST_LINKER_LIBPATHS,
which, unfortunately, can't be used as is, so we add a variant for
substitution in .bat files.
Differential Revision: https://phabricator.services.mozilla.com/D18034
--HG--
extra : moz-landing-system : lando
Now that it's automatically derived and that in-tree mozconfigs are
setting DIA_SDK_PATH, we don't need to set WIN_DIA_SDK_BIN_DIR on
automation, or elsewhere, for that matter.
Differential Revision: https://phabricator.services.mozilla.com/D17915
--HG--
extra : moz-landing-system : lando
Clang 8 seems to generate destructor decls from different source
locations which breaks an assertion in the code. This patch updates the
code to remove the assertion and more robustly handle the new
declarations.
Differential Revision: https://phabricator.services.mozilla.com/D17902
--HG--
extra : moz-landing-system : lando
We reject MSVC compilers < 2017 already, there's no point checking for
smaller versions after that.
Differential Revision: https://phabricator.services.mozilla.com/D17770
--HG--
extra : moz-landing-system : lando
And remove all the variables that configure will figure out for us as a
consequence. This has the side effect of making the automation builds a
little more like local builds, in that they don't rely on preset PATH,
LIB, etc. for the build to work.
Differential Revision: https://phabricator.services.mozilla.com/D17789
--HG--
extra : moz-landing-system : lando
We currently rely on WIN_DIA_SDK_BIN_DIR being passed, but we can
actually derive it from the DIA SDK directory. So we now do that, except
when it's given explicitly.
While in the vicinity, move the dia2.h check to python configure.
With WIN_DIA_SDK_BIN_DIR being derived and not set when dia2.h is not
found, we don't really need MSVC_HAS_DIA_SDK anymore, so we just check
for WIN_DIA_SDK_BIN_DIR to determine whether to build dump_syms or not.
One exception to the above is when WIN_DIA_SDK_BIN_DIR is passed in,
which we only keep for the in-tree mozconfigs for now. We'll remove that
possibility after bug 1523201.
Depends on D17892
Differential Revision: https://phabricator.services.mozilla.com/D17893
--HG--
extra : moz-landing-system : lando
It turns out, we don't need to `mk_add_options export` the variables
from the in-tree mozconfigs. If anything, that causes problems when
trying to simplify the mozconfigs, because it makes the variables
exported from .mozconfig.mk, overriding what configure may change and
store in autoconf.mk.
All the variables are handled by configure in a way that makes them
available in autoconf.mk, so there's no loss there, and with the
python/shell-based mozconfig loader, it turns out we don't need to go
through extra normalization via cmd.
autospider.py, being its own pseudo-mozconfig parser, still does need
it, though, but it was hooking into it already, so just inline that.
Differential Revision: https://phabricator.services.mozilla.com/D17769
--HG--
extra : moz-landing-system : lando
Previously, we hardcoded HostX64 because configure would autodetect a
x86 host on x64 machines, but the x86 MSVC compiler wouldn't be
suitable.
While the x86 MSVC compiler might still not be suitable, now that
configure detects x64 hosts properly, when the configure host is
detected as x86, we can't even execute the x64 compiler, so we can at
least try with the x86 one correctly.
Differential Revision: https://phabricator.services.mozilla.com/D17788
--HG--
extra : moz-landing-system : lando
The combination of bug 1515579 and bug 1520394 made things harder for
old-configure, because it doesn't necessarily have a complete view of
the search PATH that has been used. This doesn't actually cause problems
on non-Windows builds because things work out fine, but on Windows,
some of the executions of clang-cl in old-configure insist on being able
to find a MSVC install. That, again, doesn't currently cause problems in
general on local builds because clang-cl finds it through the registry
(presumably), and on automation, because it's in the `mk_add_options
export`'ed PATH, but the latter is due to change.
Depends on D17772
Differential Revision: https://phabricator.services.mozilla.com/D17783
--HG--
extra : moz-landing-system : lando
Also, while here, replace subprocess.check_output with check_cmd_output.
Depends on D17771
Differential Revision: https://phabricator.services.mozilla.com/D17772
--HG--
extra : moz-landing-system : lando
Bug 1520394 changed things such that the configure sandbox is using a
copy of os.environ. So when mozconfig injects environment changes, they
only affect the sandbox. Which means when the which module uses
os.environ to get PATH, it gets the original unmodified environment.
So instead of relying on the which module getting PATH itself, we feed
it with it. It's worth noting that the which module adds `.` on Windows,
but we don't copy this behavior, because in the context of configure,
it's actually not important (`.` would be the topobjdir).
Differential Revision: https://phabricator.services.mozilla.com/D17771
--HG--
extra : moz-landing-system : lando
And use it as an alternative to vswhere, instead of the current late
check. This allows to use VC_PATH when using a MSVC archive that is not
installed through the VS installer, and not have to care about PATH.
Depends on D17786
Differential Revision: https://phabricator.services.mozilla.com/D17787
--HG--
extra : moz-landing-system : lando
- Only expose it as well as --with-visual-studio-version when the host
system is Windows.
- Don't run vswhere twice (once for host and once for target).
Depends on D17785
Differential Revision: https://phabricator.services.mozilla.com/D17786
--HG--
extra : moz-landing-system : lando
config.guess doesn't actually figure out the real host system on
Windows. All it does is end up finding the CPU for which the msys used
for the shell is built for. Which has the unfortunate effect that if you
build on a 64-bits host, configure pretends the host is 32-bits.
So instead, rely on some environment variables that should be set
mostly everywhere, and derive the corresponding triplet.
This has two effects:
- configure now knows that the host is 64-bits when it is.
- consequently, configure will build a 64-bits Firefox on 64-bits hosts
by default (since by default, the target is derived from the host), and
that's actually a desired side effect.
Depends on D17620
Differential Revision: https://phabricator.services.mozilla.com/D17621
--HG--
extra : moz-landing-system : lando
- We assume the output from config.guess is sufficiently normalized for
our needs.
- If we partially derive the target from the host (with only the cpu
given for --target), we assume the derived value doesn't need
canonicalization.
Differential Revision: https://phabricator.services.mozilla.com/D17620
--HG--
extra : moz-landing-system : lando
In fact, "simply" use whatever python configure does to find a shell to
execute config.guess and config.sub, and get both the mozconfig content
and the real, canonicalized target alias. This has the side effect of
making builds with --target=$cpu use a complete obj-$cpu-$os default
objdir instead of obj-$cpu. This will also allow to change the
host-guessing logic without having to duplicate code.
Differential Revision: https://phabricator.services.mozilla.com/D17618
--HG--
extra : moz-landing-system : lando
which, at this point, is only the AC_SUBST, so we don't even need to
give it to old-configure anymore.
Depends on D17207
Differential Revision: https://phabricator.services.mozilla.com/D17208
--HG--
extra : moz-landing-system : lando
which, at this point, is only the AC_SUBST, so we don't even need to
give it to old-configure anymore.
Depends on D17207
Differential Revision: https://phabricator.services.mozilla.com/D17208
--HG--
extra : moz-landing-system : lando
Because old-configure is only refreshed when, essentially,
old-configure.in changes, hardcoded (absolute) paths don't necessarily
match the build environment of the current build.
So instead, use an environment variable that we pass from python
configure when invoking old-configure.
Also do dummy changes to old-configure.in so that old-configure is
refreshed at least once to get the environment-based value.
Differential Revision: https://phabricator.services.mozilla.com/D17077
--HG--
extra : moz-landing-system : lando
Back when those were added, option defaults could not indirectly depend
on `target` or `host`, but that changed with bug 1322025.
As a side effect, this allows to turn them into straight js_options
without resorting to the manual add_old_configure_arg.
Differential Revision: https://phabricator.services.mozilla.com/D16664
--HG--
extra : moz-landing-system : lando
Rust ships with some natvis files that enable nicer display of Rust standard
library types in Microsoft's debuggers. rustc will add the right linker
options to include them when it invokes the linker but since we don't link
libxul with rustc we need to explicitly pass them to the linker ourselves.
This change locates all natvis files in the Rust sysroot and adds them
to the libxul link line.
Differential Revision: https://phabricator.services.mozilla.com/D16544
--HG--
extra : moz-landing-system : lando
Add a dummy change to old-configure.in so that old-configure is
force-refreshed.
Differential Revision: https://phabricator.services.mozilla.com/D16797
--HG--
extra : moz-landing-system : lando
Back when those were added, option defaults could not indirectly depend
on `target` or `host`, but that changed with bug 1322025.
As a side effect, this allows to turn them into straight js_options
without resorting to the manual add_old_configure_arg.
Differential Revision: https://phabricator.services.mozilla.com/D16664
--HG--
extra : moz-landing-system : lando
Before bug 1322025, running configure --help along --host and/or
--target would show the options that are specific to the
host/target/cross-compilation (e.g. those with when=target_is_windows)
which was convenient. Bug 1322025 unfortunately broke that, and this
restores the feature.
Differential Revision: https://phabricator.services.mozilla.com/D16673
--HG--
extra : moz-landing-system : lando
Instead, use the actual log file name as configured through python
logging. This doesn't make a practical difference now, but will when
subconfigure logs to the toplevel log.
Depends on D16666
Differential Revision: https://phabricator.services.mozilla.com/D16667
--HG--
extra : moz-landing-system : lando
We're going to want to log some more. At the moment, this just adds the
output of subconfigure to top-level config.log, but this will become
more useful down the road.
Depends on D16665
Differential Revision: https://phabricator.services.mozilla.com/D16666
--HG--
extra : moz-landing-system : lando
Back when those were added, option defaults could not indirectly depend
on `target` or `host`, but that changed with bug 1322025.
As a side effect, this allows to turn them into straight js_options
without resorting to the manual add_old_configure_arg.
Differential Revision: https://phabricator.services.mozilla.com/D16664
--HG--
extra : moz-landing-system : lando
Use an I/O wrapper on the configure output handler to add the "js/src>"
prefix.
Depends on D16643
Differential Revision: https://phabricator.services.mozilla.com/D16644
--HG--
extra : moz-landing-system : lando
It was manually removed because autoconf's AC_OUTPUT_SUBDIRS passes it,
but we've not been using AC_OUTPUT_SUBDIRS for a while.
Depends on D16637
Differential Revision: https://phabricator.services.mozilla.com/D16638
--HG--
extra : moz-landing-system : lando
The code only actually handles autoconf 2.5 cache, and was useful when
we have such subconfigures, that would die when things changed in ways
that would make the subconfigure abort on its own. But we have no such
things left, and while we may want to clear the cache for old-configure
(for e.g. bug 1520232), this is not helping to get there.
Differential Revision: https://phabricator.services.mozilla.com/D16636
--HG--
extra : moz-landing-system : lando
Instead, include the module and inline its main function.
Depends on D16622
Differential Revision: https://phabricator.services.mozilla.com/D16623
--HG--
extra : moz-landing-system : lando
This happens to remove the last use of perl from configure.
Depends on D16621
Differential Revision: https://phabricator.services.mozilla.com/D16622
--HG--
extra : moz-landing-system : lando
The File class is now used only in one place, only for its mtime
property, which can be replaced with os.path.getmtime.
Depends on D16402
Differential Revision: https://phabricator.services.mozilla.com/D16403
--HG--
extra : moz-landing-system : lando
Because subconfigure.py is only used for js/src and we're not going to
add any more subconfigures, we can hardcode js/src into the script, and
remove the branches that aren't taken as a consequence.
Depends on D16382
Differential Revision: https://phabricator.services.mozilla.com/D16402
--HG--
extra : moz-landing-system : lando
It was useful to run the prepare phase and the actual subconfigure
separately, but it's not anymore after all the recent simplications.
Depends on D16381
Differential Revision: https://phabricator.services.mozilla.com/D16382
--HG--
extra : moz-landing-system : lando
Because we only ever run one subconfigure, the machinery to execute
several is not useful anymore. Inlining it allows to simplify the code
too, because it doesn't need to be generic anymore. This also removes
the last remaining bits of acwinpaths.m4.
Also remove now unused support for --list in build/subconfigure.py.
Depends on D16380
Differential Revision: https://phabricator.services.mozilla.com/D16381
--HG--
extra : moz-landing-system : lando
We only run it for js/src now, so get_config_files always returns empty lists.
Depends on D16378
Differential Revision: https://phabricator.services.mozilla.com/D16379
--HG--
extra : moz-landing-system : lando
This only affects the compiler version in the build system; not the
version reported by the compiler.
--HG--
extra : histedit_source : 37fc1a2090d872ca5e95d022e9f93b620b2d8dbb
When doing cross-compiles for Desktop, system pkg-config is going to be
looking for .pc files for the host, but systems setup for cross-compiles
come with a toolchain-prefixed pkg-config, which we should be looking
for.
Differential Revision: https://phabricator.services.mozilla.com/D16318
--HG--
extra : moz-landing-system : lando
When doing cross-compiles for Desktop, system pkg-config is going to be
looking for .pc files for the host, but systems setup for cross-compiles
come with a toolchain-prefixed pkg-config, which we should be looking
for.
Differential Revision: https://phabricator.services.mozilla.com/D16318
--HG--
extra : moz-landing-system : lando
MOZ_D3D_CPU_SUFFIX and MOZ_HAS_WINSDK_WITH_D3D are not used in the
build, and nothing includes d3d10.h except some angle code in a
preprocessed branch that is only taken for a macro we never define,
so we don't move the code corresponding to those. We also simplify the
detection code, which is convoluted now that it doesn't search for
multiple different DLLs.
Differential Revision: https://phabricator.services.mozilla.com/D16295
--HG--
extra : moz-landing-system : lando
In bug 1494022 there are problems using an SDK that is too new, which
can cause developers to spend many hours debugging an issue that we
could've checked up front.
This patch gets the version from the SDKSettings.plist file at the root
of the SDK directory, and checks it against min and max supported
versions.
The current min version is 10.11, since that is what we build with in
automation. The current max version is 10.13, since 10.14+ currently has
an issue with a black screen at startup.
Differential Revision: https://phabricator.services.mozilla.com/D16140
--HG--
extra : moz-landing-system : lando
Because MSVC compilers only support one architecture, we need to search
"cl" in different toolchain search paths for each of the host and
target, especially when they are different.
Likewise for the library paths for the linker. Ideally we'd pass
-LIBPATH both for host and target, but that has implications for rust
that I don't want to have to figure just now.
Depends on D15263
Differential Revision: https://phabricator.services.mozilla.com/D15264
--HG--
extra : moz-landing-system : lando
While it was preferable to use lld-link with msvc for performance
reasons when we defaulted to compile with msvc, now that we build with
clang-cl by default, it's now better to use msvc's linker when using
msvc's compiler, especially for aarch64, which is the last place where
we actually need msvc.
Differential Revision: https://phabricator.services.mozilla.com/D15262
--HG--
extra : moz-landing-system : lando
In bug 1259382, some workarounds were added to make the build system
alter PATH and not use absolute paths for toolchain programs, because
autoconf and the build system doesn't deal with spaces in those very
well. But later in bug 1290040, we made find_program return Windows
short paths (without spaces), which alleviates the need for those
workarounds.
We still, however, and unfortunately, need to alter PATH to account for
the fact that MSVC DLLs are not necessarily alongside the compiler
executables...
Depends on D15181
Differential Revision: https://phabricator.services.mozilla.com/D15182
--HG--
extra : moz-landing-system : lando
Like for other windows platforms. This currently doesn't make a
difference, but will with next change.
Differential Revision: https://phabricator.services.mozilla.com/D15181
--HG--
extra : moz-landing-system : lando
LLVM_PROFDATA needs the toolchain search dir, per bug 1515579.
Also, most of the options actually don't do anything useful with
artifact builds. In fact, the only one that artifact builds would need
is MOZ_PGO. So we move to options back to toolchain.configure, somewhere
late enough ; except MOZ_PGO, that we move to the top-level
moz.configure (because we don't need a separate file for one option).
Differential Revision: https://phabricator.services.mozilla.com/D16152
--HG--
extra : moz-landing-system : lando
Firefox uses multiple processes. It has intentional leaks, and when
running with ASAN, we have suppressions to eliminate those. When running
ASAN builds through CI tests, when Firefox exits, each of the processes
(parent and child) exits and goes through its leaks and when there are
(which is a given), the ASAN runtime runs llvm-symbolizer to symbolicate
and match against suppressions. So each process runs llvm-symbolizer. At
the same time.
Some of the addresses to symbolicate are in libxul. Which contains all
DWARF info, making it a ~1GB monster. Oh, and because you're lucky,
things align perfectly such that libxul size is a multiple of the page
size. That makes llvm-symbolizer pread() the file instead of mmap()ing
it. Did I say there are multiple processes? So suddenly you have n
processes simultaneously allocating and filling 1GB of memory each, on
CI machines that have enough memory for the job they usually run, but
not enough for a sudden rush of n GB.
And things go awry. When you're lucky and the OOM killer didn't take
care of killing the CI entirely, symbolication couldn't happen and the
suppressions are not matched, and leaks are reported.
This all turns out it originates in how llvm-symbolicate chooses between
pread() and mmap(), which turns out is just defaults not being made for
binary files.
Differential Revision: https://phabricator.services.mozilla.com/D16010
--HG--
extra : moz-landing-system : lando
In bug 1514288 we started to save the output from running Firefox during
PGO builds into log files, but they aren't correctly uploaded if the run
fails. This presents a problem for categorizing failed PGO builds if the
profileserver returns an error code (eg: bug 1517939), since the error
messages could be hidden away in log files that aren't uploaded.
Instead, we can generate them directly into the artifacts directory so
that they are always uploaded. Additionally, the log files are displayed
if the run fails so that the error messages are displayed in the log for
easier bug categorization.
Differential Revision: https://phabricator.services.mozilla.com/D15881
--HG--
extra : moz-landing-system : lando
Instead of per-document. This also allows to reuse this thread-pool if needed
for other stuff, like parallel CSS parsing (#22478), and to share more code with
Gecko, which is always nice.
This cherry-picks https://github.com/servo/servo/pull/22487, with a few minor
fixes to the build that are landing as part of the sync associated to this bug,
and an lsan exception tweak to point to the right module since it's moving.
The output from running the browser during PGO builds can have innocuous
error messages in them, but show up in treeherder as potential messages
to include when filing bugs. We can just save the output from these runs
as files and upload them as artifacts instead, so they don't show up in
the build log but are available for inspection if necessary.
MozReview-Commit-ID: 3VdVCKVkZNI
Differential Revision: https://phabricator.services.mozilla.com/D15154
--HG--
extra : moz-landing-system : lando
In bug 1259382, some workarounds were added to make the build system
alter PATH and not use absolute paths for toolchain programs, because
autoconf and the build system doesn't deal with spaces in those very
well. But later in bug 1290040, we made find_program return Windows
short paths (without spaces), which alleviates the need for those
workarounds.
We still, however, and unfortunately, need to alter PATH to account for
the fact that MSVC DLLs are not necessarily alongside the compiler
executables...
Depends on D15181
Differential Revision: https://phabricator.services.mozilla.com/D15182
--HG--
extra : moz-landing-system : lando
Like for other windows platforms. This currently doesn't make a
difference, but will with next change.
Differential Revision: https://phabricator.services.mozilla.com/D15181
--HG--
extra : moz-landing-system : lando
We remove --disable-libjpeg-turbo because that's only useful when Yasm
is too old, and the required version is now almost 8 years old, so we
can reasonably require people to upgrade rather than workaround with a
--disable option.
The valid_yasm_version function can seem overkill, but that's because
future moves of other things to python configure will pile up.
Differential Revision: https://phabricator.services.mozilla.com/D15184
--HG--
extra : moz-landing-system : lando
Now that we're not even building host static libraries, we don't need
variables for the tools used to build them.
Ironically, we weren't even running HOST_RANLIB.
Depends on D15172
Differential Revision: https://phabricator.services.mozilla.com/D15173
--HG--
extra : moz-landing-system : lando
So far, the main subject of cross-compiles was to cross compile from one
OS to another (e.g. {linux,osx} -> android), but there are a few useful
cases where the OS doesn't change, and, with --host being guessed, we
can just have developers pass --target=$cpu instead of a complete
target triplet. This can be useful to do x86 Linux builds on x86-64
Linux hosts, or aarch64 Windows builds on x86-64 Windows hosts.
Differential Revision: https://phabricator.services.mozilla.com/D15063
--HG--
extra : moz-landing-system : lando
Remove the version check for WINDRES, because, as per bug 454112, it
didn't actually work, and, making it work actually causes problems
because llvm's windres, used with mingw clang, has version 0.1.
Differential Revision: https://phabricator.services.mozilla.com/D15070
--HG--
extra : moz-landing-system : lando
vswhere only searches in Community, Professional and Enterprise, but one can
also install BuildTools only, which has a different product name.
Differential Revision: https://phabricator.services.mozilla.com/D15056
--HG--
extra : moz-landing-system : lando
It will be useful to run tests like try_compile, with different flags and different
kinds of sources.
Depends on D14949
Differential Revision: https://phabricator.services.mozilla.com/D14950
--HG--
extra : moz-landing-system : lando
Although this task technically doesn't build a toolchain, the set of
steps it needs to do is very similar to what a toolchain build does, so
we're shoehorning this task into the toolchain kind. The task basically
runs `cargo vendor` on the gfx/wr/Cargo.lock file (if/when it changes)
and exports a tarball of the resulting vendored crates. This allows
downstream tasks that build stuff in gfx/wr to not have to re-fetch
these crates from crates.io on every test run.
Differential Revision: https://phabricator.services.mozilla.com/D14406
--HG--
extra : moz-landing-system : lando
The SQLite in Debian 7 (3.7.13) lacks support for common table
expressions (the WITH keyword), which was introduced in SQLite
3.8.3. The Mercurial SQLite storage backend currently relies on
CTEs. Even if a future Mercurial doesn't require CTE, it is likely
that it will still use CTE if available for performance reasons.
So, it is in our best interest to give Mercurial access to a
modern SQLite. Plus, using a modern SQLite and avoiding potential
bugs in old versions seems prudent.
This commit introduces a SQLite package backport for Debian 7
so we can use the new SQLite feature. We had to minimally patch
the build to work with an older version of TCL that isn't using
multiarch.
I observed libsqlite3 being installed as part of building various
other packages (such as Python). I initially added the package as
a dependency so packages would be built against a more modern
SQLite. But glandium doesn't believe it matters, since nothing
should be doing build-time feature detection. Python is the most
important downstream package (since Mercurial uses its SQLite).
I audited the CPython build system and did not see any build-time
SQLite feature detection or version sniffing. So I think we'll be
fine building against an older SQLite.
Differential Revision: https://phabricator.services.mozilla.com/D14194
--HG--
extra : moz-landing-system : lando
We don't care anymore: the Gradle build fetches an appropriate version
of the build-tools package, and the one build-tool we actually use --
zipalign -- isn't version sensitive.
Differential Revision: https://phabricator.services.mozilla.com/D14290
--HG--
extra : moz-landing-system : lando
There's no need -- it will be fetched by Gradle when needed. The only
wrinkle is that the user might be asked to confirm license agreements,
which could be a hassle (and could cause problems in CI, but there we
install relevant platforms in advance).
Differential Revision: https://phabricator.services.mozilla.com/D14288
--HG--
extra : moz-landing-system : lando
The build system no longer invokes these directly: they're all fetched
and invoked by Gradle and its plugins.
Differential Revision: https://phabricator.services.mozilla.com/D14287
--HG--
extra : moz-landing-system : lando
CPP/HOST_CPP were probably not necessary already, but now that we leave
it to configure to figure out the appropriate compiler flags, we don't
need to set HOST_CC/HOST_CXX to remove the flags from CC/CXX.
Differential Revision: https://phabricator.services.mozilla.com/D14382
Rather than manually passing -isysroot to clang. Ideally, we shouldn't
need to fill BINDGEN_CFLAGS from the mozconfig, but we're not quite
there yet.
Differential Revision: https://phabricator.services.mozilla.com/D14381
Because we now set the sysroot include flags early in python configure,
we don't need to set CPP/CXXCPP. We also skip the explicit compiler test
because more complete tests follow anyways.
Differential Revision: https://phabricator.services.mozilla.com/D14380
This matches more closely cross toolchains prefixes (as can be seen in
e.g. media/libvpx/libvpx/README for x86_64-darwin*-gcc), and leaves it
to the build system to figure out the right --target to pass to clang on
its own.
Differential Revision: https://phabricator.services.mozilla.com/D14376
We're always setting -dead_strip on mac builds, per
cross-mozconfig.common, we might as well not do that and revert bug
638149, which disabled adding -dead_strip with LTO: that is apparently
not a problem anymore.
Differential Revision: https://phabricator.services.mozilla.com/D14373
Currently, when Firefox crashes when running the PGO profile, the error
is ignored, and the result is a build that is not optimized as expected.
So instead of such things going unnoticed, we make profileserver.py
return an error code, further failing the build.
Unfortunately, this does not make the output useful to know what went
wrong exactly, but the most important thing is that the build fails
instead of going through and leading to bad results.
Summary:
I've chosen linux64-debug since it's the most visible build I usually do, but I
could do another build task or something, or use the static analysis builds, or
what not. Just let me know if there's a better way to do this.
Caveat: This might make updating Rust toolchains a bit more painful. I think
this is better and we should just deal with warnings before updating toolchains,
but I don't know if there'd be strong opposition to that.
Note that this does _not_ affect third-party code since Cargo passes
`--cap-lint warn` automatically for those.
Proof that it works:
* https://treeherder.mozilla.org/#/jobs?repo=try&revision=4ad1e4e1392f71b574cff683e90c7b13bf8781d1
* https://treeherder.mozilla.org/#/jobs?repo=try&revision=57604f92624bbe49037eee87c56fdb6bf2b5017d
Reviewers: #firefox-build-system-reviewers, ted
Reviewed By: #firefox-build-system-reviewers, ted
Subscribers: reviewbot, glandium, ted
Bug #: 1513009
Differential Revision: https://phabricator.services.mozilla.com/D14083
But keep the crashreporter disabled at runtime because it doesn't work
yet.
This has the side effect of creating the artifacts with the
crashreporter symbols and pdbs.
Differential Revision: https://phabricator.services.mozilla.com/D14550
--HG--
extra : moz-landing-system : lando
This patch is based on the cmake cache files for Android checked in to the
clang repo.
Differential Revision: https://phabricator.services.mozilla.com/D14004
--HG--
extra : moz-landing-system : lando
This is an egregious hack. The web-platform-tests were not meant to run in the shell.
The eight tests that are included are the ones that we pass with flying colors.
In most of the others, we still have a failure or two.
Differential Revision: https://phabricator.services.mozilla.com/D11533
--HG--
rename : dom/imptests/testharness.js => js/src/jit-test/lib/w3c-testharness.js
extra : moz-landing-system : lando
This is an egregious hack. The web-platform-tests were not meant to run in the shell.
The eight tests that are included are the ones that we pass with flying colors.
In most of the others, we still have a failure or two.
Differential Revision: https://phabricator.services.mozilla.com/D11533
--HG--
rename : dom/imptests/testharness.js => js/src/jit-test/lib/w3c-testharness.js
extra : moz-landing-system : lando
This pulls a newer version of the nasm package, and patches
out doc building in order to make it compile on debian 7.
Differential Revision: https://phabricator.services.mozilla.com/D13510
--HG--
extra : moz-landing-system : lando
This removes several patches that have been upstreamed.
--HG--
extra : rebase_source : 8462b42b8342935cfb5ecb912fac06c67b9165b9
extra : amend_source : 271b5d4a5b37fa74555ef87cb0c4444f8f7468f0
David Major
Chris Manchester
Emilio Cobos Álvarez
Mike Hommey
Andi-Bogdan Postelnicu
Geoff Brown
Andreea Pavel
James Willcox
Johann Hofmann
Thomas Daede
Ted Mielczarek
Ciure Andrei
Kartikaya Gupta
Bogdan Tara
Mike Shal
Cosmin Sabou
Masatoshi Kimura
Chris Hartjes
Marco Castelluccio
Razvan Maries
Daniel Varga
Coroiu Cristina
Dorel Luca
Nathan Froyd
shindli
arthur.iakab
Kris Maglione
Csoregi Natalia
Andrew Halberstadt
Tom Ritter
Wes Kocher
Sylvestre Ledru
Makoto Kato
Gurzau Raul
Alex Chronopoulos
Mark Banner
Sebastian Hengst
Tom Prince
Ryan VanderMeulen
Jan Henning
Brindusan Cristian
Margareta Eliza Balazs
Ehsan Akhgari
Nick Alexander
Dustin J. Mitchell
Narcis Beleuzu
Marco Bonardo
Noemi Erli
Gregory Szorc
Jason Orendorff
Jacek Caban