The current way they are generated makes it so that it's not trivial for
the backends to figure that one depends on the other. While we should
eventually make things such that using FINAL_TARGET_PP_FILES works for
that, it's currently simpler to just use GENERATED_FILES.
--HG--
rename : build/application.ini => build/application.ini.in
extra : rebase_source : cad3460b65d99bdae3e6f1d9dd611e0a602ebfed
This patch does a few things:
1) Change all the in-tree tooltool manifests to contain sccache2 instead of the existing Python sccache
2) Change mozconfig.cache to point at sccache.
3) Lightly tweak the --with-cccache configure option to support sccache, and detect whether we're using ccache or sccache and set an option appropriately.
4) Add a MOZ_SCCACHE_VERBOSE_STATS option, and add a target in the top-level Makefile to make sccache spit out its stats at the end of the build. This is useful to see the cache hits/errors until we get something better.
5) Add MOZ_USING_SCCACHE to the build telemetry. Not that I think it will be immediately useful, but for future use.
MozReview-Commit-ID: 9lrdLwNj5Bm
--HG--
extra : rebase_source : d323457df10d0ee0ac5811940e518d9422a7e070
At the same time, remove HOST_LD. It was only used for MSVC builds,
which don't support cross-compile anyways, so we can, at least for now,
use LINK for both host and target.
--HG--
extra : rebase_source : 9ee9e7e1bd3edefc043fa63d5c03f2a242f76982
It turns out that, in practice, the LD variable is not used by the build
system, except on Windows, where it's used to feed the default for LINK,
which is then re-injected as LD.
The upcoming changes are going to normalize the use of LD/LINK.
--HG--
extra : rebase_source : 2a1a9924e963e082e119ff3874e8ff24247f4f94
This patch contains a number of changes to the gyp_reader code:
* Add three new flags to GYP_DIRS:
** no_chromium, to skip forcing the includes/etc needed for chromium gyp files
** no_unified, to force building all sources without unification
** action_overrides, to pass scripts used when mapping gyp actions to moz.build GENERATED_FILES
* Handle the flags mentioned above in read_from_gyp
* Handle actions in gyp targets by mapping them to GENERATED_FILES, using scripts specified in the action_overrides flag. We don't try to handle the generic action case, we require special-casing for each action.
* Handle a subset of copies in gyp targets by mapping them to EXPORTS, just enough to handle the use of them for NSS exports.
* Handle shared_library and executable gyp targets
* Handle gyp target dependencies/libraries as USE_LIBS/OS_LIBS
* Handle generated source files
* Handle .def files in sources by mapping them to SYMBOLS_FILE
* Special-case some include_dirs:
** Map `<(PRODUCT_DIR)/dist/` to $DIST/include (to handle include paths for NSS exports)
** Map include_dirs starting with topobjdir to objdir-relative paths, to handle passing the NSPR include path to NSS
* split /build/gyp.mozbuild into two parts, with gyp_base.mozbuild containing generic bits, and gyp.mozbuild containing chromium-specific bits
MozReview-Commit-ID: FbDmlqDjRp4
--HG--
extra : rebase_source : d3fb470c589f92d8c956b9ecd550fb8df79ff5bc
GCC and Clang will colorize compiler output automatically if stdout is a
TTY. Unfortunately, when the build backend is invoked via `mach`,
stdout is not a TTY.
6e9a4c0b9cd8 (bug 1315785) changed mach so it exports an environment
variable indicating whether mach's original stdout is a TTY. This was
later used to add color flags to `cargo` invocations.
Building on that work, this patch adds color flags to compiler
invocations if the compiler supports color and a mach TTY is
detected. The result is that compiler output from `mach build`
will be colorized automatically if Clang or a modern version of
GCC is used.
The only issue I see with this is that Clang doesn't "unset" its color
sequences when printing a newline. As a result, mach's time line
prefixing can sometimes inherit "bold" or other stylings. AFAICT this is
only a minor cosmetic concern. GCC does not exhibit this issue.
MozReview-Commit-ID: 5Icu6aXGZBL
--HG--
extra : rebase_source : 5b2bf5a287fdf8075b3d7dde36b91f3c65b60728
The stable cargo now has all the features we need; pulling
a separate cargo nightly package is unnecessary.
MozReview-Commit-ID: FqWvSK0swsQ
--HG--
extra : rebase_source : 3659f8fa6d4a9a70c0731650aeced899519617f9
This should have been done when autoland came into existence.
MozReview-Commit-ID: HaWNq41mDvn
--HG--
extra : rebase_source : 1f8905a45388a147970b65d6d7e226600cf8afc2
fx-team is read only. We shouldn't be performing any more builds. The
configuration can be removed.
MozReview-Commit-ID: 3cbEnnOKwaZ
--HG--
extra : rebase_source : 1e3b69721f0615327c3882767f0b90ee10bce1cf
Importing 'os' in python configure functions, on Windows, changes the
separate the various os.path functions use, and that can have
unexpected, badly handled, consequences. While on the long term, it is
desirable to make @imports('os') modify os.path to use the same base
functions as if there were no @imports, let's go with the simpler
workaround of restoring the non-{isfile,isdir,exists} os.path functions
from b6be0e9e3e1e.
--HG--
extra : rebase_source : a1857b5dce2aa818c72a77d0d9727ac6ce16cb8f
We actually target armv7, so we need this distinct triple
with the rust toolchain, while it's implicit with the
android ndk.
MozReview-Commit-ID: LUwdpOaWB6M
--HG--
extra : rebase_source : 820de90d0844e1519f9e02a583c8cc3abf1dfdc0
We want functions without an @imports to not have any side effects, and
to not use external resources. So remove the few functions we expose from
os.path without @imports('os') that do.
--HG--
extra : rebase_source : a9485ec269d4de5785d66d7772eda4fae5a84b4a
clang-cl would normally derive its MSVC emulation bits from the
installed MSVC version, but we don't have an installed MSVC in this
scenario, so we have to use command-line options instead. We use
similar options for Gecko builds.
In a taskcluster world, we cannot used fixed directories, since we don't
know the absolute path of the directory we're building in ahead of time.
(We could pass it in to the build script, or discover it in the script
itself, but that wouldn't really solve the next problem.) This change
does make the builds not reproducible, but as we're using clang-cl
purely for secondary purposes on Windows, rather than for shipping
Firefox binaries (as we would on Mac, say), I don't feel bad about
punting the reproducibility issue down the road a bit.
We cannot depend on a fixed location for cl.exe in a taskcluster world.
We therefore need to make our build-clang.py script accomodate relative
path names for cc/cxx and assume those are binaries that should be
looked up on PATH.
We also need to modify the Linux build script so that the virtualenv is
used to look up the new 'which' package.
The toolchain checks are now such that we can stop special casing x86-64
<-> x86 cross compiling. OTOH, depending on the target, the toolchain
prefix can be of the form `cpu-os` or `cpu-machine-os`. We were only
using the former, and we change this to allow to try both.
Finally, the toolchain prefix being a gcc thing, it applies on all
target platforms where we try to use gcc.
However, the status quo is kept for the value of TOOLCHAIN_PREFIX
exposed to old-configure and the build system, until the various tools
paths (such as ar, readelf, etc.) are handled in python configure.
--HG--
extra : rebase_source : 1fe2e0708e317f061a03e2b4f058bbd08b5525f1
Use concurrent.futures to provide a thread pool so we upload
files concurrently. This should make uploading complete faster.
If timestamps in mozharness logs can be trusted, uploading on Linux
currently takes ~45s. Hopefully this shaves 10+s from the build.
MozReview-Commit-ID: KTww31XWN7o
--HG--
extra : rebase_source : 1a487cd1a0772e996eacaefcd9d0b37dc96fbffa
In preparation of using a thread pool to call DoSCPFile().
MozReview-Commit-ID: Ab2gk8UTQWK
--HG--
extra : rebase_source : b1c97953f5b0be91bb16eac19ca6862e715ce7d4
Most files go to the same directory. This should cut down on the number
of scp sessions we establish as part of directory creation.
MozReview-Commit-ID: LpKwgUZhLEO
--HG--
extra : rebase_source : f7ed0490a0fd8d0f01066090af16d751864039df
We'll call this multiple times in a subsequent commit. I hate DRY
violations.
MozReview-Commit-ID: Khai4EZkPCS
--HG--
extra : rebase_source : f58ac817c4956e22821ef36ae89b3e889ca4b005
A side effect of ffi.m4 was that it would un-set CFLAGS when compiling with MSC.
This incidentally worked around the fact that if CFLAGS are set in jemalloc's
configure, it will not set certain variables that are necessary to configure
jemalloc, but are unlikely to be set manually by someone building Firefox.
MozReview-Commit-ID: 2l54YhSAUaL
Now that we access WPT related files from a source checkout, we no
longer need the web-platform tests zip file produced or consumed by
automation. So stop producing it.
MozReview-Commit-ID: Ea8KjKZJ5Yx
--HG--
extra : rebase_source : f22506a02fdd5e78434cdc5d1c1f274db1cd04e2
I wanted to be able to do some VCS interaction from a mach command, and we
didn't have anything suitable, so I tore up mozversioncontrol and replaced
it with a framework to hang new features off of. I've only implemented
the bits I need currently (get_modified_files and add_remove_files),
but it should be straightforward to add more functionality there.
This patch also adds a `repository` property to `MozbuildObject`, which will
return a `Repository` object for the topsrcdir to make using these helpers
even easier for `MozbuildObject`-derived classes.
MozReview-Commit-ID: Gw6Ixp1ltiN
--HG--
extra : rebase_source : e619d6642eb86c3f96e679ac22a3e561dfdbb56a
CLOSED TREE
--HG--
extra : amend_source : 1f0c7bbb5aa8a3dab38f0785e13e32f59e8f8c79
extra : histedit_source : ca99420cac7019a4b6fd6aab781b93151092a8bc%2C0ef091317a27688c734f20417875406726e35de7
It is unfortunately not possible to include it last (or close to last,
before old.configure), but at least putting it after toolchain related
includes will be helpful.
--HG--
extra : rebase_source : bd027a87bc350c60dc1ba3308e2cc3b10782b506
To support android/aarch64, I want to remove the requirement of system's libffi.
MozReview-Commit-ID: Lc3POx09Cks
--HG--
extra : rebase_source : 384852e7b9e61d0d7a950159535e3ddc8457e889
Now that we access WPT related files from a source checkout, we no
longer need the web-platform tests zip file produced or consumed by
automation. So stop producing it.
MozReview-Commit-ID: Ea8KjKZJ5Yx
--HG--
extra : rebase_source : ee6ec00689696a710faf390d3dec5c5d02d8ec74
This generalizes the monkey-patching of the main module added
in bug 914563 to allow multiprocessing to be used from config.status
(which triggers the same bug as when it's used with `mach` as main).
MozReview-Commit-ID: AdOdpKzmbsp
Google's projects using GYP use arm64 for target_arch instead of aarch64. So we should use it for moz.build generator.
MozReview-Commit-ID: J4SLKhCqyUo
--HG--
extra : rebase_source : c902736ba0248eb5a3dfe94c174cb96374ebb94c
extra : histedit_source : 52620e73d6457078b28402dc6ef33f78f1c5425d
I recently discovered that build-tools 23.0.3 doesn't seem to exist on Fedora. Although
I fixed mozboot to download 23.0.1 instead, the builds were still failing because of
configure required 23.0.3.
This seems like an artificial limitation, as building with 23.0.1 seems to work just fine.
This patch will allow either 23.0.1 or 23.0.3.
Note: It would probably better to check for some "minimum" version of build-tools I think
GNU sort has a -V option we could use, but I don't know how cross-platform this is.
MozReview-Commit-ID: 8W0P3yyAHu1
--HG--
extra : rebase_source : 166c1b66962cac6af92c4ce6d3cc1780d9049800
Let AnnotationProcessor accept an output prefix argument, so that we can
generate two different sets of bindings for different jars - one set for
GeckoView code and one set for Fennec code.
Provide a single mach command to automatically generate the static
database of CSS properties that devtools uses for the inspector
and various editors.
MozReview-Commit-ID: 8E2jwxF0KbM
--HG--
rename : devtools/shared/css/moz.build => devtools/shared/css/generated/moz.build
extra : rebase_source : ab1815f321d460886168d95ddb739a579599b8c7
extra : histedit_source : f2bfecdcc128a87abcf3c0284a54c53bdeff1c87
aarch64 doesn't require compiler flag when using NEON. To use NEON on aarch64 with BUILD_ARM_NEON, we should move NEON's flags to NEON_FLAGS like SSE2_FLAGS.
MozReview-Commit-ID: PGDjcHdTfH
--HG--
extra : rebase_source : d21bd6f743f363fdd183bda9b9f308fcf100186b
Visual Studio 2015 Update 3 has been out for a few months. It appears
stable. So let's start using it.
As part of this, we also update the Windows SDK to the version
corresponding with the Windows 10 Anniversary Update (10.14393.0).
MozReview-Commit-ID: C36sRlKqa8t
--HG--
extra : rebase_source : 2fd46d6053d3eaf62dd8b2b291881c5172cc6056
The Visual Studio installer now prompts to install Windows 10 SDK
10.14393.0, which corresponds to the SDK released with the Windows 10
Anniversary Update. Since it is the latest SDK available, let's start
packaging it instead of the older SDK in our standalone toolchain
archive.
MozReview-Commit-ID: 29T6hMHX18x
--HG--
extra : rebase_source : b22dba5f922dcbeb4147ac1a744c772e82e0e9ed
Provide a single mach command to automatically generate the static
database of CSS properties that devtools uses for the inspector
and various editors.
MozReview-Commit-ID: 8E2jwxF0KbM
--HG--
extra : rebase_source : 2cd21cd08f431ff933c3fd89ebca3e6684fb80f8
Some compilers on some platforms by default #define some of the values
we're using in the source we use in get_compiler_info(). Namely,
mingw-gcc #defines WINNT by default, and the WINNT in the source is then
replaced by 1, breaking the check.
The C preprocessor, fortunately, doesn't expand macros inside C strings.
So instead of `%KERNEL WINNT`, we output `%KERNEL "WINNT"`, and strip
out the double quotes. For good measure, we do this for all values in
the source used in get_compiler_info().
--HG--
extra : rebase_source : dd4cc2b8c3bf0cb508b09598706b74ccc12162be
Update linux32 tooltool manifest to use a gecko build of rustc and cargo
for x86_64-unknown-linux-gnu host targeting both x86_64 and i586.
rustc built with --enable-llvm-static-stdcpp --disable-docs
--enable-debuginfo --release-channel=stable from 'stable' branch
rust 1.11.0 (commit 9b21dcd6a89f38e8ceccb2ede8c9027cb409f6e3)
Pass --target i585-unknown-linux-gnu when building for 32-bit linux.
We mostly want this for official builds, but Debian needs it too,
in both cases to support old machines without SSE2 instruction set
support, so while it means developers will have to `rustup target add
i585-unknown-linux-gnu` when building for this architecture that is
not a common task (most linux devs will be on 64-bit) and it reduces
variance and surprise if binaries are distributed.
MozReview-Commit-ID: 3mAjWxYGpwZ