This patch adds support for configuring Gonk/B2G with Android-specific
build scripts. This removes duplicated code and simplifies maintenance
of B2G.
The B2G builds will now use libc++ for Gecko; instead of the obsolete
STLport. A side-effect of this patch is the removal of any compile-time
dependency on B2G's bionic library.
MozReview-Commit-ID: 7V6BmC7jlrs
Build slaves on automation are based on Centos 6, which doesn't have a
recent enough version of libstdc++ for our new requirements. But since
we're building with a recent GCC or clang with its own libstdc++, we do
have such a libstdc++ available somewhere, and the compiler picks it
when invoking the linker.
Problems start happening when we execute some of the built programs
during the build, like host tools (e.g. nsinstall), or target programs
(xpcshell, during packaging). In that case, we need the compiler's
libstdc++ to be used. Which required adding the GCC or clang library
directory to LD_LIBRARY_PATH.
Unconveniently enough, the clang 3.5 tooltool package we're using for
ASAN builds until we can update to at least 3.8 (bug 1278718) doesn't
contain libstdc++.so. So for those builds, pull the GCC package from
tooltool as well, and pick libstdc++ from there.
The wizard has been ported to the version-control-tools repository
and in-tree consumers have been switched to consume it from there. This
code is now dead. Kill it.
References to the now-defunct code have been removed/updated.
MozReview-Commit-ID: 5fpCXdNIp8L
--HG--
extra : rebase_source : 6c1e2363793fe2cd3a506ce5d962788657871203
extra : histedit_source : c40d2203aaa54bbd48e4e2b46178e277dcdc2e3f
This begins the consolidation of `mach mercurial-setup` into
`mach bootstrap`. The first step is to move the content of the
mach_commands.py file into the bootstrapper's.
I'm not crazy about adding the sys.path entry for tools/mercurial.
I intend to clean this up later.
MozReview-Commit-ID: Cq56wPG8sO1
--HG--
extra : rebase_source : 48d6d2631760c9333bf99285673430948085630e
extra : histedit_source : e062f6fbc0ae9678347801b4a1f1c9b6912afd52
I never really liked this. Other people had even more visceral
reactions. Let's get rid of it.
The code for touching a file when it runs has also been removed because
the only thing it was used for was the nagging feature.
MozReview-Commit-ID: ERUVkEYgkzx
--HG--
extra : rebase_source : 1c1ed9c00eb2164d19e4405f2b8becf59680d1ed
extra : histedit_source : 9f2ebc64443140c0bc853ee5a3418f4e0f03db7b
Before bug 1269513, yasm_version returned a Version object, and it
doesn't anymore, which made the assignment of _YASM_*_VERSION skipped
silently.
Then, configure would go through the yasm version checks as if they were
false and skipping over the AC_MSG_ERRORs, printing out:
/builds/slave/try-m64-0000000000000000000000/build/src/old-configure: line 19615: test: : integer expression expected
which is why this went undetected: the version checks were simply
ignored.
Some shells, however, evaluated the yasm version checks as true, hitting
the AC_MSG_ERRORs.
When target is Android, -mandroid is default parameter from gcc 4.6 So we don't need add this options.
Also clang doesn't support this argument.
MozReview-Commit-ID: AuA3Y9vlgWE
--HG--
extra : rebase_source : c1866f56f131e666cc321d21fda1317532d46101
At this point, the only remaining part of the postflight for OSX
universal builds dates back to bug 834228. Since then, many things have
changed, one of them being that automation build steps have dependencies
that can be expressed through make dependencies.
While this is not directly related to bug 1244446, fixing this bug gets
more complicated if postflight needs to happen before some of the
automation build steps.
clang complains that a constexpr definition of methods[] cannot refer to
members of the incomplete Impl template parameter, and rightly so.
Making these const is sufficient for our purposes, and that enables us
to move the declaration outside of the class, where it will be
instantiated lazily (presumably at the point when |Impl| is a complete
class definition). We also need to declare the length of methods[], as
other parts of the code require knowing the length of methods[] at
compile time.
At this point, the only remaining part of the postflight for OSX
universal builds dates back to bug 834228. Since then, many things have
changed, one of them being that automation build steps have dependencies
that can be expressed through make dependencies.
While this is not directly related to bug 1244446, fixing this bug gets
more complicated if postflight needs to happen before some of the
automation build steps.
We hit an issue where adding a new env var, MOZ_DISABLE_TELEMETRY, added env10
and caused crashes. I suspect the issue is that there are is now a double-digit
number of env vars (bug 1277390). Here, we do the quick fix by removing
MOZ_DISABLE_TELEMETRY & repurposing MOZ_DISABLE_SWITCHBOARD to be generic.
While we're at it, we simplify the code by making the setDisabled methods a
strict getter without checking for how many times they're called.
MozReview-Commit-ID: 19DDbVYRZ2
--HG--
extra : rebase_source : 1590ae4f49bf725ab8a3bb26f10dab324903aa8c
When building a desktop version of Firefox with MOZ_LINKER enabled, the
zlib library is necessary for mozglue. The mozglue library is statically
linked to programs on desktop builds of Firefox, and the required setup
for those things is done in the GeckoProgram template, along with adding
the necessary zlib linkage.
Not sure how events went through but the current definitions in
mozglue/build/moz.build and config/external/zlib/moz.build make it that
USE_LIBS += ['mozglue'] currently implies zlib being linked in that case
without it being done explicitly in GeckoProgram, so remove that.
So far, we relied on the module being copied over in the virtualenv, and
the module itself would try to find config.status in parent directories
of its own location. Unfortunately, this falls short when the source
tree's build/ directory appears early in the sys.path.
With this change, we don't copy the module to the virtualenv anymore,
and try to find config.status in parent directories of the python
executable, which, when running from the virtualenv, will be equivalent
to the current behavior.
Previously, we required both or none of MOZ_SOURCE_REPO and
MOZ_SOURCE_CHANGESET to be defined. This logic was established in
51029f4d82d3 (bug 1247162).
There appears to be no good reason why we require MOZ_SOURCE_CHANGESET
if MOZ_SOURCE_REPO is defined. After all, if we have a checkout we should
be able to resolve the revision.
This commit changes the logic to resolve the changeset when not defined.
We still error if MOZ_SOURCE_REPO is defined but we can't resolve the
changeset. I can't imagine this breaking anything.
This change will be necessary to appease TaskCluster tasks once mozharness
is changed in a subsequent commit to define MOZ_SOURCE_REPO. Buildbot and
TC each have their own way of specifying the source revision. Rather than
change mozharness, it feels easier to just have the build system derive
things. This decision is further justified by the fact there is a chicken
and egg problem in mozharness: the environment variable dict is resolved
before source directory population. So, we'd need to teach mozharness
about TC's VCS mechanism, which it currently has no knowledge of. I'd rather
not do that.
MozReview-Commit-ID: ANaoGbPGWj2
--HG--
extra : rebase_source : fd09b282dc1d88478eb76e37796b210cccecaf3a
We no longer need to support systems without SSE2, so we
can move to the standard i686 target.
Bump CLOBBER since we're having trouble with cached
`--target i586-pc-windows-msvc` in RUSTC settings in configure.
MozReview-Commit-ID: 6eaPwnYSzrR
The `taskgraph` package generates TaskCluster task graphs based on collections
of task "kinds". Initially, there is only one kind, the "legacy" kind, which
reads the YAML files from `testing/taskcluster/tasks` to generate the task
graph.
Try syntax is implemented by filtering the tasks in the taskgraph after it has
been created, then extending the result to include any prerequisite tasks.
A collection of `mach taskgraph` subcommands are provided for developers to
extend or debug the task-graph generation process.
MozReview-Commit-ID: 1TJCns4XxZ8
--HG--
rename : testing/taskcluster/docs/index.rst => taskcluster/docs/index.rst
extra : rebase_source : 7b9125281d66044db9bd8e4a1fade16136f384b9
extra : histedit_source : 47640d27080acda0279270babbcf33f5badb0d1c
There shouldn't be any noise in this number, either a patch adds or removes
constructors or it doesn't. Choosing 0.25% is kind of arbitrary, but it should
be good enough since it will make us alert for an increase of one constructor
so long as we have less than 400 constructors, and currently we have 98.
This patch removes the checks for compat.h, sys/bittypes.h, gnu/libc-version.h,
X11/XKBlib.h, sys/sysmacros.h, and sys/cdefs.h from old-configure, because they
are not checked meaningfully in the tree.
The check for memory.h is removed, because while there are checks for
HAVE_MEMORY_H in the tree, they are in places this is set by a third party
build system.
The check for io.h is also removed, because while there are checks for
HAVE_IO_H, they're only relevant on windows, where this is set manually in
old-configure.
MozReview-Commit-ID: MOI50CP00k
For most cases, this replaces a value that was set in a way that would ignore
an environment variable, so this restores behavior for values that were
set in confvars.sh.
MozReview-Commit-ID: E31hm8uKq4D
We no longer support MacOS X versions 10.6-10.8. Bumping the
default MACOSX_DEPLOYMENT_TARGET to 10.7 lets us start landing
changes which are incompatible with earlier SDKs.
Currently we build on 10.7, so moving to 10.9 must wait until
our infrastructure is reconfigured to run build jobs on more
recent MacOS versions.
MozReview-Commit-ID: B0CcWVOnnv3
This will allow MOZ_MUST_USE to be used for a different and more common case.
MozReview-Commit-ID: 4dQsdWjJfc6
--HG--
extra : rebase_source : 390ab56ef83d71eb6d28759a0195a79a78b153bd
Thanks to gkw for pointing this out.
DONTBUILD (NPOTB)
MozReview-Commit-ID: 4Hd3Nse2VCP
--HG--
extra : rebase_source : f50253b8eb28306ecd62635f292fb3f2b65574c8
To support generating zip archives with more flexibility.
MozReview-Commit-ID: LmAgAXUfn3x
--HG--
extra : rebase_source : 2b9ddbda0c3dab2a498b2fc217e728a9cfefc134
We were unconditionally adding them, now actually check what the
compilers default to and add the flags if they are necessary.
This will, in the future, allow finer grained policy changes, where
we can decide that C++11 and C++14 are fine, downgrading compilers
that do C++17, etc.
At the same time, allow to enable jemalloc 4 with --enable-jemalloc=4.
MOZ_JEMALLOC4 will be deprecated later.
This also changes the semantics for freebsd, where the system jemalloc
is used, relying on MOZ_MEMORY being unset (default on freebsd) and
MOZ_JEMALLOC4 to be set. In this new setup, MOZ_JEMALLOC4 implies
--enable-jemalloc=4, which still works because of the corresponding
changes to old-configure.
While forgetting about it was warned about, having to add every new
environment option to wanted_mozconfig_variables is cumbersome. It turns
out there is a hackish way to make things work without that list, which,
all things considered, is not worse than the hacks around the
wanted_mozconfig_variables function, and are certainly an improvement as
it doesn't require an ever growing list of environment options.
At the same time, we improve things slightly by deriving HOST_CC from CC
in a smarter way, as well as CXX from CC, which we weren't doing
previously.
Many related things are not moved at the same time to keep the patch
somehow "small".
So far, everything was essentially executed at "declaration". This
made the sandbox code simpler, but to improve on the tooling around
python configure (for tests and introspection), we need to have more
flexibility, which executing everything at declaration doesn't give.
With this change, only @depends functions depending on --help, as
well as templates, are executed at the moment the moz.configure
files are included in the sandbox. The remainder is executed at the
end.
Ideally, we'd use the tarballs from
http://llvm.org/releases/download.html but I didn't feel like modifying
the script more than I already did to make it work at all (bug 1262735).
The new tarball for Linux was built on
https://tools.taskcluster.net/task-inspector/#LCUn8aEgTBeRJ11a3qTlDQ/0
The new tarball for Mac was built on a loaner, after installing cmake
and ninja, as well as building ld64 127.2 from source because the
installed version would assert while building clang. The latter required
manually adding some missing headers to /usr/include. TSAN was also
disabled because it requires APIs that are not available on the OSX
version on the build slaves (e.g. pthread_introspection_hook_install).
Building clang also required using a mac clang from tooltool, the system
one lacking support for atomics.
The check is for the clang plugin, which is a host library. It is especially
important to use the host compiler for this check on OSX-on-Linux cross builds.
Both point to the same path for cc and cxx, but not for gcc_dir, which
makes no sense. That's the only significant difference between both, so
just merge them both, and use the merged file in the taskcluster build
script.
Instead, copy libgcc from the GCC used to build clang at each stage.
When passing --gcc-toolchain, the flag ends up appearing in the output
of llvm-config, and completely defeats the purpose of copying libgcc in
clang/lib/gcc.
Since build-clang.py requires a gcc_dir to be set, and we're using GCC
from there to build clang, we might as well copy libgcc from there
instead of building a fresh GCC. On the taskcluster job building clang,
GCC comes from a tooltool package that is already the result of
build-gcc anyways.
When passing -DCMAKE_C_COMPILER="gcc -flags" to CMake, ninja then tries
to literally call "gcc -flags", not "gcc" "-flags", and a "gcc -flags"
file obviously doesn't exist, so the build fails.
This fixes a regression from bug 1042132. Hopefully, this also doesn't
regress bug 1042132 itself.
Runtime configs have been implemented for awhile, but disabled. This patch
enables configuration. Config files will be loaded in the following order
(later files override earlier ones):
1a. $MACHRC
1b. $MOZBUILD_STATE_PATH/machrc (if $MACHRC is unset)
2. topsrcdir/machrc
3. CLI via --settings
Note: .machrc may be used instead of machrc if desired.
MozReview-Commit-ID: IntONAZLGML
--HG--
extra : rebase_source : ff79b129eaea7cca5064d30fa6ddc76fceb9669b
Valgrind-test's output handler buffers valgrind's output to detect
errors and emit proper messages that are detected by automation log
parsing. When an error is detected, the buffered output is reinjected,
but there was a typo that prevented that from actually happening,
which ate the first lines of stack traces.
Also fixes bug 926980 - load ICU data from an archive file.
Stop invoking ICU's autoconf build system. Instead, have hand-authored
moz.build files under config/external/icu to build what we need. In addition,
we'll commit a pre-built copy of the ICU data file (currently icudt56l.dat)
under config/external/icu/data to avoid having to build ICU host tools to
generate it. config/external/icu/data also contains some assembly files
which can generate an object file containing the ICU data file contents
so that the JS shell (or standalone JS builds) can be linked directly to
the data without having to deal with the external data file. This requires
yasm or GNU as.
Various bits of packaging have been updated to account for the ICU data file.
XPCOM initialization now sets the ICU data directory so ICU can locate its
data file.
The update-icu.sh script has been modified to read the list of C/C++ source
files out of the ICU Makefiles and update `sources.mozbuild` files under
config/external/icu, as well as build a local copy of ICU using its
autoconf build system to generate the ICU data file to be committed in-tree.
MozReview-Commit-ID: 8Pfkzqt6S1W
--HG--
extra : rebase_source : 31426cddddb5543e0191059ba2f2eb069abe7727
- HW_THREADS doesn't appear to be doing anything.
- USE_ARM_KUSER is not used since bug 675078. This also removes the configure
flag that sets it.
- HAVE_SETBUF and HAVE_SNPRINTF are leftover from bug 944935.
- MOZ_MEMORY_GONK is leftover from bug 804303.
- DEVELOPER_OPTIONS, INTEL_CC, INTEL_CXX, MOZ_ENABLE_QTMOBILITY,
GTK_CONFIG are or even were never used outside configure.
- MOZ_PROFILELOCKING which gradually became a no-op over the years. This
also removes the configure flag that sets it.
- XULRUNNER_STUB_NAME is xulrunner-only, and xulrunner is gone. This
also removes the configure flag that sets it.
- The only use of MOZ_CAN_RUN_PROGRAMS was removed in bug 780561.
- AR_LIST and AR_DELETE have not been used since bug 584474.
- MOZ_COMPONENT_NSPR_LIBS is leftover from bug 1036894.
- MOZ_PNG_ARM_NEON_CHECK is not used since bug 980488.
- MOZ_WEBRTC_LEAKING_TESTS has been no-oped by bug 825510.
- VPX_NEED_OBJ_INT_EXTRACT and NO_INTEGRATED_AS_CFLAGS are not used since
bug 1151175.
- WCHAR_CFLAGS is not used since bug 904985.
Because closing the handler is not enough (it eventually reopens
itself), the dumping of config.log overwrites the file, and confuses
the process of reading it to dump it in the first place, showing
incomplete logs. The intent from the start was that nothing would be
logged in the FileHandler, so on top of closing it, actively remove it.
In bug 1254861, we unsupported clang < 3.3, picking 3.3 essentially
because that's the smallest version we had on automation. Bug 1254854
changed that, and the smallest version on automation is now 3.5.
Now, the motivation to unsupport an old version of clang again is that
recent versions don't have the problem with __float128 used in libstdc++
headers (bug 654493). In fact, starting with clang 3.4, the hack we have
in place is not necessary.
So let's just drop support for clang 3.3 instead of keeping that hack
around longer.
As mentioned in bug 1254854, Ubuntu 12.04 LTS has clang 3.4 packages.
When reading config.log, with old-configure output, we may get non-ascii
strings, but that currently fails because we're using plain open() to
read it. So use encoded_open() instead (which does the same job for
other files in the same script).
Because the build system can be encapsulated in mach, python configure
can have a pipe as stdout/stderr, and in that case, sys.stdout/stderr
have an ascii encoding, failing to print out anything that doesn't
fit in ascii, consequently failing to print the things we've read from
config.log. So reopen stdout and stderr with the right encoding in
the configure output handler.
This moves test installation for test files out of the monolithic install
manifest for $objdir/_tests, and determines the test and support files
to install based on the object derived from all-tests.json. Additionally,
the files resulting from TEST_HARNESS_FILES are installed, as some tests
will depend on them.
As a result, the time to install tests when invoking the test runner will
scale with the number of tests requested to run rather than the entire set
of tests in the tree, resulting in significantly less overhead.
MozReview-Commit-ID: LeIrUVh1yD4
Update win32 builds to use a Repack of today's (2016 April 1)
rustc nightly for i686-pc-windows-msvc host and
i586-pc-windows-msvc target.
This should properly support machines without sse2 instructions.
The lone remaining INSTALL_TARGETS is for moztt, which relies on
including a Makefile from that repository. We could potentially add a
mozbuild file there instead.
MozReview-Commit-ID: FXJDg3xPW1x
The hack was supposed to avoid clobbers when landing bug 967556, which
might have worked somehow back then, but doesn't do anything useful
anymore. In fact, it now breaks the display of some results in
old-configure.
Gonk, Android, and the generic cross-compilation setup all were using a
different yet similar way to prefix the toolchain. The latter was even
wrong, since the target and target alias usually don't match actual
toolchain prefixes (which don't include the machine part of the target).
Note that this removes force-setting cross_compiling to yes in
old-configure, which wasn't working because every AC_TRY_COMPILE
resets it with $ac_cv_prog_cc_cross or $ac_cv_prog_cxx_cross.
Recent config.sub better handles all the broken android triplets (which
would have worked around bug 1257793), and added the -ios variant that
we've been relying on since bug 1257051, but that bug 1257648 broke by
using config.sub, which didn't support it)
The new config.guess interestingly now returns pc instead of unknown as
maching type on linux, which will, unfortunately, make objdir names
change when they are not manually set.
The initial goal of templates was to provide a way to write shorter
constructs for some generic tasks during configure. But the limitations
of the sandbox and the properties of templates made them used for more
general functions.
Consequently, this led to templates having to be available from
anywhere, which, in turn, led to difficult to introspect constructs.
With bug 1257823, we've made almost everything use set_config and
similar functions from the global scope, but we don't enforce that
those primitives are only used at the global scope.
This change does that: it enforces that primitives are only used at
the global scope. Or in templates.
Now, since templates were used for other purposes than generic uses
of other primitives, we now allow non-template functions to be declared.
Those can be used everywhere, but don't have access to the sandbox
primitives.
So far, we've been using the lowercase of the variable name, but it's
not enough for some special cases. Those special cases could do their
own business, but then, they'd have to duplicate 90% of check_prog,
which is less desirable.
While DummyFunction is descriptive of what the instances are (and they
can't even be called), the various uses of isintance(obj, DummyFunction)
are kind of confusing, especially when they are in moz.configure land
(and this bug is about to add another one).
And since the file is also used for old-configure, close our handle on
the file before spawning old-configure, and make old-configure append
there instead of truncating the file.
The reason the "checking" string always appears is that @depends
functions are always called, regardless of the value of the dependency.
This introduces a new decorator @depends_true, which works like
@depends, but the decorated function is not called unless one of the
dependency value resolves to True.
The new decorator can also be used to replace many cases where we do
@depends(foo)
def bar(foo):
if foo:
...
This commit switches Windows builds from Visual Studio 2013
to Visual Studio 2015 Update 1.
Previously, Visual Studio was installed on the builders as part
of the base system image. Starting with this commit, we obtain
Visual Studio from a pre-generated, self-contained archive
containing the executables, Windows SDK, and other support
files. This means that new Windows toolchains can be installed
without having to modify configuration of machines in automation!
The mozconfigs for Visual Studio 2015 are a bit different from
existing mozconfigs.
Because it appears to be completely redundant and not necessary,
the LIBPATH variable has been dropped.
The order of paths in PATH, LIB, and INCLUDE has changed. The new
order more accurately reflects what would be defined by
vcvarsall.bat.
As part of switching to Visual Studio 2015, the Universal CRT is
now required. So, the 2015 mozconfigs export WIN_UCRT_REDIST_DIR
to define the location to those files.
The switch to Visual Studio 2015 also involves the switch from
the Windows 8.1 SDK to the Windows 10 SDK. However, we still
target an old version of Windows, so this hopefully shouldn't
have any significant fallout.
It's worth noting that switching to Visual Studio 2015 makes
builds - especially PGO builds - significantly faster. Our
PGO build times in automation are ~1 hour faster. Our regular
builds appear to be a few minutes faster.
MozReview-Commit-ID: Pa5GW8V87Q
--HG--
extra : rebase_source : bff4fad17f781d8d21bdb941bdd500955d1e9f08
extra : amend_source : faa3038c290fdf5cdd3e24a45ba2a37490f68c17
extra : source : 56b27306d3257445f70374aa78fc5bd42ef300ce
For the same reasons as set_config is being moved to the global scope,
we're moving set_define to the global scope here. An additional change
is that set_define is now part of the sandbox itself instead of being
defined within the sandbox, which makes it share the implementation
details with set_config.
The way set_config is set currently makes it difficult to introspect
moz.configure files to know what configuration items are being set,
because they're hidden in the control flow of functions.
This makes some of the moz.configure more convoluted, but this is why
there are templates, and we can improve the recurring cases afterwards.
compiler-opts.m4's check for `MOZ_CXX_SUPPORTS_WARNING(-W, unreachable-code-return, ac_cxx_has_wunreachable_code_return)` is broken. The C/C++ code that configure emits for MOZ_CXX_SUPPORTS_WARNING() actually contains a -Wunreachable-code-return warning and, thus, doesn't actually detect that clang supports the -Wunreachable-code-return flag.
This configure code in MOZ_CXX_SUPPORTS_WARNING():
AC_TRY_COMPILE([],
[return(0);],
$3="yes",
$3="no")
generates something like:
int main() {
return(0);
; return 0; }
where the second return, automatically emitted by configure, is unreachable and causes a -Wunreachable-code-return warning.
The fix is to remove the redundant return(0) from MOZ_CXX_SUPPORTS_WARNING(). This allows clang's -Wunreachable-code-return flag to be detected, but then -Wunreachable-code-return breaks other configure checks, including third-party libraries' configure checks (in particular jemalloc) that also have redundant `return(0)`. So all the third-party libraries' configure checks would need to be fixed upstream, which seems like more hassle than the value of the -Wunreachable-code-return warnings.
The way functions are being sandboxed in moz.configure land is that
their global namespace is being replaced with a limited and identifiable
dict. And we avoid re-wrapping a function that already received this
treatment.
The problem is that template functions have their global namespace
replaced, and any function that is defined within the template inherits
that global namespace. So when it comes time to wrap those functions
defined in templates with e.g. depends, we detect that they're already
wrapped although they are not, because we look if their global namespace
is of the recognizable type we use when replacing it.
So instead of looking at the global namespace type, keep track of all
functions that are wrapped.
This also adds a GRADLE_FLAGS environment variable for use in
automation.
Manually tested.
MozReview-Commit-ID: 8nDkqz2VnJn
--HG--
extra : rebase_source : 32626a7dc0c0a6a440e300d92c31670f14319325
extra : amend_source : fe134e25f079851b4c648b53a7a485ee20c15c18
Now that MOZ_BUILD_APP is set to js when building js/src, we can
distinguish those builds with MOZ_BUILD_APP==js instead of BUILDING_JS.
Consequently, remove BUILDING_JS.
subconfigure.py shortcuts js/src/configure to avoid a msys transition
breaking the environment. But js/src/configure passes an extra
--enable-project=js that subconfigure.py currently doesn't pass.
And because in some cases, MOZ_BUILD_APP is set in the environment
(notably, in OSX universal builds, through client.mk), we need to
unset it before calling the subconfigure.
Now that MOZ_BUILD_APP is set to js when building js/src, we can
distinguish those builds with MOZ_BUILD_APP==js instead of BUILDING_JS.
Consequently, remove BUILDING_JS.
This change adds a `Version` type to moz.configure which is a small
wrapper around `distutils.version.Version`. It's suitable for wrapping
version numbers in configure checks and doing equality or greater-than
less-than comparisons in a sensible way.
MozReview-Commit-ID: BOL6yvemulG
--HG--
extra : rebase_source : 3b463eac0499086f8acffda0d01418b6ab17f3d6
extra : amend_source : aebd6e40c408d9f868623b2f53fcdf7455e2fff5
In Python parens around an expression without a trailing comma is not a tuple,
so ('foo') == 'foo'. This is really easy to screw up with check_progs, which
coerces progs to a list and would give you ['f','o','o'] in this case. This
patch enforces that the progs argument is a tuple or list and errors if it
is not.
MozReview-Commit-ID: 7BJZuF9B8D5
--HG--
extra : rebase_source : 5db9a6b4bb9ef7195c2513407810093bff5e9174
extra : amend_source : f67ab46c2ac00a2a95cfc67e9763ac12b690ac14
old-configure and js/src/old-configure interestingly didn't handle both
the same way. But vtune support is only actually implemented in js/src,
so only the rules from js/src/old-configure matter (nothing was
enforcing the decistion from old-configure to js/src/old-configure), and
this is what is implemented here.
Until we stop relying on the raw_cpu and raw_os values from target and
host, we need to keep normalizing them like old-configure.in did, which
involves filtering them through config.sub.
Bug 1038639 removed the option, but since autoconf doesn't barf for
unknown options, the option MOZ_ARG_WITH_STRING was kept to emit a
AC_MSG_ERROR. This is not necessary anymore.
This aligns with the triplets used by clang/llvm. Technically, this
won't break iOS builds still using -darwin triplets until we move
MOZ_IOS_SDK to moz.configure and actively reject non iOS targets with
the iOS sdk.
Also allow to distinguish iOS and OSX with target.os.
Previously, Windows toolchains and related dependencies (SDKs, etc)
were installed on Windows builders by people responsible for
maintaining those machines.
This commit takes a step in a new direction. We introduce a
script (complete with documentation) that can produce a zip
archive (or any archive format if people want to implement
support) of the toolchain files. Basically, you install
Visual Studio 2015 Community, run the script, and produce
a self-contained zip file containing everything from Microsoft
you need to build Firefox. With a copy of this archive and
an installation of MozillaBuild, it is possible to build
Firefox on a fresh Windows installation. No time-consuming
Visual Studio installation needed.
The goal is to upload these archives to tooltool and have
our Windows builders download and extract them at run-time.
At which time, we can remove all the other Visual Studio
and SDK files from builders because they don't need to be
baked into the image.
We may find tooltool's caching isn't good enough and we have
to more aggressively caching the standalone toolchain files.
But that is a problem for another day. Whatever happens,
we'll need the functionality in this script to produce
a self-contained archive of the toolchain.
There are certainly files in the produced archive that aren't
needed. I think perfect is the enemy of done and we can prune
the archive over time, if wanted.
MozReview-Commit-ID: EckEK1a6vA3
--HG--
extra : rebase_source : c328be792b2bfb4b3cb8acb50e4868277cb59974
extra : source : 4c980771e574e899a1b05319ad11fb6cffb00087
Currently, if a @depends function doesn't have a return statement or
return None, a result is automatically set from all the set_config()
called from the function.
As we're going to move set_config to the global namespace, and as this
feature is only used once, and it's only used for something that was
written before ReadOnlyNamespace was exposed to the sandbox, we can
"safely" get rid of it.
Because some of the existing mozconfigs may be setting some variables,
we need to inject those that are handled by moz.configure now. It likely
doesn't matter for the variables currently in moz.configure, but it will
soon become important when more things are moved to moz.configure.
In fact, it is necessary for GENISOIMAGE and DSYMUTIL that we're going
to move in this bug, set in automation mozconfigs.
The implementation is cumbersome and quite horrible. We could do better
by changing the execution model in mozbuild.configure, which is probably
necessary for other reasons as well, but that requires more work and
testing.
It's an opt-in flag that allows to display where the build is in
terminal window titles. The fact that it's opt-in and likely unknown
makes it very low-value, and the fact that it was added in an era where
builds were not very well parallelized made it have a meaning, but now
that builds are parallelized, its meaningfulness is diminished.
Let's just remove it.
With all the things that still depend on all the variables derived from
--host and --target in both old-configure and moz.build, we still need
to keep variables such as OS_ARCH, OS_TARGET, CPU_ARCH, OS_TEST, etc.
Eventually, we'd settle on the output of split_triplet.
This /tries/ to preserve the current values for all these variables,
while also trying to make things a little more consistent. It also
effectively rejects OSes such as HPUX or AIX, because it is unclear
the decades old accumulated scripts related to them still do anything
useful, and we might as well have them start again from scratch, which,
in the coming weeks, will be even easier.
The data we get out of mozconfig can be unicode, and needs to be in a
form the shell used to run old-configure will be able to recognize,
which is likely utf-8 on UNIX (that's what we settled on), and mbcs on
Windows.
So far, we've been passing down all configure_args from mozconfig as
well as every flag appearing on sys.argv. This is overly broad and
causes problems for some options, like --enable-application.
However, we don't need all these options to be passed.
For the top-level old-configure, we need to pass the flags it can
handle, as well as the flags that we want passed down to
js/src/configure.
For js/src/old-configure, we only need to pass the flags it can handle.
The flags an old-configure can handle is defined by the list of flags
in @old_configure_options. The list of flags to pass down to
js/src/configure is defined by extra_old_configure_args.
And since the mozconfig configure_args are being injected into python
configure processing, the list of values we get in old_configure includes
the mozconfig configure_args.
While the long term goal is that js and top-level use the same configure
and the same overall setup, including the possibility to use mozconfigs,
figuring out what we want to do wrt mozconfig vs. command line and
environment variable is not a clear-cut case, and it's more important to
fix the immediate problem mozconfig causes to js developers by
"temporarily" returning to the previous behavior of not loading the
mozconfig for the js configure.
This was already done in the case of running it as a subconfigure, this
extends the exception. Unfortunately, there is no direct way to tell
whether the running configure is the js configure. The indirect way is
to look at the OLD_CONFIGURE path, which points to
js/src/old-configure.
I expect we'll have figured things out for mozconfigs well before
old-configure dies.
The implementation is a bit circumvoluted, but we do need to share
options between the top-level and js/src configures, possibly with
different defaults, and to properly pass things down from one to
the other. Until we are further down the road and can actually merge
both configures, this is a necessary evil.
This spins up a separate process to submit telemetry data rather than
blocking the execution the current mach command.
Although the initial Python process needs to wait for the second process
to complete prior to exiting, it releases control of the console once it
finishes executing Python code, so from the user's perspective, mahc command
completion is not blocked by submitting telemetry data.
MozReview-Commit-ID: FlKDYd6rNPc
--HG--
extra : rebase_source : d10f3f2cdfe6e8c8cdd8b1e02ce9261178b528e6
Because --enable-application is the current way to do things, transpose
it to configure.py, but since --enable-application=js doesn't make
sense, make it an alias of a new --enable-project option.
This only partially moves --enable-application out of old-configure.in
because there are a lot of other things intertwined with it.
This moves all the reading mozconfig, finding autoconf, refreshing the
old configure, and running the old configure into sandboxed
moz.configure. This effectively bootstraps the sandboxed python configure.
The nice side effect is that now we can have actual dicts for defines
and substs from the start, which simplifies so things, although it
requires adjustments to some unit tests.
This catches the exception and prevents further attempts to post to
telemetry in this command invocation.
MozReview-Commit-ID: JDW0zUS8fjF
--HG--
extra : rebase_source : 04b9399a2504d08bae33f2f8ff390e02c9a53fe9
32-bit PGO builds need to modify the PATH to find pgortXXX.dll. We were
doing this for Visual Studio 2013 (VC12) in 2 locations. We weren't
doing it for Visual Studio 2015. This resulted in a failure to launch
PGO instrumented executables because pgort140.dll could not be found.
This commit refactors the PATH munging to support Visual Studio 2015.
MozReview-Commit-ID: 4EKf8gjcNH6
--HG--
extra : rebase_source : 2772558b06202d26583401bc41e56da8c5a69ef4
-Wno-psabi has not been necessary since we updated the build machines to Android NDK r8c three years ago in bug 826133.
-Wsometimes-initialized is not necessary because it is implicitly enabled by clang's -Wuninitialized, which is enable by -Wall.
-Wcast-align is very noisy and we apparently only enable it for gcc on 32-bit x86 builds?
-Wno-unused-local-typedef is no longer necessary. This is the clang flag; gcc's flag is -Wno-unused-local-typedefs, with an 's'. Suppressing gcc's warning was recently deemed unnecessary and WONTFIX'd in bug 1243604. Unsurprisingly, we no longer need it on clang either.
-Wrange-loop-analysis is no longer necessary because it is implicitly enabled by -Wloop-analysis, which is enabled by the previous commit.
-Wsign-compare is enabled by gcc's -Wall (for C++ only), but not by clang's -Wall. -Wsign-compare is enabled for C and C++ by gcc's -Wextra and clang's -Wextra, which we don't use.
Previously on first `mach` run on a system, we'd display a prompt with a
20s countdown timer after which the default state directory
(~/.mozbuild) was created. Users had to wait 20s or ctrl+c and
make the directory by hand. Either way, they'd have to re-invoke
mach to run whatever command they were trying to run.
This was annoying.
This commit makes the following changes:
1) The countdown timer is replaced with "press RETURN/ENTER to continue"
2) The requirement to re-invoke mach has been removed
On first run, people now see a message telling them to press
RETURN/ENTER. Then the directory is created and whatever mach command
they requested to run is run.
While I was here, I also changed the permissions on the newly created
directory to match what it was a few lines above. Without, we're relying
on umask, which may be permissive. UNIX convention is to use a more
restrictive umask insider user directories. So this feels like the right
behavior.
MozReview-Commit-ID: IodN13xAJ8P
--HG--
extra : rebase_source : 7da66e2f03aaf4e6807c54ac20854a4f41092f2a
Thomas Zimmermann
Mike Hommey
Sebastian Kaspari
Eric Rahm
Gregory Szorc
Karl Tomlinson
Kearwood (Kip) Gilbert
Dustin J. Mitchell
Christoph Diehl
Chris Peterson
Carsten "Tomcat" Book
Ms2ger
Makoto Kato
Nathan Froyd
Michael Comella
Ehsan Akhgari
Wes Kocher
Alessio Placitelli
Gijs Kruitbosch
Chris Manchester
Ralph Giles
Ryan VanderMeulen
Michael Layzell
Andrzej Hunt
Sebastian Hengst
Trevor Saunders
Trevor Saunders
Steve Fink
Ben Kelly
Mike Shal
Jonathan Kingston
Andrew Halberstadt
Jan Beich
Nicholas Nethercote
Kim Moir
Andrew McCreight
Vlad Ciobancai
dimi
Ted Mielczarek
Jeff Gilbert
George Wright
aleth
Edmund Wong
Hector Zhao
Nick Alexander
Geoff Brown
Blake Winton
Nicolas B. Pierron
Joshua Cranmer
Dan Minor
Patrick Cloke
Myk Melez
Andi-Bogdan Postelnicu
William Lachance
Andrea Marchesini
Joel Maher
Henrik Skupin
Andreas Tolfsen