At the same time, allow to enable jemalloc 4 with --enable-jemalloc=4.
MOZ_JEMALLOC4 will be deprecated later.
This also changes the semantics for freebsd, where the system jemalloc
is used, relying on MOZ_MEMORY being unset (default on freebsd) and
MOZ_JEMALLOC4 to be set. In this new setup, MOZ_JEMALLOC4 implies
--enable-jemalloc=4, which still works because of the corresponding
changes to old-configure.
While forgetting about it was warned about, having to add every new
environment option to wanted_mozconfig_variables is cumbersome. It turns
out there is a hackish way to make things work without that list, which,
all things considered, is not worse than the hacks around the
wanted_mozconfig_variables function, and are certainly an improvement as
it doesn't require an ever growing list of environment options.
At the same time, we improve things slightly by deriving HOST_CC from CC
in a smarter way, as well as CXX from CC, which we weren't doing
previously.
Many related things are not moved at the same time to keep the patch
somehow "small".
So far, everything was essentially executed at "declaration". This
made the sandbox code simpler, but to improve on the tooling around
python configure (for tests and introspection), we need to have more
flexibility, which executing everything at declaration doesn't give.
With this change, only @depends functions depending on --help, as
well as templates, are executed at the moment the moz.configure
files are included in the sandbox. The remainder is executed at the
end.
- HW_THREADS doesn't appear to be doing anything.
- USE_ARM_KUSER is not used since bug 675078. This also removes the configure
flag that sets it.
- HAVE_SETBUF and HAVE_SNPRINTF are leftover from bug 944935.
- MOZ_MEMORY_GONK is leftover from bug 804303.
- DEVELOPER_OPTIONS, INTEL_CC, INTEL_CXX, MOZ_ENABLE_QTMOBILITY,
GTK_CONFIG are or even were never used outside configure.
- MOZ_PROFILELOCKING which gradually became a no-op over the years. This
also removes the configure flag that sets it.
- XULRUNNER_STUB_NAME is xulrunner-only, and xulrunner is gone. This
also removes the configure flag that sets it.
- The only use of MOZ_CAN_RUN_PROGRAMS was removed in bug 780561.
- AR_LIST and AR_DELETE have not been used since bug 584474.
- MOZ_COMPONENT_NSPR_LIBS is leftover from bug 1036894.
- MOZ_PNG_ARM_NEON_CHECK is not used since bug 980488.
- MOZ_WEBRTC_LEAKING_TESTS has been no-oped by bug 825510.
- VPX_NEED_OBJ_INT_EXTRACT and NO_INTEGRATED_AS_CFLAGS are not used since
bug 1151175.
- WCHAR_CFLAGS is not used since bug 904985.
Because closing the handler is not enough (it eventually reopens
itself), the dumping of config.log overwrites the file, and confuses
the process of reading it to dump it in the first place, showing
incomplete logs. The intent from the start was that nothing would be
logged in the FileHandler, so on top of closing it, actively remove it.
When reading config.log, with old-configure output, we may get non-ascii
strings, but that currently fails because we're using plain open() to
read it. So use encoded_open() instead (which does the same job for
other files in the same script).
Because the build system can be encapsulated in mach, python configure
can have a pipe as stdout/stderr, and in that case, sys.stdout/stderr
have an ascii encoding, failing to print out anything that doesn't
fit in ascii, consequently failing to print the things we've read from
config.log. So reopen stdout and stderr with the right encoding in
the configure output handler.
Gonk, Android, and the generic cross-compilation setup all were using a
different yet similar way to prefix the toolchain. The latter was even
wrong, since the target and target alias usually don't match actual
toolchain prefixes (which don't include the machine part of the target).
Note that this removes force-setting cross_compiling to yes in
old-configure, which wasn't working because every AC_TRY_COMPILE
resets it with $ac_cv_prog_cc_cross or $ac_cv_prog_cxx_cross.
The initial goal of templates was to provide a way to write shorter
constructs for some generic tasks during configure. But the limitations
of the sandbox and the properties of templates made them used for more
general functions.
Consequently, this led to templates having to be available from
anywhere, which, in turn, led to difficult to introspect constructs.
With bug 1257823, we've made almost everything use set_config and
similar functions from the global scope, but we don't enforce that
those primitives are only used at the global scope.
This change does that: it enforces that primitives are only used at
the global scope. Or in templates.
Now, since templates were used for other purposes than generic uses
of other primitives, we now allow non-template functions to be declared.
Those can be used everywhere, but don't have access to the sandbox
primitives.
So far, we've been using the lowercase of the variable name, but it's
not enough for some special cases. Those special cases could do their
own business, but then, they'd have to duplicate 90% of check_prog,
which is less desirable.
While DummyFunction is descriptive of what the instances are (and they
can't even be called), the various uses of isintance(obj, DummyFunction)
are kind of confusing, especially when they are in moz.configure land
(and this bug is about to add another one).
And since the file is also used for old-configure, close our handle on
the file before spawning old-configure, and make old-configure append
there instead of truncating the file.
The reason the "checking" string always appears is that @depends
functions are always called, regardless of the value of the dependency.
This introduces a new decorator @depends_true, which works like
@depends, but the decorated function is not called unless one of the
dependency value resolves to True.
The new decorator can also be used to replace many cases where we do
@depends(foo)
def bar(foo):
if foo:
...
For the same reasons as set_config is being moved to the global scope,
we're moving set_define to the global scope here. An additional change
is that set_define is now part of the sandbox itself instead of being
defined within the sandbox, which makes it share the implementation
details with set_config.
The way set_config is set currently makes it difficult to introspect
moz.configure files to know what configuration items are being set,
because they're hidden in the control flow of functions.
This makes some of the moz.configure more convoluted, but this is why
there are templates, and we can improve the recurring cases afterwards.
The way functions are being sandboxed in moz.configure land is that
their global namespace is being replaced with a limited and identifiable
dict. And we avoid re-wrapping a function that already received this
treatment.
The problem is that template functions have their global namespace
replaced, and any function that is defined within the template inherits
that global namespace. So when it comes time to wrap those functions
defined in templates with e.g. depends, we detect that they're already
wrapped although they are not, because we look if their global namespace
is of the recognizable type we use when replacing it.
So instead of looking at the global namespace type, keep track of all
functions that are wrapped.
This also adds a GRADLE_FLAGS environment variable for use in
automation.
Manually tested.
MozReview-Commit-ID: 8nDkqz2VnJn
--HG--
extra : rebase_source : 32626a7dc0c0a6a440e300d92c31670f14319325
extra : amend_source : fe134e25f079851b4c648b53a7a485ee20c15c18
Currently, if a @depends function doesn't have a return statement or
return None, a result is automatically set from all the set_config()
called from the function.
As we're going to move set_config to the global namespace, and as this
feature is only used once, and it's only used for something that was
written before ReadOnlyNamespace was exposed to the sandbox, we can
"safely" get rid of it.
This change adds a `Version` type to moz.configure which is a small
wrapper around `distutils.version.Version`. It's suitable for wrapping
version numbers in configure checks and doing equality or greater-than
less-than comparisons in a sensible way.
MozReview-Commit-ID: BOL6yvemulG
--HG--
extra : rebase_source : 3b463eac0499086f8acffda0d01418b6ab17f3d6
extra : amend_source : aebd6e40c408d9f868623b2f53fcdf7455e2fff5
In Python parens around an expression without a trailing comma is not a tuple,
so ('foo') == 'foo'. This is really easy to screw up with check_progs, which
coerces progs to a list and would give you ['f','o','o'] in this case. This
patch enforces that the progs argument is a tuple or list and errors if it
is not.
MozReview-Commit-ID: 7BJZuF9B8D5
--HG--
extra : rebase_source : 5db9a6b4bb9ef7195c2513407810093bff5e9174
extra : amend_source : f67ab46c2ac00a2a95cfc67e9763ac12b690ac14
old-configure and js/src/old-configure interestingly didn't handle both
the same way. But vtune support is only actually implemented in js/src,
so only the rules from js/src/old-configure matter (nothing was
enforcing the decistion from old-configure to js/src/old-configure), and
this is what is implemented here.
Until we stop relying on the raw_cpu and raw_os values from target and
host, we need to keep normalizing them like old-configure.in did, which
involves filtering them through config.sub.
Bug 1038639 removed the option, but since autoconf doesn't barf for
unknown options, the option MOZ_ARG_WITH_STRING was kept to emit a
AC_MSG_ERROR. This is not necessary anymore.
This aligns with the triplets used by clang/llvm. Technically, this
won't break iOS builds still using -darwin triplets until we move
MOZ_IOS_SDK to moz.configure and actively reject non iOS targets with
the iOS sdk.
Also allow to distinguish iOS and OSX with target.os.
Because some of the existing mozconfigs may be setting some variables,
we need to inject those that are handled by moz.configure now. It likely
doesn't matter for the variables currently in moz.configure, but it will
soon become important when more things are moved to moz.configure.
In fact, it is necessary for GENISOIMAGE and DSYMUTIL that we're going
to move in this bug, set in automation mozconfigs.
The implementation is cumbersome and quite horrible. We could do better
by changing the execution model in mozbuild.configure, which is probably
necessary for other reasons as well, but that requires more work and
testing.
It's an opt-in flag that allows to display where the build is in
terminal window titles. The fact that it's opt-in and likely unknown
makes it very low-value, and the fact that it was added in an era where
builds were not very well parallelized made it have a meaning, but now
that builds are parallelized, its meaningfulness is diminished.
Let's just remove it.
With all the things that still depend on all the variables derived from
--host and --target in both old-configure and moz.build, we still need
to keep variables such as OS_ARCH, OS_TARGET, CPU_ARCH, OS_TEST, etc.
Eventually, we'd settle on the output of split_triplet.
This /tries/ to preserve the current values for all these variables,
while also trying to make things a little more consistent. It also
effectively rejects OSes such as HPUX or AIX, because it is unclear
the decades old accumulated scripts related to them still do anything
useful, and we might as well have them start again from scratch, which,
in the coming weeks, will be even easier.
The data we get out of mozconfig can be unicode, and needs to be in a
form the shell used to run old-configure will be able to recognize,
which is likely utf-8 on UNIX (that's what we settled on), and mbcs on
Windows.
So far, we've been passing down all configure_args from mozconfig as
well as every flag appearing on sys.argv. This is overly broad and
causes problems for some options, like --enable-application.
However, we don't need all these options to be passed.
For the top-level old-configure, we need to pass the flags it can
handle, as well as the flags that we want passed down to
js/src/configure.
For js/src/old-configure, we only need to pass the flags it can handle.
The flags an old-configure can handle is defined by the list of flags
in @old_configure_options. The list of flags to pass down to
js/src/configure is defined by extra_old_configure_args.
And since the mozconfig configure_args are being injected into python
configure processing, the list of values we get in old_configure includes
the mozconfig configure_args.
While the long term goal is that js and top-level use the same configure
and the same overall setup, including the possibility to use mozconfigs,
figuring out what we want to do wrt mozconfig vs. command line and
environment variable is not a clear-cut case, and it's more important to
fix the immediate problem mozconfig causes to js developers by
"temporarily" returning to the previous behavior of not loading the
mozconfig for the js configure.
This was already done in the case of running it as a subconfigure, this
extends the exception. Unfortunately, there is no direct way to tell
whether the running configure is the js configure. The indirect way is
to look at the OLD_CONFIGURE path, which points to
js/src/old-configure.
I expect we'll have figured things out for mozconfigs well before
old-configure dies.
The implementation is a bit circumvoluted, but we do need to share
options between the top-level and js/src configures, possibly with
different defaults, and to properly pass things down from one to
the other. Until we are further down the road and can actually merge
both configures, this is a necessary evil.
Because --enable-application is the current way to do things, transpose
it to configure.py, but since --enable-application=js doesn't make
sense, make it an alias of a new --enable-project option.
This only partially moves --enable-application out of old-configure.in
because there are a lot of other things intertwined with it.
This moves all the reading mozconfig, finding autoconf, refreshing the
old configure, and running the old configure into sandboxed
moz.configure. This effectively bootstraps the sandboxed python configure.
Mike Hommey
Chris Manchester
Hector Zhao
Wes Kocher
Nick Alexander
Ted Mielczarek
Carsten "Tomcat" Book
Blake Winton
Nicolas B. Pierron
Patrick Cloke