Now that we access WPT related files from a source checkout, we no
longer need the web-platform tests zip file produced or consumed by
automation. So stop producing it.
MozReview-Commit-ID: Ea8KjKZJ5Yx
--HG--
extra : rebase_source : ee6ec00689696a710faf390d3dec5c5d02d8ec74
Move packaging for Marionette from make to test_archiver by using root manifest files.
MozReview-Commit-ID: 1cxEBYQeJ2Z
**
--HG--
extra : rebase_source : 372a407d9207bfbccbfb88c689df60b8cc1abcaf
This patch also reworks how we set all the boolean flags in mozinfo.py.
MozReview-Commit-ID: IhjXCFsx5J1
--HG--
extra : rebase_source : d984d9ecaed765126600c8d85847a82585ab3e67
Some compilers on some platforms by default #define some of the values
we're using in the source we use in get_compiler_info(). Namely,
mingw-gcc #defines WINNT by default, and the WINNT in the source is then
replaced by 1, breaking the check.
The C preprocessor, fortunately, doesn't expand macros inside C strings.
So instead of `%KERNEL WINNT`, we output `%KERNEL "WINNT"`, and strip
out the double quotes. For good measure, we do this for all values in
the source used in get_compiler_info().
--HG--
extra : rebase_source : dd4cc2b8c3bf0cb508b09598706b74ccc12162be
.mozbuild/last_log.json is opened by the process doing the deleting.
On Windows, you can't delete an opened file.
So stop clobbering the .mozbuild directory.
This directory only holds non-essential artifacts from mach/build
invocations. I don't think keeping it around will break anything.
MozReview-Commit-ID: 9b0AC2ywAAZ
--HG--
extra : rebase_source : 9236e3e282d1db2595e8b36353008bc98e2ae6cf
Some substs values are lists, and some CONFIGURE_SUBST_FILES use them,
and don't expect the substitution to use a pythonic expression of the
value. So serialize those lists.
--HG--
extra : rebase_source : 848962c326236607609bca2b92180c8f6f4ad079
The "remove_objdir" method has been rewritten to support partial
clobber. It still defaults to full clobber. But the "full" argument
can be passed as False to limit to a partial clobber where currently
the "msvc" directory (contains Visual Studio files) is not clobbered.
On Windows, there might be a regression with this change because
we'll be invoking N winrm.exe processes and new processes have
a non-trivial overhead on Windows. However, it is hopefully unlikely
that new processes are more overhead than deleting hundreds of thousands
of files.
MozReview-Commit-ID: 7yeMttztwic
--HG--
extra : rebase_source : 646992be199e1059f0b09cf8bed3334085293fe0
My Python editor told me rmtree has been marked as deprecated. Use
mozfile.remove instead.
MozReview-Commit-ID: FYkXk1fnxvC
--HG--
extra : rebase_source : df392e8832e52501950f09acda208943c9e3d707
This required implementing a utility function to resolve the binary
type. I used GetBinaryTypeW via ctypes because this seems the fastest.
I arbitrarily limited the function to testing 32-bit and 64-bit Windows
executables because hopefully those are the only executables we'll
ever encounter. We can expand the binary detection later, if needed.
This includes support for running on non-Windows platforms.
MozReview-Commit-ID: CYwyDWQrePc
--HG--
extra : rebase_source : 8fd7ca7f253d9e9e18d64784652a5ff934ad2272
And with indentation so it is easier for humans to read.
MozReview-Commit-ID: Kkd6vmfLNUf
--HG--
extra : rebase_source : 9f2b4ad2223f361dd6dd1ec9cdda71bd5a8560e3
Until now, HAVE_64BIT_BUILD was entirely determined by a compiler check.
But we didn't run the check on e.g. artifact builds, while relying on
its result for some non-compilation related things, leading to subtle
discrepancies.
This changes the configure check to derive HAVE_64BIT_BUILD from bitness
determined by the target CPU, and double checked with a compiler check.
--HG--
extra : rebase_source : 5dc0cf2369ed4457bdd9a15736a70265a771d919
Currently, it returns either None or the contents of the compiler's stdout,
which is always expected to be an empty string, and is not very useful. So
instead, return True in the latter case.
--HG--
extra : rebase_source : ee69cdeab38d27178ce759591fb394da65e694ac
As of Python 3, decimal notations of octal values for permission modes
are no longer permitted and will result in a `SyntaxError` exception
(`invalid token`).
Using the proper octal notation which is also Python 2.7 compatible will
fix this issue.
--HG--
extra : rebase_source : 2e897c51f04ad0ee69071f84b98df224f3af72d3
The base compiler check in python configure does some preprocessing,
which ensures the compiler works to some extent. Autoconf used to have
a more complete test, doing a compile/link. We do have plenty of tests
afterwards that do that anyways, but it's better if we fail early if
the toolchain fails somehow.
This refactors try_compile such that the *_compiler variable themselves
can be used to trigger compiler tests. Eventually, we'll want something
similar for preprocessing and possibly other invocations.
This also removes similar tests from build/autoconf/toolchain.m4.
--HG--
extra : rebase_source : c60d1d6e39b6bd2a377516687affd9b8932ebc12
While on automation, there is no MSVC to find through the registry, the
story is different on local builds, and that can interfere with tests.
Specifically, it breaks test_toolchain_configure.py because it's not
expecting the registry to provide a valid path to an almost valid
compiler, and then fails because that compiler doesn't match the
expected target CPU.
And because build/moz.configure/toolchain.configure also affects the PATH
environment variable, subsequent tests end up failing even earlier
because executing the empty mozconfig with the modified environment then
fails because of the unicode value of PATH.
This change implements enough of _winreg to make the get_registry_values
function return nothing.
--HG--
extra : rebase_source : f70e40ddabaed1197f6cddce67832bb112f1969d
Back when it was added, it was used, but it is not anymore, outside
test_base.py.
--HG--
extra : rebase_source : f0b9a4dab2985e89e9950eda774ae853c7de764c
We've been reading the mozconfig in MozbuildObject.from_environment to
check whether the mozconfig topobjdir matches the detected topobjdir.
Since bug 1278415, everything using the buildconfig python module now
calls MozbuildObject.from_environment, which reads the mozconfig. A lot
of things to that during the build. But none of them actually need the
data from the mozconfig, and the topobjdir match test has been breaking
things randomly on multiple occasions.
The topobjdir match test, however, really only needs to happen once:
when a mach command starts. So we can move the test to MachCommandBase,
where it belongs, and anything actively using MozbuildObject.mozconfig
will have the mozconfig read, but everything else won't.
On a Linux64 opt build, this brings down the number of times the
mozconfig is read during `mach build` from 979 to 9.
--HG--
extra : rebase_source : 6b340f1fcf73a3c3987033c37f8f14ef06a44f04
This patch is really two separate changes.
The first change is that rust crates are large, standalone entities that
may contain multitudes of source files. It therefore doesn't make sense
to keep them in SOURCES, as we have been doing. Moving to use cargo
will require a higher-level approach, which suggests that we need a
different, higher-level representation for Rust sources in the build
system.
The representation here is to have the build system refer to things
defined in Cargo.toml files as the entities dealt with in the build
system, and let Cargo deal with the details of actually building things.
This approach means that adding a new crate to an existing library just
requires editing Rust and Cargo.toml files, rather than dealing with
moz.build, which seems more natural to Rust programmers. By having the
source files for libraries (and binaries in subsequent iterations of
this support) checked in to the tree, we can also take advantage of
Cargo.lock files.
The second is that we switch the core build system over to building via
cargo, rather than invoking rustc directly.
We also clean up a number of leftover things from the Old Way of doing
things. A number of tests are added to confirm that we'll only permit
crates to be built that have dependencies in-tree.
rlibs are going to be less important once we start building with cargo:
the focus will move to crates instead, so that's what we should call the
moz.build object.
Through an oversight, we listed librul.a twice when linking libxul: once
as part of the "objects" we were linking, and once as a static library.
This duplication is unnecessary and would cause problems later when we
try to generate librul.a via cargo, as cargo will put it someplace
different from where we expect and the two names will conflict. Let's
have rules.mk be the single source of truth for how librul.a is named,
and then the code to link libxul can simply refer to that name.
This patch is really two separate changes.
The first change is that rust crates are large, standalone entities that
may contain multitudes of source files. It therefore doesn't make sense
to keep them in SOURCES, as we have been doing. Moving to use cargo
will require a higher-level approach, which suggests that we need a
different, higher-level representation for Rust sources in the build
system.
The representation here is to have the build system refer to things
defined in Cargo.toml files as the entities dealt with in the build
system, and let Cargo deal with the details of actually building things.
This approach means that adding a new crate to an existing library just
requires editing Rust and Cargo.toml files, rather than dealing with
moz.build, which seems more natural to Rust programmers. By having the
source files for libraries (and binaries in subsequent iterations of
this support) checked in to the tree, we can also take advantage of
Cargo.lock files.
The second is that we switch the core build system over to building via
cargo, rather than invoking rustc directly.
We also clean up a number of leftover things from the Old Way of doing
things. A number of tests are added to confirm that we'll only permit
crates to be built that have dependencies in-tree.
rlibs are going to be less important once we start building with cargo:
the focus will move to crates instead, so that's what we should call the
moz.build object.
Through an oversight, we listed librul.a twice when linking libxul: once
as part of the "objects" we were linking, and once as a static library.
This duplication is unnecessary and would cause problems later when we
try to generate librul.a via cargo, as cargo will put it someplace
different from where we expect and the two names will conflict. Let's
have rules.mk be the single source of truth for how librul.a is named,
and then the code to link libxul can simply refer to that name.
Mingw python has a different os.path setup from native python, and has
os.sep and os.altsep reversed. In that case, the normsep function was
doing the wrong thing, leading to all sorts of problems.
While fixing this, also ensure the corresponding unit test covers this
peculiarity, even when running under the native win32 python.
--HG--
extra : rebase_source : 8fb18e0d4dc669c1d7e069f73fc44c22d419d43c
This never was a problem since auto-logging was broken on Windows, but
having mach clobber auto-log is a problem on Windows since it tries to
remove the log file while itself has it open, which fails.
--HG--
extra : rebase_source : 6da9f3e6148c201eade01d4598ce9c4becd051a0
We were relying on the log manager's terminal to know whether we're
running on a terminal to turn on auto-logging, but the log manager's
terminal is always None on Windows because blessings imports curses,
which doesn't work on Windows.
So instead, just use a plain os.isatty() call.
The same problem also applies to the show-log command to trigger the
pager (less). At the same time, fix the environment setting for LESS,
as on Windows, unicode literals are not allowed in the environment.
--HG--
extra : rebase_source : 1462849608c8cb86afcb080184efb50caf6d2f9e
The best kind of patch: bulk deletion. This removes two separate but
similar build flags, and an unsupported integration piece. The first
packaged Fennec's resources into an ill-defined GeckoView archive; the
second built on the first to produce an Android ARchive for external
consumption. Neither of these artifacts are supported or have
consumers; in fact, they mislead potential consumers, since they're
known to be broken. The Gradle build work under the --with-gradle
flag, and significant follow-up, is the path forward if Mozilla wants
to invest in packaging GeckoView on Android for external consumption.
That is, rather than hacking together AAR files, we'll use the
well-supported Gradle mechanisms for building and publishing such
libraries.
MozReview-Commit-ID: 4Z1jJ8z0cyJ
--HG--
extra : rebase_source : b8e65f39c286313fe8963bf3832d9b6977ef188f
I have a need to create tar archives deterministically
and reproducibly. Since we already have similar functionality in
mozpack for producting zip/jar archives, I figured it made sense for
this functionality to live in mozpack.
I made the functionality as simple as possible: we only accept
files from the filesystem and the set of files must be known in
advance. No class to hold/buffer state: just a simple function
that takes a mapping of files and writes to a stream.
MozReview-Commit-ID: If0NTcA7wpc
--HG--
extra : rebase_source : 9cbea36347ba2840dd5bff9dffefd994a73a0725
Mike Shal
Carsten "Tomcat" Book
Phil Ringnalda
Matthew Gregan
Ted Mielczarek
Chris Manchester
Gregory Szorc
Henrik Skupin
Michael Layzell
Yen Chi Hsuan
Mike Hommey
Sebastian Hengst
Shing Lyu
Wes Kocher
Al Tsai
Xidorn Quan
Myk Melez
Ryan VanderMeulen
Elias Probst
Nathan Froyd
Ehsan Akhgari
Nick Alexander
Geoff Brown