Add a check to `mach bootstrap` that a compatible version of
the rust toolchain is installed and in path. Note that we use
a separate minimum version from the one checked at configure
time, defined in build/moz.configure/rust.configure, because
this script may be running stand-alone.
If we don't find `rustc` in PATH, we check for it in CARGO_HOME
and suggest adding that to PATH.
If we don't find `rustc` but find a `rustup`, we call it with
the --version switch and report if that fails. This will detect
the older `rustup.rs` script.
MozReview-Commit-ID: EPfQhLz4Dvo
--HG--
extra : rebase_source : 2ea4acdfbfdc2a436f386eae7bc3cbcbc485aa1b
Also use the abstracted helper method for reading the current
mercurial version too. This changes the regex from what was in
use before, but should work for normal version numbers.
MozReview-Commit-ID: IZfC65Jg6T8
--HG--
extra : rebase_source : d61a73b7b1b438d8c846523e5e1f639950fe5473
Move version parsing to a helper method so it can be used
for more than one executable.
MozReview-Commit-ID: 4gOgdgYFbFx
--HG--
extra : rebase_source : 944f562c0d5a6a105a0c27af6f4d7dfc214f3c01
This will allow tests, etc to conditionally run depending on Stylo's
presence.
MozReview-Commit-ID: 3WHxNawP1qC
--HG--
extra : rebase_source : d4d983d37d3bfac1f9d465a3023b5bd5df52cd56
extra : source : 6baa88f7b1f4183ea57b9f8928556975042cad29
The build system's TestResolver does a pretty good job of getting the right manifestparser
based tests to run, but it isn't perfect. Notably, it ignores the 'disabled' key. We filter
the tests through manifestparser here to make sure the build system didn't miss anything.
For context, this is also what the other test harnesses (e.g mochitest) do as well.
MozReview-Commit-ID: FaHb4nvuoK9
--HG--
extra : rebase_source : 476b7068c3ddf7d9757d605de5843e21547c6c33
This deprecates PYTHON_UNIT_TESTS and replaces it with PYTHON_UNITTEST_MANIFESTS.
In the build system, this means python unittests will be treated the same as all
other test suites that use manifestparser. New manifests called 'python.ini' have
been created for all test directories containing python unittests.
MozReview-Commit-ID: IBHG7Thif2D
--HG--
extra : rebase_source : 11a92a2bc544d067946bbd774975140147458caa
This fixes a UnicodeDecodeError when sys.stdout's encoding can't handle unicode.
MozReview-Commit-ID: 3INna8MRje5
--HG--
extra : rebase_source : bb8f3b5cbdbb6601b6946d5fbb7396848f22b5c6
This patch does a few things:
1) Change all the in-tree tooltool manifests to contain sccache2 instead of the existing Python sccache
2) Change mozconfig.cache to point at sccache.
3) Lightly tweak the --with-cccache configure option to support sccache, and detect whether we're using ccache or sccache and set an option appropriately.
4) Add a MOZ_SCCACHE_VERBOSE_STATS option, and add a target in the top-level Makefile to make sccache spit out its stats at the end of the build. This is useful to see the cache hits/errors until we get something better.
5) Add MOZ_USING_SCCACHE to the build telemetry. Not that I think it will be immediately useful, but for future use.
MozReview-Commit-ID: 9lrdLwNj5Bm
--HG--
extra : rebase_source : d323457df10d0ee0ac5811940e518d9422a7e070
In order for VS2017 to recognize solution and project files as
belonging to VS2017 and not some earlier version, we need to bump
a few versions strings.
The added code path can't be hit yet because MSVS_VERSION cannot
be "2017" until configure detects VS2017. I hacked up MSVS_VERSION
locally and verified that solutions produced with the new code load
in VS2017.
FWIW, the solution loads significantly faster in VS2017 compared to
VS2015!
MozReview-Commit-ID: 2vYAgd6wOsG
--HG--
extra : rebase_source : a29b956bb6eef9071cb46d30460a42ef17658026
This replaces the "return_code" property on the LintRoller object with a list of "failed"
linters. This is a bit more useful as it lets us know exactly which linters had a problem
(whereas previously we just knew *something* went wrong). This patch pushes determining
the return code back into cli.py, which I think is fine.
In addition, we now pass the list of failed linters into the formatter. This allows us to
clarify exactly how many linters hit a failure which is a lot better than a seemingly
"successful" summary message.
Finally I also removed the "no files to lint" message because I've seen several people
confuse it for an error. I'll probably add it back as a debug log message when we switch
to using mozlog for output.
MozReview-Commit-ID: 4wyCeOZdOf8
--HG--
extra : rebase_source : 23810a8ab8dd9cbbad6b9e965ccff7214f947fbc
This patch contains a number of changes to the gyp_reader code:
* Add three new flags to GYP_DIRS:
** no_chromium, to skip forcing the includes/etc needed for chromium gyp files
** no_unified, to force building all sources without unification
** action_overrides, to pass scripts used when mapping gyp actions to moz.build GENERATED_FILES
* Handle the flags mentioned above in read_from_gyp
* Handle actions in gyp targets by mapping them to GENERATED_FILES, using scripts specified in the action_overrides flag. We don't try to handle the generic action case, we require special-casing for each action.
* Handle a subset of copies in gyp targets by mapping them to EXPORTS, just enough to handle the use of them for NSS exports.
* Handle shared_library and executable gyp targets
* Handle gyp target dependencies/libraries as USE_LIBS/OS_LIBS
* Handle generated source files
* Handle .def files in sources by mapping them to SYMBOLS_FILE
* Special-case some include_dirs:
** Map `<(PRODUCT_DIR)/dist/` to $DIST/include (to handle include paths for NSS exports)
** Map include_dirs starting with topobjdir to objdir-relative paths, to handle passing the NSPR include path to NSS
* split /build/gyp.mozbuild into two parts, with gyp_base.mozbuild containing generic bits, and gyp.mozbuild containing chromium-specific bits
MozReview-Commit-ID: FbDmlqDjRp4
--HG--
extra : rebase_source : d3fb470c589f92d8c956b9ecd550fb8df79ff5bc
We recently switched make check to call into |mach python-test| rather than invoking python
itself for each test file. But this ended up slowing down the tests as they were no longer
being run in parallel. This patch adds a --jobs flag to python-tests and runs test files in
parallel.
Note: if more than one job is used, output per test will be buffered and printed at the end
to avoid interleaving. This has the unfortunate side effect of making |mach python-test| look
like it is hanging, especially if running a very long file like mozbase's test.py. For this
reason, we still use -j1 by default so output will continue to be streamed. In automation we
will use multiple processes though.
MozReview-Commit-ID: 3u0wOFmyQLI
--HG--
extra : rebase_source : d08ac412023731c46226c7adbf5f6e798b9a345a
We need the fix from https://github.com/agronholm/pythonfutures/issues/25
to allow sending KeyboardInterrupts to thread pools.
MozReview-Commit-ID: 5VfBttLbKOr
--HG--
extra : rebase_source : 8e3aa7a1d6fbbaa7b94cea35b196b35e103a1e33
Some mach commands may want to re-use a requirements.txt file rather than installing packages
individually. This enables --require-hashes which means all packages and dependencies must be
listed with their hashes. For more details, see:
https://pip.pypa.io/en/stable/reference/pip_install/#hash-checking-mode
MozReview-Commit-ID: 3lOutbcSzIY
--HG--
extra : rebase_source : d07ac21bd1f2f0009465f9e004208464b22de01b
This is a most minimal gtest conversion possible. It leaves in place
significant amounts of non-typical-for-gtest code.
Notable changes:
- All the mock Link and URLSearchParams method definitions are no longer
needed.
- The changes adds a new constructor for Link that doesn't set mHistory.
Without that, leaked URLs occur at shutdown.
- The output printed by the test is slightly streamlined, mostly by omitting
the test filename.
- It disables TestMediaFormatReader.cpp, which was causing problems. That test
is slated for removal in bug 1318225 anyway.
--HG--
rename : toolkit/components/places/tests/cpp/mock_Link.h => toolkit/components/places/tests/gtest/mock_Link.h
rename : toolkit/components/places/tests/cpp/moz.build => toolkit/components/places/tests/gtest/moz.build
rename : toolkit/components/places/tests/cpp/places_test_harness.h => toolkit/components/places/tests/gtest/places_test_harness.h
rename : toolkit/components/places/tests/cpp/places_test_harness_tail.h => toolkit/components/places/tests/gtest/places_test_harness_tail.h
rename : toolkit/components/places/tests/cpp/test_IHistory.cpp => toolkit/components/places/tests/gtest/test_IHistory.cpp
extra : rebase_source : b7def3f9afce3a44e99f5ed35cb220f7814551cd
On Windows, the rustup installer doesn't create ~/.cargo/env
but instead pokes .cargo/bin into the path in the Windows
registry. This doesn't necessarily propagate to the msys
evironment however, so some PATH manipulation may still
be necessary.
Split our path advice to be clear in both the new install
and unconfigured path cases, and amend our path update
advice to not mention .cargo/env if it isn't present.
MozReview-Commit-ID: 9IHhS6UYCqq
--HG--
extra : rebase_source : 898615106078882f335385ac0b50eff1612377f0
The current state of python configure sandbox execution is that if a
template imports a module, and a function defined in the template tries
to use the module, it doesn't work. Ideally, it would, but rather than
try to fix this corner case, we remove the unit tests that assume it
works (and consequently pass for half bad reasons), and add a unit test
so that the behavior doesn't change unwillingly.
--HG--
extra : rebase_source : 579ba2bc7c19d4fe7df11bbdb1ceb6171a1ee857
The way it works now, `mach` commands often invoke subprocesses where
the subprocesses' stdio file descriptors are pipes so the mach command
can e.g. parse output.
Processes like clang, gcc, and cargo determine if they can send color
codes to {stderr, stdout} by seeing if those file descriptors are TTYs.
When e.g. `make` is executed via `mach`, this test fails because those
descriptors are pipes (even though they eventually end up on a TTY).
We can't wire the file descriptors to the TTY because `mach` needs
to analyze output. We don't want users defining process flags to force
color in their mozconfigs because color codes would still be sent
if stdout was not a TTY.
This patch sets the MACH_STDOUT_ISATTY environment variable in all mach
commands when stdout is a TTY. Subsequent processes can then look for
this variable to determine whether to override color settings, print
terminal control codes, etc.
MozReview-Commit-ID: GxXP2mQssjC
--HG--
extra : rebase_source : 4b99547b453cb7dd5cb590a71ed554ce2bc4759d
Currently, environment variables set when running mach commands will
propagate after the command is finished. This can allow unwanted state
to bleed through.
This likely isn't an issue today, but isolating state during code
execution is generally a good practice. So do that.
MozReview-Commit-ID: AdaomGub5EF
--HG--
extra : rebase_source : ce81987a1f6de3a16bce6a9e45b9dc8e8eb29b4b
Also allow when=True/False to avoid the chicken-egg problem of using
a generic `when` to use in replacement of @depends('--help') for things
like @dependable.
--HG--
extra : rebase_source : f1571a5b904efb66a361b90f3b7e1edbaa75772e
--help dependencies currently help identify functions that will run when
running configure --help, which we don't want to have spreading too
much. OTOH, when such functions have no side effect, it's not really
that important to have them explicitly marked.
So, allow missing --help dependencies for functions that:
- don't use @imports
- don't have a closure
- don't use global variables
This is a first step towards entirely removing the --help markings (the
end goal being that --help dependencies will indicate actual --help
dependencies). As such, we don't really care about updating the lint
error message.
--HG--
extra : rebase_source : e81ec9b51ff01c2ee75722904e551286aa0b2bec
We want functions without an @imports to not have any side effects, and
to not use external resources. So remove the few functions we expose from
os.path without @imports('os') that do.
--HG--
extra : rebase_source : a9485ec269d4de5785d66d7772eda4fae5a84b4a
Missing such dependencies shouldn't impair running configure itself
after local modifications, but they are currently required for
(mostly) documentation purpose. Which means they are better done in
the linter.
--HG--
extra : rebase_source : 6bfff2342cda2ed1351f561c9eb9623f1fb4e4c4
Previously if an Arch Linux user had a different package extension configured
in `/etc/makepkg.conf` building AUR packages during bootstrap would fail.
Forcing the extension by providing it as an environment variable makes sure
building doesn't fail regarding of a user's configuration.
MozReview-Commit-ID: 4aryYS0XVr7
--HG--
extra : rebase_source : 4c466e49f729de625e814a92325c6d38e6d1e0b4
Both BuildMonitor and BuildOutputManager are doing similar things and
wrapping the build with forms of monitoring. These classes are only
used in the `mach build` command.
Since the BuildOutputManager takes an instance of BuildMonitor as an
argument and since BuildOutputManager behaves as a context manager,
it makes sense for BuildOutputManager to control the lifetime of
BuildMonitor's collection.
So, we teach BuildOutputManager's context manager to start the
BuildMonitor and ensure its resource collection is stopped when it
exits.
MozReview-Commit-ID: 3l9Hwz0Xe7o
--HG--
extra : rebase_source : 600b0231fce050465bdccb2bc11680f77bbc35d9
r?gps for the build changes and ochameau for the rest.
This results in about a 28% speed-up for Jetpack mochitest runs, for me.
MozReview-Commit-ID: K30q7BfvTLs
--HG--
extra : rebase_source : 8074947062c73a329f1d8a255a9601147ba1ba0c
For devtools-html, devtools will progressively stop using DTD files and will use only properties
files. To avoid retranslating already localized strings in every locale, this script can be used
by localization teams to automatically migrate strings moved in the scope of the devtools-html
project.
MozReview-Commit-ID: KNDfCoPXOM9
--HG--
extra : rebase_source : 48fbf2ae3daab89891262e460c979b6c141a2caa
ccache 3.3 added two fields in the status.
1. "cache hit rate"
https://github.com/ccache/ccache/issues/94
2. "cleanups performed"
0769ea3630
We just teach the parser the two fields. Nothing else.
MozReview-Commit-ID: 9kZrDrpCopd
--HG--
extra : rebase_source : ecb55fa9187abb37e668f0d25b40afd41597f7f8
If a linter returns a status code instead of a list of results, mozlint will throw away
that status code and move on to the next linter. We should make sure that status code
bubbles all the way up to the cli.
MozReview-Commit-ID: 9fXpmtlUUT1
--HG--
extra : rebase_source : 84c80d1b24d7233d0c4c6e38b23e775110776d1f
In our current Rust world, we have the following dependency structure:
xul.so --------------------------+
|
xul-gtest.so -+--> xul.a --------+-> gkrust
|
+--> gkrust-gtest
This structure results in link errors with multiply-defined symbols
between gkrust-gtest and gkrust with newer Rust releases when linking
xul-gtest.so. So we have to do something different.
Our new structure is:
xul.so --------------------------+
|
xul-gtest.so -+--> xul.a --------+-> gkrust --+-> gkrust-shared
| |
+--> gkrust-gtest --------------+
and we enforce that a given shared library can only have at most one
Rust library that it depends on. Said Rust library is assumed to
include all significant Rust dependencies of the dependent static
libraries as well. (In the above structure, gkrust is simply a wrapper
around gkrust-shared, so gkrust-gtest doesn't have to include gkrust as
a dependency.)
This moves reading of gyp files to a separate process from the main
reader/emitter pipeline, causing gyp contexts to be consumed after other
contexts.
MozReview-Commit-ID: CevPcpe5cFI
Due to an apparent bug, DIRS are being ouptut inconsistently for directories
with code built by gyp files (as of this writing, they are output for child
directories of media/webrtc but not for some other directories). The DIRS
variable no longer drives compilation, so this was essentially a no-op.
MozReview-Commit-ID: IMfjyrcsWyv
This generalizes the monkey-patching of the main module added
in bug 914563 to allow multiprocessing to be used from config.status
(which triggers the same bug as when it's used with `mach` as main).
MozReview-Commit-ID: AdOdpKzmbsp
This patch allows specifying an objdir path in `SYMBOLS_FILE`, with the
requirement that the file is also listed in `GENERATED_FILES`. This is used
for handling NSS' .def files with their special processing.
I added tests for the existing `SYMBOLS_FILE` case as well as the new case
and a test for the error if the file is not listed in `GENERATED_FILES`.
MozReview-Commit-ID: 3cVbKplltfb
--HG--
extra : rebase_source : 0ae4180dbe9383b09f14554bfda8aec20d7cc094
Now that we access WPT related files from a source checkout, we no
longer need the web-platform tests zip file produced or consumed by
automation. So stop producing it.
MozReview-Commit-ID: Ea8KjKZJ5Yx
--HG--
extra : rebase_source : f22506a02fdd5e78434cdc5d1c1f274db1cd04e2
When attempting to run mach from a make target, there were failures on OSX due to an objdir
mismatch:
https://dxr.mozilla.org/mozilla-central/rev/7c576fe3279d87543f0a03b844eba7bc215e17f1/python/mozbuild/mozbuild/base.py#656
The two detected objdirs were "<objdir>" and "<objdir>/x86_64". I believe this should not
count as a mismatch, otherwise it would not be possible to run a mach command from the
x86_64 directory.
MozReview-Commit-ID: CXDEABNWX8L
--HG--
extra : rebase_source : 9daeaf372d872921e7b2854eee0313b5adc0e707
This command is mostly just a wrapper around `cargo vendor`, but it runs
it the right way and will install the cargo-vendor tool if necessary.
Additionally, the mach command will by default error if there are uncommited
changes to files other than Cargo.{toml,lock} in your working copy, and it
will stage changes to the vendored crates in your VCS after the update.
MozReview-Commit-ID: 2fMDBawNUMO
--HG--
extra : rebase_source : fe72f26aec795eb663c554933432e700ac5089c3
I wanted to be able to do some VCS interaction from a mach command, and we
didn't have anything suitable, so I tore up mozversioncontrol and replaced
it with a framework to hang new features off of. I've only implemented
the bits I need currently (get_modified_files and add_remove_files),
but it should be straightforward to add more functionality there.
This patch also adds a `repository` property to `MozbuildObject`, which will
return a `Repository` object for the topsrcdir to make using these helpers
even easier for `MozbuildObject`-derived classes.
MozReview-Commit-ID: Gw6Ixp1ltiN
--HG--
extra : rebase_source : e619d6642eb86c3f96e679ac22a3e561dfdbb56a
This is a little tricky since tup expects rules in a single Tupfile to
be topologically sorted, and the emitter emits the GeneratedFile object
before it emits the FinalTargetPreprocessedFile object. We work around
this by delaying when we process the application.ini.h GeneratedFile
object, but it's a hack.
MozReview-Commit-ID: 8nVm76nZOKb
--HG--
extra : rebase_source : a82cfbeef739aac791998fb6d45cf88eb7116e32
Now that we access WPT related files from a source checkout, we no
longer need the web-platform tests zip file produced or consumed by
automation. So stop producing it.
MozReview-Commit-ID: Ea8KjKZJ5Yx
--HG--
extra : rebase_source : ee6ec00689696a710faf390d3dec5c5d02d8ec74
Move packaging for Marionette from make to test_archiver by using root manifest files.
MozReview-Commit-ID: 1cxEBYQeJ2Z
**
--HG--
extra : rebase_source : 372a407d9207bfbccbfb88c689df60b8cc1abcaf
Ralph Giles
Gregory Szorc
Carsten "Tomcat" Book
Chris Peterson
Andrew Halberstadt
Ted Mielczarek
Gabriele Svelto
Mike Shal
Ryan VanderMeulen
Phil Ringnalda
Kris Maglione
aleth
Wes Kocher
Benjamin Smedberg
Dan Minor
Henrik Skupin
Iris Hsiao
Matthew Gregan
Sebastian Hengst
Mike Hommey
Nicholas Nethercote
Axel Hecht
Kim Moir
Trevor Saunders
Chris Manchester
Dustin J. Mitchell
Eugen Sawin
Leo Schwarz
Francesco Lodolo (:flod)
Grisha Kruglov
Julian Descottes
Ting-Yu Lin
Nathan Froyd
Kartikaya Gupta
Michael Kaply
Xidorn Quan
Michael Layzell
Yen Chi Hsuan