Make the $PYTHON3 build var point to a full virtualenv bootstrapped with
the same libraries as the $PYTHON Python 2 build var. This allows us to
upgrade build tasks from $PYTHON to $PYTHON3.
This patch adds some debug logging and documentation to the Python
2 virtualenv so that it is easier to diagnose issues that may arise
from running two different Python interpreters in re-entrant
multiprocess routines.
Differential Revision: https://phabricator.services.mozilla.com/D50819
--HG--
extra : moz-landing-system : lando
Homebrew on OS X will change Python's sys.executable to a custom value
which messes with mach's virtualenv handling code. Override Homebrew's
changes with the correct sys.executable value.
Differential Revision: https://phabricator.services.mozilla.com/D54602
--HG--
extra : moz-landing-system : lando
Changes:
Remove `environment` from the python3 blacklist.
Remove the option to specify `configure` for the formatting argument, since it has never been supported (the method has never been implemented).
Differential Revision: https://phabricator.services.mozilla.com/D54395
--HG--
extra : moz-landing-system : lando
The FileAvoidWrite class does a bunch of stuff with bytes and strings
that doesn't work in Python 3. Make sure the object is handling only
bytes or only strings under the hood so Python 3 is happy.
The FileAvoidWrite unit tests written with MockedOpen don't work in
Python 3 either. Swap them out for vanilla pytest tests without
the MockedOpen dependency.
Differential Revision: https://phabricator.services.mozilla.com/D51341
--HG--
extra : moz-landing-system : lando
This will install ipython into the default virtualenv if it doesn't exist. Unless --no-virtualenv
is specified in which case an error will be printed.
Differential Revision: https://phabricator.services.mozilla.com/D53030
--HG--
extra : moz-landing-system : lando
`os.path.abspath` on Windows doesn't return a consistent value -- the capitalization can vary based on environmental factors that I don't fully understand. Regardless, normalize case where this function is called in config_status.
Differential Revision: https://phabricator.services.mozilla.com/D53889
--HG--
extra : moz-landing-system : lando
If configure is invoked manually, clobber.py is bypassed and the CLOBBER
file doesn't get touched. The clobber check in Makefile.in gets
triggered causing the build to stop.
Moving the objdir/CLOBBER file creation into configure.py should cause
it to be created regardless of how configure is invoked.
Depends on D53290
Differential Revision: https://phabricator.services.mozilla.com/D53291
--HG--
extra : moz-landing-system : lando
Bug 1596868 was triggered by forgetting a comma, turning an intended
tuple into a string, which Python dutifully iterated over as a sequence
of single characters. Let's add some basic typechecking so these sort
of mistakes don't happen again.
Differential Revision: https://phabricator.services.mozilla.com/D53815
--HG--
extra : moz-landing-system : lando
Now since we use unicode_literals by bug 1210157, mercurial version check will be error due to `TypeError: environment can only contain strings`.
So we don't use unicode for os.environment['PATH'] when appending Java path.
Differential Revision: https://phabricator.services.mozilla.com/D53139
--HG--
extra : moz-landing-system : lando
There's a lot of formatting that is irrelevant for Fluent,
and it's hard to diff usefully on the text file.
Instead, normalize the Fluent files by parsing them, and serialzing
them with Junk included.
Differential Revision: https://phabricator.services.mozilla.com/D53202
--HG--
extra : moz-landing-system : lando
The minidump-analyzer tool was originally conceived to be used from the crash
report client and as such was installed in the crash reporter client
application bundle on macOS. It was later adapted to work from Firefox itself
but this caused linking problems when invoked from the Firefox app bundle.
This patch moves the minidump-analyzer into the Firefox app bundle and adapts
the relevant code to find it there.
The minidump-analyzer was also not signed like the rest of our executables and
this patch addresses that issue too.
Differential Revision: https://phabricator.services.mozilla.com/D52910
--HG--
extra : moz-landing-system : lando
We need this for "full" C++17 support (everything is supported, but some
C++17 features still have bugs) and this change also brings Linux into
parity with our Mac requirements.
MANUAL PUSH: build toolchains on inbound to avoid clogging autoland
Differential Revision: https://phabricator.services.mozilla.com/D51450
This loader uses 'reader.find_variables_from_ast' to parse all *_MANIFESTS variables from
moz.build files using the abstract syntax tree. This means it will find all such variables
regardless of the current buildconfig.
Differential Revision: https://phabricator.services.mozilla.com/D51834
--HG--
extra : moz-landing-system : lando
Despite what the comment says, the finder *does* pick up the root moz.build. So
we end up processing it twice. This was never caught before because we only ever
used this function to read Sphinx related variables, of which the root moz.build
contains none.
Differential Revision: https://phabricator.services.mozilla.com/D51832
--HG--
extra : moz-landing-system : lando
This also moves the 'mach' docs from /python/mach to /mach. The reason being
that 'mach' doesn't really have anything to do with Python other than its
implemented in it.
Differential Revision: https://phabricator.services.mozilla.com/D52253
--HG--
extra : moz-landing-system : lando
Changes:
- use regular strings instead of byte strings when matching
- encode files in UTF-8 prior to hashing
Depends on D51414
Differential Revision: https://phabricator.services.mozilla.com/D51728
--HG--
extra : moz-landing-system : lando
Changes:
- change `urlparse` import to `urllib.parse` to make switch over to python3
Depends on D51414
Differential Revision: https://phabricator.services.mozilla.com/D51648
--HG--
extra : moz-landing-system : lando
Changes:
- update sections of `generate_sources_mozbuild.py` and `cmakeparser.py` to be python3 compatible
- change import of `urlparse` to be python3 compatible
Depends on D51414
Differential Revision: https://phabricator.services.mozilla.com/D51692
--HG--
extra : moz-landing-system : lando
The help view copies strings from the main menubar. When we moved the original DTD string
to ftl, there were performance implications for using it in browser.xhtml, so it was
only added once needed. The help view copies attributes from the items in the main menubar's
help menu, and so didn't copy the label for this item, resulting in the broken
behaviour.
To fix this, it's enough to have the string in the markup. As we've moved the other strings
into menubar.ftl, I'm taking the opportunity to move this string there, too, next to its
sibling string to report deceptive sites.
Differential Revision: https://phabricator.services.mozilla.com/D51850
--HG--
extra : moz-landing-system : lando
Add test annotations to the ReftestManifest and TestResolver. For example, a
test description from the TestResolver might now contain 'skip-if': 'skiaContent';
similar to the content provided for manifestparser tests; this will allow
'mach test-info report' to filter tests based on reftest manifest test
annotations.
Also add the referenced-test field which identifies the test file associated
with test entries for reference files; this will allow test-verify to
run the correct reftest when only the reference file has been modified.
Differential Revision: https://phabricator.services.mozilla.com/D51113
--HG--
extra : moz-landing-system : lando
This allows subclasses of MozbuildObject to define additional instance
arguments and still use 'from_environment'.
Differential Revision: https://phabricator.services.mozilla.com/D51174
--HG--
extra : moz-landing-system : lando
On Windows, './mach bootstrap' downloads 8 artifacts taking up 1GiB+ of
space. Given the current limits of 6 artifacts and 1GiB, this means a
full run of './mach bootstrap' doesn't actually use anything from the
cache. Each new artifact is over the limits, so the oldest artifact gets
removed from the cache before downloading the next one. Doubling the
limits here should give us some space to work with.
Depends on D50676
Differential Revision: https://phabricator.services.mozilla.com/D50677
--HG--
extra : moz-landing-system : lando
Artifacts are fetched to a local cache, and if the artifact in the cache
exists it won't be re-downloaded. However, the messaging suggested that
artifacts were always being downloaded when they were just re-used from
the cache, leading to confusion.
Differential Revision: https://phabricator.services.mozilla.com/D50676
--HG--
extra : moz-landing-system : lando
Separating these changes out into a separate patch/bug makes them a
little easier to verify. Turning on C++17 support should then be just a
matter of updating expected results.
Differential Revision: https://phabricator.services.mozilla.com/D47324
--HG--
extra : moz-landing-system : lando
Changes:
- remove the `unicode_literals` import from `test_archive.py` as it was causing failures when dealing with file paths
- mark failing tests with `xfail` annotations, to be investigated at a later date
Differential Revision: https://phabricator.services.mozilla.com/D49498
--HG--
extra : moz-landing-system : lando
Since bootstrap now pulls mozilla-unified it makes sense to reference the
unified repo as well as central.
Differential Revision: https://phabricator.services.mozilla.com/D49478
--HG--
extra : moz-landing-system : lando
This is needed to release a new mozlog with the PRECONDITION_FAILED
test and subtest status for use in web-platform-tests.
Update all in-tree dependencies on mozlog to >=5.0. These were found
with `hg grep 'mozlog.*[0-9]'`.
Only testing/web-platform/tests/tools/wptrunner/requirements.txt
remains on 4.2.0, and it will be updated in upstream wpt after mozlog
5.0 has been released.
Differential Revision: https://phabricator.services.mozilla.com/D50456
--HG--
extra : moz-landing-system : lando
Add code to clean up Git and Hg repositories, and invoke that where `./mach vendor rust` would throw an error. Unfortunately, `cargo vendor` also updates the repo's root `Cargo.lock` file in-place and while we could `git checkout`/`hg revert` that file for the user, `Cargo.lock` may have had pre-existing changes that would be overwritten by such a change. Instead of a potentially destructive update in the error case, I've opted to add an extra error message to tell you how to deal with that one file.
Differential Revision: https://phabricator.services.mozilla.com/D49494
--HG--
extra : moz-landing-system : lando
Changes:
- remove `uuid` from the python3 blacklist in `mach`
- enable `test_telemetry.py` for python3
- adjust test outcome expectation for Windows + python3, suspect `mozpack.path` is not filtering Windows path correctly in python3 environment
- switch file read mode between `r` and `rb` depending on version of python
Differential Revision: https://phabricator.services.mozilla.com/D45903
--HG--
extra : moz-landing-system : lando
The BuildReader's 'find_sphinx_variables' function is hardcoded to look for the
SPHINX_TREES and SPHINX_PYTHON_PACKAGE_DIRS variables. But it's otherwise
implemented to find any arbitrary variable (as long as they are a simple list
or dict).
This change simply moves the variable name(s) to the function call. The comment
about possibly wanted to use a higher level AST library to parse other kinds of
variables still applies, but for now this change is good enough to suit my
needs.
Differential Revision: https://phabricator.services.mozilla.com/D48424
--HG--
extra : moz-landing-system : lando
Fennec is removed, but mach install still tries to install Fennec. It should
install GVE instead.
Differential Revision: https://phabricator.services.mozilla.com/D48811
--HG--
extra : moz-landing-system : lando
Some generated files are missing in generated-sources.json so just add them.
Differential Revision: https://phabricator.services.mozilla.com/D48085
--HG--
extra : moz-landing-system : lando
This leaves the testharness files, because they are used in various mochitests.
Differential Revision: https://phabricator.services.mozilla.com/D49132
--HG--
extra : moz-landing-system : lando
This leaves the testharness files, because they are used in mochitest-chrome
tests in dom/animation/test.
Differential Revision: https://phabricator.services.mozilla.com/D49132
--HG--
extra : moz-landing-system : lando
This leaves the testharness files, because they are used in mochitest-chrome
tests in dom/animation/test.
Differential Revision: https://phabricator.services.mozilla.com/D49132
--HG--
extra : moz-landing-system : lando
Some fields of /etc/os-release are optional, so do not throw KeyError's when they're missing.
Differential Revision: https://phabricator.services.mozilla.com/D48984
--HG--
extra : moz-landing-system : lando
Māris Fogels
Botond Ballo
Edwin Takahashi
Mark Striemer
Cosmin Sabou
Andrew Halberstadt
Zibi Braniecki
Nathan Froyd
Ricky Stewart
Chris Manchester
Geoff Lankow
Mike Shal
Makoto Kato
Alexander Surkov
Andi-Bogdan Postelnicu
Axel Hecht
Tom Schuster
Gabriele Svelto
Razvan Maries
Sylvestre Ledru
Jesse Schwartzentruber
Nick Thomas
Kirk Steuber
Csoregi Natalia
Tom Prince
Gijs Kruitbosch
Geoff Brown
Bryce Seager van Dyk
Philip Jägenstedt
Anthony Ramine
Marco Castelluccio
Dorel Luca
Wes Kocher
Francesco Lodolo (:flod)
Calixte Denizet
Edouard Oger
Ms2ger
shindli
Kenny Levinsen