This patch allows us to enable Python 3 tests and skip any tests that fail so that we can work on adding support for Python 3 without risking regressing any existing support. It will also eventually allow us to skip tests from running against Python 2, when we decide to drop support for it. To skip a test against Python 3, add "skip-if = python == 3" to the [DEFAULT] or test file section of a manifest file.
MozReview-Commit-ID: KYzjW6PQw2Q
--HG--
extra : rebase_source : 9e0670efe237b8953aab96e95cfa3c9ae678351c
Bug 1467041 changed the default for --enable-release such that it is
affected by the milestone. Test both possible cases for milestones.
--HG--
extra : rebase_source : 5bfeadf4cd1e8c672cf78e4922ebb91bdafecad8
90dca0906337 accidentally broke `mach artifact toolchain --from-build`
because that code is attempting to load toolchain tasks in isolation.
The new "use_fetches" transform added to toolchain tasks requires
that "fetch" tasks are already processed and their references are
available to toolchain tasks.
This commit adds a mechanism to effectively disable the "use_fetches"
transform when called by `mach artifact toolchain`. It is a hack. I
suspect future planned work around artifacts/fetches will necessitate
additional changes to the `mach artifact toolchain` code. But this
can be deferred to a later day: this commit unbusts `mach artifact
toolchain` and isn't super hacky, so it seems more reasonable than
backing out fetch tasks completely.
Differential Revision: https://phabricator.services.mozilla.com/D1588
This populates $OBJDIR/dist/host/bin as part of |mach artifact install|.
Conceptually, the mar and mbsdiff utilities should be grouped (in the
same way that the test-related binaries are grouped). However, it's
difficult to achieve that with the current structure of the code, so
this fetches mar and mbsdiff and produces $HASH-mar.processed.jar and
$HASH-mbsdiff.processed.jar files.
MozReview-Commit-ID: 3ks5xsUEKp5
--HG--
extra : rebase_source : 5fcf186decc95537cbaa90ffedb86774eab050d2
This upgrades sphinx to version 1.7.5, which contained a couple backwards
incompatible changes that needed fixing.
This also leaves sphinx-js at version 2.1 as upgrading that to 2.5 seems to
introduce an intermittent in the Doc task.
MozReview-Commit-ID: FRUTcXs5yzb
--HG--
extra : rebase_source : e874a2e9c637b7cec710203f75f4dd989a5681a1
This patch allows executing |mach python-test| against Python 3 by specifying the optional |--three| command line option. When this option is present, pipenv will be used to manage a virtual environment using Python 3 and attempt to run the tests. When it is not present, pipenv will not be used, and everything will work as it did before this patch.
My original plan was to use pipenv regardless of the target version of Python, however I encountered several issues running some of the tests against our Python packages. Once all tests have been patched to run against Python 3, then we should be able to use pipenv when running them against Python 2.
Note that this patch allows tests to run against Python 3, but there are plenty of issues preventing them from passing. With this patch in place we can start to add Python 3 support to our packages and have the tests running in CI to ensure we don't regress back to just supporting Python 2.
MozReview-Commit-ID: BuU5gZK83hL
IHG: changed taskcluster/ci/source-test/python.yml
--HG--
extra : rebase_source : ca2b15d905f7a5c895a2fd8916144841f5d205de
This patch allows executing |mach python-test| against Python 3 by specifying the optional |--three| command line option. When this option is present, pipenv will be used to manage a virtual environment using Python 3 and attempt to run the tests. When it is not present, pipenv will not be used, and everything will work as it did before this patch.
My original plan was to use pipenv regardless of the target version of Python, however I encountered several issues running some of the tests against our Python packages. Once all tests have been patched to run against Python 3, then we should be able to use pipenv when running them against Python 2.
Note that this patch allows tests to run against Python 3, but there are plenty of issues preventing them from passing. With this patch in place we can start to add Python 3 support to our packages and have the tests running in CI to ensure we don't regress back to just supporting Python 2.
MozReview-Commit-ID: BuU5gZK83hL
IHG: changed taskcluster/ci/source-test/python.yml
--HG--
extra : rebase_source : e1a64c0ffa8fe5cce71a041579601d4a72e37779
Creates a standard library for working with moz.yaml files, and adds a `mach
vendor manifest --verify` command that loads and verifies manifest schema.
The list of permitted licenses is one I derived from about:license, pending an
authoritative list from legal.
Differential Revision: https://phabricator.services.mozilla.com/D1208
The crash reporter symbol files are the easiest cross-platform way to
find static initializers. While some types of static initializers (e.g.
__attribute__(constructor) functions) don't appear there in a notable
way, the static initializers we do care the most about for tracking do
(static initializers from C++ globals). As a matter of fact, there is
only a difference of 2 compared to the currently reported count of 125
on a linux64 build, so this is a good enough approximation. And allows
us to easily track the count on Android, OSX and Windows builds, which
we currently don't do.
The tricky part is that the symbol files are in
dist/crashreporter-symbols/$lib/$fileid/$lib.sym, and $fileid is hard to
figure out. There is a `fileid` tool in testing/tools, but it is a
target binary, meaning it's not available on cross builds (OSX,
Android).
So the simplest is just to gather the data while creating the symbol
files, which unfortunately requires to go through some hoops to make it
happen for just the files we care about.
--HG--
extra : rebase_source : 458fed1ffd6f9294eefef61f10ff7a284af0d986
This is slightly ugly, but is unfortunately necessary due to do the nature of
l10n repacks. Hopefully this can go away once we move to bundling lancpack
add-ons rather than repacking in the future.
MozReview-Commit-ID: JZUblVsEbZI
--HG--
extra : rebase_source : 60c9ced2184a52f52c7f2a8820021b14b1a66abf
This one looks to be pretty straight-forward. It irritates me that
the jar.mn entry doesn't explicitly say that the result is coming from
the object directory, like
locale/browser/bookmarks.html (!bookmarks.html)
but that's for another day.
MozReview-Commit-ID: Cw8E0VJhSxv
--HG--
extra : rebase_source : a1045a5b564b0094b562729bc7234e69ec7a786d
To vendor a Python package, run ``mach vendor python [PACKAGE]``, where
``[PACKAGE]`` is one or more package names along with a version number in the
format ``pytest==3.5.1``. The package will be installed, transient dependencies
will be determined, and a ``requirements.txt`` file will be generated with the
full list of dependencies. The requirements file is then used with ``pip`` to
download and extract the source distributions of all packages into the
``third_party/python`` directory.
If you're familiar with ``Pipfile`` you can also directly modify this in the in
the top source directory and then run ``mach vendor python`` for your changes
to take effect. This allows advanced options such as specifying alternative
package indexed (see below), and
`PEP 508 specifiers <https://www.python.org/dev/peps/pep-0508/>`_.§
MozReview-Commit-ID: CRWoFamUy7V
--HG--
extra : rebase_source : e71ab6ecc74a168faa9df7cc4c2273cd8bf9e0d2
To vendor a Python package, run ``mach vendor python [PACKAGE]``, where
``[PACKAGE]`` is one or more package names along with a version number in the
format ``pytest==3.5.1``. The package will be installed, transient dependencies
will be determined, and a ``requirements.txt`` file will be generated with the
full list of dependencies. The requirements file is then used with ``pip`` to
download and extract the source distributions of all packages into the
``third_party/python`` directory.
If you're familiar with ``Pipfile`` you can also directly modify this in the in
the top source directory and then run ``mach vendor python`` for your changes
to take effect. This allows advanced options such as specifying alternative
package indexed (see below), and
`PEP 508 specifiers <https://www.python.org/dev/peps/pep-0508/>`_.§
MozReview-Commit-ID: CRWoFamUy7V
--HG--
extra : rebase_source : e71ab6ecc74a168faa9df7cc4c2273cd8bf9e0d2
This relies on the fact that providing multiple --version-script
combines them all, so we effectively create a new symbol version
that has no global symbol, but hides the std:🧵:_M_start_thread
symbols.
This version script trick happens to work with BFD ld, gold, and lld.
The downside is that when providing multiple --version-script's, ld
doesn't want any of them to have no version at all. So for the libraries
that do already have a version script (through SYMBOLS_FILE), we use a
version where there used to be none, using the library name as the
version. Practically speaking, this binds the libraries a little closer
than they used to be, kind of non-flat namespace on OSX (which is the
default there), meaning the dynamic linker will actively want to use
symbols from those libraries instead of a system library that might
happen to have the same symbol name.
--HG--
extra : rebase_source : a7f672c35609d993849385ddb874ba791b34f929
We wrap STL headers so as to ensure our inline infallible operator new
definitions are used, but for some reason, STL wrappers were disabled
in whatever code we used gyp for first, and that ended up spreading as
more things used gyp, and now with gn, whose handling cargo culted
STL disabling from the gyp code.
We remove all this disabling, because it turns out it causes no build
problem anymore.
--HG--
extra : rebase_source : 0dd5dc3dbf613a519db74072ebda04d05ea1214c
Dave Hunt
assiya
Chris Manchester
shindli
Peter Simonyi
Sofia Carillo
Mike Hommey
Gregory Szorc
Ciure Andrei
Nick Alexander
Andrew Halberstadt
Andreea Pavel
Csoregi Natalia
Francesco Lodolo (:flod)
Chris AtLee
Dorel Luca
byron jones
Kris Maglione
Gurzau Raul