This switches most tests over to use pytest as the runner instead of unittest (taking
advantage of the fact that pytest can run unittest based tests).
There were a couple tests that had failures when swithing to pytest:
config/tests/unit-expandlibs.py
xpcom/idl-parser/xpidl/runtests.py
For these tests, I added a runwith='unittest' argument so that they still run the
same way as before. Once we fix them to use pytest, the unittest logic in mozunit.py
can be deleted.
MozReview-Commit-ID: Gcsz6z8MeOi
--HG--
extra : rebase_source : 3c762422ce0af54cbbe7d9fc20085a2d1ebe7057
This is a quick and dirty hack to get treeherder to show pytest failures. Long term, we might
want to investigate using something like pytest-mozlog. But the benefit of this approach is
we get to keep pytest's fantastic default log format.
MozReview-Commit-ID: Gcsz6z8MeOi
--HG--
extra : rebase_source : 00ee7973eadf86c081b548d5e79c48ca951e25a6
We set up the temporary directory here so that it persists across multiple test invocations, as each
test is run in its own subprocess.
MozReview-Commit-ID: 7SNip54AqEI
--HG--
extra : rebase_source : 22a6eccf3ce63d23217d4551ee45e72efe63aa9a
Because test_objects was a generator, using it in the condition always returned True,
even if no tests were found. But extending test_objects to the manifest, converts it
to a list. So this patch simply moves the 'no tests' check a bit later on.
MozReview-Commit-ID: JpETWD1WQWH
--HG--
extra : rebase_source : 874d7a17d4dfa4a9de4f9daf54b51d1763d55044
This formats the marionette-harness python tests to be a regular |mach python-test| suite. Though
we add subsuite=marionette, this is just for automation purposes. The new preferred way to run the
marionette harness tests locally is:
./mach python-test testing/marionette
They will also run if running the full suite.
The mozbase packages.txt file modifies mozlog to use 'setup.py' instead of 'pth'. The reason for
this is that the marionette-harness tests use the pytest_mozlog pytest plugin for formatting
their results (converts pytest format into something resembling the standard tbpl logging format).
In order for this plugin to get picked up however, mozlog's setup.py file needs to be processed.
MozReview-Commit-ID: Ata99evHxbd
--HG--
extra : rebase_source : 22382e3d65ce8454a1682cfced0d03477762e8fe
This formats the marionette-harness python tests to be a regular |mach python-test| suite. Though
we add subsuite=marionette, this is just for automation purposes. The new preferred way to run the
marionette harness tests locally is:
./mach python-test testing/marionette
They will also run if running the full suite.
The mozbase packages.txt file modifies mozlog to use 'setup.py' instead of 'pth'. The reason for
this is that the marionette-harness tests use the pytest_mozlog pytest plugin for formatting
their results (converts pytest format into something resembling the standard tbpl logging format).
In order for this plugin to get picked up however, mozlog's setup.py file needs to be processed.
MozReview-Commit-ID: Ata99evHxbd
--HG--
extra : rebase_source : 16ed70edd38a53c3279d8632d7cba3df4d5216c3
This adds the ability to use manifestparser subsuites to |mach python-test|.
Subsuites are based on the premise of a "default" set that gets run when no
subsuites are explicitly specified. When a test is labelled with a subsuite,
that test is removed from the default set and will only run if that subsuite
is explicitly specified. This will allow us to chunk python unittests out of
'make check' piecemeal. The default set will run in 'make check', and
individual tasks (e.g mozbase), will specify a subsuite explicitly.
The |mach python-test| implementation is slightly different. By default,
subsuites are not considered if developers do not pass in --subsuite. This
means running |mach python-test| without arguments will still run the full set
of tests, and similarly, passing in test paths will *just work*.
If for some reason a developer needs to actually run the default set, a special
"default" subsuite has been create, so they can use
|mach python-test --subsuite default|. This default subsuite is also what 'make
check' will explicitly invoke.
MozReview-Commit-ID: FaHb4nvuoK9
--HG--
extra : rebase_source : 2ecbc902bb6bafa7cd78ac0e380a10bad7c14351
In bug 1308472 we are seeing 'make -k check' fail intermittently with
the only apparent error message something like:
make: *** [check] Error 245
This debug should make it more clear which test (if any) is responsible
for setting the return code in 'mach python-test', and whether or not
'mach python-test' is actually reaching the end of the function.
MozReview-Commit-ID: 6IQrZQqs8ij
--HG--
extra : rebase_source : 4d2d4b8b139d7f16bda2b22ce79ab2c84e8fe4c7
The build system's TestResolver does a pretty good job of getting the right manifestparser
based tests to run, but it isn't perfect. Notably, it ignores the 'disabled' key. We filter
the tests through manifestparser here to make sure the build system didn't miss anything.
For context, this is also what the other test harnesses (e.g mochitest) do as well.
MozReview-Commit-ID: FaHb4nvuoK9
--HG--
extra : rebase_source : 476b7068c3ddf7d9757d605de5843e21547c6c33
This deprecates PYTHON_UNIT_TESTS and replaces it with PYTHON_UNITTEST_MANIFESTS.
In the build system, this means python unittests will be treated the same as all
other test suites that use manifestparser. New manifests called 'python.ini' have
been created for all test directories containing python unittests.
MozReview-Commit-ID: IBHG7Thif2D
--HG--
extra : rebase_source : 11a92a2bc544d067946bbd774975140147458caa
We recently switched make check to call into |mach python-test| rather than invoking python
itself for each test file. But this ended up slowing down the tests as they were no longer
being run in parallel. This patch adds a --jobs flag to python-tests and runs test files in
parallel.
Note: if more than one job is used, output per test will be buffered and printed at the end
to avoid interleaving. This has the unfortunate side effect of making |mach python-test| look
like it is hanging, especially if running a very long file like mozbase's test.py. For this
reason, we still use -j1 by default so output will continue to be streamed. In automation we
will use multiple processes though.
MozReview-Commit-ID: 3u0wOFmyQLI
--HG--
extra : rebase_source : d08ac412023731c46226c7adbf5f6e798b9a345a
In the `python-test` mach command and the mozharness script for
the Marionette harness tests, use the vendored-in Pytest
instead of installing from pip.
Add the Marionette harness test requirements file to the
file_patterns in the definition of the marionette-harness taskcluster
job, as changes to the requirements should trigger the job to run.
MozReview-Commit-ID: J5pln2WB4GY
--HG--
extra : rebase_source : 5144ccfabb84eb0da244b24f6d27b59ae183c174
So a few changes here:
- node_modules is downloaded using tooltool so that we dont need to rely on external infrastructure.
- We have a npm-shrinkwrap.json file that version locks all of our node packages.
- eslint, eslint-plugin-mozilla etc. are now all installed locally.
In reality this means that we don't hit the network and we don't force users into installing global packages.
./mach eslint --setup has also been improved. We install packages locally and display the path of the user's eslint binary (useful for configuring editors).
eslint-plugin-mozilla has been moved from testing/eslint-plugin-mozilla to /testing/eslint/eslint-plugin-mozilla.
The node_modules directory for eslint and other plugins is located in testing/eslint/.
MozReview-Commit-ID: 4SFSxzka6BS
--HG--
rename : testing/eslint-plugin-mozilla/LICENSE => testing/eslint/eslint-plugin-mozilla/LICENSE
rename : testing/eslint-plugin-mozilla/docs/balanced-listeners.rst => testing/eslint/eslint-plugin-mozilla/docs/balanced-listeners.rst
rename : testing/eslint-plugin-mozilla/docs/import-browserjs-globals.rst => testing/eslint/eslint-plugin-mozilla/docs/import-browserjs-globals.rst
rename : testing/eslint-plugin-mozilla/docs/import-globals.rst => testing/eslint/eslint-plugin-mozilla/docs/import-globals.rst
rename : testing/eslint-plugin-mozilla/docs/import-headjs-globals.rst => testing/eslint/eslint-plugin-mozilla/docs/import-headjs-globals.rst
rename : testing/eslint-plugin-mozilla/docs/index.rst => testing/eslint/eslint-plugin-mozilla/docs/index.rst
rename : testing/eslint-plugin-mozilla/docs/mark-test-function-used.rst => testing/eslint/eslint-plugin-mozilla/docs/mark-test-function-used.rst
rename : testing/eslint-plugin-mozilla/docs/no-aArgs.rst => testing/eslint/eslint-plugin-mozilla/docs/no-aArgs.rst
rename : testing/eslint-plugin-mozilla/docs/no-cpows-in-tests.rst => testing/eslint/eslint-plugin-mozilla/docs/no-cpows-in-tests.rst
rename : testing/eslint-plugin-mozilla/docs/reject-importGlobalProperties.rst => testing/eslint/eslint-plugin-mozilla/docs/reject-importGlobalProperties.rst
rename : testing/eslint-plugin-mozilla/docs/var-only-at-top-level.rst => testing/eslint/eslint-plugin-mozilla/docs/var-only-at-top-level.rst
rename : testing/eslint-plugin-mozilla/lib/globals.js => testing/eslint/eslint-plugin-mozilla/lib/globals.js
rename : testing/eslint-plugin-mozilla/lib/helpers.js => testing/eslint/eslint-plugin-mozilla/lib/helpers.js
rename : testing/eslint-plugin-mozilla/lib/index.js => testing/eslint/eslint-plugin-mozilla/lib/index.js
rename : testing/eslint-plugin-mozilla/lib/processors/xbl-bindings.js => testing/eslint/eslint-plugin-mozilla/lib/processors/xbl-bindings.js
rename : testing/eslint-plugin-mozilla/lib/rules/.eslintrc => testing/eslint/eslint-plugin-mozilla/lib/rules/.eslintrc
rename : testing/eslint-plugin-mozilla/lib/rules/balanced-listeners.js => testing/eslint/eslint-plugin-mozilla/lib/rules/balanced-listeners.js
rename : testing/eslint-plugin-mozilla/lib/rules/import-browserjs-globals.js => testing/eslint/eslint-plugin-mozilla/lib/rules/import-browserjs-globals.js
rename : testing/eslint-plugin-mozilla/lib/rules/import-globals.js => testing/eslint/eslint-plugin-mozilla/lib/rules/import-globals.js
rename : testing/eslint-plugin-mozilla/lib/rules/import-headjs-globals.js => testing/eslint/eslint-plugin-mozilla/lib/rules/import-headjs-globals.js
rename : testing/eslint-plugin-mozilla/lib/rules/mark-test-function-used.js => testing/eslint/eslint-plugin-mozilla/lib/rules/mark-test-function-used.js
rename : testing/eslint-plugin-mozilla/lib/rules/no-aArgs.js => testing/eslint/eslint-plugin-mozilla/lib/rules/no-aArgs.js
rename : testing/eslint-plugin-mozilla/lib/rules/no-cpows-in-tests.js => testing/eslint/eslint-plugin-mozilla/lib/rules/no-cpows-in-tests.js
rename : testing/eslint-plugin-mozilla/lib/rules/reject-importGlobalProperties.js => testing/eslint/eslint-plugin-mozilla/lib/rules/reject-importGlobalProperties.js
rename : testing/eslint-plugin-mozilla/lib/rules/var-only-at-top-level.js => testing/eslint/eslint-plugin-mozilla/lib/rules/var-only-at-top-level.js
rename : testing/eslint-plugin-mozilla/moz.build => testing/eslint/eslint-plugin-mozilla/moz.build
rename : testing/eslint-plugin-mozilla/package.json => testing/eslint/eslint-plugin-mozilla/package.json
extra : rebase_source : cf689f6cc170b9a22018c981a768f545f952e019
This accounts for default unittest and pytest output formatting,
in addition to mozunit.
MozReview-Commit-ID: 749CD0xQezX
--HG--
extra : rebase_source : 7a451c61d1ec41303b859b8fff4ec3dd2f84064c
The most common issue I'm hearing with eslint is people who have an outdated
node installed. This does a quick check to verify the version is high enough
before linting.
MozReview-Commit-ID: Em0jn18OUYo
--HG--
extra : rebase_source : 5325eb5f556f93e09d48fb123e0abb625aa77b84
Also removes related unused variables in mach_commands.py.
--HG--
extra : commitid : IiDVMuEZtA5
extra : rebase_source : 575a51dd0ad5450323b4da5f441f8e5d721e41d6
Currently mach treats the first argument to eslint as the path and moves it to
the end of the arguments but this breaks usage like "mach eslint -f json browser".
It used to be necessary to change to the directory you wanted to lint but now
the .eslintignore is at the top level we just run from the top level. This means
the path argument doesn't need to be special anymore.
--HG--
extra : commitid : 5ozct0pVSC4
extra : rebase_source : 22132a240d8e6f4d099dbcdeb793958d7173e154
extra : amend_source : 2b9931b4283e1c84f699027e13eccc33fcdec978
This was the only import of glob from all mach_commands.py files. Kill
it.
With this commit, there are no modules imported by a single
mach_commands.py outside of testing/web-platform/mach_commands.py.
--HG--
extra : commitid : 4CJqlwDqOVg
extra : rebase_source : 9dbbd69291d64b894a399523864562107c10872e
This removes ambiguity as to which modules are being imported, making
import slightly faster as Python doesn't need to test so many
directories for file presence.
All files should already be using absolute imports because mach command
modules aren't imported to the package they belong to: they instead
belong to the "mach" package. So relative imports shouldn't have been
used.
--HG--
extra : commitid : 6tFME1KKfTD
extra : rebase_source : 78728f82f5487281620e00c2a8004cd5e1968087
Back when mozpack.path was added, it was used as:
import mozpack.path
mozpack.path.func()
Nowadays, the common idiom is:
import mozpack.path as mozpath
mozpath.func()
because it's shorter.
$ git grep mozpath\\. | wc -l
423
$ git grep mozpack.path\\. | wc -l
123
This change was done with:
$ git grep -l mozpack.path\\. | xargs sed -i 's/mozpack\.path\./mozpath./g'
$ git grep -l 'import mozpack.path$' | xargs sed -i 's/import mozpack.path$/\0 as mozpath/'
$ (pat='import mozpack.path as mozpath'; git grep -l "$pat" | xargs sed -i "1,/$pat/b;/$pat/d")
The reason to use '+' prefixing was to distinguish between options to the
mach command itself, and options that are passed down to whatever the
command does (like mach run passing down args to the built application).
That makes things unnecessarily awkward, and quite non-standard.
Instead, use standard '-' prefixing, and pass all the unknown arguments
down. If there is overlap between the known arguments and arguments supported
by the underlying tool (like -remote when using mach run), it is possible to
use '--' to mark all following arguments as being targetted at the underlying
tool.
For instance:
mach run -- -remote something
would run
firefox -remote something
while
mach run -remote something
would run
firefox something
As allow_all_arguments is redundant with the presence of a argparse.REMAINDER
CommandArgument, allow_all_arguments is removed. The only mach command with a
argparse.REMAINDER CommandArgument without allow_all_arguments was "mach dmd",
and it did so because it didn't want to use '+' prefixes.
The Python-related mach commands were written before we had a virtualenv
API exposed to the mach command context. This patch updates those
commands to use the newer APIs. As a bonus, these commands now work
without running configure!
--HG--
extra : rebase_source : ea394d6fc0c5fa2d3a3a6ed25fc59ce6be40690c
extra : amend_source : e841d57a2578c93b778ef73c68c35a8cc7cfde44
Andrew Halberstadt
Sebastian Hengst
Mike Shal
Anjana Vakil
Michael Ratcliffe
Alexandre Poirot
J. Ryan Stinnett
Hector Zhao
Bob Silverberg
Maja Frydrychowicz
Dave Townsend
Ted Mielczarek
Patrick Brosset
Kris Maglione
Felipe Gomes
Jared Wein
Nick Alexander
Gregory Szorc
Mike Hommey
Nathan Froyd