Since we're adding specific 'task_regexes' for each new suite definition,
this will allow us to schedule tests of these subsuites with
|mach try fuzzy <path>|.
MozReview-Commit-ID: 2mDSneV95lG
--HG--
extra : rebase_source : 467b9d885e92c1c855ed547f2a7496b1062f2dc2
The end goal here is to be able to use |mach try fuzzy <path>| with tests that
belong to a subsuite. To do this, we need a unique 'task_regex' value for each
subsuite so that we can map a test path back to a set of tasks.
This removes the TEST_FLAVORS dict (which was mostly just a redefinition of the
data in TEST_SUITES), and instead provides two new private mappings:
<flavor> -> suite definition
(<flavor>, <subsuite>) -> suite definition
To retrieve a suite definition given a flavor/subsuite, consumers can now call
get_suite_definition.
MozReview-Commit-ID: 2pe1v1IHUVy
--HG--
extra : rebase_source : 6fff947ba214112ccf16c894174a6a0e2487111a
This removes a lot of redundant alias definitions by calling lower() on the
user input. It also adds a couple of new aliases that look like they might
be useful.
MozReview-Commit-ID: 3Aix4LPB8wg
--HG--
extra : rebase_source : c4bdc327bd737a18f03952bb360af35608d091f1
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
This enables the syntax like:
./mach try fuzzy dom/indexedDB
This will open up the fzf interface like normal, except only tasks
that have tests under dom/indexedDB will be selectable (and there
will only be one chunk per configuration).
This can be combined with -q/--query like normal:
./mach try fuzzy dom/indexedDB -q "!pgo !cov !asan"
When the tasks get scheduled, only the tests under the specified
path(s) will run within the harness.
MozReview-Commit-ID: IHRXXi5mB4G
--HG--
extra : rebase_source : 8a89f255591e6dfa31b1420196c4698f2015d10c
This removes the subcommands for "./mach geckodriver", reverting
it back to have the meaning of running the geckodriver binary.
The build- and test commands are now integrated with mach, which
means you can run "./mach build testing/geckodriver" and "./mach
test testing/geckodriver" to run tests. This is backed by a new
top-level "./mach geckodriver-test" command, which we will not be
announcing.
MozReview-Commit-ID: CiQsfNqrvIp
--HG--
extra : rebase_source : 6c492b7e1128e4858e42ae4bb35ab4b29564dbeb
The code in |mach test| for test resolving, should get merged with the TestResolver
class in moztest.resolve. This way it can be shared with other modules and we'll
have a single canonical place for all our test resolving logic.
MozReview-Commit-ID: IHRXXi5mB4G
--HG--
extra : rebase_source : 6f96d06412ab8fa152ac5d9bdd15acbcdc9695c4
The TestMetadata and TestResolver classes aren't technically part of the build
system. The only connection is that they consume some build system output.
The next patch in this series is going to be merging in a bunch of other test
resolving logic from other parts of the tree. Moving this out first allows us
to keep that extra logic out of mozbuild.
MozReview-Commit-ID: 1eq4SjFVCyW
--HG--
rename : python/mozbuild/mozbuild/test/test_testing.py => testing/mozbase/moztest/tests/test_resolve.py
extra : rebase_source : 7ff11f9ec455547533082d20cb5371845f7a4f21
This includes code for downloading a Firefox binary, downloading + setting up a tests.zip and
running output through mozharness' output parsers. This is all stuff that will also be required
for the reftest selftests.
I couldn't think of a better location to put this stuff, suggestions welcome.
MozReview-Commit-ID: 59TSbsugT5T
--HG--
extra : rebase_source : a328f6bc90e73fe23f9054933cd01a30065419f6
This is needed before we can upgrade to flake8 3.3.0, as that version starts flagging these errors.
These files were modified by running:
autopep8 --select E305 --in-place -r <dir>
on the affected directories. I did it one dir at a time and verified the result after each.
MozReview-Commit-ID: FmlsfiKIbtr
--HG--
extra : rebase_source : 9df32258cadff5d27a0e72113c57f782756c0b18
The subsuite is added conditionally because we only have the capability of
running source-check tasks on linux at the moment. Once taskcluster support
for windows and mac has matured a bit and the taskcluster configs support
source-check there, we should apply the subuite unconditionally.
MozReview-Commit-ID: Kk9Irz3fn14
--HG--
extra : rebase_source : b9266a06583083c36477d4e93f5462ee614cdb71
Registered callback handlers for tests should receive the correct
test status when the test has been finished, and not always "Error".
This change allows those callbacks to run specific code for individual
test results, eg. only do screenshots for failures.
MozReview-Commit-ID: FfbCRR0Jvjb
--HG--
extra : rebase_source : c73253acbb666ca62b23f806145c20f0a70c934c
Registered callback handlers for tests should receive the correct
test status when the test has been finished, and not always "Error".
This change allows those callbacks to run specific code for individual
test results, eg. only do screenshots for failures.
MozReview-Commit-ID: FfbCRR0Jvjb
--HG--
extra : rebase_source : 98c69eea450f35312fd43bb7237a9d00e90636c4
The mozbase unittests don't use mozunit, so their output is confusing in the log.
This makes mozbase output consistent with the rest of the python unittests.
MozReview-Commit-ID: AIs5mza8Rn6
--HG--
extra : rebase_source : 10f65e612f5b3cebb921c47699f5a8be7cd2ba5a
added testing/mozbase to tools/lint/flake8.lint
fixed a first batch of PEP8 errors/warnings
at first the commad autopep8 -i --max-line-length 99 -r -j 8 .
has been used to fix simpler problems, run from testing/mozbase
some of the issues can not easily fixed :
- undefined 'names' in code for example isLinux - isLinux and isBsd "fixed" with # noqa
- undefined 'message' resolved with return fmt.format(...
- undefined 'structured' resolved replacing those with mozlog
- long comments - some remaining - addressed with # noqa
- package level import everything - addressed with # flake8: noqa
restored testing/mozbase/mozdevice/mozdevice/Zeroconf.py
fixed issues reported on mozreview
fixed ')' in testing/mozbase/mozprocess/mozprocess/qijo.py imports
finally fixed multiline string at testing/mozbase/manifestparser/tests/test_manifestparser.py:114
^^^ and again, but now with ./mach python-test --path-only testing/mozbase/manifestparser/tests/test_manifestparser.py passing
fixed testing/mozbase/manifestparser/tests/test_convert_directory.py assert
fixed this error:
10:15:21 INFO - return lambda line: stack_fixer_module.fixSymbols(line)
10:15:21 INFO - TypeError: fixSymbols() takes exactly 2 arguments (1 given)
fixed two spaces lint error even of # noqa comments
restored assignement to lambda with # noqa to silence the lint error
global noqa for testing/mozbase/manifestparser/tests/test_filters.py
stupid is/is not error...
MozReview-Commit-ID: 1FpJF54GqIi
--HG--
extra : rebase_source : 3cf0277fb36a296e3506aeacc2ff05e1b03f9eac
This properly implements the structured logging protocol in the unittest
adapter.
The patch also adds an optional argument `test_list` to StructuredTestLogger
which will include an array of all tests when sending the suite_start message.
---
testing/mozbase/moztest/moztest/adapters/unit.py | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
This properly implements the structured logging protocol in the unittest
adapter.
The patch also adds an optional argument `test_list` to StructuredTestLogger
which will include an array of all tests when sending the suite_start message.
---
testing/mozbase/moztest/moztest/adapters/unit.py | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
--HG--
extra : rebase_source : f8c9852485129bd034317c1b801a3f33a56162cf
Andrew Halberstadt
Andreas Tolfsen
Tom Prince
Steve Armand
Ganti Sai Sarath Chandra
Henrik Skupin
Wes Kocher
Francesco Pischedda
Julien Pagès
Chris Manchester
Andreas Tolfsen
Ryan VanderMeulen
James Graham
Jonathan Griffin
Jeff Hammel