Граф коммитов

177 Коммитов

Автор SHA1 Сообщение Дата
Geoff Brown 9ae75067b3 Bug 1596599 - Minor update for 'mach test-info report --show-manifests'; r=jmaher
Puppeteer tests have t['manifest'] = None; allowing for this avoids downstream pain
when reporting manifests.
(Also removed a stray debugging print statement.)

Differential Revision: https://phabricator.services.mozilla.com/D53102

--HG--
extra : moz-landing-system : lando
2019-11-15 10:38:06 +00:00
Geoff Brown c97e7a5294 Bug 1595257 - Add annotations report to 'mach test-info report'; r=jmaher
This "annotations" report provides a different view of manifest data,
concentrating on the manifest annotations, like "skip-if". What conditions
are used in manifests? How many times does each occur?

Differential Revision: https://phabricator.services.mozilla.com/D52926

--HG--
extra : moz-landing-system : lando
2019-11-14 19:11:07 +00:00
Geoff Brown a52ff2f347 Bug 1591464 - In |mach test-info tests| display bugzilla info sooner; r=bc
Very minor enhancement to 'mach test-info tests <test-name>' such that the list of
related bugs is displayed earlier, before any of the ActiveData queries, so that
all the quickly-retrieved info is displayed before the annoying ActiveData delay.
I had to split set_test_name() into two parts since the first part is required
for the bugzilla report; the second part is not required for bugzilla and involves
a lengthy ActiveData query.

Differential Revision: https://phabricator.services.mozilla.com/D52087

--HG--
extra : moz-landing-system : lando
2019-11-07 15:32:31 +00:00
Ted Campbell 729b33d7fe Bug 1593486 - Cleanup spidermonkey test code in mach. r=froydnj
Remove some overly clever code and make the spidermonkey-related tests more
consistent with each other.

Differential Revision: https://phabricator.services.mozilla.com/D51901

--HG--
extra : moz-landing-system : lando
2019-11-06 18:53:07 +00:00
Geoff Brown 9212846e2c Bug 1594237 - Remove robocop support from 'mach test-info tests'; r=bc
A minor optimization for 'mach test-info tests'.

Differential Revision: https://phabricator.services.mozilla.com/D51946

--HG--
extra : moz-landing-system : lando
2019-11-06 00:29:36 +00:00
Ted Campbell 0a584d961b Bug 1593486 - Cleanup ./mach jsapi-tests implementation r=sfink,froydnj
Standardize the jsapi-tests logic to match the other spidermonkey check
commands in testing/mach_commands.py.

Differential Revision: https://phabricator.services.mozilla.com/D51530

--HG--
extra : moz-landing-system : lando
2019-11-05 16:24:54 +00:00
Ted Campbell cabef1d645 Bug 1593486 - Add ./mach jit-test command r=sfink,froydnj
This allows running the jit_test.py harness using mach. It works very
similarly to the existing 'mach jstests' command.

Differential Revision: https://phabricator.services.mozilla.com/D51518

--HG--
extra : moz-landing-system : lando
2019-11-04 19:46:43 +00:00
Geoff Brown 106bb99f61 Bug 1584310 - Improve reftest annotation reporting in 'mach test-info report'; r=bc
Treat the reftest 'fails-if' as a synonym for manifest parser 'fail-if'.
For any annotation types requested by --filter-keys, display those annotations
in the test-level report (--show-tests).

Differential Revision: https://phabricator.services.mozilla.com/D51166

--HG--
extra : moz-landing-system : lando
2019-10-30 20:33:44 +00:00
Geoff Brown 3b5009d168 Bug 1588935 - Improve efficiency of 'mach test-info report'; r=bc
Pull calls to mozbuild_reader().files_info() out of loops, instead passing
all the paths of interest at once.

Differential Revision: https://phabricator.services.mozilla.com/D49358

--HG--
extra : moz-landing-system : lando
2019-10-16 13:10:39 +00:00
Cosmin Sabou 09a3569500 Bug 1586027 - Fix lint error. r=lint-fix 2019-10-04 00:03:33 +03:00
Geoff Brown 66ce37dbd1 Bug 1586027 - Improve help for 'mach test-info'; r=bc
Adds 'tests' sub-command for 'mach test-info'.
'mach help test-info' shows all sub-commands.
'mach help test-info <subcommand>' shows sub-command descriptions,
including help for the options previously excluded.

Differential Revision: https://phabricator.services.mozilla.com/D48079

--HG--
extra : moz-landing-system : lando
2019-10-03 20:39:28 +00:00
Geoff Brown a61a7cc10b Bug 1586035 - Improve platform naming in 'mach test-info'; r=bc
Instead of choosing specific run.types of interest, like 'e10s', concatenate
all returned strings, eliminating known exceptions, like 'chunked'. Notably
this correctly displays "linux64/debug-e10s-service-worker".

Differential Revision: https://phabricator.services.mozilla.com/D48091

--HG--
extra : moz-landing-system : lando
2019-10-03 20:39:28 +00:00
Geoff Brown c25975635c Bug 1580914 - Ensure config environment before generating test-info report; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D46359

--HG--
extra : moz-landing-system : lando
2019-09-18 22:04:39 +00:00
Geoff Brown aa4ae1c9d2 Bug 1581843 - Improve sorting in 'mach test-info report'; r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D46152

--HG--
extra : moz-landing-system : lando
2019-09-17 15:25:27 +00:00
Andrew Halberstadt f58125d411 Bug 1568638 - [test-info] Create --output-file directory if it doesn't exist, r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D43513

--HG--
extra : moz-landing-system : lando
2019-08-27 19:08:06 +00:00
Geoff Brown dee3c435aa Bug 1575652 - Show test skip counts in 'mach test-info'; r=jmaher
Report both failures and skipped test counts.
(Also incidentally fixes platform name reporting of fission tests.)

Differential Revision: https://phabricator.services.mozilla.com/D44074

--HG--
extra : moz-landing-system : lando
2019-08-29 23:30:29 +00:00
James Graham 92fb46bfa6 Bug 1564425 - Improve error handling of mach addtest, r=bgrins
Ensure that if we try to create a mochitest of unsupported file type we return a
useful error

Differential Revision: https://phabricator.services.mozilla.com/D38224

--HG--
extra : moz-landing-system : lando
2019-08-26 21:06:02 +00:00
Mihai Alexandru Michis 60087ba0c3 Bug 1573585 - Fix unindented block. a=lint-fix CLOSED TREE 2019-08-23 00:05:01 +03:00
Edwin Gao ca724248e4 Bug 1573585 - ensure debugger name is prefixed to the application command when using mach test r=gbrown,Ehsan
If test is run using the `mach test` harness, populate the `extra_args` with `--debugger=<name>` that is expected by the underlying harnesses (`wpt,`, `mochitest`)  in order to maintain compatibility.

Differential Revision: https://phabricator.services.mozilla.com/D42952

--HG--
extra : moz-landing-system : lando
2019-08-22 20:51:11 +00:00
Geoff Brown fd801b55c9 Bug 1572794 - Add summary and filter-values regex to 'mach test-info report'; r=jmaher
Optional summary provides a convenient synopsis of matching tests. filter-values regex
was requested for in bug 1568638, like --filter-values='(?<!!)fission'.

Differential Revision: https://phabricator.services.mozilla.com/D42570

--HG--
extra : moz-landing-system : lando
2019-08-20 16:19:40 +00:00
Geoff Brown 16c83acf2b Bug 1572794 - Add --no-component-report option to 'mach test-info report'; r=jmaher
Keep the existing report format by default, but provide an option to generate a
report that is not broken down by component.

Differential Revision: https://phabricator.services.mozilla.com/D41427

--HG--
extra : moz-landing-system : lando
2019-08-10 21:52:39 +00:00
Geoff Brown 9207b87bbb Bug 1572794 - Add 'flavor' and 'subsuite' args for 'mach test-info report'; r=jmaher
These were already supported by the test resolver; let's use them. More tweaks planned...

Differential Revision: https://phabricator.services.mozilla.com/D41414

--HG--
extra : moz-landing-system : lando
2019-08-09 17:31:56 +00:00
Axel Hecht c3a71a9957 Bug 1321281, add test framework for Fluent migration recipes, r=flod,ahal
Basic test infrastructure for Fluent migration recipes, automate creating
a reference from recipe data, and running the recipe dry and wet.

There are no hard failures for diffs, as it might just be OK.

Differential Revision: https://phabricator.services.mozilla.com/D40200

--HG--
extra : moz-landing-system : lando
2019-08-08 11:54:30 +00:00
Geoff Brown bbd039411d Bug 1404472 - Add mach subcommand for per-component test-info report; r=jmaher
This provides the basic functionality, but certainly not all the bells and whistles. Future work may add summaries and totals, better reftest support, better web-platform support, more flexible filtering, etc.

Differential Revision: https://phabricator.services.mozilla.com/D39224

--HG--
extra : moz-landing-system : lando
2019-07-31 09:21:59 +00:00
Edwin Gao a1cc4534a7 Bug 1519369 - support 'debugger' parser argument for mach test command r=ahal,gbrown
Differential Revision: https://phabricator.services.mozilla.com/D39229

--HG--
extra : moz-landing-system : lando
2019-07-26 15:18:18 +00:00
Andrew Halberstadt 3336efcb3f Bug 1473498 - Fixing Py3 compatibility errors reachable from testing/mach_commands.py r=gbrown
Differential Revision: https://phabricator.services.mozilla.com/D37762

--HG--
extra : moz-landing-system : lando
2019-07-16 01:21:55 +00:00
Andrew Halberstadt 3486ba642c Bug 1563797 - Use 'backports.shutil_which' instead of 'which' across the tree r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D37097

--HG--
extra : moz-landing-system : lando
2019-07-11 14:03:39 +00:00
Jan de Mooij ae0157f590 Bug 1562129 part 2 - Add --blinterp-eager to jit-test 'all' configuration. r=sfink
To not make jstests even slower, this adds --jitflags=jstests and uses it for
jstests in automation. For jstests we can use fewer jit configurations than
for jit-tests.

This also removes the bigint and bigintdebug autospider variants. BigInt is now
enabled by default so we no longer need the configure flag etc.

Differential Revision: https://phabricator.services.mozilla.com/D36307

--HG--
extra : moz-landing-system : lando
2019-07-08 11:22:04 +00:00
Kartikaya Gupta bb83950948 Bug 1558598 - Force WebRender on or off in the cppunit harness. r=ahal
Also adds it to the mach command, which is a little weird, because the
mach command doesn't expose the option but does parse it via the cpp unit
argument parser. So I just exposed it on the mach command and after that
it Just Works for mach.

Differential Revision: https://phabricator.services.mozilla.com/D35859

--HG--
extra : moz-landing-system : lando
2019-06-29 09:49:23 +00:00
Geoff Brown 694a310a53 Bug 1558221 - Update 'mach test-info long-tasks'; r=jmaher
Follow-up on ekyle's kind suggestion. I also found the type of some of the
returned data has changed, producing incorrect zero counts. A simple type
conversion gets everything working again:

$ ./mach test-info long-tasks

Tasks nearing their max-run-time on mozilla-central,mozilla-inbound,autoland between 2019-06-03 and 2019-06-10
test-linux64-asan/opt-web-platform-tests-e10s-10       : 52 of 212 runs (24.0%) exceeded 90% of max-run-time (7200 s)
test-linux64/debug-telemetry-tests-client-e10s         : 34 of 211 runs (16.0%) exceeded 90% of max-run-time (600 s)
test-linux64-asan/opt-telemetry-tests-client-e10s      : 76 of 696 runs (10.0%) exceeded 90% of max-run-time (600 s)

Differential Revision: https://phabricator.services.mozilla.com/D34362

--HG--
extra : moz-landing-system : lando
2019-06-10 17:40:33 +00:00
Aaron Klotz 104c07093b Bug 1238305: Modify cppunittests to look up breakpad symbols for logged stack traces; r=ted
--HG--
extra : rebase_source : 159c245748d0192207ffa97465e254ea1a1538e2
2019-05-03 13:00:46 -06:00
Chris Manchester caa48376ad Bug 1547380 - Reject attempts to run the `rusttests` command from artifact builds. r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D29436

--HG--
extra : moz-landing-system : lando
2019-04-30 23:03:14 +00:00
Brian Grinstead 8ca66379e9 Bug 1547996 - Fix mochitest-browser-chrome with mach addtest;r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D29364

--HG--
extra : moz-landing-system : lando
2019-04-30 15:20:51 +00:00
Geoff Brown b57f56b05d Bug 1445677 - Fix 'mach test-info' platform handling; r=jmaher
The format of platform and type data seems to have evolved over time; these
changes seem to handle the new formats well.

Differential Revision: https://phabricator.services.mozilla.com/D28766

--HG--
extra : moz-landing-system : lando
2019-04-25 09:20:06 +00:00
Andrew Halberstadt 48648a7344 Bug 1523303 - Align mozharness suite names with the ones in 'moztest.resolve', r=gbrown
This officially makes 'moztest.resolve' the source of truth when it comes to
suite names. It aligns that file with the names used in both the
desktop_unittest and android_emulator_unittest scripts.

Differential Revision: https://phabricator.services.mozilla.com/D27555

--HG--
extra : moz-landing-system : lando
2019-04-22 22:32:34 +00:00
James Graham 7b45ebc476 Bug 1543244 - Refactor mach addtest support, r=ahal,bgrins
Add addtest support for per-suite arguments and multiple files. Also
support opening the created test in an editor. This allowed supporting
the wpt suite and replaces `mach wpt-create`.

 # Create a wpt test
 ./mach addtest testing/web-platform/tests/accelerometer/test.html

 # Create a wpt reftest
 ./mach addtest --suite wpt-reftesttesting/web-platform/tests/css/example.html --ref example-ref.html

The files created will be opened in the default editor if --editor is
supplied or a specified editor if the argument is given a value.

Differential Revision: https://phabricator.services.mozilla.com/D26339

--HG--
extra : moz-landing-system : lando
2019-04-11 09:44:32 +00:00
Julian Descottes 28c84500a6 Bug 1543007 - Restore ./mach test command;r=pbro
Differential Revision: https://phabricator.services.mozilla.com/D26661

--HG--
extra : moz-landing-system : lando
2019-04-09 09:22:22 +00:00
Brian Grinstead 05f5ec26a4 Bug 1540285 - Add a test generator script into `./mach addtest` r=ahal
Instead of:
perl gen_template.pl -b=123456 --type=plain > path/to/test_bug123456.html

You can do:
./mach addtest --suite="mochitest-plain" > path/to/test_bug123456.html

But you can also pass in a new file path and let it guess the suite/doc:

```
# mochitest-chrome tests
./mach addtest js/xpconnect/tests/chrome/test_chrome.html
./mach addtest js/xpconnect/tests/chrome/test_chrome.xhtml
./mach addtest js/xpconnect/tests/chrome/test_chrome.xul

# mochitest-plain tests
./mach addtest js/xpconnect/tests/mochitest/test_plain.html
./mach addtest js/xpconnect/tests/mochitest/test_plain.xhtml
./mach addtest js/xpconnect/tests/mochitest/test_plain.xul

# mochitest-browser tests
./mach addtest browser/base/content/test/alerts/browser_foo.js

# xpcshell tests
./mach addtest browser/components/extensions/test/xpcshell/test_xpcshell.js
```

This also changes the mochitest template files in the following ways:
- removes the bug # boilerplate
- remove some unnecessary attributes in the template
- removes the th.template
- adds the browser.template for browser-chrome tests

Differential Revision: https://phabricator.services.mozilla.com/D25482

--HG--
rename : testing/mochitest/static/chrome.template.txt => testing/mochitest/static/chromehtml.template.txt
rename : testing/mochitest/static/test.template.txt => testing/mochitest/static/plainhtml.template.txt
rename : testing/mochitest/static/xhtml.template.txt => testing/mochitest/static/plainxhtml.template.txt
rename : testing/mochitest/static/xul.template.txt => testing/mochitest/static/plainxul.template.txt
extra : moz-landing-system : lando
2019-04-08 19:09:42 +00:00
Geoff Brown fa1f40ea6a Bug 1500995 - Allow for missing 'failures' data in 'mach test-info'; r=me,a=test-only 2018-12-21 14:18:15 -07:00
Chris Manchester b78d8dbce3 Bug 1505859 - Implement a mach wrapper for rusttests. r=firefox-build-system-reviewers,ted
Differential Revision: https://phabricator.services.mozilla.com/D11515

--HG--
extra : moz-landing-system : lando
2018-11-10 15:00:28 +00:00
Geoff Brown 79f5bdc311 Bug 1481449 - Require at least one test argument for 'mach test'; r=ahal 2018-08-31 09:59:07 -06:00
Geoff Brown ee452067ce Bug 1329286 - Remove 'mach find-test-chunk'; r=ahal 2018-08-27 07:19:02 -06:00
Geoff Brown 24afcca748 Bug 1469720 - Add a 'mach test-info' subcommand to find long-running tasks; r=jmaher 2018-08-07 10:05:37 -06:00
Andrew Halberstadt 44f4128a40 Bug 1477276 - Fix regression preventing |mach help| without an objdir r=bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D2267

--HG--
extra : moz-landing-system : lando
2018-07-20 14:58:06 +00:00
Geoff Brown dfb845682a Bug 1476363 - Add --show-tasks support to 'mach test-info'; r=jmaher 2018-07-17 12:54:19 -06:00
Ms2ger 3787678d13 Bug 1460546 - Add a mach jstests command; r=dminor 2018-07-05 14:42:52 +02:00
Andrew Halberstadt 851c87fc61 Bug 1445975 - Add a basic mach command for running jsshell benchmarks, r=jmaher
Run with:
./mach jsshell-bench ares6

This requires a non-artifact build with ac_add_options --enable-js-shell. Otherwise,
a custom JS shell can be specified with --binary.

MozReview-Commit-ID: C8WAxG9pIkk

--HG--
extra : rebase_source : 99163261b771bfb2f903892f4c1260c8b241ce71
2018-05-15 14:41:16 -04:00
Jan de Mooij d047348566 Bug 1063728 - Run SpiderMonkey style checkers as part of the build instead of make check. r=chmanchester 2018-04-26 09:09:51 +02:00
Jan de Mooij 2c4c4c3296 Bug 1064316 part 3 - Fix |mach check-spidermonkey| to use correct cwd. r=gps 2018-04-24 16:17:50 +02:00
Andrew Halberstadt 20f38147c2 Bug 1445624 - [test] Make sure verbose logging is enabled when running a single test, r=gbrown
This sets verbose=True (used by the mach formatter) and compact=False (used by tbplformatter) if
only a single test is specified with either |mach test| or |mach mochitest|.

This way all TEST_STATUS and log lines will be visible when developers are writing a new test.

MozReview-Commit-ID: 2nuKs9DLnx0

--HG--
extra : rebase_source : 1fc594b465a2a94dfcd85d56d042969af47f6f65
2018-03-19 16:04:02 -04:00