Currently mochitest looks for a test_start log action and stores the name of
the test in an instance variable. In the event of a timeout/hang, this string
is used in the log to help people try and narrow down where it happened.
However, there are a couple cases it can be misleading:
1) If the hang happens during shutdown, it looks like it was the last test's fault (it probably isn't)
2) If the hang happens in-between tests, it looks like the test timed out
We should make the string more descriptive. I propose that when we get a
test_end action, we change it to <test> (finished). And when we get the test
end action of the last test, we change it to "Last test finished".
This way it narrows down where the hang happened even further.
MozReview-Commit-ID: B6g8JrIJuJw
Differential Revision: https://phabricator.services.mozilla.com/D1885
--HG--
extra : moz-landing-system : lando
This serves two purposes:
1) It makes web-platform-tests pref downloading/handling a little more robust. When
run externally, it now downloads the entire testing/profiles directory. When loading
prefs it will look for both prefs_general.js (to support older versions of Firefox)
and profiles.json (for support moving forward).
This way we can add/remove/rename pref files under these directories without needing
to worry about breaking upstream wpt.
2) It provides developers an overview of which harnesses are using which base profiles.
Instead of hunting through test harness code to find this information, they can glance
at profiles.json.
MozReview-Commit-ID: AMzdnD8aGA2
--HG--
extra : rebase_source : 6fa0a802680417e49fcef99f3d03de7458a8fcba
This makes mochitest use mozprofile's new 'merge' feature instead of
explicitly loading the user.js preference file.
This means that any extensions that get dropped into
testing/profiles/common/extensions, will automatically run as part of the
mochitest run. This can be useful for testing how extensions impact tests
locally or on try.
In the near future, all our other test harnesses will also start using this
profile directory.
MozReview-Commit-ID: 34aSqdnkHqx
--HG--
extra : rebase_source : 977b0eb6404648e387428004ec3f3085af6f86fd
This moves testing/profiles/prefs_general.js to
testing/profiles/common/user.js. It also adds an 'extensions' folder to the
common profile. Dropping extension files here will get them installed in all
test harnesses (useful for testing on try).
The idea is that all test harnesses will eventually use this 'common' profile.
We'll also create some new per harness profiles, e.g testing/profiles/mochitest
and testing/profiles/reftest. This way there will be a single location
developers can go to set preferences, both for a specific harness, and across
all harnesses.
MozReview-Commit-ID: 8sqBqLiypgU
--HG--
rename : testing/profiles/prefs_general.js => testing/profiles/common/user.js
extra : rebase_source : 72a4a4b691e93c77479c7e70647b0ca373862c51
This makes mochitest use mozprofile's new 'merge' feature instead of
explicitly loading the user.js preference file.
This means that any extensions that get dropped into
testing/profiles/common/extensions, will automatically run as part of the
mochitest run. This can be useful for testing how extensions impact tests
locally or on try.
In the near future, all our other test harnesses will also start using this
profile directory.
MozReview-Commit-ID: 34aSqdnkHqx
--HG--
extra : rebase_source : 52b9c02aa712ac5136b6535e5b07fb56eae2412c
This moves testing/profiles/prefs_general.js to
testing/profiles/common/user.js. It also adds an 'extensions' folder to the
common profile. Dropping extension files here will get them installed in all
test harnesses (useful for testing on try).
The idea is that all test harnesses will eventually use this 'common' profile.
We'll also create some new per harness profiles, e.g testing/profiles/mochitest
and testing/profiles/reftest. This way there will be a single location
developers can go to set preferences, both for a specific harness, and across
all harnesses.
MozReview-Commit-ID: 8sqBqLiypgU
--HG--
rename : testing/profiles/prefs_general.js => testing/profiles/common/user.js
extra : rebase_source : 7599913e547533f2f57b597ad0f238c6cd391c82
(This also fixes Bug 879740 and Bug 1204543.)
build/pgo/certs contains an NSS database set that has a bunch of hand-generated
certificates, and many of these hand-generated certificates are specifically
depended upon for a variety of unit tests. This patch changes all of these to
use the "pycert.py" and "pykey.py" utilities that produce deterministic keys
and certificates.
The naming convention here is new, and defined in the README. It is based on
the mochitest runtest.py naming convention that imports .ca and .client
PEM-encoded certificates.
Unfortunately, the updates to build/pgo/genpgocert.py to generate these files
depends on OpenSSL in order to produce PKCS12 archives for pk11tool to import
into NSS. This could be done with pure-NSS tooling, but it'd require some new
command line functionality, which is out-of-scope for this change.
Note that build/pgo/genpgocert.py no longer takes arguments when run. It's not
run automatically anywhere that I can see, but could (reasonably) be, now.
Differential Revision: https://phabricator.services.mozilla.com/D971
--HG--
extra : amend_source : bc389b9b0a807a4889feb14db439daa28635dfe9
You can still run them on a --disable-stylo build, as long as that works
(presumably not for long).
I think I haven't missed anything, but please double-check.
MozReview-Commit-ID: 3BIAEjgTLo5
This allows us to specifically whitelist browser mochitests which still rely
on unsafe CPOWs, and run them in a separate Sandbox global with permissive
CPOWs enabled.
The test harness and most of the in-tree tests will run with permissive CPOWs
disabled, like the rest of the browser.
MozReview-Commit-ID: CxIkuxr5PXJ
--HG--
extra : rebase_source : 897c951e5ea84db58e92c8b627679f029ebf4a42
This was causing problems for the test-verify mode. Sometimes the 'log' option
can contain a class instance and then 'verify' attempts to deepcopy that (which
fails).
Since the 'log' option is only used in the Mochitest constructor, it's probably
simplest to just remove it from the main options object right at the start.
MozReview-Commit-ID: 9UQAYxr2Zvm
--HG--
extra : rebase_source : e10b9419f65b0209e650de1afb5765072833d780
In order to facilitate testing of enterprise policies that must be applied via a JSON at startup, the path to the test data will now be made available during testing.
MozReview-Commit-ID: IUhXXsiPRYW
--HG--
extra : rebase_source : 297bff6b0bf276edfff3a1920c733c89c62b085c
extra : source : eb38c57de57a288f67156271c09bd867b1fb2643
In order to facilitate testing of enterprise policies that must be applied via a JSON at startup, the path to the test data will now be made available during testing.
MozReview-Commit-ID: IUhXXsiPRYW
--HG--
extra : rebase_source : 0b7c405dc3ab3ad70280b78745944b879e923a37
extra : source : eb38c57de57a288f67156271c09bd867b1fb2643
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
The marionette.logging preference has been deprecated for some time.
We will want to use the official marionette.log.level preference instead.
Supported preferences are listed in
testing/marionette/prefs/marionette.js.
MozReview-Commit-ID: 4Z6OgqITe14
--HG--
extra : rebase_source : 62eabd701e79002b3828c3452e634966a2846643
In order for |mach test| and |mach mochitest| to log an overall summary,
every test harness invocation they make needs to use the same structured
logger (otherwise the affected suite will be missing from the summary).
MozReview-Commit-ID: 8LJw7r8SItk
--HG--
extra : rebase_source : 1417dce3817bae94ad61a5250065c6cbc35857e4
Suite names are currently only used by formatters to print out
an overall summary from |mach test| and |mach mochitest|. So
this doesn't need to be exact and can be tweaked further at a
later date.
If multiple test invocations have the same suite name, their
results will be merged in the overall summary. If a suite name
is missing, the summary will contain a placeholder name.
MozReview-Commit-ID: K1xpb9hUQRX
--HG--
extra : rebase_source : cc8cc8b36255d939dd5dffd3c5444c34951ac8e2
This is no longer necessary because the formatters now have the ability
to dump failure summaries which include failing tests and subtest results.
This also needs to be removed because it is throwing off the summary counts.
MozReview-Commit-ID: GbQkk0xQRds
--HG--
extra : rebase_source : 7bfbd81e1e7019ab639a98069fd7f0da994bfff7
This makes things slightly more inconvenient (having to set two
environment variables instead of one for the simplest case) until a few
patches down the line, when DMD is statically linked, at which point it
will get down to one environment variable every time.
--HG--
extra : rebase_source : 08dc3c05318b572ae1026227d0369fa8bf21b20f
This makes things slightly more inconvenient (having to set two
environment variables instead of one for the simplest case) until a few
patches down the line, when DMD is statically linked, at which point it
will get down to one environment variable every time.
--HG--
extra : rebase_source : 08dc3c05318b572ae1026227d0369fa8bf21b20f
When psutil is available, use it, but also add any processes found in the
process log to the list of children. Hopefully this will ensure that all
browser children are successfully killed.
When running mochitests in automation, we expect no more than 1 instance of
Firefox to be running at any time: the browser under test. In local testing
scenarios, additional browser instances might be running.
Warn when mochitests find additional 'firefox' instances running before launching
a browser for testing, to make it easier to detect anomalies in automation.
mochitests needs access to TESTING_JS_MODULES which are installed in $(OBJDIR)\_tests\modules\
MozReview-Commit-ID: CMgDlj4uKeP
--HG--
extra : rebase_source : 0a32b71db56bd68fc369d58117075dabf0465727
Bob Clary
Brian Hackett
Andrew Halberstadt
Chris Manchester
Dorel Luca
Emilio Cobos Álvarez
Bogdan Tara
J.C. Jones
Geoff Brown
Kris Maglione
Bryce Van Dyk
Kirk Steuber
Cosmin Sabou
Paul Adenot
Andreas Tolfsen
Alex Chronopoulos
Matthew Noorenberghe
Mike Hommey
Csoregi Natalia
Joel Maher
Jan de Mooij
Coroiu Cristina
Henrik Skupin