We are seeing failures starting websocketProcessBridge due to the port being
in use. It is most likely a previously started websocketProcessBridge process that
is using the port.
On Windows, mozprocess.kill() calls TerminateJobObject/TerminateProcess and
GetExitCodeProcess, but these are asynchronous and don't wait for the process to
actually exit. Adding a wait call should guarantee the process has exited before
continuing which will hopefully ensure the port is free by the time we start an
additional websocketProcessBridge.
MozReview-Commit-ID: HGyjEsy1Ons
--HG--
extra : rebase_source : f56034e1fc0febae07d9b2728eded0a48975baca
Set the idle preference to be the current time, so that idle-daily won't kick in for 24 hours.
MozReview-Commit-ID: 6OJCSm8RaeZ
--HG--
extra : rebase_source : 71217263ddd5b9299e8463254f48ad2d9918b8a2
The main bug this fixes is that on reftest, the objdir needs to be added to the
whitelist on Windows. However, this only happens when running on Linux for some
reason.
Changing the --work-path and --obj-path arguments to --sandbox-read-whitelist
was more of a drive-by cleanup than anything necessary.
MozReview-Commit-ID: Dq8ZLETMzeM
--HG--
extra : rebase_source : 3d2cdda125205e76f86235eb373074899fe0789a
If Firefox crashes while mochitests or reftests are running, Marionette
will trigger an IOError exception which currently gets logged immediately,
and causes no post-test checks to be performed. This results in missing
crash and leak checks, and an unclear failure message on Treeherder.
With this change only the IOError from Marionette gets deferred until
all post-test checks are done. This fixes the failure messages, and will
put PROCESS_CRASH or leak log output first.
MozReview-Commit-ID: JCYP5LlPE1m
--HG--
extra : rebase_source : a4a9455402b01db8ef1dbafccc7a726d2927ec03
This adds new test verification steps for mochitest/reftest/xpcshell tests
with MOZ_CHAOSMODE=3 (thread scheduling and network thread scheduling).
Enabling all chaos mode features seems to destabilize test verification
so I am only enabling these features for now.
This patch makes it possible to disable mochitest-devtools tests on linux64-jsdcov through the 'coverage' flag.
MozReview-Commit-ID: 1gkgbFo19MZ
--HG--
extra : rebase_source : 9c6d311cb467e4424d42848e1caf024f9702b8cd
Currently the mochitest and reftest runners are forcing a timeout of 60s
(or 900s for valgrind or debug builds) when calling `start_session` of
Marionette. While this method still offers a timeout parameter, the
timeout should be set via the `startup_timeout` argument for Marionette.
Reason is that Marionette doesn't control the browser instance and
is getting called right after the application gets launched. As such
slow running builds can cause timeouts once it takes longer than 60s
for the Marionette server to get started.
By using `startup_timeout` the timeout will even be configurable via
the command line and mozharness config files.
MozReview-Commit-ID: EV7GklBcJjU
--HG--
extra : rebase_source : 9d3c623c49deb92d68c40ba4410c812c864e06bd
Bug 1394910 modified the runtests.py return code when one or more tests
failed, but that causes mozharness to emit a starrable error. This patch
returns to the old behavior of normally returning 0 from runtests.py when
tests fail. To allow for test verification, verifyTests() is updated to
explicitly check for failed tests, instead of relying on the runTests()
return code.
This renames killNamedOrphans to killNamedProc and removes the check for
parent proc id, so that any xpcshell or ssltunnel process is killed before
starting a mochitest or reftest run. For reftests, this is moved out of
the desktop harness and into the remote/android harness, since desktop
reftests do not use xpcshell or ssltunnel.
This will only work if runByManifest is enabled, otherwise the harness will error out. It's also
illegal to set this on an individual test, it must be on the entire manifest.
MozReview-Commit-ID: LWYa3Sk1uyW
--HG--
extra : rebase_source : ea4add087c795324745a0a5fc18b3ef50b2292b0
Two small changes: 1. Test verification uses a copy of the runtests.py
options, so that any changes made to the options object in one step do
not affect the next step. 2. Use mochitest url format
.../tests?testname=xxx instead of .../xxx?... when a single test is
being run, regardless of --repeat; this is required so that other
options, notably --keep-open, are respected.
--verify runs tests many times, in various configurations, in hopes of
finding intermittent failures. This option is added mostly for the -
coming soon! - test-verify task in automation, but can be used from
mach too, as a convenient way to stress tests of interest.
Test harnesses may use STYLO_FORCE_ENABLED, so we need to fold this into mozinfo
for mochitest annotations.
MozReview-Commit-ID: JyadTE7apX6
--HG--
extra : rebase_source : 4eda1f150c012a8f5c0a58fdc921d8a6e03c4e68
This patch renames the mozinfo flag 'coverage' to 'ccov' to avoid ambiguity in whether a test is being skipped for linux64-ccov or for linux64-jsdcov. It also removes the 'runtests.py' mozinfo hack and renames all occurrences of 'coverage' that are used for skipping tests in linux64-ccov.
MozReview-Commit-ID: IF2640bDQP7
--HG--
extra : rebase_source : 614020325e30d1ca9e01aaf08479b8a4ffaec888
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.
It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.
Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.
MozReview-Commit-ID: C5ofhfJdpCS
--HG--
extra : rebase_source : 952043a441b41b2f58ec4abc51ac15fa71fc142f
The mochitest harness uses testEnd multiple times to log various failures. This can result
in several testEnd messages, which will soon cause mozlog to spit out an error. Instead,
these should be testStatus.
This also starts using mozlog's assertion_count log action to log test assertions (again,
instead of testEnd).
MozReview-Commit-ID: FFsyicSso5Y
--HG--
extra : rebase_source : 5e8d02714fcf24f2b86a9867b0403bbda0d00f91
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.
It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.
Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.
MozReview-Commit-ID: C5ofhfJdpCS
--HG--
extra : rebase_source : 64aa804965d24bb90b103c00c692a2ac6859e408
Geoff Brown
Henrik Skupin
Dan Minor
Mark Banner
Brendan Dahl
Sebastian Hengst
Andrew Halberstadt
Steve Armand
Greg Mierzwinski
Kris Maglione
Wes Kocher
Botond Ballo
J. Ryan Stinnett
Eric Rahm
Gian-Carlo Pascutto
Bob Owen
Carsten "Tomcat" Book
Xidorn Quan
Haik Aftandilian
Alex Gaynor
Christian Holler