Test harnesses may use STYLO_FORCE_ENABLED, so we need to fold this into mozinfo
for mochitest annotations.
MozReview-Commit-ID: JyadTE7apX6
--HG--
extra : rebase_source : 4eda1f150c012a8f5c0a58fdc921d8a6e03c4e68
This patch renames the mozinfo flag 'coverage' to 'ccov' to avoid ambiguity in whether a test is being skipped for linux64-ccov or for linux64-jsdcov. It also removes the 'runtests.py' mozinfo hack and renames all occurrences of 'coverage' that are used for skipping tests in linux64-ccov.
MozReview-Commit-ID: IF2640bDQP7
--HG--
extra : rebase_source : 614020325e30d1ca9e01aaf08479b8a4ffaec888
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.
It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.
Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.
MozReview-Commit-ID: C5ofhfJdpCS
--HG--
extra : rebase_source : 952043a441b41b2f58ec4abc51ac15fa71fc142f
The mochitest harness uses testEnd multiple times to log various failures. This can result
in several testEnd messages, which will soon cause mozlog to spit out an error. Instead,
these should be testStatus.
This also starts using mozlog's assertion_count log action to log test assertions (again,
instead of testEnd).
MozReview-Commit-ID: FFsyicSso5Y
--HG--
extra : rebase_source : 5e8d02714fcf24f2b86a9867b0403bbda0d00f91
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.
It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.
Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.
MozReview-Commit-ID: C5ofhfJdpCS
--HG--
extra : rebase_source : 64aa804965d24bb90b103c00c692a2ac6859e408
The mochitest harness uses testEnd multiple times to log various failures. This can result
in several testEnd messages, which will soon cause mozlog to spit out an error. Instead,
these should be testStatus.
This also starts using mozlog's assertion_count log action to log test assertions (again,
instead of testEnd).
MozReview-Commit-ID: FFsyicSso5Y
--HG--
extra : rebase_source : f6f3ab02e1c9ef693ea6a857f3e6dc863a075135
For a while now it has been making the content process sandbox less strict.
MozReview-Commit-ID: Am6fGzViaLk
--HG--
extra : rebase_source : 0bc037f205896c866559a7ab1f7e2c042c3142db
This is case that got hit with the new mochitest selftest harness. In this scenario, several MochitestDesktop instances
(which call commandline.setup_logging in their constructor) are instantiated in the same interpreter. Because mozlog
implicitly saves the logger state, this meant that setup_logging kept appending duplicate handlers to the existing ones.
I believe that the intent of 'setup_logging' is to get a brand new logger, so it should ensure logger state is reset
on subsequent calls.
MozReview-Commit-ID: Jqyhbj7nC6z
--HG--
extra : rebase_source : f267489bef99f3ac3d657357002a0001610a038f
This means they will be copied to $PROFILE/extensions, which the sandbox allows
access to; if they are installed as temporary addons, loading frame scripts in
the content process tries to read from wherever they happen to be on disk. This
breaks running tests with a packaged build once we have full read-restrictions
for the content process sandbox.
MozReview-Commit-ID: 7ZiiM9FMXfG
--HG--
extra : rebase_source : d2cf3a2d06df9099dc6056fae351200eaa1d0ca9
Our linux32-debug build is very slow to startup when running mochitests on aws.
Sometimes we see similar behavior on other linux platforms. Intermittently,
in this environment, startup takes longer than the 120 seconds that marionette
waits, resulting in test failures in bug 1261598. Increasing the marionette
startup timeout to 180 seconds appears to effectively avoid these failures.
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.
r=haik,froydnj
MozReview-Commit-ID: KNvAoUs5Ati
--HG--
extra : rebase_source : 81ba8bfee0ca96979cf8e30d75cdd47f06bc10ea
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.
r=haik,froydnj
MozReview-Commit-ID: KNvAoUs5Ati
--HG--
extra : rebase_source : f637acff32fc8582732de932503dd696abc57877
This eliminates a 2 minute timeout seen at the end of Android mochitests
and reftests. Attempts to shutdown the web server were failing because
they were directed at IP 10.0.2.2 -- the loopback address for the
Android emulator.
The gecko messages are now in the "process_output" action, rather than
in the "log" action (except for a few legacy cases), so examine both
when looking for LSAN messages.
MozReview-Commit-ID: 82r1p8WLwFa
--HG--
extra : rebase_source : 5af1c529e58f5ba90a3fd222e3cbbc67a850a08c
By using --wait-for-jsdebugger, we can allow the test suite to start
automatically and also ensure the JS debugger has a chance to connect.
There's still an extra click to get the tests running though (at least on macOS)
because the test harness needs to be focused and the extra process used for the
Browser Toolbox removes the focus from it.
MozReview-Commit-ID: 1Eg7lqG3KST
--HG--
extra : rebase_source : 5df1ac74a3e083ac6c2293fb1cf8a27509b81274
Mochitest currently converts unstructured logs (e.g output from gecko) to 'info' messages. But
this means those messages won't be validated against mozharness' error logs. This change first
gets unstructured messages logged as process_output, and also ensures the StructuredOutputParser
in mozharness checks process_output messages against the error list.
MozReview-Commit-ID: KPTQnulwzyK
--HG--
extra : rebase_source : c9038ff87843b1cb46de506e245715f24a60ed63
Mochitest currently converts unstructured logs (e.g output from gecko) to 'info' messages. But
this means those messages won't be validated against mozharness' error logs. This change first
gets unstructured messages logged as process_output, and also ensures the StructuredOutputParser
in mozharness checks process_output messages against the error list.
MozReview-Commit-ID: KPTQnulwzyK
--HG--
extra : rebase_source : 52f2f048aee5bd40cde29030e7668b321366e9ec
Mochitest currently converts unstructured logs (e.g output from gecko) to 'info' messages. But
this means those messages won't be validated against mozharness' error logs. This change first
gets unstructured messages logged as process_output, and also ensures the StructuredOutputParser
in mozharness checks process_output messages against the error list.
MozReview-Commit-ID: KPTQnulwzyK
--HG--
extra : rebase_source : bdcdbf5567355f28ab88d17b27f44d5dfa0467c2
This patch allows the use of the flag '--jscov-dir-prefix' for mochitest plain tests to enable code coverage collection with the JS Debugger. It also enables the mochitest-plain tests for the linux64-jsdcov build platform.
MozReview-Commit-ID: 6RqMEZ1I0D7
--HG--
extra : rebase_source : 351754541801f69f7c54807f6bdd3a3d1baf9222
Include the current flavor in the "No tests found" error message. Also suggest
what may have went wrong.
MozReview-Commit-ID: 5LEQFVDoJrT
--HG--
extra : rebase_source : 6abe8c6f467a11ff8abc33a37eefab80505e3cfb
This fixes a regression in bug 1332573 where if no tests are found by the getActiveTests
function, the status variable never gets set. This ensures that we always set status, and
that it will be set to a non-zero return code if any of the calls to runApp fail.
MozReview-Commit-ID: 20M7FcBs0DF
--HG--
extra : rebase_source : d33a1a052986df628a90146e6de29aa415d8a1ad
Haik Aftandilian
J. Ryan Stinnett
Andrew Halberstadt
Greg Mierzwinski
Alex Gaynor
Sebastian Hengst
Christian Holler
Carsten "Tomcat" Book
Phil Ringnalda
Shing Lyu
Geoff Brown
Chris Manchester
James Graham
Ting-Yu Chou
Wes Kocher
Madeleine Chercover
Iris Hsiao
Xidorn Quan
Bob Clary
Luke Wagner