This also starts running the selftests on linux debug builds, since that's the only place that we
can test assertions and leaks.
MozReview-Commit-ID: JTdTLOLWn5r
--HG--
extra : rebase_source : 643f8d15a550fa39f4d3daba75c60560deefc96e
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.
It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.
Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.
MozReview-Commit-ID: C5ofhfJdpCS
--HG--
extra : rebase_source : 952043a441b41b2f58ec4abc51ac15fa71fc142f
The whitelisting function thisTestLeaksUncaughtRejectionsAndShouldBeFixed was replaced by expectUncaughtRejection, and existing calls did not take effect anymore.
MozReview-Commit-ID: 3uOxkgWYWEz
--HG--
extra : rebase_source : 6b27b77da78743880529adae7adf4ec2aa64dfea
extra : intermediate-source : 5809a30a0c4817509beb1078cf336d8367b8107a
extra : source : 6f0394b523a66dab444b8551deb8f3c6c81d8f31
The mochitest harness uses testEnd multiple times to log various failures. This can result
in several testEnd messages, which will soon cause mozlog to spit out an error. Instead,
these should be testStatus.
This also starts using mozlog's assertion_count log action to log test assertions (again,
instead of testEnd).
MozReview-Commit-ID: FFsyicSso5Y
--HG--
extra : rebase_source : 5e8d02714fcf24f2b86a9867b0403bbda0d00f91
"Automation Error" is not sufficient to turn a mozharness job orange.
mochitest-plain failures of this type normally cause job failure by not
printing out the test suite summary. This patch uses the same technique
for geckoview tests: If a crash is detected, do not print a test summary,
so that mozharness will subsequently fail the job.
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.
It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.
Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.
MozReview-Commit-ID: C5ofhfJdpCS
--HG--
extra : rebase_source : 64aa804965d24bb90b103c00c692a2ac6859e408
This also adds a utility function for synthesizing native touch
events to Eventutils.js.
I did not add a test for searchbar because of intermittent issues
with showing the contextmenu (that are not reproducible manually).
I believe this is rather related to searchbar functionality than
my patches.
MozReview-Commit-ID: Dqm92Saosxz
--HG--
extra : rebase_source : e59df4f487f60cea137fbf8aea71a854a5706de9
This also adds a utility function for synthesizing native touch
events to Eventutils.js.
I did not add a test for searchbar because of intermittent issues
with showing the contextmenu (that are not reproducible manually).
I believe this is rather related to searchbar functionality than
my patches.
MozReview-Commit-ID: Dqm92Saosxz
--HG--
extra : rebase_source : d5c4333609b68773e62447bd3158cadfa89b803b
SpecialPowers.loadChromeScript() sends a script to the child process,
then creates a sandbox, and runs the script in that sandbox. There are
various sandboxOptions that can be passed when creating a sandbox, and
it would be nice to have that functionality for loadChromeScript.
I just need this for wantGlobalProperties, but I might as well make it
as general as possible. I'm not sure all of the types it can take can
actually be serialized across processes, but I guess that's okay.
MozReview-Commit-ID: GoJjXdjizFk
--HG--
extra : rebase_source : 9c2bc190dbf5a080978953cffd64205e8b816367
For a while now it has been making the content process sandbox less strict.
MozReview-Commit-ID: Am6fGzViaLk
--HG--
extra : rebase_source : 0bc037f205896c866559a7ab1f7e2c042c3142db
The browser-chrome test suite now detects and reports unhandled rejections of native Promises, in addition to those created by Promise.jsm. The whitelisting mechanism is updated to use primarily the PromiseTestUtils.expectUncaughtRejection function. Tests will fail if a rejection that is not whitelisted occurs, or if a whitelisted rejection does not occur anymore.
MozReview-Commit-ID: 1beGB5GG8Ty
--HG--
extra : rebase_source : b6573f8e2001f91d0e5a50f6376b191459549e94
extra : intermediate-source : 0411e687044ecc7b56684196238e6e6e68a9d685
extra : source : 8d53be05afc59519c5ce8cfae96d284a972fda71
This is needed before we can upgrade to flake8 3.3.0, as that version starts flagging these errors.
These files were modified by running:
autopep8 --select E305 --in-place -r <dir>
on the affected directories. I did it one dir at a time and verified the result after each.
MozReview-Commit-ID: FmlsfiKIbtr
--HG--
extra : rebase_source : 9df32258cadff5d27a0e72113c57f782756c0b18
This will create a mochitest selftest harness based on |mach python-test|. There
is also a basic test that checks whether TEST-PASS and TEST-UNEXPECTED-FAIL work.
MozReview-Commit-ID: Jqyhbj7nC6z
--HG--
extra : rebase_source : d73b37305590a415e350ee45785a85635e7d4209
This is case that got hit with the new mochitest selftest harness. In this scenario, several MochitestDesktop instances
(which call commandline.setup_logging in their constructor) are instantiated in the same interpreter. Because mozlog
implicitly saves the logger state, this meant that setup_logging kept appending duplicate handlers to the existing ones.
I believe that the intent of 'setup_logging' is to get a brand new logger, so it should ensure logger state is reset
on subsequent calls.
MozReview-Commit-ID: Jqyhbj7nC6z
--HG--
extra : rebase_source : f267489bef99f3ac3d657357002a0001610a038f
This means they will be copied to $PROFILE/extensions, which the sandbox allows
access to; if they are installed as temporary addons, loading frame scripts in
the content process tries to read from wherever they happen to be on disk. This
breaks running tests with a packaged build once we have full read-restrictions
for the content process sandbox.
MozReview-Commit-ID: 7ZiiM9FMXfG
--HG--
extra : rebase_source : d2cf3a2d06df9099dc6056fae351200eaa1d0ca9
The whitelisting function thisTestLeaksUncaughtRejectionsAndShouldBeFixed was replaced by expectUncaughtRejection, and existing calls did not take effect anymore.
MozReview-Commit-ID: 3uOxkgWYWEz
--HG--
extra : rebase_source : 5a10a3ebbfe0ce2a801330041f95447c313a9a70
extra : source : 6f0394b523a66dab444b8551deb8f3c6c81d8f31
The browser-chrome test suite now detects and reports unhandled rejections of native Promises, in addition to those created by Promise.jsm. The whitelisting mechanism is updated to use primarily the PromiseTestUtils.expectUncaughtRejection function. Tests will fail if a rejection that is not whitelisted occurs, or if a whitelisted rejection does not occur anymore.
MozReview-Commit-ID: 1beGB5GG8Ty
--HG--
extra : rebase_source : 64395c5fdf25deebd60dfbf2cf5df3cbf7ca8abb
extra : amend_source : 0a3f13419c050662680f2bd110d724b3bf991732
extra : source : 8d53be05afc59519c5ce8cfae96d284a972fda71
The browser-chrome self-test files now use the setExpectedFailuresForSelfTest function to specify the exact number of assertion failures that will be triggered. Also, most failures are now intercepted when specifying the "fail-if" property in a "browser.ini" manifest, while previously only those triggered using the "ok" function were intercepted. This allows re-enabling several browser-chome self-tests.
MozReview-Commit-ID: DlDjWaJPfvH
--HG--
extra : rebase_source : d9977dac6ecbd2b28f5697d22ce6edf4e1d4f899
extra : source : 6da58c7bb247d3e879012bea8d848eb68f16e36e
The whitelisting function thisTestLeaksUncaughtRejectionsAndShouldBeFixed was replaced by expectUncaughtRejection, and existing calls did not take effect anymore.
MozReview-Commit-ID: 3uOxkgWYWEz
--HG--
extra : rebase_source : 3a7720091180a770b32b595f8094c0d20170166d
The browser-chrome test suite now detects and reports unhandled rejections of native Promises, in addition to those created by Promise.jsm. The whitelisting mechanism is updated to use primarily the PromiseTestUtils.expectUncaughtRejection function. Tests will fail if a rejection that is not whitelisted occurs, or if a whitelisted rejection does not occur anymore.
MozReview-Commit-ID: 1beGB5GG8Ty
--HG--
extra : rebase_source : 59e5b84cb431f3ca28287d30a3da8fbea1363ec5
The browser-chrome self-test files now use the setExpectedFailuresForSelfTest function to specify the exact number of assertion failures that will be triggered. Also, most failures are now intercepted when specifying the "fail-if" property in a "browser.ini" manifest, while previously only those triggered using the "ok" function were intercepted. This allows re-enabling several browser-chome self-tests.
MozReview-Commit-ID: DlDjWaJPfvH
--HG--
extra : rebase_source : 498914c84ab69dd484fb5487ad9967073c331fd3
Several mochitests call `assertSnapshots`, which prints a reftest-like output
for use with the reftest analyzer. Update these lines to check failure patterns
before printing, like the regular assertion methods do.
MozReview-Commit-ID: CfChoar7bp8
Several mochitests call `assertSnapshots`, which prints a reftest-like output
for use with the reftest analyzer. Update these lines to check failure patterns
before printing, like the regular assertion methods do.
MozReview-Commit-ID: CfChoar7bp8
Alex Gaynor
Sebastian Hengst
Andrew Halberstadt
Christian Holler
Carsten "Tomcat" Book
Paolo Amadini
Geoff Brown
Johann Hofmann
Ryan VanderMeulen
Tooru Fujisawa
Marco Bonardo
Andrew McCreight
Florian Quèze
Edgar Chen
Mark Banner
Joel Maher
J. Ryan Stinnett
Phil Ringnalda