This just adds two basic tests, one for a passing test and another for a
failing one. In mochitest, we use privileged APIs to also tests crashes,
assertions, asan and leaks. But these APIs aren't available to reftests
so I'm not sure how we can test these things.
I figure it's not worth holding the framework up on this though, I'll file
a follow-up to figure out something to do for that.
MozReview-Commit-ID: 59TSbsugT5T
--HG--
extra : rebase_source : 72ecd817017c8b7d55eab879db4f6ad5fecc54c0
This includes code for downloading a Firefox binary, downloading + setting up a tests.zip and
running output through mozharness' output parsers. This is all stuff that will also be required
for the reftest selftests.
I couldn't think of a better location to put this stuff, suggestions welcome.
MozReview-Commit-ID: 59TSbsugT5T
--HG--
extra : rebase_source : a328f6bc90e73fe23f9054933cd01a30065419f6
dm.killProcess correctly tries to use 'am force-stop' in preference to kill()
to end a process. But some clients of killProcess specify a kill signal and
use killProcess for purposes other than endding the process, for example, to trigger
crash dumps. To allow for these cases, it is better to not use force-stop when a
signal is specified.
This switches most tests over to use pytest as the runner instead of unittest (taking
advantage of the fact that pytest can run unittest based tests).
There were a couple tests that had failures when swithing to pytest:
config/tests/unit-expandlibs.py
xpcom/idl-parser/xpidl/runtests.py
For these tests, I added a runwith='unittest' argument so that they still run the
same way as before. Once we fix them to use pytest, the unittest logic in mozunit.py
can be deleted.
MozReview-Commit-ID: Gcsz6z8MeOi
--HG--
extra : rebase_source : 3c762422ce0af54cbbe7d9fc20085a2d1ebe7057
The poll() call in SystemResourceMonitor.stop might fail even though
there is something to read from the pipe, in some corner cases, and
python won't let us know about it. In that case, an exception is thrown,
leaving the SystemResourceMonitor (and its callers) in a weird state. In
practice, this leads BuildMonitor.__exit__ to recall stop, which then
fails.
So when poll() throws an exception, we pretend there's still something
to read, and we try to read anyways. If there is something to read,
recv() will return it, otherwise, it will throw an exception of its own,
which we catch, pretending we're done.
Furthermore, when there is nothing to read from the pipe, poll() simply
returns False, and our loop never sets `done` to True, and we then hit
an assert, which doesn't have its place here, so we remove it.
Finally, the other end of the pipe might have died at any time, making
sending over the pipe fail, so we also protect against that.
With all these changes, it feels like the reason to backout bug 1239939
in bug 1272782 should have been dealt with, and we can drop the timeout
again.
--HG--
extra : rebase_source : ac72dd5b2602cf3ffddfb429f95e02380f939893
The poll() call in SystemResourceMonitor.stop might fail even though
there is something to read from the pipe, in some corner cases, and
python won't let us know about it. In that case, an exception is thrown,
leaving the SystemResourceMonitor (and its callers) in a weird state. In
practice, this leads BuildMonitor.__exit__ to recall stop, which then
fails.
So when poll() throws an exception, we pretend there's still something
to read, and we try to read anyways. If there is something to read,
recv() will return it, otherwise, it will throw an exception of its own,
which we catch, pretending we're done.
Furthermore, when there is nothing to read from the pipe, poll() simply
returns False, and our loop never sets `done` to True, and we then hit
an assert, which doesn't have its place here, so we remove it.
Finally, the other end of the pipe might have died at any time, making
sending over the pipe fail, so we also protect against that.
With all these changes, it feels like the reason to backout bug 1239939
in bug 1272782 should have been dealt with, and we can drop the timeout
again.
--HG--
extra : rebase_source : fededf989fe9021654b67d5a070f7e49aa717f3c
Currently manifestparser will only look for line continuations *after* looking for a key. This means
that line continuations cannot contain key separators. For example, this:
[test]
foo=
bar=baz
gets treated as:
{'name': 'test', 'foo': '', 'bar': 'baz'}
Here, bar=baz will be treated as a new key/value pair despite the indentation. This patch switches
the order around, so we look for a continuation first. Now, it is only treated as a continuation if
the indent is greater than the indent of the preceding key.
So this manifest:
[test]
foo=bar
baz=fleem
is a continuation and results in:
{'name': 'test', 'foo': 'bar\nbaz=fleem'}
But this manifest:
[test]
foo=bar
baz=fleem
is not a continuation, and yields:
{'name': 'test', 'foo': 'bar', 'baz': 'fleem'}
MozReview-Commit-ID: FAMP5TUIo9q
--HG--
extra : rebase_source : 624c53cfe0565374c1224dd86a3fffc8831279d3
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.
It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.
Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.
MozReview-Commit-ID: C5ofhfJdpCS
--HG--
extra : rebase_source : 952043a441b41b2f58ec4abc51ac15fa71fc142f
This is needed before we can upgrade to flake8 3.3.0, as that version starts flagging these errors.
These files were modified by running:
autopep8 --select E305 --in-place -r <dir>
on the affected directories. I did it one dir at a time and verified the result after each.
MozReview-Commit-ID: FmlsfiKIbtr
--HG--
extra : rebase_source : 9df32258cadff5d27a0e72113c57f782756c0b18
We've seen a couple of enormous log files (200MB+). This largely happened due to a bug in the test harness that resulted
in suite_start being called over and over again. In each of these instances, mozlog logged something like:
log.error("Suite start called multiple times: {}".format(data))
The problem is that 'data' contained every single test id in the suite, which was *a lot*. Dumping all test ids in that
error message is not useful for debugging. This patch limits the size of the 'suite_start' data in the error message to
100 characters.
MozReview-Commit-ID: GnPizNrZ2QJ
--HG--
extra : rebase_source : 985d484544da9ea4cce445ce406fe085f86f112b
This is case that got hit with the new mochitest selftest harness. In this scenario, several MochitestDesktop instances
(which call commandline.setup_logging in their constructor) are instantiated in the same interpreter. Because mozlog
implicitly saves the logger state, this meant that setup_logging kept appending duplicate handlers to the existing ones.
I believe that the intent of 'setup_logging' is to get a brand new logger, so it should ensure logger state is reset
on subsequent calls.
MozReview-Commit-ID: Jqyhbj7nC6z
--HG--
extra : rebase_source : f267489bef99f3ac3d657357002a0001610a038f
This is a minor convenience for downloading the installer from a url. It uses requests (which should be
available everywhere in-tree).
MozReview-Commit-ID: 8IfiVkYNr06
--HG--
extra : rebase_source : cb8798cf3adb61008a5dac3794043950e48c3d6a
This allows subprocesses to log to a shared stream via a queue, so that we
avoid the overhead of a multiprocessing Lock around all log access, but still
avoid races where two processes try to log simultaneously. It's mostly useful
where one process is responsible for the majority of logging, but some messages
will be generated in child processes.
MozReview-Commit-ID: ABl6cvpb6qI
--HG--
extra : rebase_source : 5c749074c1646c7abb865a71b31b3056137ef398
The devicemanager killProcess() is updated to use force-stop first, then
use kill if force-stop does not work.
Browser test harnesses are updated to check if killProcess() worked, and
warn if it failed.
Various minor improvements to aid debugging:
- recommend --verbose on most common failure
- in verbose mode, display platform
- in verbose mode, display file creation date of binaries
- in verbose mode, display sdk binary versions
- remind of x86 vs arm emulator and need for corresponding apk
Kartikaya Gupta
Andrew Halberstadt
Myk Melez
Cosm
Geoff Brown
Ganti Sai Sarath Chandra
Wes Kocher
Eric Rahm
Mike Hommey
Phil Ringnalda
William Lachance
Dave Hunt
Tarek Ziadé
Sylvestre Ledru
Alex Gaynor
Sebastian Hengst
Christian Holler
James Graham
Carsten "Tomcat" Book
Dão Gottwald
Ting-Yu Chou
Maja Frydrychowicz