Guard against flaky results from the remote is_file and reduce adb traffic
by waiting once for the log file to be created rather than checking for it
throughout the test run.
(Minor ride-along change to reftest.jsm resolves javascript error introduced
by bug 1575266.)
Differential Revision: https://phabricator.services.mozilla.com/D55026
--HG--
extra : moz-landing-system : lando
Whereas:
- desktop tests don't make this check;
- the check for directory existence has been troublesome and almost never useful;
- bug classification of this condition has been troublesome;
- if a startup crash actually did occur before crashreporter init, there would still be an indication in logcat and possibly a tombstone, and the "No test summary found" check would definitely be triggered;
Let's stop checking for minidumps directory creation.
Differential Revision: https://phabricator.services.mozilla.com/D54755
--HG--
extra : moz-landing-system : lando
Improve support for Python 3 by running modernize over adb.py and a few other related changes.
Differential Revision: https://phabricator.services.mozilla.com/D19306
--HG--
extra : moz-landing-system : lando
Other than the MediaPipelineFactory references, nothing else is obsolete, so this is a
very minor cleanup, just to improve readability a bit.
Differential Revision: https://phabricator.services.mozilla.com/D25124
--HG--
extra : moz-landing-system : lando
The existing ANR and tombstone checks in remoteautomation are used
by mochitests and reftests but are awkward for other harnesses like cppunit
that do not use remoteautomation...and easily missed. This patch moves that
code to the mozharness AndroidMixin, treating ANRs and tombstones like the
logcat: Make sure any old logs are deleted when the mozharness script starts,
then move any logs found at the end of the run to the upload directory.
https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=f3de1e9836da2b0c9232f5d92c751b979459e19b
demonstrates tombstone artifacts for the Android 7.0 cppunit tests.
Differential Revision: https://phabricator.services.mozilla.com/D21774
--HG--
extra : moz-landing-system : lando
This makes the changes necessary to use TestRunnerActivity when geckoview
is installed and requested, but we do not yet attempt to run any such
test tasks in automation.
getTopActivity calls 'adb shell dumpsys window windows' which intermittently
times out. Instead of ending the test run when that happens, let's try
again, in hopes of recovering. If recovery fails and the browser is
subsequently found running, log an explanation before killing it.
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.
It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.
Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.
MozReview-Commit-ID: C5ofhfJdpCS
--HG--
extra : rebase_source : 952043a441b41b2f58ec4abc51ac15fa71fc142f
sutagent is no longer built or used; devicemanagerSUT is completely
unused. After this change, devicemanagerADB is the only implementation of
devicemanager, and test harness options like --dm_trans are eliminated.
Android tests use dumpsys to determine the current "top activity";
if Firefox is not in the foreground, tests are considered complete.
But dumpsys is heavy-weight and can fail, for reasons unknown. With
this patch, test harnesses continue to use dumpsys to determine the
top activity, but call it much less often: If the harness has just
received new test output, the harness assumes that tests are in
progress and does not check the top activity.
We hit an issue where adding a new env var, MOZ_DISABLE_TELEMETRY, added env10
and caused crashes. I suspect the issue is that there are is now a double-digit
number of env vars (bug 1277390). Here, we do the quick fix by removing
MOZ_DISABLE_TELEMETRY & repurposing MOZ_DISABLE_SWITCHBOARD to be generic.
While we're at it, we simplify the code by making the setDisabled methods a
strict getter without checking for how many times they're called.
MozReview-Commit-ID: 19DDbVYRZ2
--HG--
extra : rebase_source : 1590ae4f49bf725ab8a3bb26f10dab324903aa8c
There is an ImportError on Android, as well as a log related
regression from the structured log patch once that is fixed.
MozReview-Commit-ID: KxSEotr38qO
--HG--
extra : rebase_source : 15d8421aab813d9e0dbf6d00611f921aaa779a49
Structured logs bring many benefits. We can stop parsing the logs for magic strings, we
can modify the format without breaking things, and we can stream results into systems like
ActiveData. The structured logs originate primarily in reftest.js. StructuredLog.jsm is
used to generate the JSON-based log stream. Finally OutputHandler in the python harness
reads structured output from stdout, and formats it into human readable form.
MozReview-Commit-ID: G3ZLkMRl6p7
--HG--
extra : commitid : J3ui9XYWR3Q
extra : rebase_source : 77ed0ba842cc8e557141fb3494212b06868c728a
extra : amend_source : 735d48225a2e627e0fe45fc11b50b6c49a885a4b
extra : source : d1779fe421c3c7cd8e3d191816776390dc104f37
Structured logs bring many benefits. We can stop parsing the logs for magic strings, we
can modify the format without breaking things, and we can stream results into systems like
ActiveData. The structured logs originate primarily in reftest.js. StructuredLog.jsm is
used to generate the JSON-based log stream. Finally OutputHandler in the python harness
reads structured output from stdout, and formats it into human readable form.
--HG--
extra : commitid : J3ui9XYWR3Q
extra : rebase_source : 6bae978126dbd5beddc39332c7cbce0c1354cd87
extra : amend_source : 735d48225a2e627e0fe45fc11b50b6c49a885a4b
Geoff Brown
Marian Raiciof
Bob Clary
Sylvestre Ledru
Mike Hommey
Csoregi Natalia
John Lin
Christian Holler
Michael Comella
Andrew Halberstadt
Margaret Leibovic
Phil Ringnalda
Wes Kocher