Граф коммитов

116 Коммитов

Автор SHA1 Сообщение Дата
Geoff Brown 27c0a1415f Bug 1507207 - Improve support for e10s mode testing on Android; r=bc 2018-11-27 09:41:13 -07:00
Geoff Brown fa50b896b4 Bug 1509534 - Simplify Android runApp; r=bc 2018-11-27 09:41:12 -07:00
Geoff Brown a6aa854ab5 Bug 1503587 - Check for and kill errant crashreporter after Android browser tests; r=bc 2018-10-31 17:23:47 -06:00
Bob Clary 129bad1f48 Bug 1482887 - Remote automation should kill already running instances rather than dying, r=gbrown. 2018-08-18 10:47:12 -07:00
Bob Clary fbd6d3dfca Bug 1482878 - Use screencap to capture on physical android device screenshots, r=gbrown. 2018-08-18 10:47:12 -07:00
Bob Clary be2ccf78f3 Bug 1482874 - Handle Android 8.1 change for anr stack directory to contain individual files for traces, r=gbrown. 2018-08-18 10:47:12 -07:00
Geoff Brown 8b4653a7b7 Bug 1480546 - Enhance logging for Android automation app wait; r=bc 2018-08-07 10:05:37 -06:00
Bob Clary 745c5ff6c2 Bug 1477807 - Distinguish ADBTimeoutErrors from other exceptions in Android remote tests, r=gbrown. 2018-07-27 08:27:16 -07:00
Geoff Brown bea4bee944 Bug 1472032 - Increase Android browser time-out to 110 minutes; r=me,a=test-only
This time-out has traditionally been 90 minutes. It mustn't exceed the 120 minute
task time-out for android browser tests. Let's try 110 minutes.
2018-06-29 16:05:58 -06:00
Geoff Brown 4d05f461aa Bug 1464866 - Fix a typo in Android time-out message; r=me,a=test-only 2018-05-28 11:29:34 -06:00
Sylvestre Ledru dcfef841a7 bug 1463425 - Fix flake8/pep8 issue by hand in build/ r=gps
MozReview-Commit-ID: AZdcEWyVV6e

--HG--
extra : rebase_source : b1c45028c8d46be5ba590a27a2f9f20e248a26b1
2018-05-21 23:58:19 +02:00
Sylvestre Ledru 8cd16bb55b bug 1463425 - autopep8 on build/ r=gps
MozReview-Commit-ID: ETzx4HsjbEF

--HG--
extra : rebase_source : 7e27a4cfe2bb358d513a18a33c245bcc6d559c46
2018-05-21 23:56:34 +02:00
Geoff Brown 3680684f70 Bug 1452694 - Use TestRunnerActivity for geckoview mochitest/reftest; r=bc
This makes the changes necessary to use TestRunnerActivity when geckoview
is installed and requested, but we do not yet attempt to run any such
test tasks in automation.
2018-04-10 13:26:11 -06:00
Geoff Brown b4dddb3c35 Bug 1440719 - Set MOZ_CRASHREPORTER_SHUTDOWN during tests, other than browser-chrome and non-e10s; r=ted 2018-03-28 12:43:52 -06:00
Geoff Brown cf1401dff9 Bug 1448697 - Try to use force-stop instead of kill to end remote applications; r=bc
This is important for running browser tests on some non-privileged devices, where
kill may not be permitted.
2018-03-26 12:21:46 -06:00
Geoff Brown 6083932cff Bug 1440714 - Convert Android browser test harnesses to adb.py; r=bc
This affects Android robocop, mochitest (all flavors) and reftests (all flavors).
2018-03-23 18:06:27 -06:00
Geoff Brown 385ab244a3 Bug 1440674 - Simplify some code in remoteautomation.py; r=jmaher
remove RemoteAutomation._product and product options;
remove some unused code paths;
remove "blobber" references;
remove incorrect and/or unhelpful comments;
s/_devicemanager/_dm/ (shorter, improve readability)
2018-02-23 09:47:57 -07:00
Geoff Brown 0531bf800b Bug 1436262 - Improve efficiency of detection of end of test Android test runs; r=jmaher
Once a test summary is seen, check for application completion more often,
so that we can complete the test run sooner.
2018-02-07 13:33:44 -07:00
Geoff Brown 8b8a2994e8 Bug 1435433 - Print master test summary after android mochitests when runByManifest; r=jmaher 2018-02-07 13:33:43 -07:00
Mike Hommey 2b2a874b47 Bug 1420355 - Statically link DMD. r=njn
--HG--
extra : rebase_source : 8e7cf975d096116b666532f3fe8aa5a7f61b5725
2017-11-28 08:10:48 +09:00
Csoregi Natalia 4ce8d0124c Backed out 7 changesets (bug 1420355) for mass failures on OS X and Android. r=backout on a CLOSED TREE
Backed out changeset a7ed89e13a4c (bug 1420355)
Backed out changeset fd6702e6e0a0 (bug 1420355)
Backed out changeset 0479dda078a2 (bug 1420355)
Backed out changeset e69357ccca9e (bug 1420355)
Backed out changeset 3742a4b69ba2 (bug 1420355)
Backed out changeset 451cd087922f (bug 1420355)
Backed out changeset d80b5c4e1dd0 (bug 1420355)
2017-11-29 03:08:46 +02:00
Mike Hommey 7c62087a43 Bug 1420355 - Statically link DMD. r=njn
--HG--
extra : rebase_source : 46800c9c0c5006a5a32f11abc209da27e65ae0f5
2017-11-28 08:10:48 +09:00
John Lin 93c62b59f7 Bug 1417006 - remove tombstone files but keep the directory. r=dminor
MozReview-Commit-ID: Ds87KZ47vf1

--HG--
extra : rebase_source : 0578a58107b39abb546758cd8aa21c89e7b4b81f
2017-11-13 16:10:35 +08:00
Geoff Brown b6387e7f30 Bug 1412297 - Retry devicemanager getTopActivity once; r=jmaher
getTopActivity calls 'adb shell dumpsys window windows' which intermittently
times out. Instead of ending the test run when that happens, let's try
again, in hopes of recovering. If recovery fails and the browser is
subsequently found running, log an explanation before killing it.
2017-10-31 15:58:31 -06:00
Geoff Brown 3c8e31bdb7 Bug 1402347 - Be more exact in Android tests when checking run time; r=jmaher 2017-09-22 11:32:38 -06:00
Christian Holler b8ee76ebd8 Bug 1373256 - Changes to support -fsanitize=integer in the codebase. r=froydnj
The -fsanitize=integer analysis from UBSan can be helpful to detect signed and unsigned integer overflows in the codebase. Unfortunately, those occur very frequently, making it impossible to test anything with it without the use of a huge blacklist. This patch includes a blacklist that is broad enough to silence everything that would drain performance too much. But even with this blacklist, neither tests nor fuzzing is "clean". We can however in the future combine this with static analysis to limit ourselves to interesting places to look at, or improve the dynamic analysis to omit typical benign overflows.

It also adds another attribute that can be used on functions. It is not used right now because it was initially easier to add things to the compile-time blacklist to get started.

Finally, it includes a runtime suppression list and patches various parts in the test harnesses to support that. It is currently empty and it should not be used on frequent overflows because it is expensive. However, it has the advantage that it can be used to differentiate between signed and unsigned overflows while the compile-time blacklist cannot do that. So it can be used to e.g. silence unsigned integer overflows on a file or function while still reporting signed issues. We can also use this suppression list for any other UBSan related suppressions, should we ever want to use other features from that sanitizer.

MozReview-Commit-ID: C5ofhfJdpCS

--HG--
extra : rebase_source : 952043a441b41b2f58ec4abc51ac15fa71fc142f
2017-04-09 12:59:26 +02:00
Geoff Brown df122d5bfa Bug 1340584 - Remove devicemanagerSUT and references to sutagent; r=wlach
sutagent is no longer built or used; devicemanagerSUT is completely
unused. After this change, devicemanagerADB is the only implementation of
devicemanager, and test harness options like --dm_trans are eliminated.
2017-03-21 12:20:01 -06:00
Geoff Brown 839ad41abf Bug 1319196 - Call dumpsys less during Android tests; r=jmaher
Android tests use dumpsys to determine the current "top activity";
if Firefox is not in the foreground, tests are considered complete.
But dumpsys is heavy-weight and can fail, for reasons unknown. With
this patch, test harnesses continue to use dumpsys to determine the
top activity, but call it much less often: If the harness has just
received new test output, the harness assumes that tests are in
progress and does not check the top activity.
2017-01-06 16:35:29 -07:00
Geoff Brown d1584879a8 Bug 1319196 - Take screenshot when dumpsys fails; r=jmaher 2016-12-13 11:28:06 -07:00
Geoff Brown 094fe98143 Bug 1309436 - Add test name to java-exception failure messages; r=ted 2016-10-26 11:31:41 -06:00
Geoff Brown 306a94e967 Bug 1290114 - Make Android test logging more responsive; r=ahal 2016-07-29 08:29:53 -06:00
Michael Comella af565b6cbb Bug 1277214 - Move MOZ_DISABLE_* to single MOZ_IN_AUTOMATION env var. r=grisha
We hit an issue where adding a new env var, MOZ_DISABLE_TELEMETRY, added env10
and caused crashes. I suspect the issue is that there are is now a double-digit
number of env vars (bug 1277390). Here, we do the quick fix by removing
MOZ_DISABLE_TELEMETRY & repurposing MOZ_DISABLE_SWITCHBOARD to be generic.

While we're at it, we simplify the code by making the setDisabled methods a
strict getter without checking for how many times they're called.

MozReview-Commit-ID: 19DDbVYRZ2

--HG--
extra : rebase_source : 1590ae4f49bf725ab8a3bb26f10dab324903aa8c
2016-06-01 16:46:44 -07:00
Michael Comella 1e85dedec9 Bug 1270191 - Disable telemetry upload during tests. r=grisha
MozReview-Commit-ID: GoQQRfGRvd4

--HG--
extra : rebase_source : c2d56638e078012766e8f7c9b64de6d3988e6dbc
2016-05-31 17:04:40 -07:00
Andrew Halberstadt 6e6c1c8aae Bug 1245092 - Install reftest and specialpowers extensions at runtime via AddonManager.installTemporaryAddon, r=jgriffin
MozReview-Commit-ID: 5HSa3nFVTeF

--HG--
rename : layout/tools/reftest/reftest.js => layout/tools/reftest/reftest.jsm
extra : rebase_source : 924a3e937b47cac03674acdb56334882a2373f50
2016-03-09 14:38:13 -05:00
Margaret Leibovic e0825d6416 Bug 1247324 - Disable Switchboard in automation. r=gbrown,mfinkle
MozReview-Commit-ID: ItqHBIEdD0Q

--HG--
extra : rebase_source : 639eebe0ee5854beef82695729d73dc001af8fac
2016-02-18 12:03:51 -05:00
Andrew Halberstadt dee8b08ce5 Bug 1231784 - Install specialpowers and mochikit extensions at runtime via AddonManager.loadTemporaryAddon(), r=jgriffin
MozReview-Commit-ID: 9I56x6Vnbf7

--HG--
rename : testing/mochitest/b2g_start_script.js => testing/mochitest/start_b2g.js
extra : commitid : CrUflHO8tRa
extra : rebase_source : c50792dfe82e688b52aa07150ddab03e5d0d3f36
extra : amend_source : 1dcd8ae7d74b3397e965672b6dc05d431ab4a916
extra : source : edcbd7c0aa36be0846f2e8e989fb57c9e4449cc4
2016-01-25 09:55:57 -05:00
Andrew Halberstadt 97c3a77bc1 Bug 1247054 - Fix |mach reftest| on Android, r=gbrown
There is an ImportError on Android, as well as a log related
regression from the structured log patch once that is fixed.

MozReview-Commit-ID: KxSEotr38qO

--HG--
extra : rebase_source : 15d8421aab813d9e0dbf6d00611f921aaa779a49
2016-02-09 16:19:44 -05:00
Andrew Halberstadt f94d7e71d8 Bug 1034290 - Use structured log output for test results in reftest, r=jmaher
Structured logs bring many benefits. We can stop parsing the logs for magic strings, we
can modify the format without breaking things, and we can stream results into systems like
ActiveData. The structured logs originate primarily in reftest.js. StructuredLog.jsm is
used to generate the JSON-based log stream. Finally OutputHandler in the python harness
reads structured output from stdout, and formats it into human readable form.

MozReview-Commit-ID: G3ZLkMRl6p7

--HG--
extra : commitid : J3ui9XYWR3Q
extra : rebase_source : 77ed0ba842cc8e557141fb3494212b06868c728a
extra : amend_source : 735d48225a2e627e0fe45fc11b50b6c49a885a4b
extra : source : d1779fe421c3c7cd8e3d191816776390dc104f37
2016-02-05 15:44:20 -05:00
Phil Ringnalda 66e6da78e6 Back out d1779fe421c3 (bug 1034290) for UnicodeDecodeErrors in things run by the reftest harness
CLOSED TREE
2016-02-05 08:18:47 -08:00
Andrew Halberstadt 8a462fd84b Bug 1034290 - Use structured log output for test results in reftest, r=jmaher
Structured logs bring many benefits. We can stop parsing the logs for magic strings, we
can modify the format without breaking things, and we can stream results into systems like
ActiveData. The structured logs originate primarily in reftest.js. StructuredLog.jsm is
used to generate the JSON-based log stream. Finally OutputHandler in the python harness
reads structured output from stdout, and formats it into human readable form.

--HG--
extra : commitid : J3ui9XYWR3Q
extra : rebase_source : 6bae978126dbd5beddc39332c7cbce0c1354cd87
extra : amend_source : 735d48225a2e627e0fe45fc11b50b6c49a885a4b
2015-10-29 15:01:44 -04:00
Wes Kocher aaf89c475e Backed out 3 changesets (bug 1231784) for breaking 10.10 opt jetpack tests
Backed out changeset 653df21ca861 (bug 1231784)
Backed out changeset 0aac20bc3619 (bug 1231784)
Backed out changeset edcbd7c0aa36 (bug 1231784)

--HG--
rename : testing/mochitest/start_b2g.js => testing/mochitest/b2g_start_script.js
extra : commitid : DLurSs6PJ1G
2016-02-03 17:38:12 -08:00
Andrew Halberstadt ae410fb5a9 Bug 1231784 - Install specialpowers and mochikit extensions at runtime via AddonManager.loadTemporaryAddon(), r=jgriffin
--HG--
rename : testing/mochitest/b2g_start_script.js => testing/mochitest/start_b2g.js
extra : commitid : CrUflHO8tRa
extra : rebase_source : 0a041c36b59c79c1bea755b22fe07017cb3d0aa5
extra : amend_source : 1dcd8ae7d74b3397e965672b6dc05d431ab4a916
2016-01-25 09:55:57 -05:00
Geoff Brown b0a570ab95 Bug 1156425 - Allow Android test jobs to run for up to 90 minutes; r=RyanVM 2015-08-27 11:29:00 -06:00
Andrew Halberstadt 2be5ff0c10 Bug 1014760 - Move mozlog.structured to mozlog; Move mozlog to mozlog.unstructured, r=jgraham
Mozlog currently has two implementations. The top level package is based on the logging module and is
deprecated. The newer structured logging implementation lives in mozlog.structured. This patch swaps the
two, so the top level mozlog module contains the recommended implementation, while mozlog.unstructured
contains the old one.

--HG--
rename : testing/mozbase/docs/mozlog_structured.rst => testing/mozbase/docs/mozlog.rst
rename : testing/mozbase/mozlog/mozlog/structured/commandline.py => testing/mozbase/mozlog/mozlog/commandline.py
rename : testing/mozbase/mozlog/mozlog/structured/formatters/__init__.py => testing/mozbase/mozlog/mozlog/formatters/__init__.py
rename : testing/mozbase/mozlog/mozlog/structured/formatters/base.py => testing/mozbase/mozlog/mozlog/formatters/base.py
rename : testing/mozbase/mozlog/mozlog/structured/formatters/errorsummary.py => testing/mozbase/mozlog/mozlog/formatters/errorsummary.py
rename : testing/mozbase/mozlog/mozlog/structured/formatters/html/__init__.py => testing/mozbase/mozlog/mozlog/formatters/html/__init__.py
rename : testing/mozbase/mozlog/mozlog/structured/formatters/html/html.py => testing/mozbase/mozlog/mozlog/formatters/html/html.py
rename : testing/mozbase/mozlog/mozlog/structured/formatters/html/main.js => testing/mozbase/mozlog/mozlog/formatters/html/main.js
rename : testing/mozbase/mozlog/mozlog/structured/formatters/html/style.css => testing/mozbase/mozlog/mozlog/formatters/html/style.css
rename : testing/mozbase/mozlog/mozlog/structured/formatters/html/xmlgen.py => testing/mozbase/mozlog/mozlog/formatters/html/xmlgen.py
rename : testing/mozbase/mozlog/mozlog/structured/formatters/machformatter.py => testing/mozbase/mozlog/mozlog/formatters/machformatter.py
rename : testing/mozbase/mozlog/mozlog/structured/formatters/tbplformatter.py => testing/mozbase/mozlog/mozlog/formatters/tbplformatter.py
rename : testing/mozbase/mozlog/mozlog/structured/formatters/unittest.py => testing/mozbase/mozlog/mozlog/formatters/unittest.py
rename : testing/mozbase/mozlog/mozlog/structured/formatters/xunit.py => testing/mozbase/mozlog/mozlog/formatters/xunit.py
rename : testing/mozbase/mozlog/mozlog/structured/handlers/__init__.py => testing/mozbase/mozlog/mozlog/handlers/__init__.py
rename : testing/mozbase/mozlog/mozlog/structured/handlers/base.py => testing/mozbase/mozlog/mozlog/handlers/base.py
rename : testing/mozbase/mozlog/mozlog/structured/handlers/bufferhandler.py => testing/mozbase/mozlog/mozlog/handlers/bufferhandler.py
rename : testing/mozbase/mozlog/mozlog/structured/handlers/statushandler.py => testing/mozbase/mozlog/mozlog/handlers/statushandler.py
rename : testing/mozbase/mozlog/mozlog/structured/logtypes.py => testing/mozbase/mozlog/mozlog/logtypes.py
rename : testing/mozbase/mozlog/mozlog/structured/reader.py => testing/mozbase/mozlog/mozlog/reader.py
rename : testing/mozbase/mozlog/mozlog/structured/scripts/__init__.py => testing/mozbase/mozlog/mozlog/scripts/__init__.py
rename : testing/mozbase/mozlog/mozlog/structured/scripts/format.py => testing/mozbase/mozlog/mozlog/scripts/format.py
rename : testing/mozbase/mozlog/mozlog/structured/scripts/logmerge.py => testing/mozbase/mozlog/mozlog/scripts/logmerge.py
rename : testing/mozbase/mozlog/mozlog/structured/scripts/unstable.py => testing/mozbase/mozlog/mozlog/scripts/unstable.py
rename : testing/mozbase/mozlog/mozlog/structured/stdadapter.py => testing/mozbase/mozlog/mozlog/stdadapter.py
rename : testing/mozbase/mozlog/mozlog/structured/structuredlog.py => testing/mozbase/mozlog/mozlog/structuredlog.py
rename : testing/mozbase/mozlog/mozlog/logger.py => testing/mozbase/mozlog/mozlog/unstructured/logger.py
rename : testing/mozbase/mozlog/mozlog/loggingmixin.py => testing/mozbase/mozlog/mozlog/unstructured/loggingmixin.py
rename : testing/mozbase/mozlog/mozlog/loglistener.py => testing/mozbase/mozlog/mozlog/unstructured/loglistener.py
extra : commitid : 3JWk75JY4N0
extra : rebase_source : 229bad7a7bf8ead013ce62d128c0896a75cad393
2015-07-16 10:38:40 -04:00
Geoff Brown 65e4c8cee0 Bug 1179981 - A new test harness for robocop; r=jmaher 2015-07-09 12:30:01 -06:00
Geoff Brown a9b255a8eb Bug 1175540 - Reduce timeouts for many adb devicemanager calls; r=mcote 2015-06-21 13:58:34 -06:00
Nick Alexander 97786c5695 Bug 1169476 -- Implement |mach robocop --serve|. r=gbrown
This adds a flag to |mach robocop| that does everything to run a
Robocop test except launch the actual test.  Instead of launching the
test, it starts the mochi.test server and launches Fennec with a test
profile; then it sits and waits forever.

This allows regular Java IDEs (IntelliJ, but previously Eclipse) to
run Robocop tests like regular instrumentation tests, "injecting" them
into the prepared testing environment.  It's quite nice!

--HG--
extra : rebase_source : a5ab08222110a20291aebe70ef1fda0d340dbe7d
extra : source : e91ac9a35f86928fcd519911476ee7d68d06f921
2015-05-29 17:18:07 -07:00
Carsten "Tomcat" Book a723ed3ad7 Backed out changeset e91ac9a35f86 (bug 1169476) for android rc2 crashes 2015-06-01 09:38:42 +02:00
Nick Alexander f242708503 Bug 1169476 -- Implement |mach robocop --serve|. r=gbrown
This adds a flag to |mach robocop| that does everything to run a
Robocop test except launch the actual test.  Instead of launching the
test, it starts the mochi.test server and launches Fennec with a test
profile; then it sits and waits forever.

This allows regular Java IDEs (IntelliJ, but previously Eclipse) to
run Robocop tests like regular instrumentation tests, "injecting" them
into the prepared testing environment.  It's quite nice!

--HG--
extra : rebase_source : 15e9e8d5c311312e2eb317936e5d154237c1f9a3
extra : histedit_source : 050ac958ae8580f45e5011a0d353bf13e65d5ff3
2015-05-25 14:27:21 -07:00
Geoff Brown 51f4882b20 Bug 1156425 - Allow Android test jobs to run for up to 75 minutes; r=ryanvm 2015-05-04 10:32:46 -06:00