Граф коммитов

72 Коммитов

Автор SHA1 Сообщение Дата
Eashan Bhatt a5933d77a4 test_runner: Deprecate --private-key
The workaround for using private key has been fixed on the chromeos
  side and is no longer needed here.

BUG=chromium:972869
TEST=manual

Change-Id: I2bedc47785a0f50cd198017155f294b45fa09e04
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1717996
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Eashan Bhatt <eashanbhatt@google.com>
Cr-Original-Commit-Position: refs/heads/master@{#680983}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 35bc6451f74d62cf8a52e301501ba7bd485ea7d8
2019-07-25 20:38:54 +00:00
Ben Pastene cdb8b3fe48 Capture system logs when running host-side tests on CrOS DUTs.
Bug: 608034
Change-Id: Iec72254dd2949c329be60980f27dd7c940b26fcc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1717158
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#680869}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 908863ca0e08af19ae9a30f29821b6340847a6a8
2019-07-25 16:20:03 +00:00
Keiichi Watanabe 5d238c68de Remove modprobe for vivid on Chrome OS VM
We no longer need to load vivid explicitly on Chrome OS VM
because it's already loaded at the start-up time by upstart.

Bug: 904730, 852302
Test: capture_unittests passed at chromeos-amd64-generic-rel on CQ
Change-Id: Id75594458c7ca92c406cf4074845763c0672f944
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1710286
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Keiichi Watanabe <keiichiw@chromium.org>
Auto-Submit: Keiichi Watanabe <keiichiw@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#679872}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: b34a6dc8903ad13a9484ee3d33492b1d14ccfd06
2019-07-23 04:28:24 +00:00
Ben Pastene f9514f2deb chromeos: Make additional args to the sanity test non-fatal.
It currently breaks the with/without patch retry flow:
https://ci.chromium.org/p/chromium/builders/try/chromeos-kevin-rel/820
https://chromium-swarm.appspot.com/task?d=true&id=46179d96935c9210

So let's ignore any --test-launcher-* args. The sanity tests is quick
and (hopefully) non-flaky enough to not need any of those.

R=jbudorick

Change-Id: I577508a63eaa6dbed3a248301bc0e39a8c19bcc7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1707048
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#678346}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 8841d1d9d5317cfbad6087c7cde43598ecdb6b30
2019-07-17 17:37:22 +00:00
Ben Pastene f52a9d0c5a Refactor out references to VMs in build configs for simplechrome tests.
This renames the test "cros_vm_sanity_test" to "cros_browser_sanity_test"
since we also run it on chromebooks.

This also renames the target "cros_vm_launcher" to "cros_test_wrapper"
since we use that wrapper script to launch tests on chromebooks. (Used
mainly for telemetry tests.)

Bug: 866062
Change-Id: I4e6d0b7934ca15f132692121ddac94c56fb3d258
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1689981
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#675399}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 4534c39e40005e35370b4f3b1aed54c4fe4756f8
2019-07-08 22:55:34 +00:00
Ben Pastene f50d8c8e98 Add bpastene@ to //build/chromeos/OWNERS
R=jbudorick

Change-Id: I415d23b8cdab2a3762a5ee294ed101320d99ea3f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1677230
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#675273}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 25deffcf604f5255c9b47eba10d59cf1214657a2
2019-07-08 17:24:30 +00:00
Ben Pastene a6157df4ce chromeos: Point vpython to /usr/local/tmp instead of /tmp during tests.
/tmp will soon no longer be mounted with exec permissions in test images.
We've been using /tmp to install vpython binaries during some browser
tests, so move them to /usr/local/tmp, which should remain mounted with
exec permissions.

R=jbudorick, vapier

Bug: 936818
Change-Id: I0c6ed52f0c1d91d2e905674a74b193a604145f9b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1684632
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#674094}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 215b9c188f1e5aa78c2b25cad7140c726c17d102
2019-07-02 16:46:47 +00:00
Ben Pastene ff7a641a07 chromeos: Add a test runner arg that stops the UI on the device.
And enable it for some UI tests.

Bug: 947531
Change-Id: I1545ce6f718214a65690cc6044e2398d9a0bc218
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1635350
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#665248}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 4dad0e4bd5ea5659770c2e5c4e6b8d127a21c6a5
2019-05-31 20:43:22 +00:00
Ben Pastene 2a57b80f66 chromeos: Ensure cros_run_test's exit code is propagated if it's non-zero.
eg: https://chromium-swarm.appspot.com/task?id=4527b399d8f02d10
Something went wrong there, and cros_run_test return non-zero, but no
failed Tast tests were found, so the test runner returned zero.

This'll prevent that from happening.

Bug: 947531
Change-Id: Ie43912771820f3920c30b82f82d6d54555868164
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1638777
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#665223}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 72d151e8d9b1fb4b4172bc15e24ce42cf61c47cd
2019-05-31 19:43:05 +00:00
Ben Pastene 41416a0150 chromeos: Remove references to Tast's dep:chrome_login.
It's been folded into dep:chrome.

Bug: 963618
Change-Id: Id98b1e01116538f2c9c2f9ddccd9ac023e33d44c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1637479
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#664937}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 904ac73be37df0248f99b5528f8960834ff13a1f
2019-05-30 23:03:46 +00:00
Ben Pastene 57ee4e3bef chromeos: Run all Tast tests via the host-side tast bin.
*except for coverage tests, since they need special setup.

This removes support for the '--use-host-tast-bin' arg, since we'll
now default to always using.

Bug: 923426
Change-Id: Ifb57f53153c4d4db73e98e98cce1ef9a49548679
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1585002
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#655245}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: d33514e31c554ba8e861ad03ac6dc66d5a325cf2
2019-04-30 16:05:42 +00:00
Ben Pastene 70dcfa3e46 chromeos: Switch Tast result parser to use streamed_results.jsonl.
This should let us parse some results even if something goes wrong
and the final results.json doesn't get generated for some reason.

Bug: 923426
Change-Id: I028896c9edc7aeb8b3ec6339b8867f0b29ae095f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1575105
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#654678}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ba744c00f6bd238921962db91c7545edc34eebe9
2019-04-26 23:59:19 +00:00
Ben Pastene 26539ae5b0 chromeos: Add a few more dirs to the list we don't copy over during tests.
These shouldn't be used on the device during tests (only the host).

Bug: 866062
Change-Id: I1d8ecf6cc9b416c34cc90bb55377bb8185556fa1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1572266
Auto-Submit: Ben Pastene <bpastene@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#652277}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 9220d4b37fc0bf0c27d386bfbe213f343f96eb36
2019-04-18 20:23:44 +00:00
Ben Pastene 2c6f5922db chromeos: Restructure test runner args a bit.
This makes the "test_type" arg the very first arg. Previously, anything
that came before it couldn't be passed in from the generated test script.

eg: running `./$out/bin/run_gtest --device $DUT_IP` would have no effect
since "--device" came before the "test_type" arg (which itself was
set in the generated script).

Bug: 866062
Change-Id: I7712f11b23f78f28e9831fe506b9467db8c217e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1572959
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#652208}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: fef7399c060df4ae25879ac8fb05070f1b0ccf95
2019-04-18 17:41:26 +00:00
Ben Pastene 6cacb8c92b chromeos: Parse test results when using the host-side Tast runner.
Bug: 923426
Change-Id: I26920165476cbc5a6cb7d03329e66d346528f1d1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1565617
Reviewed-by: Dan Erat <derat@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#651744}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: a4bda7b436add2044c64c0b83753e8c8a9012662
2019-04-17 16:08:24 +00:00
Ben Pastene 26445a002a Add an arg to cros test runner to optionally use host-side Tast bin.
And enable it on the fyi bot. This'll let me safely iterate on it w/o
breaking things again.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:chromeos-kevin-rel

Bug: 923426
Change-Id: I3f2a8973e7d7c1d35001643fe26e1cf6c08f9477
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1565154
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#650054}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 86fbe06b11705f92a5a8ca700c7e27becdcd337b
2019-04-11 21:28:18 +00:00
Ben Pastene 340e3d29a3 Force all tast tests to use local_test_runner.
This is a functional revert of https://chromium-review.googlesource.com/c/chromium/src/+/1553805

Turns out `cros_run_test --tast` doesn't exit non-zero if one of the tests
fail. So we've been ignoring failures since that landed. This reverts back
to using local_test_runner, which does reliably exit non-zero on failure.

Hard-coding the conditional instead of reverting the original CL to
avoid merge conflicts and for easier follow-up fixes.

Bug: 923426
Change-Id: Iaa49011061f42f062e79d0d6867f0741314d473b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1558494
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#648835}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 41e078e8a655634994f8169a5965990926bba98c
2019-04-08 20:19:50 +00:00
Ben Pastene 3ebe394d21 Capture Tast results in the CrOS system logs dir.
Practically speaking, the logs dir is just the isolated out dir.

This'll collect a number of *.json files for the tests, which I'll convert
into standard chromium json results in a follow-up.

Bug: 923426
Change-Id: I488b740d733a192c97de11bc2ec4dd06f2ffbd1e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1554797
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#648786}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: c4a5c357a442b5e31c07e215ebb31eec77441fe7
2019-04-08 19:14:07 +00:00
Ben Pastene e4948d79f2 Use the host-side tast bin to run tast tests instead of the DUT-side local_test_runner.
This is done by using cros_run_test's "--tast" feature instead of using
its "--remote-cmd" option. The tast bin has a lot more features, one of
which is better result reporting. Leveraging these more accurate results
will come in a follow-up.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:chromeos-kevin-rel

Bug: 923426
Change-Id: I6e30c62c0a1bba0444d3b6d25a0e59a70de50263
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1553805
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#648032}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 5084d89d50df91778c36cf3fd6e09b047b50a24d
2019-04-05 01:53:38 +00:00
Ben Pastene 71333e009f Fix args for interactive_ui_tests on cros VM fyi bot.
I typo'ed the dbus one. And we don't need a to fake a src checkout since
we already essentially have one on disk thanks to the fact that we scp
all of the test's runtime_deps.

R=jbudorick

Bug: 946685
Change-Id: I16ee3cf79f400ad7de883cb3b60da448019f2338
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1546198
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#646943}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: c99736ed42b4138cb54152f17a667507b7842c5d
2019-04-02 20:42:45 +00:00
Ben Pastene 127e5f5304 Add interactive_ui_tests' SplitViewResize on the FYI cros VM test bot.
And remove a couple old suites from the fyi bot.

Bug: 946685
Change-Id: I621e6482c39b9af47173db644a5a1866741fa7eb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1548054
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#646560}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 39af4ef6f76f173ae2d057e08493c85e913024b0
2019-04-01 23:35:09 +00:00
Ben Pastene 9337a8c07a Set the LLVM env var for chrome when running coverage tests on CrOS.
Since chrome-on-cros is launched/managed by initctl, it has its own
method for configuring the environment. This is done by chrome_dev.conf

Bug: 934059
Change-Id: I90c5262b4d27142b308ba2aa2e6d5b0f514d0cec
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1546310
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#645884}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 86bc16b2c84f1882f46274bb909f14e1758fde86
2019-03-29 19:06:48 +00:00
Ben Pastene 5a1bd0a035 Run cros tast and sanity tests via on_device scripts.
Instead of passing in the single command directly to cros_run_test. (Or
in the case of vm_sanity, letting cros_run_test do that for us.)

This will let us add additional pre-test setup commands for both tests.
(None for now, but coverage tests will use some.)

Bug: 934059
Change-Id: Iba15f4817293c4a1c5db2dabc89f4a97fee49664
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1542298
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#645790}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ca17d127f40b91a53d0b53e59a30c57f2830ef9a
2019-03-29 16:08:54 +00:00
Ben Pastene dfcb7da0a3 Generalize on_device_script logic for cros remote tests.
This will let us call tast and sanity tests via a shell script on the
device. (Actually switching them will come in a follow-up CL.)

This will be useful for coverage tests, since they need special
setup on the device's filesystem prior to the tests.

Bug: 934059
Change-Id: I2a907d176faf4ce12411bd2157188f909e494cc5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1534857
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#644398}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 7c4a2d847e2e4603eda4ddacde557170e0cf7fb3
2019-03-26 17:24:57 +00:00
Ben Pastene db7c8237c4 Generalize LLVM coverage setup in cros remote tests.
We're planning on enabling coverage on both the sanity test, as well as
tast tests. This generalizes the coverage setup for gtests in the parent
class.

And prepare parts of the on_device_script logic in gtests for
generalizing as well. (I'm going to eventually call tast and sanity
tests via on_device_scripts, but that will come in later CLs.)

Bug: 934059
Change-Id: I913a2f1268810db764d98b660292afd194223e08
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1533310
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#643898}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ae38d86f1734ba8595385d8d0c6b946fb5825405
2019-03-25 17:47:38 +00:00
Ben Pastene 9531b49c2d Mount Chrome when deploying it to DUTs during CrOS tests.
Instead of deploying it directly to /opt/google/chrome/, this will deploy
it to another drive (which should have more space) and mount it to
/opt/google/chrome.

This fixes a regression introduced in crrev.com/c/1520955

NOTRY=true

Bug: 934059
Change-Id: Icf2c64c8be412108dc917cfdee53c63eec6a3d76
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1534148
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#643007}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: db20a90825bb53bbebc53927d1cf74570f5ca9ba
2019-03-21 17:56:47 +00:00
Ben Pastene 2a1991f5f2 Strip symbols from chrome in the CrOS sanity tests when running in VMs.
Like for tast tests, sometimes we run out of room when deploying the
browser during the sanity test. So let's start stripping symbols.

Change-Id: I1e737f6fa71b919c2760d31d9b96f8aab1250636
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1519099
Reviewed-by: Roberto Carrillo <robertocn@chromium.org>
Reviewed-by: Yuke Liao <liaoyuke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#640358}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: c353f3b77dc89975be6f6234c050db8c6b654532
2019-03-13 16:27:27 +00:00
Ben Pastene effe12640f Refactor out references to VMs in the cros test runner.
It's now used for device tests too, so the hard-coded references to
VMs are a bit out of place.

Bug: 866062
Change-Id: I78621e2c615307e3f05a78b5fe40366124a0541b
Reviewed-on: https://chromium-review.googlesource.com/c/1385004
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#635632}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 2057647e5a5ddb73090b64c1b5703e3c3cb49ba9
2019-02-26 19:14:46 +00:00
Roberto Carrillo ba245a9302 [code coverage] Pass profile location var to vm and exfil data.
This change passes the LLVM_PROFILE_FILE var from the test_runner
environment down to the environment inside the VM where actual test is
run.

The location is set to /tmp inside the VM, where the test can write to,
and passed to the test runner for extracting the data out into the
results directory (which should be included in the isolated output IIUC)

R=liaoyuke,stgao,jbudorick
BUG=934059

Change-Id: Id85f54120355176925cc39ce52947fcfe83c2e62
Reviewed-on: https://chromium-review.googlesource.com/c/1480552
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>
Reviewed-by: Yuke Liao <liaoyuke@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Shuotao Gao <stgao@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#634355}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 47c6af43932bfda385a880fd3cb9acef6f6e4ca5
2019-02-21 21:43:10 +00:00
Daniel Erat 67f5e221d1 chromeos: Pass -waituntilready when running Tast tests.
Make test_runner.py pass -waituntilready to
local_test_runner when running Tast tests. This
instructs local_test_runner to wait for necessary
services to be running before executing tests (which is
often needed in freshly-booted VMs).

This flag is currently used when running Tast tests on
Chrome OS builders.

Bug: 914258
Change-Id: I1b22a739eea5ed363e29fe14fe2763b5a0f312b5
Reviewed-on: https://chromium-review.googlesource.com/c/1385450
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Dan Erat <derat@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#627573}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 2f5bdbc8855c6fc084b217aa7701f4ad0c74d7e7
2019-01-30 20:26:06 +00:00
Ben Pastene 164e0de77d Skip VM-incompatible tast tests when running in cros VMs.
Some tests aren't meant to run in VMs. Passing this flag to
local_test_runner will skip them.

Bug: 914688
Change-Id: I867680329a9a6d43633bccfbb5f2a7d457242a50
Reviewed-on: https://chromium-review.googlesource.com/c/1398842
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#620444}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 86155b6373ea23e702481b30ce15b95052cce754
2019-01-07 20:12:58 +00:00
Ben Pastene 6d711bec5b Create a different target for tast tests that run on devices.
There's a bot up that's running the tast test suite on kevin devices:
https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.ci/chromeos-kevin-rel-hw-tests

A few of them are perma-failing. This creates a new target which has them
disabled.

Bug: 866062
Change-Id: Ia715a164ae96bfbc2ee54d85482a25cdc216de96
Reviewed-on: https://chromium-review.googlesource.com/c/1382508
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#618044}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: acc68d72f338b3405a1ae422cabc06cb1119df68
2018-12-20 00:49:10 +00:00
Ben Pastene 4f2483068e Don't strip symbols from Chrome binaries when running CrOS tests on devices.
VMs can't handle the symbols, but hopefully real devices can.

R=jbudorick

Bug: 866062
Change-Id: Ia7a22cecca4046a1e13efc47bcf3a8b296385c8c
Reviewed-on: https://chromium-review.googlesource.com/c/1383042
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#617682}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: dbb96a90540551b2878b7b0dd4fbfca7ef0ff0cf
2018-12-19 00:11:19 +00:00
Ben Pastene b423fb4501 Add support to chromeos test runner for running tests on devices.
Adds a "--use-vm" option (when using VMs) and a "--device" option (when
using real devices.)

Bug: 866062
Change-Id: Ic8fd34f9c34e5bc8b42b8e900220d30a8b19e645
Reviewed-on: https://chromium-review.googlesource.com/c/1380332
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#617222}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 627e2941f115368c5d648f559380cc94f8ef0f9e
2018-12-17 20:56:02 +00:00
Ben Pastene 34827de4f2 Rename CrOS test configs to be VM agnostic.
Bug: 866062
Change-Id: I8d0acef39ad13a699195d1202490448085ced7e7
Reviewed-on: https://chromium-review.googlesource.com/c/1378914
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#617167}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 4ab9865b4d071e7afd9c0f5d2f624312359fdcd9
2018-12-17 18:33:18 +00:00
Ben Pastene 270883ba5c Strip symbols when deploying chrome to cros VMs during tast tests.
It's looking like we don't have enough disk space in the VM for this.
I'll look into increasing partition sizes before turning this back on.

TBR=dpranke

Bug: 913153
Change-Id: I7d0ac50ed45171f69d294596a0c35b3cd46e4a94
Reviewed-on: https://chromium-review.googlesource.com/c/1369036
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#615172}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 7a1760fa9078626f62c18733b569bf298ccb871d
2018-12-10 18:07:58 +00:00
Ben Pastene 84a0041531 Don't fail when unknown args are passed down to cros VM tast tests.
This is to prevent failures like https://ci.chromium.org/p/chromium/builders/luci.chromium.try/chromeos-amd64-generic-rel/141038

I'm tired of playing catch-up with all the additional gtest args we
keep adding to test retries.

Change-Id: I86e1180b62a8f47264c47c0bcb8c2d24a589928d
Reviewed-on: https://chromium-review.googlesource.com/c/1353996
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#612365}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 567c92dfe381af6d3bebedd72d36071aa7015b46
2018-11-29 21:35:55 +00:00
Keiichi Watanabe 6102e79bc9 testing/buildbot: Use vivid for capture_unittests on ChromeOS VM in CQ
Load vivid before running capture_unittests on chromeos-amd64-generic-rel.
This makes it possible to run test cases that require a webcam.

Bug: 852302, 904730
Change-Id: I5d4f5a1dd7a8ea27ef7eabd438fd766621573096
Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/1282424
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#608688}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: a81bdb917ecd5c3bd158865d35338a37ca300995
2018-11-16 07:12:14 +00:00
Ben Pastene 5839d1c9c6 Ignore --gtest_repeat arg for browser-sanity and tast cros VM tests.
See the retries on something like:
https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.try/chromeos-amd64-generic-rel/110795

Failing due to the unknown args. This will just ignore them.

Bug: 876587
Change-Id: Ibfe012d74af2dff8ee9b35c3dc921ba53c671b83
Reviewed-on: https://chromium-review.googlesource.com/c/1284443
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#600613}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 7c2fd4105eb2e1f8941433c0cb48a42e7a5ed93b
2018-10-18 00:09:36 +00:00
Ben Pastene c8da52084a Don't strip symbols from Chrome when running cros VM tests.
When testing a chrome that crashes, we end up w/ unhelpful stack traces:
https://isolateserver.appspot.com/browse?namespace=default-gzip&digest=0c2f5a159649297fb9ed196b997e6cb423eb6fcc&as=ui.20181005-105914

With this option, they become symbolized and much more helpful:
https://isolateserver.appspot.com/browse?namespace=default-gzip&digest=27003a6fb30bbe684e69c2e8e362a67983a6627f&as=ui.20181005-134512

Needs https://chromium-review.googlesource.com/c/chromiumos/chromite/+/1266021
before this can land.

Bug: 876587
Change-Id: Ibea029dad2a7c9385ec533d1ef8f3b043696981c
Reviewed-on: https://chromium-review.googlesource.com/c/1266116
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#598376}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ff7150fe8707d9ca0bb3f00f49d46077f7df0021
2018-10-10 17:15:46 +00:00
Ben Pastene c71b994c1b Bump cros VM sanity test timeout to 10min and reduce retry count by 1.
If chrome crashes on startup, the test itself will fail gracefully
after ~7 min. Bumping timeout let's us hit that reliably, and lets the
test do some cleanup. (Instead of killing it early.)

R=jbudorick

Change-Id: I9c23198e3267b3a438b9a41daadb4ba70e20de4d
Reviewed-on: https://chromium-review.googlesource.com/c/1262799
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#597159}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 87570ef7e393af95c2db06a532f87f6b9ae1d5d1
2018-10-05 16:24:41 +00:00
Ben Pastene dc9a4dfd0b Centralize env var tweaking for cros VM tests.
And hard-code enable_nacl, which needs to be set to deploy nacl libs
to the VM.

Change-Id: Iffa9d62c02e33e1822a20f85a5ee2a4c11a7907d
Reviewed-on: https://chromium-review.googlesource.com/c/1258056
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#595989}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 5d46ab5b5ed12a4872f5e6499f8524a623d4cf82
2018-10-02 21:22:06 +00:00
Ben Pastene 79a68d8054 Support conditionals for tast cros vm tests.
And add a new test "chrome_all_tast_tests" that uses a conditional. And
add that test to the FYI bot.

Bug: 876587
Change-Id: I9518389338b99207ba045bfc62d87dda02c4e3e9
Reviewed-on: https://chromium-review.googlesource.com/c/1256209
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#595905}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 09de394786c06d9f1143770e4fe744aba93cd585
2018-10-02 18:06:42 +00:00
Ben Pastene 03dca66b3c Remove post-test filesystem cleanup from cros VM tests.
This is now done automatically after crrev.com/c/1252005

Change-Id: Iddcd43de19376bf2d6d522dd2ceed5060b880bf6
Reviewed-on: https://chromium-review.googlesource.com/1256102
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#595875}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ccdc07b71c9ded73cc0564cd6e42196d40ce0bfa
2018-10-02 16:44:54 +00:00
Ben Pastene 943188ae3c Add a cros-tast target for chrome login tests. Enable it on cros VM fyi bot.
Bug: 876587
Change-Id: I340c12102a697d6f1b710594111536de564a338c
Reviewed-on: https://chromium-review.googlesource.com/1239354
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#595169}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 33015a01eb411a769dd83e80b86399be1d117e72
2018-09-28 18:59:14 +00:00
Ben Pastene 3e80aacb27 Boot from copy-on-write images for cros VM tests.
With this, any changes to the filesystem a test makes will only last
for as long as the VM is alive. Since we launch+shutdown for every
test, this perfectly isolates consecutive tests from one another. No
more out-of-disk errors!

Bug: 887753
Change-Id: Ifdf4af22e288583135cfe99d363eb3e79045e08d
Reviewed-on: https://chromium-review.googlesource.com/1252005
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#595138}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 9ed482697ef1ba1baaeee47263c9c2a613c299b6
2018-09-28 17:33:44 +00:00
Ben Pastene e7faeb2615 Apply env var modifications for every type of cros VM test.
This is needed for both sanity test and tast tests, so this applies
the env var changes for every type of test. Normal GTest tests don't
need it, but they'll get it anyway. Shouldn't be harmful.

Change-Id: Ida2bd856fc75a089f899c4e4cfdc56de90d5fa48
Reviewed-on: https://chromium-review.googlesource.com/1244876
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#594146}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 5bf0ad43a852111739ff3e0bd0f3ae05d898ef41
2018-09-25 23:55:50 +00:00
Ben Pastene 9c064848ce Add a Tast mode to cros VM test runner.
Tast is a new suite of tests. It's invoked via "local_test_runner" in
the VM. You pass it a set of test names and it'll exercise some test
cases. We shouldn't use much more than the browser flavored ones.

Bug: 876587
Change-Id: I1d6ae9b6b5a929daadb6c12751cb0cd632a24d95
Reviewed-on: https://chromium-review.googlesource.com/1237673
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#594075}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 70849d7634f014baa057b61d6647b574b638b94d
2018-09-25 20:13:25 +00:00
Ben Pastene 7d34066755 Pass in a dummy USE env var for cros VM tests.
That USE val gates pushing 200_percent.pak to the VM. Without that
resource, chrome can crash. A follow-up CL will fix all these env vars.

TBR=jbudorick@chromium.org

Change-Id: I61e417253f9293639b62f6d295f59bf556084d44
Reviewed-on: https://chromium-review.googlesource.com/1228778
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#591782}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 6df796565e5adcf86a64497f48ac65b5a32c4250
2018-09-17 19:37:05 +00:00
Ben Pastene db6c7d551c When deleting crash dirs in cros VM tests, print out the file name.
R=jbudorick@chromium.org

Change-Id: Ic593c71edc288deb6aa9480b2bf9542722b38f0f
Reviewed-on: https://chromium-review.googlesource.com/1217914
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#590485}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 3bfe197a3f89034407a8d451f9ec7663c2d434a0
2018-09-11 21:46:53 +00:00