Граф коммитов

99 Коммитов

Автор SHA1 Сообщение Дата
Ben Pastene dfcb7da0a3 Generalize on_device_script logic for cros remote tests.
This will let us call tast and sanity tests via a shell script on the
device. (Actually switching them will come in a follow-up CL.)

This will be useful for coverage tests, since they need special
setup on the device's filesystem prior to the tests.

Bug: 934059
Change-Id: I2a907d176faf4ce12411bd2157188f909e494cc5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1534857
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#644398}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 7c4a2d847e2e4603eda4ddacde557170e0cf7fb3
2019-03-26 17:24:57 +00:00
Ben Pastene db7c8237c4 Generalize LLVM coverage setup in cros remote tests.
We're planning on enabling coverage on both the sanity test, as well as
tast tests. This generalizes the coverage setup for gtests in the parent
class.

And prepare parts of the on_device_script logic in gtests for
generalizing as well. (I'm going to eventually call tast and sanity
tests via on_device_scripts, but that will come in later CLs.)

Bug: 934059
Change-Id: I913a2f1268810db764d98b660292afd194223e08
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1533310
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#643898}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ae38d86f1734ba8595385d8d0c6b946fb5825405
2019-03-25 17:47:38 +00:00
Ben Pastene 9531b49c2d Mount Chrome when deploying it to DUTs during CrOS tests.
Instead of deploying it directly to /opt/google/chrome/, this will deploy
it to another drive (which should have more space) and mount it to
/opt/google/chrome.

This fixes a regression introduced in crrev.com/c/1520955

NOTRY=true

Bug: 934059
Change-Id: Icf2c64c8be412108dc917cfdee53c63eec6a3d76
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1534148
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#643007}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: db20a90825bb53bbebc53927d1cf74570f5ca9ba
2019-03-21 17:56:47 +00:00
Ben Pastene 2a1991f5f2 Strip symbols from chrome in the CrOS sanity tests when running in VMs.
Like for tast tests, sometimes we run out of room when deploying the
browser during the sanity test. So let's start stripping symbols.

Change-Id: I1e737f6fa71b919c2760d31d9b96f8aab1250636
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1519099
Reviewed-by: Roberto Carrillo <robertocn@chromium.org>
Reviewed-by: Yuke Liao <liaoyuke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#640358}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: c353f3b77dc89975be6f6234c050db8c6b654532
2019-03-13 16:27:27 +00:00
Ben Pastene effe12640f Refactor out references to VMs in the cros test runner.
It's now used for device tests too, so the hard-coded references to
VMs are a bit out of place.

Bug: 866062
Change-Id: I78621e2c615307e3f05a78b5fe40366124a0541b
Reviewed-on: https://chromium-review.googlesource.com/c/1385004
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#635632}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 2057647e5a5ddb73090b64c1b5703e3c3cb49ba9
2019-02-26 19:14:46 +00:00
Roberto Carrillo ba245a9302 [code coverage] Pass profile location var to vm and exfil data.
This change passes the LLVM_PROFILE_FILE var from the test_runner
environment down to the environment inside the VM where actual test is
run.

The location is set to /tmp inside the VM, where the test can write to,
and passed to the test runner for extracting the data out into the
results directory (which should be included in the isolated output IIUC)

R=liaoyuke,stgao,jbudorick
BUG=934059

Change-Id: Id85f54120355176925cc39ce52947fcfe83c2e62
Reviewed-on: https://chromium-review.googlesource.com/c/1480552
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>
Reviewed-by: Yuke Liao <liaoyuke@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Shuotao Gao <stgao@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#634355}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 47c6af43932bfda385a880fd3cb9acef6f6e4ca5
2019-02-21 21:43:10 +00:00
Daniel Erat 67f5e221d1 chromeos: Pass -waituntilready when running Tast tests.
Make test_runner.py pass -waituntilready to
local_test_runner when running Tast tests. This
instructs local_test_runner to wait for necessary
services to be running before executing tests (which is
often needed in freshly-booted VMs).

This flag is currently used when running Tast tests on
Chrome OS builders.

Bug: 914258
Change-Id: I1b22a739eea5ed363e29fe14fe2763b5a0f312b5
Reviewed-on: https://chromium-review.googlesource.com/c/1385450
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Dan Erat <derat@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#627573}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 2f5bdbc8855c6fc084b217aa7701f4ad0c74d7e7
2019-01-30 20:26:06 +00:00
Ben Pastene 164e0de77d Skip VM-incompatible tast tests when running in cros VMs.
Some tests aren't meant to run in VMs. Passing this flag to
local_test_runner will skip them.

Bug: 914688
Change-Id: I867680329a9a6d43633bccfbb5f2a7d457242a50
Reviewed-on: https://chromium-review.googlesource.com/c/1398842
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#620444}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 86155b6373ea23e702481b30ce15b95052cce754
2019-01-07 20:12:58 +00:00
Ben Pastene 6d711bec5b Create a different target for tast tests that run on devices.
There's a bot up that's running the tast test suite on kevin devices:
https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.ci/chromeos-kevin-rel-hw-tests

A few of them are perma-failing. This creates a new target which has them
disabled.

Bug: 866062
Change-Id: Ia715a164ae96bfbc2ee54d85482a25cdc216de96
Reviewed-on: https://chromium-review.googlesource.com/c/1382508
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#618044}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: acc68d72f338b3405a1ae422cabc06cb1119df68
2018-12-20 00:49:10 +00:00
Ben Pastene 4f2483068e Don't strip symbols from Chrome binaries when running CrOS tests on devices.
VMs can't handle the symbols, but hopefully real devices can.

R=jbudorick

Bug: 866062
Change-Id: Ia7a22cecca4046a1e13efc47bcf3a8b296385c8c
Reviewed-on: https://chromium-review.googlesource.com/c/1383042
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#617682}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: dbb96a90540551b2878b7b0dd4fbfca7ef0ff0cf
2018-12-19 00:11:19 +00:00
Ben Pastene b423fb4501 Add support to chromeos test runner for running tests on devices.
Adds a "--use-vm" option (when using VMs) and a "--device" option (when
using real devices.)

Bug: 866062
Change-Id: Ic8fd34f9c34e5bc8b42b8e900220d30a8b19e645
Reviewed-on: https://chromium-review.googlesource.com/c/1380332
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#617222}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 627e2941f115368c5d648f559380cc94f8ef0f9e
2018-12-17 20:56:02 +00:00
Ben Pastene 34827de4f2 Rename CrOS test configs to be VM agnostic.
Bug: 866062
Change-Id: I8d0acef39ad13a699195d1202490448085ced7e7
Reviewed-on: https://chromium-review.googlesource.com/c/1378914
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#617167}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 4ab9865b4d071e7afd9c0f5d2f624312359fdcd9
2018-12-17 18:33:18 +00:00
Ben Pastene 270883ba5c Strip symbols when deploying chrome to cros VMs during tast tests.
It's looking like we don't have enough disk space in the VM for this.
I'll look into increasing partition sizes before turning this back on.

TBR=dpranke

Bug: 913153
Change-Id: I7d0ac50ed45171f69d294596a0c35b3cd46e4a94
Reviewed-on: https://chromium-review.googlesource.com/c/1369036
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#615172}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 7a1760fa9078626f62c18733b569bf298ccb871d
2018-12-10 18:07:58 +00:00
Ben Pastene 84a0041531 Don't fail when unknown args are passed down to cros VM tast tests.
This is to prevent failures like https://ci.chromium.org/p/chromium/builders/luci.chromium.try/chromeos-amd64-generic-rel/141038

I'm tired of playing catch-up with all the additional gtest args we
keep adding to test retries.

Change-Id: I86e1180b62a8f47264c47c0bcb8c2d24a589928d
Reviewed-on: https://chromium-review.googlesource.com/c/1353996
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#612365}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 567c92dfe381af6d3bebedd72d36071aa7015b46
2018-11-29 21:35:55 +00:00
Keiichi Watanabe 6102e79bc9 testing/buildbot: Use vivid for capture_unittests on ChromeOS VM in CQ
Load vivid before running capture_unittests on chromeos-amd64-generic-rel.
This makes it possible to run test cases that require a webcam.

Bug: 852302, 904730
Change-Id: I5d4f5a1dd7a8ea27ef7eabd438fd766621573096
Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/1282424
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#608688}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: a81bdb917ecd5c3bd158865d35338a37ca300995
2018-11-16 07:12:14 +00:00
Ben Pastene 5839d1c9c6 Ignore --gtest_repeat arg for browser-sanity and tast cros VM tests.
See the retries on something like:
https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.try/chromeos-amd64-generic-rel/110795

Failing due to the unknown args. This will just ignore them.

Bug: 876587
Change-Id: Ibfe012d74af2dff8ee9b35c3dc921ba53c671b83
Reviewed-on: https://chromium-review.googlesource.com/c/1284443
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#600613}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 7c2fd4105eb2e1f8941433c0cb48a42e7a5ed93b
2018-10-18 00:09:36 +00:00
Ben Pastene c8da52084a Don't strip symbols from Chrome when running cros VM tests.
When testing a chrome that crashes, we end up w/ unhelpful stack traces:
https://isolateserver.appspot.com/browse?namespace=default-gzip&digest=0c2f5a159649297fb9ed196b997e6cb423eb6fcc&as=ui.20181005-105914

With this option, they become symbolized and much more helpful:
https://isolateserver.appspot.com/browse?namespace=default-gzip&digest=27003a6fb30bbe684e69c2e8e362a67983a6627f&as=ui.20181005-134512

Needs https://chromium-review.googlesource.com/c/chromiumos/chromite/+/1266021
before this can land.

Bug: 876587
Change-Id: Ibea029dad2a7c9385ec533d1ef8f3b043696981c
Reviewed-on: https://chromium-review.googlesource.com/c/1266116
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#598376}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ff7150fe8707d9ca0bb3f00f49d46077f7df0021
2018-10-10 17:15:46 +00:00
Ben Pastene c71b994c1b Bump cros VM sanity test timeout to 10min and reduce retry count by 1.
If chrome crashes on startup, the test itself will fail gracefully
after ~7 min. Bumping timeout let's us hit that reliably, and lets the
test do some cleanup. (Instead of killing it early.)

R=jbudorick

Change-Id: I9c23198e3267b3a438b9a41daadb4ba70e20de4d
Reviewed-on: https://chromium-review.googlesource.com/c/1262799
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#597159}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 87570ef7e393af95c2db06a532f87f6b9ae1d5d1
2018-10-05 16:24:41 +00:00
Ben Pastene dc9a4dfd0b Centralize env var tweaking for cros VM tests.
And hard-code enable_nacl, which needs to be set to deploy nacl libs
to the VM.

Change-Id: Iffa9d62c02e33e1822a20f85a5ee2a4c11a7907d
Reviewed-on: https://chromium-review.googlesource.com/c/1258056
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#595989}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 5d46ab5b5ed12a4872f5e6499f8524a623d4cf82
2018-10-02 21:22:06 +00:00
Ben Pastene 79a68d8054 Support conditionals for tast cros vm tests.
And add a new test "chrome_all_tast_tests" that uses a conditional. And
add that test to the FYI bot.

Bug: 876587
Change-Id: I9518389338b99207ba045bfc62d87dda02c4e3e9
Reviewed-on: https://chromium-review.googlesource.com/c/1256209
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#595905}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 09de394786c06d9f1143770e4fe744aba93cd585
2018-10-02 18:06:42 +00:00
Ben Pastene 03dca66b3c Remove post-test filesystem cleanup from cros VM tests.
This is now done automatically after crrev.com/c/1252005

Change-Id: Iddcd43de19376bf2d6d522dd2ceed5060b880bf6
Reviewed-on: https://chromium-review.googlesource.com/1256102
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#595875}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ccdc07b71c9ded73cc0564cd6e42196d40ce0bfa
2018-10-02 16:44:54 +00:00
Ben Pastene 943188ae3c Add a cros-tast target for chrome login tests. Enable it on cros VM fyi bot.
Bug: 876587
Change-Id: I340c12102a697d6f1b710594111536de564a338c
Reviewed-on: https://chromium-review.googlesource.com/1239354
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#595169}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 33015a01eb411a769dd83e80b86399be1d117e72
2018-09-28 18:59:14 +00:00
Ben Pastene 3e80aacb27 Boot from copy-on-write images for cros VM tests.
With this, any changes to the filesystem a test makes will only last
for as long as the VM is alive. Since we launch+shutdown for every
test, this perfectly isolates consecutive tests from one another. No
more out-of-disk errors!

Bug: 887753
Change-Id: Ifdf4af22e288583135cfe99d363eb3e79045e08d
Reviewed-on: https://chromium-review.googlesource.com/1252005
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#595138}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 9ed482697ef1ba1baaeee47263c9c2a613c299b6
2018-09-28 17:33:44 +00:00
Ben Pastene e7faeb2615 Apply env var modifications for every type of cros VM test.
This is needed for both sanity test and tast tests, so this applies
the env var changes for every type of test. Normal GTest tests don't
need it, but they'll get it anyway. Shouldn't be harmful.

Change-Id: Ida2bd856fc75a089f899c4e4cfdc56de90d5fa48
Reviewed-on: https://chromium-review.googlesource.com/1244876
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#594146}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 5bf0ad43a852111739ff3e0bd0f3ae05d898ef41
2018-09-25 23:55:50 +00:00
Ben Pastene 9c064848ce Add a Tast mode to cros VM test runner.
Tast is a new suite of tests. It's invoked via "local_test_runner" in
the VM. You pass it a set of test names and it'll exercise some test
cases. We shouldn't use much more than the browser flavored ones.

Bug: 876587
Change-Id: I1d6ae9b6b5a929daadb6c12751cb0cd632a24d95
Reviewed-on: https://chromium-review.googlesource.com/1237673
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#594075}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 70849d7634f014baa057b61d6647b574b638b94d
2018-09-25 20:13:25 +00:00
Ben Pastene 7d34066755 Pass in a dummy USE env var for cros VM tests.
That USE val gates pushing 200_percent.pak to the VM. Without that
resource, chrome can crash. A follow-up CL will fix all these env vars.

TBR=jbudorick@chromium.org

Change-Id: I61e417253f9293639b62f6d295f59bf556084d44
Reviewed-on: https://chromium-review.googlesource.com/1228778
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#591782}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 6df796565e5adcf86a64497f48ac65b5a32c4250
2018-09-17 19:37:05 +00:00
Ben Pastene db6c7d551c When deleting crash dirs in cros VM tests, print out the file name.
R=jbudorick@chromium.org

Change-Id: Ic593c71edc288deb6aa9480b2bf9542722b38f0f
Reviewed-on: https://chromium-review.googlesource.com/1217914
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#590485}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 3bfe197a3f89034407a8d451f9ec7663c2d434a0
2018-09-11 21:46:53 +00:00
Ben Pastene 6533d0538d Cleanup cros VM crash logs after test runs and not before.
This script that runs in the VM only runs after all the test binaries &
deps have been pushed. Consequently if a test generates crash dumps, it
won't get cleaned up until the next test pushes all its deps (which
might not fit in the VM since there's left-over crash dumps.)

This fixes that by ensuring that the test that generates the dumps also
cleans them up.

Bug: 880878
Change-Id: Icfe1559d244053998cace31bfc19f5c1606770b9
Reviewed-on: https://chromium-review.googlesource.com/1211706
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#589336}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 35a2d44eb0b1b201f7ec2616f6bc4539d9873a23
2018-09-06 22:23:23 +00:00
Ben Pastene 6ad0d20ebd Delete crash dumps and logs that persist from tests on cros VMs.
The following two dirs can accumulate cruft over a VM's life time:
- /var/log/chrome/
- /var/spool/crash/

This clears them before each test. Should become unnecessary when
crbug.com/878526 is closed out.

Bug: 880878
Change-Id: I5414cb65016294938e9e68db495946bbb05cad48
Reviewed-on: https://chromium-review.googlesource.com/1208254
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#589007}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: dfc6b1dc264ece4083ecfaa7a454e2e604000fd6
2018-09-05 21:36:12 +00:00
Ben Pastene 4755315ada Generate a single shell script for cros VM tests.
And invoke that instead of invoking a long list of chained commands.

This generates a temporary shell script on the host at test-time,
pushes it over to the VM, then removes it after the test on the host.
(The VM gets auto cleaned-up.)

Bug: 732531, 876587
Change-Id: I7a2e5749d329d6f7a8716724c8c1060d4f8f80d3
Reviewed-on: https://chromium-review.googlesource.com/1149189
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#586014}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 7f275fd94375783f830c24aae5a4a6cc07da93da
2018-08-24 22:01:39 +00:00
Ben Pastene ce3ea9594f Deploy a locally built browser when running telemetry tests in cros VMs.
This adds a --deploy option to the vm test runner and binds it to true
every time we build the cros_vm_launcher. This assumes that everything
that uses that target also builds chrome and also wants it deployed to
the VM (which is true atm).

Also rebalance telemetry unit test shard count on the FYI bot.

Bug: 836447
Change-Id: I012932edac7bf40c369f69378e3ad365dd9e1ab6
Reviewed-on: https://chromium-review.googlesource.com/1155833
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#580969}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 22d25c856a70980c251952e873d923d9a3516b52
2018-08-06 20:18:34 +00:00
Ben Pastene da2375258a Change CWD during cros VM tests only for remote tests.
It screws with the host cmd for host-side tests:
https://chromium-swarm.appspot.com/task?id=3ef65521090b3a10

R=jbudorick

Change-Id: If903a70eeb5f6044ee888d436cbbc56848c5a6c3
Reviewed-on: https://chromium-review.googlesource.com/1153779
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#578833}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 49db3d87a8a7ed203b45dd85ee76400126add61f
2018-07-27 22:29:35 +00:00
Ben Pastene fbbcdb1c8f Ignore --gtest_filter arg for cros browser sanity VM test.
If it fails in a trybot run, the "without patch" rerun will add the
gtest arg, which the test runner doesn't like:
https://chromium-swarm.appspot.com/task?id=3eecdd4cb9de5f10

Cq-Include-Trybots: luci.chromium.try:ios-simulator-full-configs;master.tryserver.chromium.mac:ios-simulator-cronet
Change-Id: If3dbee3dc5e37589f8f143edfb935d1870f45c9b
Reviewed-on: https://chromium-review.googlesource.com/1150962
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#578364}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 61befea71d1cbab1daf476855dc3b3a82b9f68e7
2018-07-26 17:59:13 +00:00
Ben Pastene 393f434bf5 Refactor cros VM test runner with more test abstraction.
Now with more OOP! Class structure is:

RemoteTest
   | --- GTestTest
   | --- BrowserSanityTest

Doesn't add anything new. Just splits out sanity-test and gtest-test
logic into their own classes (and resuses what it can in a parent class)

I'm planning on making some changes to how GTests are launched in the
VM. This'll make that easier.

I'm also thinking about reusing some logic in here when running hardware
tests. This will make that easier as well.

Bug: 732531, 866062
Change-Id: I80a6e29c657b1db6c7911cb873c64be094c9b5ef
Reviewed-on: https://chromium-review.googlesource.com/1145796
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#577996}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 60f80879af74355bff3f1b95a8ec1a24e74f8655
2018-07-25 18:53:13 +00:00
Ben Pastene e2cea10e24 Add retries+timeouts to cros VM sanity test.
Gives the test 5min to finish, and retries up to two more times. I'm
pretty confident this'll cover all current flakes.

Change-Id: I1a706b315f3ee029a6530510facd8b2dfd9de4f5
Reviewed-on: https://chromium-review.googlesource.com/1102121
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#568067}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: e0bc8908a84224ea30bd4353a5d42feca3936afd
2018-06-18 18:00:38 +00:00
Ben Pastene 4c5013a73a Catch unknown gtest args for cros VM tests and pass them down.
The retry-without-patch recipe flow is currently broken (whoops!!)
https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.try/chromeos-amd64-generic-rel/25559

Bug: 840967
Change-Id: I0beeb5657f79475089fd6bb355263d7b8543796c
Reviewed-on: https://chromium-review.googlesource.com/1101316
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#567431}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: f42d036f2f2aac22764b9ead9069fd752c2273ce
2018-06-14 21:37:21 +00:00
Ben Pastene c95229837d Run chromeos VM tests as a non-root user.
And enable base_unittests on the CQ.

Bug: 840967
Change-Id: I485b9d785ef02fe9918e0f2c310e3b63ffab1b4a
Reviewed-on: https://chromium-review.googlesource.com/1098314
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#566948}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 20ea78e677f0adf7440297034fb68997054c7ab8
2018-06-13 19:33:34 +00:00
Ben Pastene fda68003c1 Fix return code handling in cros VM tests.
TBR=jbudorick@chromium.org

Change-Id: Ie11ee477222ea041c0f5d558306d52b3b83c2c32
Reviewed-on: https://chromium-review.googlesource.com/1093267
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#565672}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 7aebaa45cb7ddc98c7e052f3fdaa0efe62627c95
2018-06-08 17:40:05 +00:00
Ben Pastene d299c7b40b Gracefully kill child procs of CrOS VM tests on SIGTERM.
We collect logs from the VM after tests. The problem is, if the test
hangs indefinitely, we won't get a chance to grab the logs before
swarming kills us when we reach the timeout.

Swarming sends a SIGTERM, waits for grace_period, then SIGKILLs the task
if it's still running. This CL will catch the SIGTERM and kill what
should be the frozen test process. This will let cros_run_vm_test pull
the logs before we get SIGKILLed.

Bug: 848402
Change-Id: I48be1de865e3b287584978b5461f15e2bae49dfd
Reviewed-on: https://chromium-review.googlesource.com/1087800
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#565357}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: b602383ee8c4cf2007dc300fc9860e36fb8b0f2c
2018-06-07 18:25:02 +00:00
Ben Pastene b642b71807 Add option to CrOS VM test runner to collect VM logs after the test.
And enable it for the sanity test.

This won't work when the test times out and swarming kills it (ie:
we won't pull the logs from the VM before we die.) The fix for that will
come in a follow-up CL.

Bug: 848402
Change-Id: Ie2396adb7e284d5069162a0c8aea145c12d85be1
Reviewed-on: https://chromium-review.googlesource.com/1086416
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#564723}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 417c70dc0081dcae1559905de877ed6f404c23ac
2018-06-06 00:11:51 +00:00
Ben Pastene 08fc263540 Add option to cros test runner to deploy vpython to VM.
This will:
- copy a dir containing vpython over onto the VM
- put the dest dir onto the VM's PATH env var
- initialize the vpython cache prior to running the test

And utilize this option on the fyi bot by adding a vpython cipd
package to net_unittests.

Bug: 843279
Change-Id: I19d8c68e048aed03e6424119d08a3d13ef490def
Reviewed-on: https://chromium-review.googlesource.com/1063043
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#563717}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 8f2d48dcff54fd25a4e653534fd7007a53b97563
2018-06-01 17:59:16 +00:00
Ben Pastene 0676f7632d Use "cros_run_vm_test --host-cmd" for host-side cros VM tests.
Replaces our use of cros_vm directly. Needed because going through
cros_run_vm_test.py gives us smarter VM-bootup logic. cros_vm.py lacks
this is and is rather dumb.

Also bump timeout of cros-vm bot's timeout on telemetry unittests by a
few min. (Added in this CL 'cause all of this is aimed at getting
telemetry_perf_unittests stable.)

Bug: 839187
Change-Id: I3fe6a8ec52d4a6f47b8b1ef475390346aa920ec3
Reviewed-on: https://chromium-review.googlesource.com/1062590
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#559382}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 016a7aee48cd40630a6b06177592388550fb38ec
2018-05-17 01:22:46 +00:00
Ben Pastene 74481331ca Create a simple json results file when running the cros VM sanity test.
This uses build/android's gtest result lib. The results will contain only
a single test, and will either be PASS or FAIL depending on the return code.

Bug: 832374
Change-Id: I7294206340b17fda3f9f93a5085cd95b9b5faceb
Reviewed-on: https://chromium-review.googlesource.com/1060023
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#558929}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 39f23e84c22a9815c1537e05aeb38490ef1d6ad9
2018-05-16 02:13:18 +00:00
Ben Pastene 2c4f94033e Add a sanity-test mode to the cros vm test runner.
When a test command is specified, the run_cros_vm_test will run the vm
sanity test. This test is baked into the VM and smoke checks the system
browser by loading empty pages and running some simple javascript.

Since it depends on the system browser, we need to first deploy a locally
built Chrome to the VM (via the --deploy arg). The deploy script needs to
know the gn args that built the browser to know which libs to copy over.
Since we don't have the full set of args at test-time, give it a dummy
set of args to trigger the default deploying behavior.

Bug: 832374
Change-Id: I84f7fdd94672fcb220eaa4d9625395c1d0ab7636
Reviewed-on: https://chromium-review.googlesource.com/1050981
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#558372}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 83b500c126930271f4da9c246c768d61ada04a84
2018-05-14 18:00:28 +00:00
Ben Pastene e3785a882c Remove stdout result parsing from cros-vm test runner.
Instead, it just pulls the file from the VM directly, which was added in
CL:1021531.

Bug: 828607
Change-Id: I6c71f557bf91dafc8c5f638d0c07c49bde728ed4
Reviewed-on: https://chromium-review.googlesource.com/1044755
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#556213}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 5a53981ff196802bba5e5436b4cb433a2fccc8c7
2018-05-04 21:59:11 +00:00
Ben Pastene 556bbf8211 Add a host-side mode of execution for cros vm test runner.
This will add a positional arg to run_vm_test.py that can take one of two
values:

- "host-cmd": Runs a command on the host. Just launches the vm then
immediately calls the passed in cmd on the host. It's assumed the cmd will
handle data deps, ssh commands, etc.

- "vm-test": Runs a test in the vm. Takes care of spawning the vm, pushing
the test and its deps to the vm, executing the test, and tears down the vm.
(This is the current default behavior.)

Bug: 832374
Change-Id: Ib8bd1b2e7dd393f8a4578979fa164381a8342ef2
Reviewed-on: https://chromium-review.googlesource.com/1038747
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#555279}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: e5e7dc3762a1b18646810303e106da173ac603b5
2018-05-02 01:42:14 +00:00
Ben Pastene b848556d5e Add KVM-related checks to chromeos vm test runner.
Mostly for humans. The bots should all have kvm installed and available.

Also use logging instead of print.

Bug: 828607
Change-Id: Ifee1ead435932d6d2752c0a418cd38618a66228e
Reviewed-on: https://chromium-review.googlesource.com/1026569
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#553350}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 67e8c110257594c18d66cbeff3a45321c7081f7e
2018-04-24 23:18:16 +00:00
Benjamin Pastene 4830c81ed7 Add standard gtest args to the cros vm_test wrapper.
This adds support for test sharding and json results.

See https://ci.chromium.org/buildbot/tryserver.chromium.chromiumos/chromeos-amd64-generic-rel/103454
for this patch in action.

Bug: 832374
Change-Id: I1a1bea2412e25af9d5ba7a8daf0b6cb34e7b6ae4
Reviewed-on: https://chromium-review.googlesource.com/1020535
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#552356}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: be4dea1713bc7a1c0461744289cfee9c3b88acd4
2018-04-20 16:50:54 +00:00
Benjamin Pastene 80b7a3c6ca Create a wrapper and generator script around CrOS's run_vm_test script.
Example led build of one of the simple chrome bots:
https://ci.chromium.org/swarming/task/3cb56f4f4bb79410?server=chromium-swarm.appspot.com

Bug: 828607
Change-Id: Ifcbe4059544722b48f4ab31fb2e97331b4f976f2
Reviewed-on: https://chromium-review.googlesource.com/994062
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#550845}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 3bce864eb7611eb93eb3789e7b57f8b2921c7a5c
2018-04-14 01:16:32 +00:00