This CL:
- Removes RenderUtils.
- Creates RenderTestRule with most of the RenderUtils functionality.
- Updates render tests to RenderTestRule.
- Removes the 'regenerate-goldens' flag. This is not needed because the
RenderTestRule will throw an Exception once the test method has
finished, so all image renders and diffs will be saved anyway.
- Simplifies the rendering logic.
BUG=614305
Review-Url: https://codereview.chromium.org/2901053003
Cr-Original-Commit-Position: refs/heads/master@{#474978}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 1c831a5af48331b017f2bf3d6a07abb40f143626
We are creating some new instrumentationt tests which capture
screenshots as they run. Clear the test device's screenshot
directory before they run.
Add a --ui-screenshot-dir to test_runner.py. If specified then,
once the tests have been run, the screenshots are copied to
the given directory on the host.
BUG=721289
Review-Url: https://codereview.chromium.org/2874173002
Cr-Original-Commit-Position: refs/heads/master@{#474231}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: b9e451ab1f65fec40a11146a868aa07565a74390
Still attempting to resuscitate x86 Emulator Tester on
chromium.android.fyi.
TBR=yolandyan@chromium.org,hzl@chromium.org
Change-Id: I7da60ce19b0b6d6b3d3007638bbcb7534eefd40f
Reviewed-on: https://chromium-review.googlesource.com/509290
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#473582}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: f72054ceedbf07e177d5b740aca7a2c196f9be46
In general time.strptime is not threadsafe in python: [1]. This causes
flaky errors on various bots that test Android because catapult [2]
talks to devices from different threads (for multiple reasons).
One observation in the python bug ([1]) says that only importing parts
of strptime is racy, so importing it early enough before any threading
is started would remove this kind of flake.
This workaround is used in other parts of chromium and catapult itself,
so we can probably be more preventive by doing it in a central place.
For now let's just reduce the flake.
[1] time.strptime not thread safe
https://bugs.python.org/issue7980
[2] Catapult is the home for several performance tools that span from
gathering, displaying and analyzing performance data.
https://github.com/catapult-project/catapult
BUG=724524
Review-Url: https://codereview.chromium.org/2891403002
Cr-Original-Commit-Position: refs/heads/master@{#473545}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: bfb763ee9499a5f88993910c2401ac5550e9ac38
Previously the strings inside table cells on clickable, but it takes
accurate mouse control and aiming.
In this cl, we are making the whole cell clickable,
to make the table easier to use.
BUG=717682
Review-Url: https://codereview.chromium.org/2888723002
Cr-Original-Commit-Position: refs/heads/master@{#473035}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: e111db0385aeceaf3e0c03f3f57568f908be46b5
We were mistakenly checking the existence of the directory containing
the file, rather than the actual file.
BUG=706943
Review-Url: https://codereview.chromium.org/2885413003
Cr-Original-Commit-Position: refs/heads/master@{#472931}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: a0fbb7c0d4e4b52d9170031a8883c60e19f4916c
Removing this for a few reasons. (1) We don't use much CSS from
the file we import. (2) It couples our results details page with
the waterfall CSS file (things like classnames info/interface were
used in that CSS but don't seem to do anything special just looking
at results detail page) and (3) we might want local result detail
pages eventually.
Review-Url: https://codereview.chromium.org/2886323003
Cr-Original-Commit-Position: refs/heads/master@{#472903}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: e99ade7b40d361233acfad8d735dc66a34799de6
If --render-results-dir is specified, the render result images
will get pulled from the device and stored there locally.
BUG=722779
Review-Url: https://codereview.chromium.org/2889663003
Cr-Original-Commit-Position: refs/heads/master@{#472670}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 39e3f9e3bdd92088ebf9c48bfd9395ebe38d7801
This fixes the alignment for when the render test images are a
different size than the golden image. The images are compared by
lining up the top-left corner. Setting vertical-align:top makes
it easier to visualize how the diff image was constructed in this
case.
Review-Url: https://codereview.chromium.org/2884833002
Cr-Original-Commit-Position: refs/heads/master@{#472616}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: c4f60b915d897050dac3d7532ff8c1e0139c1821
Resources were not collected for the pseudo all module before. Adding it
back, but skipping the generated resources folders.
BUG=722302
Review-Url: https://codereview.chromium.org/2888923002
Cr-Original-Commit-Position: refs/heads/master@{#472485}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: dc177240b50784d9fe81d654e129275028e6d97f
Stack traces are presented as DEBUG:I, but the previous logcat filter
does not include such logcat.
In this cl, we are including DEBUG:I into the logcat.
BUG=721811
Review-Url: https://codereview.chromium.org/2882793002
Cr-Original-Commit-Position: refs/heads/master@{#472172}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: d82dbc620a69d659b7cdf4d105e62a8cb62156dc
The word "View" is used in the MVC sense, which is not conventional for
Android code. Android uses View to mean a widget. However. EditorView is
a Dialog. Therefore, it should be named EditorDialog to follow Android
conventions.
Bug: 722382
Change-Id: I47e720098ce43def28d1db1f6a23a70e84def527
Reviewed-on: https://chromium-review.googlesource.com/505870
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Ted Choc <tedchoc@chromium.org>
Commit-Queue: Rouslan Solomakhin <rouslan@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#471913}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 9edfd334b6666cd0e42b8f6994cb7807b6ad8f0c
This should trigger these bots to start uploading screenshots
of test failures as well as uploading render test results in the
future.
Review-Url: https://codereview.chromium.org/2880063002
Cr-Original-Commit-Position: refs/heads/master@{#471595}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 4e3a7c4082bdd25b35036e77b1d8870a47e63e9a
Will let people use Android resources in Robolectric tests.
To use, specify package_name GN variable in junit_binary
targets with your apps package name.
This change will also (basically) require that you use the
generated wrappers from out_dir/bin/run_<suite name> to run
the tests since GN will be generating many arguments to
pass along to the test runner.
BUG=693573
Review-Url: https://codereview.chromium.org/2864693002
Cr-Original-Commit-Position: refs/heads/master@{#471523}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 0b6b35f669cfd191e369832a9385d939650c244b
We will be uploading the same golden images over and over for
render tests. Uploading images by md5sum so that we can avoid
storing/uploading multiple copies of the same exact file.
Also CL fixes small bug in _IsRenderTest helper function. Now
properly identifies tests with RenderTest feature annotation.
Review-Url: https://codereview.chromium.org/2876183003
Cr-Original-Commit-Position: refs/heads/master@{#471488}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: c5fdbf8734c0194ecf673f8627cfdbb0bbb9a895
With crrev.com/686094f6b082e2595bc5d429cdcdaa3b76217501, we started
writing a python script to <output dir>/<suite_name>. The stack
symbolization script was picking up that script as a possible library
for symbolization. This change prevents it from doing so.
Bug: 719984
TBR=agrieve@chromium.org
Change-Id: Id0591ad05a531b7cf2037ced1fef36dd8d767bce
Reviewed-on: https://chromium-review.googlesource.com/504909
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Michael Case <mikecase@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#471463}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 46858e064d251775e93c4f8fc7fe2da367ccaf36
This will (1) make render tests perform pixel-wise
comparison and generate diff images, (2) allow you
to specify output directory for where to dump the
result images on the device, and (3) upload the results
to be visible from buildbot status page.
Review-Url: https://codereview.chromium.org/2866103002
Cr-Original-Commit-Position: refs/heads/master@{#471144}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 49c5485e248597a4afe07101241bda3c3060cad8
Create a link to flakiness dashboard for each test case, so that engineers will see the flakiness level for that test case.
BUG=717631
Review-Url: https://codereview.chromium.org/2873723002
Cr-Original-Commit-Position: refs/heads/master@{#471122}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 20892f4eb9dad7ed0b1720dc3718dbc5db5f0d7f
Currently, the post-test screenshots are taken on the python after
the test has finished (and Activity destroyed). Moving the
screenshot capture into Android code so we can snap a pic before
Activity is gone.
Review-Url: https://codereview.chromium.org/2854823007
Cr-Original-Commit-Position: refs/heads/master@{#470816}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: fc8efc725e151ffc2f2584b75d9c52fd16cf3721
This reverts commit 477db25e38327a2891aa3c02e1b112d0308f9306.
Chromecast internal code no longer uses the files that were
causing the suppressed lint errors, so we can remove these.
BUG=669629
TEST=build internal Cast apk
Change-Id: I38d9eb6284788276f7f776f139b9f74c9f2668e5
Reviewed-on: https://chromium-review.googlesource.com/493774
Commit-Queue: Simeon Anfinrud <sanfin@chromium.org>
Reviewed-by: Stephen Lanham <slan@chromium.org>
Reviewed-by: Michael Case <mikecase@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#470361}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 064434bb26e0cf8c916b11ecd988041bdb1f1c40
By default, gsutil will look for updates every 30 days. To prevent
this update prompt from interupting tests, adding -q option.
Review-Url: https://codereview.chromium.org/2867503002
Cr-Original-Commit-Position: refs/heads/master@{#470232}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 6b2241e2042fe5365c84d84828c8db3a3625c18f
Previously, server url is 'storage.googleapis.com'. And the issue
creation page can include the current page url in the description of the
issue. However then we start to use 'storage.cloud.google.com', which
will automatically redirect to a temporary signed url when visiting the
page. Thus including the temporary signed url in the issue creation page
is not useful.
In this cl, we generate the result details link when we create the html
file. And as a result is able to include the original page url, not the
signed url in the description.
BUG=717634
Review-Url: https://codereview.chromium.org/2854083004
Cr-Original-Commit-Position: refs/heads/master@{#470078}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 29948291955ddcd6c0cba23173c06e072585db29
Bugs causing split-project and use-gradle-resources to not work
correctly. Also fix the jinja template.
BUG=620034
Review-Url: https://codereview.chromium.org/2858233002
Cr-Original-Commit-Position: refs/heads/master@{#469372}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: f011a78f3f93e0f52b89b29a4b40b2e2f879733c
Want to move resources.zip files out of gen/ directory since they
won't get packaged by several of the Builder bots. We will need
these resources.zip files for JUnit tests.
BUG=693573
Review-Url: https://codereview.chromium.org/2862603002
Cr-Original-Commit-Position: refs/heads/master@{#469191}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 54c8a0f1de1a3cf8599dde9efb7bc61f08fef03f
Reason for revert:
TileGroupTest failing on waterfall, see http://crbug.com/717820
BUG=717820
Original issue's description:
> (Reland x2) Expose resources in Robolectric/JUnit tests.
>
> Will let people use Android resources in Robolectric tests.
> To use, specify package_name GN variable in junit_binary
> targets with your apps package name.
>
> This change will also (basically) require that you use the
> generated wrappers from out_dir/bin/run_<suite name> to run
> the tests since GN will be generating many arguments to
> pass along to the test runner.
>
> BUG=693573
>
> Review-Url: https://codereview.chromium.org/2842603002
> Cr-Commit-Position: refs/heads/master@{#468822}
> Committed: bc26a6f5b4TBR=agrieve@chromium.org,jbudorick@chromium.org,nyquist@chromium.org,mikecase@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=693573
Review-Url: https://codereview.chromium.org/2859583004
Cr-Original-Commit-Position: refs/heads/master@{#468871}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: a0f8ac5d596602275b1820e3f63d4966430f0de7
Will let people use Android resources in Robolectric tests.
To use, specify package_name GN variable in junit_binary
targets with your apps package name.
This change will also (basically) require that you use the
generated wrappers from out_dir/bin/run_<suite name> to run
the tests since GN will be generating many arguments to
pass along to the test runner.
BUG=693573
Review-Url: https://codereview.chromium.org/2842603002
Cr-Original-Commit-Position: refs/heads/master@{#468822}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: bc26a6f5b4ed9ee2b84878c17e8f4149db815db7
This CL updates DEPS to enable updated Android Lint that would work for
Java 8. Also, it adds inline suppression for newly discovered Lint
errors. Additionally, it suppresses Lint deprecation warning for
"android:singleLine". Moreover, it suppresses Lint warnings "AuthLeak",
"VisibleForTests", etc. for tests, and Lint error "ByteOrderMark" in
generated android_chrome_strings.xml (crbug.com/715196).
BUG=635567,669999
Review-Url: https://codereview.chromium.org/2830843004
Cr-Original-Commit-Position: refs/heads/master@{#468545}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 5512e732f9513b2e5b3f8c47f146dfa5c3582b6d
We had been looking in lib.stripped for the library when symbolizing
debuggerd traces. The libraries in lib.stripped are not the ones with
packed relocations, though, so the resulting symbolization was wrong.
This CL generates a wrapper script around the stack symbolization
script that points to the correct library to use for symbolization.
It also contains a few tests that intentionally crash in various ways.
These are all disabled by default and should only ever be run manually
for testing out instrumentation test crash tooling.
BUG=712265
Review-Url: https://codereview.chromium.org/2840193003
Cr-Original-Commit-Position: refs/heads/master@{#468206}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 276cc56b7c04cc3a28ec6315feb00f056ea8327b
Previously when test results presentation does not receive any json
input file, the script will raise a parser error. This is confusing to
developer, because they may mistake the parser error in this script as
the cause of the crash, instead of the reason why no json input file is
given at the first place.
In this cl, test results presentation will not raise error anymore when
no json file is given.
BUG=715972
Review-Url: https://codereview.chromium.org/2845973004
Cr-Original-Commit-Position: refs/heads/master@{#467834}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 043d55636176cf1c5f9b0ff29c9c853bb538dbd5
It is no longer necessary to list all the apk targets as separate
modules now that we have the _all pseudo module. Having the separate
modules resulted in bugs where Android Studio could not distinguish
between prod code and test code since some modules viewed dirs as prod
and some as test. The --split-projects flag can be used to see the
dependency graph, and when only a single module will be generated, the
_all pseudo module will not replace it.
For android studio's _all pseudo module, move all known test dirs to
androidTest instead of main so that they are properly displayed when
filtered.
BUG=620034
Review-Url: https://codereview.chromium.org/2837863002
Cr-Original-Commit-Position: refs/heads/master@{#467704}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: e851ad470036997a9a43b0ee2710442f1b5c2096
peconn
wnwen
aberent
Kai Ninomiya
hzl
John Budorick
pasko
mikecase
paulmiller
Rouslan Solomakhin
aluo
ctzsm
Simeon Anfinrud
avayvod
finnur
zpeng
aelias