f5e315ccf1
add --writeChecksumBasedFilenames flag to render_pictures
...
BUG=skia:1455,skia:2230
R=robertphillips@google.com , robertphillips@chromium.org
Author: epoger@google.com
Review URL: https://codereview.chromium.org/202983003
git-svn-id: http://skia.googlecode.com/svn/trunk@13859 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-19 17:26:07 +00:00
341241d567
short-term fix for RunToolSelfTests
...
BUG=skia:2307
NOTREECHECKS=True
NOTRY=True
R=bensong@google.com
TBR=bensong
Author: epoger@google.com
Review URL: https://codereview.chromium.org/202203010
git-svn-id: http://skia.googlecode.com/svn/trunk@13846 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-18 05:41:33 +00:00
f11943f21d
render_pictures unittests: run over a dir with multiple SKPs, like production
...
BUG=skia:1942
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/194563004
git-svn-id: http://skia.googlecode.com/svn/trunk@13775 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-12 20:09:46 +00:00
184263325b
Updates bench alert test output to match latest changes.
...
BUG=skia:2193
TBR=epoger@google.com
NOTRY=true
Author: bensong@google.com
Review URL: https://codereview.chromium.org/167703005
git-svn-id: http://skia.googlecode.com/svn/trunk@13513 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-02-20 16:22:55 +00:00
557e3cda9a
Separates faster and slower bench alerts; sorts by delta.
...
Separate faster and slower bench alerts; sort by delta.
BUG=skia:2193
NOTRY=true
R=epoger@google.com
Author: bensong@google.com
Review URL: https://codereview.chromium.org/172683002
git-svn-id: http://skia.googlecode.com/svn/trunk@13512 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-02-20 15:58:32 +00:00
164052e9ce
Create a common utility for finding and running binaries in out/
...
NOTRY=true
BUG=skia:2019
R=epoger@google.com
Author: rmistry@google.com
Review URL: https://codereview.chromium.org/156173005
git-svn-id: http://skia.googlecode.com/svn/trunk@13367 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-02-07 18:41:49 +00:00
c3147c668c
render_pictures: adjust unittests to show specific failures
...
BUG=skia:2043,skia:2044
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/139383004
git-svn-id: http://skia.googlecode.com/svn/trunk@13100 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-01-15 20:35:54 +00:00
238771c930
[try 2] skpmaker: make the SKP more graphically interesting (add a border)
...
relanding https://codereview.chromium.org/137993008/
BUG=skia:1942
NOTRY=True
R=rmistry@google.com
TBR=rmistry
Author: epoger@google.com
Review URL: https://codereview.chromium.org/139583002
git-svn-id: http://skia.googlecode.com/svn/trunk@13088 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-01-15 16:33:31 +00:00
7e1a31db6a
Revert of skpmaker: make the SKP slightly more interesting (add a border) ( https://codereview.chromium.org/137993008/ )
...
Reason for revert:
broke http://108.170.217.252:10117/builders/Build-Mac10.6-GCC-x86-Debug/builds/2837
Original issue's description:
> skpmaker: make the SKP slightly more interesting (add a border)
>
> This will help with render_pictures self-testing
> BUG=skia:1942
>
> Committed: http://code.google.com/p/skia/source/detail?r=13086
R=rmistry@google.com
TBR=rmistry@google.com
NOTREECHECKS=true
NOTRY=true
BUG=skia:1942
Author: epoger@google.com
Review URL: https://codereview.chromium.org/139553002
git-svn-id: http://skia.googlecode.com/svn/trunk@13087 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-01-15 16:04:45 +00:00
a3b0c4fe63
skpmaker: make the SKP slightly more interesting (add a border)
...
This will help with render_pictures self-testing
BUG=skia:1942
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/137993008
git-svn-id: http://skia.googlecode.com/svn/trunk@13086 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-01-15 15:40:13 +00:00
30d4f8372c
deprecate tools/test_rendering.py and tools/test_pictures.py
...
they will be removed entirely on/after Feb 1 unless there is objection
BUG=skia:1943
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/135783003
git-svn-id: http://skia.googlecode.com/svn/trunk@13065 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-01-14 17:01:28 +00:00
11f156201a
add unittest for render_pictures binary
...
Once committed, this will be run as part of RunToolSelfTests on http://108.170.217.252:10117/builders/Housekeeper-PerCommit .
BUG=skia:1943
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/124253002
git-svn-id: http://skia.googlecode.com/svn/trunk@12941 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-01-07 17:03:40 +00:00
9e8f88b6ef
create tools/tests/run_all.py to run all our Python self-tests within tools
...
Eventually, we want all of these self-tests to be written in Python;
this change makes it easier for us to add new Python self-tests, and discourages
the addition of bash self-tests.
BUG=skia:677,skia:1943
NOTRY=True
R=rmistry@google.com , scroggo@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/112163004
git-svn-id: http://skia.googlecode.com/svn/trunk@12775 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-12-19 18:22:32 +00:00
4a28f598ec
Changes bench_graph_svg.py test to check_bench_regressions.py test; deletes deprecated chunks in syn_bucket_subdir.
...
Review URL: https://codereview.chromium.org/27377002
git-svn-id: http://skia.googlecode.com/svn/trunk@11952 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-10-24 18:08:45 +00:00
5187c4313c
More work to integrate skimage with rebaseline tools.
...
tools/skimage_main.cpp:
Add the ability to write the results to checksum based filenames,
much like GM uses. This will allow using the skpdiff server to
rebaseline images.
Write the keys in the JSON file as <original image>_<pref config>.png,
so it matches gm_json.IMAGE_FILENAME_PATTERN. Also replace '_' with
'-' in the original file name, to avoid confusing the pattern matcher.
The '_' to '-' replacement also happens on the output filename.
Read the keys in a similar manner.
In make_outname, no longer remove a suffix. This fixes a bug where
subset decoding writes multiple subsets to the same file.
tools/rebaseline.py:
Since the filenames written to json files now match
gm_json.IMAGE_FILENAME_PATTERN, enable the option to match based
on configs/tests when rebaselining skimage.
test json files:
Update to match the new format of output.
gm/gm_expectations:
Add a constructor that takes a BitmapAndDigest as input.
tools/tests/skimage_self_test.py:
Test that reading the expectations file just created by skimage with
the same file actually compares to the original file (rather than just
succeeding because expectations were missing).
Change the expectations files to match the new format.
Will require a buildbot change to use the new flag: https://codereview.chromium.org/27389002/
BUG=1466
R=epoger@google.com
Review URL: https://codereview.chromium.org/26297004
git-svn-id: http://skia.googlecode.com/svn/trunk@11902 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-10-22 00:42:46 +00:00
36c5bdb097
Truly ignore failures in skimage.
...
If the expectation is set to ignore failures, do not return a
negative 1 (indicating a failure) when a successful decode
does not match the expectation.
If the file could not be decoded at all, report this to the
user depending on the input expectations file:
No expectations:
Report failure. The user wanted to know if the file could be
decoded.
Expectations expected a valid result:
Report failure. The decode should have matched the result.
Empty expectations:
Print a message that the expectation was missing, but still
return success from skimage, since this is a newly added file
(i.e. it has no expectation yet).
Ignore failure:
Return success from skimage, since it is a known failure.
Update the self tests to ensure these behaviors.
R=epoger@google.com
Review URL: https://codereview.chromium.org/22293006
git-svn-id: http://skia.googlecode.com/svn/trunk@11790 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-10-15 20:29:37 +00:00
e209a70d1e
Suppresses failing test due to change of bench_graph_svg. Will fix later.
...
Review URL: https://codereview.chromium.org/26894004
git-svn-id: http://skia.googlecode.com/svn/trunk@11766 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-10-15 00:35:18 +00:00
bfe9167c66
Add README file so git will create empty dir
...
BUG=skia:1613
(SkipBuildbotRuns)
R=rmistry@google.com
Review URL: https://codereview.chromium.org/24112002
git-svn-id: http://skia.googlecode.com/svn/trunk@11203 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-11 17:31:49 +00:00
c206dfcf01
Add forgotten file from https://code.google.com/p/skia/source/detail?r=11063
...
Review URL: https://codereview.chromium.org/23911002
git-svn-id: http://skia.googlecode.com/svn/trunk@11064 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-03 18:31:25 +00:00
93acfd2b49
rebaseline.py: fix --add-new when there are no expectations at all
...
BUG=skia:1582
R=rmistry@google.com
Review URL: https://codereview.chromium.org/23899003
git-svn-id: http://skia.googlecode.com/svn/trunk@11063 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-03 18:27:19 +00:00
06e626d69e
rebaseline.py: add --bugs and --unreviewed flags
...
BUG=skia:1569
R=borenet@google.com
Review URL: https://codereview.chromium.org/23478011
git-svn-id: http://skia.googlecode.com/svn/trunk@11058 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-03 17:32:15 +00:00
8209b183da
rebaseline.py self-test: validate that unknown fields are maintained
...
BUG=skia:1569
R=bsalomon@google.com
Review URL: https://codereview.chromium.org/23702006
git-svn-id: http://skia.googlecode.com/svn/trunk@11008 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-08-29 18:04:20 +00:00
3f0a108586
Remove temporary workaround TODOs from tools/tests/run.sh
...
R=rmistry@google.com
Review URL: https://codereview.chromium.org/23490009
git-svn-id: http://skia.googlecode.com/svn/trunk@10989 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-08-28 19:04:05 +00:00
fd040116c5
Remove base-* directories from gm expected/actual paths; just use platform names
...
BUG=skia:1512
The buildbot master continues to set gm_image_subdir, but the slaves no longer
use that component when building GM paths (they just use builder_name instead).
But gm_image_subdir is still used for SKP storage paths under
gs://chromium-skia-gm/playback/gm-actual , for now...
Immediately before committing this CL, I will copy the actual-results.json files
from their old locations to their new locations (within the skia-autogen repo)
so that both old and new versions of rebaseline.py will be able to retrieve
actual results, like so:
svn cp base-android-galaxy-nexus/Test-Android-GalaxyNexus-SGX540-Arm7-Debug/base-android-galaxy-nexus Test-Android-GalaxyNexus-SGX540-Arm7-Debug
svn cp base-android-galaxy-nexus/Test-Android-GalaxyNexus-SGX540-Arm7-Release/base-android-galaxy-nexus Test-Android-GalaxyNexus-SGX540-Arm7-Release
svn cp base-android-nexus-10/Test-Android-Nexus10-MaliT604-Arm7-Debug/base-android-nexus-10 Test-Android-Nexus10-MaliT604-Arm7-Debug
...
svn cp base-shuttle-win7-intel-float/Test-Win7-ShuttleA-HD2000-x86-Debug/base-shuttle-win7-intel-float Test-Win7-ShuttleA-HD2000-x86-Debug
svn cp base-shuttle-win7-intel-float/Test-Win7-ShuttleA-HD2000-x86-Release/base-shuttle-win7-intel-float Test-Win7-ShuttleA-HD2000-x86-Release
Once we see this CL land successfully, I will delete the actual-results.json
files from their OLD location within skia-autogen, like so:
svn rm base-android-galaxy-nexus
svn rm base-android-nexus-10
...
svn rm base-shuttle-win7-intel-directwrite
svn rm base-shuttle-win7-intel-float
R=borenet@google.com , rmistry@google.com
Review URL: https://codereview.chromium.org/23120002
git-svn-id: http://skia.googlecode.com/svn/trunk@10821 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-08-20 16:21:55 +00:00
2a192a8577
Delete image-based rebaselining tool; we have switched to checksums
...
R=borenet@google.com
Review URL: https://codereview.chromium.org/21901004
git-svn-id: http://skia.googlecode.com/svn/trunk@10524 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-08-02 20:54:46 +00:00
ffcbdbfe6a
rebaseline.py: add --keep-going-on-failure option, off by default
...
R=borenet@google.com
Review URL: https://codereview.chromium.org/18092004
git-svn-id: http://skia.googlecode.com/svn/trunk@10109 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-16 17:35:39 +00:00
3e7399fe28
Fix Housekeeper bot after r9961
...
git-svn-id: http://skia.googlecode.com/svn/trunk@9965 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-10 17:23:47 +00:00
89fa4b9ee6
rebaseline.py: error out early if --expectations-root not pointing at gm-expected dir
...
BUG=https://code.google.com/p/skia/issues/detail?id=1403
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/18416005
git-svn-id: http://skia.googlecode.com/svn/trunk@9961 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-10 16:54:10 +00:00
9de25e3fd4
rebaseline.py: use specified configs/tests as a FILTER over actually-run tests
...
Now that we have the actual-results.json file for every bot type, we know
the full set of tests that actual results are available for. If configs and/or
tests are specified, just use those as FILTERS over the actually available
results.
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/18324018
git-svn-id: http://skia.googlecode.com/svn/trunk@9956 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-10 15:27:18 +00:00
a783f2bee8
rebaseline.py: if expectations dir contains JSON format results, update those instead of image files
...
Part of Step 3 in https://goto.google.com/ChecksumTransitionDetail
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/18348018
git-svn-id: http://skia.googlecode.com/svn/trunk@9910 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-08 17:51:58 +00:00
34b73a2da1
rebaseline.py self-tests: prepare to add tests of JSON-format rebaselining
...
For now, rebaseline.py still does not support JSON-format rebaselining, so
the one self-test reports failure. But this change will allow us to test
the JSON-format rebaselining once rebaseline.py supports it.
R=scroggo@google.com
Review URL: https://codereview.chromium.org/18348015
git-svn-id: http://skia.googlecode.com/svn/trunk@9896 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-03 17:43:11 +00:00
53953b457b
Create jsondiff tool to capture diffs between two JSON expectations files.
...
This will be used to view diffs before committing rebaselines, or to review
exactly what changed between two baseline revisions in the past.
Part of Step 3 in https://goto.google.com/ChecksumTransitionDetail
R=scroggo@google.com
Review URL: https://codereview.chromium.org/18103005
git-svn-id: http://skia.googlecode.com/svn/trunk@9878 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-02 20:22:27 +00:00
8575b4df41
Fix rebaseline.py self-test to account for r9859
...
R=robertphillips@google.com
Review URL: https://codereview.chromium.org/18559002
git-svn-id: http://skia.googlecode.com/svn/trunk@9872 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-02 18:25:06 +00:00
2b9424b342
Treat empty directory properly in skimage.
...
Instead of treating an empty directory as a file that failed to
decode, treat it as an empty directory.
Add self tests to check for this.
Builds upon from https://codereview.chromium.org/16866010 ,
which was reverted.
R=epoger@google.com
Review URL: https://codereview.chromium.org/17101029
git-svn-id: http://skia.googlecode.com/svn/trunk@9730 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-21 19:12:47 +00:00
99a8ec9705
rebaseline.py: split image-based rebaselining, which will go away soon, into its own script
...
(part of step 3 in https://goto.google.com/ChecksumTransitionDetail )
also adds new --expectations-root option
R=scroggo@google.com , senorblanco@chromium.org
Review URL: https://codereview.chromium.org/17379004
git-svn-id: http://skia.googlecode.com/svn/trunk@9689 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-19 18:56:59 +00:00
c7e1873c85
skimage: revert r9642
...
git-svn-id: http://skia.googlecode.com/svn/trunk@9647 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-18 02:53:16 +00:00
e0433f1ad3
skimage: if --readPath points at an empty dir, create an empty expectations file
...
TBR=scroggo
Review URL: https://codereview.chromium.org/17294006
git-svn-id: http://skia.googlecode.com/svn/trunk@9642 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-17 19:34:58 +00:00
74feb1593a
skimage self-tests: confirm correctness of bitmap hash values
...
R=scroggo@google.com
Review URL: https://codereview.chromium.org/16866010
git-svn-id: http://skia.googlecode.com/svn/trunk@9591 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-13 19:12:05 +00:00
e78d20798d
rebaseline.py: try to download images from Google Storage before skia-autogen
...
This will allow us to complete Step 1 of https://goto.google.com/ChecksumTransitionDetail
R=scroggo@google.com
Review URL: https://codereview.chromium.org/16311011
git-svn-id: http://skia.googlecode.com/svn/trunk@9534 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-12 17:44:14 +00:00
dad5310ab6
rebaseline.py: add --add-new option, and only add new expectations if it is set
...
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/16363006
git-svn-id: http://skia.googlecode.com/svn/trunk@9527 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-12 14:25:30 +00:00
cc2e1cf13c
rebaseline.py: improve readability of dry-run output
...
Split off of already-reviewed https://codereview.chromium.org/15789010/ ('rebaseline.py: --tests and --configs are now FILTERS within json results')
Review URL: https://codereview.chromium.org/16509006
git-svn-id: http://skia.googlecode.com/svn/trunk@9507 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-11 16:24:37 +00:00
82f317815e
rebaseline.py: in option names, replace underscores with hyphens
...
Split off of already-reviewed https://codereview.chromium.org/15789010/ ('rebaseline.py: --tests and --configs are now FILTERS within json results')
Review URL: https://codereview.chromium.org/15904018
git-svn-id: http://skia.googlecode.com/svn/trunk@9504 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-11 15:45:46 +00:00
cf5eb6ab25
Preparations for running skimage tool on bots.
...
skimage_main:
Specify the full name of the expectations file to write on the command line,
rather than the folder to write 'results.json'. This is consistent with
writeJsonSummaryPath in GM, as well as the format for reading the file.
Further, it allows more flexibility to name the file.
Do not force opaque when writing bitmaps. skimage does its comparison on
the raw bitmaps. The images are written for visual comparison, and mostly
transparent bitmaps (like resources/plane.png) aren't meaningful when
forced opaque.
Provide an option to to write only mismatches to a directory. Handled
similar to the writePath.
Report a failure if there is no expectations file. This allows writing
to the mismatchPath so we can create an initial baseline.
Pass SkBitmap by const&.
skimage_self_test:
Include the name of the json file in the parameter passed to
createExpectationsPath. The name has been changed to be more specific.
.gitignore:
Ignore the new name of the file created by skimage_self_test.
run.sh:
Run skimage_self_tests.py
BUG=http://code.google.com/p/skia/issues/detail?id=1241
R=epoger@google.com
Review URL: https://codereview.chromium.org/15813016
git-svn-id: http://skia.googlecode.com/svn/trunk@9474 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-07 12:43:15 +00:00
e762dad67c
Run skimage as part of run.sh
...
Test skimage to ensure that it does not break.
Split off from https://codereview.chromium.org/15813016/
BUG=http://code.google.com/p/skia/issues/detail?id=1241
R=epoger@google.com
Review URL: https://codereview.chromium.org/15774017
git-svn-id: http://skia.googlecode.com/svn/trunk@9473 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-07 12:40:35 +00:00
19e80c16a5
rebaseline.py self-tests: split test inputs and outputs into completely separate directories
...
Review URL: https://codereview.chromium.org/16557002
git-svn-id: http://skia.googlecode.com/svn/trunk@9459 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-06 15:11:11 +00:00
bf65378266
add more rebaseline test resources (no behavioral changes)
...
(SkipBuildbotRuns)
Review URL: https://codereview.chromium.org/15774019
git-svn-id: http://skia.googlecode.com/svn/trunk@9446 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-05 16:26:23 +00:00
99ba65a181
rebaseline.py: if --tests is not specified, get test list from actual-results.json
...
R=scroggo@google.com
Review URL: https://codereview.chromium.org/16093025
git-svn-id: http://skia.googlecode.com/svn/trunk@9443 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-05 15:43:37 +00:00
db29a310f7
rebaseline.py : add self-tests
...
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/15660014
git-svn-id: http://skia.googlecode.com/svn/trunk@9426 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-04 14:58:47 +00:00
4fed643cb9
Create a self test for skimage.
...
Runs skimage twice: once to create an expectations file, and a
second time comparing against the file. Uses the files in
resources as test files.
R=epoger@google.com
Review URL: https://codereview.chromium.org/14969007
git-svn-id: http://skia.googlecode.com/svn/trunk@9123 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-05-14 17:50:02 +00:00
bd9669ad27
bench_graph_svg: add r7618 results (NOT per-tile) to self-test source data
...
Review URL: https://codereview.appspot.com/7305098
git-svn-id: http://skia.googlecode.com/svn/trunk@7760 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-02-17 08:33:27 +00:00
commit-bot@chromium.org
bensong@google.com
scroggo@google.com
epoger@google.com