Will allow running "null" Mock ICD tests.
Also fixes a crash running the Mock ICD with the overlay.
Bug: angleproject:5736
Change-Id: I42e3943a0de00dbab3ee125f1946afd22b38fec6
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2809096
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Tim Van Patten <timvp@google.com>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
This path was broken for the white box unit tests. Also adds argparse
handling to the runner so we can more flexibly override command line
arguments.
Previously the broken calibration was causing some of the tests to run
only a single test iteration when measuring. This could lead to low
quality measurements.
Bug: angleproject:5573
Change-Id: Ic1cb2b2553774a361325f290440c40b2ff90db5e
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2672702
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Mohan Maiya <m.maiya@samsung.com>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
This will limit the total number of steps run in the test.
When using this argument we skip warmp & calibration.
Renames the prior --steps argument to --steps-per-trial.
Bug: b/172480147
Change-Id: Iae7cce26d66ca0bbdb21c1f511a6bfa246bd204e
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/2519404
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Cody Northrop <cnorthrop@google.com>
Refactors the perf tests to fix the issue of metric and story being
swapped, which causes issues when trying to convert to histograms.
Specifically, does the following:
1. Rolls the version of src/tests/perf_tests/third_party/perf/ to
Chromium 476dae823269c8d05b544271af97ad1adb0db8ee
2. Switch to using PerfResultReporter instead of PrintResult directly.
3. Split RenderTestParams::suffix into backend and story; backend is
used as part of the metric, while story is used as the story.
4. Remove the "average" metric that was being automatically reported
by ANGLEPerfTest, as reported results are automatically averaged.
5. Update the reported metric to more clearly distinguish between
test, backend, and metric. It is now name_backend.metric. e.g.
DrawCallPerf_vulkan.wall_time.
Bug: chromium:923564,chromium:924618
Change-Id: I00cc191407052f23df57dbfa53b6fb088fc26960
Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1762360
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Jonah Ryan-Davis <jonahr@google.com>
There's a lot more that can go into this perf test, but it requires
further work on the Vulkan back end.
Bug: angleproject:2999
Change-Id: Iea62bfd09639af108674dcf0a9e7c9d36ccddcef
Reviewed-on: https://chromium-review.googlesource.com/c/1437734
Commit-Queue: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
This change does a few things involving the way we run the perf tests:
- Perf test runner can capture different metrics
Useful for capturing the new "nanoSecPerIteration" metric.
- Removes the "score" metric
We'll move to the new time-based metrics. These new metrics are scaled
correctly with iteration counts.
- Runs three trials per perf test
This gives more measurements per test. Each trial is approximately one
second. First the perf tests set a fixed number of iterations after
calibrating the number of steps that we can run in one second. After
that the three trials are run. This should give more stable results.
- Apply more CPU stabilization on Windows
Use SetPriorityClass to apply more CPU priority. Also upgrade
SetThreadPriority to the highest level.
- Always build the Vulkan command buffer test
This catches build regressions more easily. We still skip the test on
non-Android platforms.
Bug: angleproject:2923
Change-Id: I7da234c5af07775ba4a232bb8d65e0138ee7073f
Reviewed-on: https://chromium-review.googlesource.com/c/1330262
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
This change does a few things:
- make perf test runner script print % variation instead of stddev
This makes it a bit more clear how much variance there is.
- stabilize CPU in the render perf tests
Setting a thread affinity and priority should stop from switching cores
during the run. Hopefully can prevent background noise from changing
the test results.
- warm up the benchmark with a few iterations
This should hopefully make the test results a bit more stable.
- output a new normalized perf result value
The new result is normalized against the number of iterations. So it
should hopefully be stable even if the number of iterations is changed.
- increases the iteration count in the draw call perf tests.
These tests were completely dominated by SwapBuffers time. Increasing
the iterations per step means we actually are bottlenecked on CPU time
instead.
Bug: angleproject:2923
Change-Id: I5ee347cf93df239ac33b83dc5effe4c21e066736
Reviewed-on: https://chromium-review.googlesource.com/c/1303679
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
This perf test highlights the performance hotspots with
State::syncProgramTextures.
Also includes a fix to the perf test runner script.
Bug: angleproject:2763
Change-Id: I69ffa0cc0d5e023944495b7a1c844770a54f7ddc
Reviewed-on: https://chromium-review.googlesource.com/1166041
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Geoff Lang <geofflang@chromium.org>
Add #!/usr/bin/python2 and the executable permission bit to all scripts
where missing.
BUG=angleproject:2209
Change-Id: Ib33017c17e579c371b89bbfbdb7136b870027dc5
Reviewed-on: https://chromium-review.googlesource.com/862987
Reviewed-by: Corentin Wallez <cwallez@chromium.org>
Commit-Queue: Frank Henigman <fjhenigman@chromium.org>
No code changes in ANGLE, just a perf test runner fix.
BUG=angleproject:1155
Change-Id: If49c58da26f9224e09f292689b2d74f7e129a844
Reviewed-on: https://chromium-review.googlesource.com/633786
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
This patch enables perf_test_runner.py to run on Linux and MacOS.
BUG=angleproject:2029
Change-Id: Ia7176a3f5fed0245ab72d5def1199623c4fb4fe2
Reviewed-on: https://chromium-review.googlesource.com/497648
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
If src/tests/tests.sln is built instead of gyp/ANGLE.sln, the test
executable is placed under src/tests/Release*.
TEST=perf_test_runner on Windows
Change-Id: I72ce55e2caa8270bef089c073840d5e2d6caa14d
Reviewed-on: https://chromium-review.googlesource.com/438348
Commit-Queue: Olli Etuaho <oetuaho@nvidia.com>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
The missing check was a bit buggy, and the paths were not updated
to the new 'gyp' folder instead of 'build'.
BUG=angleproject:1569
Change-Id: I10408252851e90b79e306c8aea175ebe6a931b61
Reviewed-on: https://chromium-review.googlesource.com/412320
Reviewed-by: Geoff Lang <geofflang@chromium.org>
The hard-coded threshold of five uses before a release was regressing
the Oort online benchmark, which seems to use an index buffer many
times, then change to a different range of indices, which would need
to check the system memory for index range validation.
Also add a performance regression test, and an update to the perf
runner script which checks for the most recent binary among the search
directories.
BUG=594066
Change-Id: Id09cc32fd00bff1c72cbe9b6fb7c210fd047a551
Reviewed-on: https://chromium-review.googlesource.com/339271
Reviewed-by: Geoff Lang <geofflang@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
This patch works using a notification scheme - whenever a Texture or
Renderbuffer changes in such a way as to recreate its RenderTarget, we
pass a signal to the Framebuffer to invalidate some internal state.
Everything is entirely tracked in the Renderer11 layer, and the GL
layer is left untouched.
A RenderTarget11 now tracks points to which it is bound, and the
Framebuffer11 is mostly responsible for managing those links.
The three locations where we notify a Framebuffer when its bound
RenderTargets might be dirty are:
1) RenderTarget11::~RenderTarget
2) EGLImageD3D::copyToLocalRendertarget
3) TextureStorage11_2D::useLevelZeroWorkaroundTexture
This patch gives about a 10% score increase in the D3D11 draw call
benchmark on my system.
BUG=angleproject:1260
Change-Id: Ide38aeadff4a2681bf5bd685e8ca3c9e2612a380
Reviewed-on: https://chromium-review.googlesource.com/327255
Reviewed-by: Geoff Lang <geofflang@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
The previous version of the script would always print the default test
name even if user specified a different one.
BUG=angleproject:596
Change-Id: I43a07e81b49f69c7c372b46bd9494d6fe440b968
Reviewed-on: https://chromium-review.googlesource.com/319091
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Tested-by: Olli Etuaho <oetuaho@nvidia.com>
1) Make it possible to run the script from any working directory
2) Abort the run if angle_perftests is executing multiple test cases
3) Print some information on which exe and test is running
4) Print the test runner output in case there was an error
BUG=angleproject:596
Change-Id: If30c2455dc39b0a776df03e6c1dda2ced90d73ba
Reviewed-on: https://chromium-review.googlesource.com/319090
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Tested-by: Olli Etuaho <oetuaho@nvidia.com>
This test runner can run the tests in an infinite loop, giving much
lower variance. It also automatically parses the output from several
runs and prints out the mean and standard deviation of the population.
BUG=None
Change-Id: Ic9f6f7d3158e93949a8e17c99ea1e3856eb34076
Reviewed-on: https://chromium-review.googlesource.com/312262
Reviewed-by: Jamie Madill <jmadill@chromium.org>
Tested-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill
Brian Sheedy
Geoff Lang
Shahbaz Youssefi
Frank Henigman
Jeff Gilbert
Shao
Olli Etuaho