This patch adds a new set of tests (essential only) to run in CI at tier 2. A command line argument is added to turn bytecode cache testing on in warm pageloads. This also disables the testing in our primary pageload tests to fix the variance and increases in runtime.
Differential Revision: https://phabricator.services.mozilla.com/D151757
This patch adds the matrix-react-bench to raptor-browsertime. It will run alongside the other benchmarks on trunk and mozilla-beta. It will also be possible to run these on chrome and chromium.
Differential Revision: https://phabricator.services.mozilla.com/D137344
In the test transforms we currently have general 'test_description_schema' and
a schema for raptor / browsertime tasks. Prior, the raptor schema was validated
*before* the test schema, and we were doing a lot of validation twice.
This revision switches the order such that the test schema is first evaluated,
and then the raptor one is. To accomplish this, we create a 'raptor' subconfig
that the 'test_description_schema' completely ignores (and leaves up to the
raptor schema to evaluate).
The benefit of this change is that all test tasks will have a single entry point
when they get into the 'test' transforms. This makes reasoning about their
configuration much easier and lays the ground work to start splitting even more
transforms out into other modules.
Differential Revision: https://phabricator.services.mozilla.com/D132069
This patch adds 4 new interactive tests for facebook and reddit. The patch updates the recordings as well as needed. Furthermore, an update to the visualmetrics.py script is needed to handle a permafailure in the reddit-post tests. Note that the reddit-post test was split in 2 since they are quite large.
Differential Revision: https://phabricator.services.mozilla.com/D130665
CLOSED TREE
This patch reduces the number of browser-cycles that a live-site test (cnn in this case) does from 25 down to 5. Furthermore, we also disable media autoplay for all live site tests. This change required an update to browsertime to prevent default prefs from being set and overriding our prefs.
Differential Revision: https://phabricator.services.mozilla.com/D127831
This patch reduces the number of browser-cycles that a live-site test (cnn in this case) does from 25 down to 5. Furthermore, we also disable media autoplay for all live site tests. This change required an update to browsertime to prevent default prefs from being set and overriding our prefs.
Differential Revision: https://phabricator.services.mozilla.com/D127831
This includes removing some OSX jobs that are run already with
WebRender, shifting Linux clang-trunk and Windows reference hardware
jobs to run with WebRender.
autoland
--- target_task_set@9d67b780469b
+++ target_task_set@ao_ci_noqr_remove
-test-macosx1015-64/debug-web-platform-tests-reftest-e10s
mozilla-central
--- target_task_set@9d67b780469b
+++ target_task_set@ao_ci_noqr_remove
-test-macosx1015-64-shippable/opt-awsy-base-fis-e10s
-test-macosx1015-64-shippable/opt-awsy-tp6-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-h264-sfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-hfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-vp9-sfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-widevine-h264-sfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-widevine-hfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-widevine-vp9-sfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-amazon-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-bing-search-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-cnn-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-fandom-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-google-slides-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-instagram-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-twitter-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-wikipedia-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-yahoo-mail-e10s
-test-macosx1015-64/debug-web-platform-tests-reftest-e10s
+test-windows10-64-ref-hw-2017-qr/opt-talos-webgl-gli-e10s
Differential Revision: https://phabricator.services.mozilla.com/D122248
This includes removing some OSX jobs that are run already with
WebRender, shifting Linux clang-trunk and Windows reference hardware
jobs to run with WebRender.
autoland
--- target_task_set@9d67b780469b
+++ target_task_set@ao_ci_noqr_remove
-test-macosx1015-64/debug-web-platform-tests-reftest-e10s
mozilla-central
--- target_task_set@9d67b780469b
+++ target_task_set@ao_ci_noqr_remove
-test-macosx1015-64-shippable/opt-awsy-base-fis-e10s
-test-macosx1015-64-shippable/opt-awsy-tp6-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-h264-sfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-hfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-vp9-sfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-widevine-h264-sfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-widevine-hfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-firefox-youtube-playback-widevine-vp9-sfr-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-amazon-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-bing-search-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-cnn-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-fandom-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-google-slides-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-instagram-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-twitter-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-wikipedia-e10s
-test-macosx1015-64-shippable/opt-browsertime-tp6-essential-firefox-yahoo-mail-e10s
-test-macosx1015-64/debug-web-platform-tests-reftest-e10s
+test-windows10-64-ref-hw-2017-qr/opt-talos-webgl-gli-e10s
Differential Revision: https://phabricator.services.mozilla.com/D122248
Jeff Muizelaar
Myeongjun Go
Johan Lorenzo
Gregory Mierzwinski
Joel Maher
Kash Shampur
Andrew Halberstadt
andrej
Kimberly Sereduck
Alexandru Michis
Csoregi Natalia
Alex Ionescu
Sebastian Hengst
Andrew Osmond
Butkovits Atila
Andi-Bogdan Postelnicu