mozilla
/
gecko-dev
зеркало из https://github.com/mozilla/gecko-dev.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Bug 1868384 - re-enable the recently disabled CaR pageload and motionmark tests. r=perftest-reviewers,sparky 

Recently there was a perma failure in chromium-as-release tests so all
non-sp3 tests were temporarily disabled. It would now seem whatever
issue was causing it upstream has been resolved so here the tests are
re-enabled. The perma fails were happening between (approx.) 29-11-2023
to 02-12-2023.

Differential Revision: https://phabricator.services.mozilla.com/D195650
This commit is contained in:
KS 2023-12-06 16:53:27 +00:00
Родитель c52c227f53
Коммит b625136948
2 изменённых файлов: 288 добавлений и 4 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

7
taskcluster/ci/test/browsertime-desktop.yml
Просмотреть файл

@ -214,7 +214,7 @@ browsertime-tp6-essential:
- --conditioned-profile=settled
- --extra-profiler-run
raptor:
apps: [firefox, chrome] # Disable non sp3 tests temporarily for CaR. Bug 1867809
apps: [firefox, chrome, custom-car]
test: tp6
run-visual-metrics:
by-test-platform:
@ -345,9 +345,8 @@ browsertime-benchmark:
custom-car:
- [speedometer, sp]
- [speedometer3, sp3]
# Disable non sp3 tests temporarily for CaR. Bug 1867809
# - [motionmark-animometer-ramp, mm-ar]
# - [motionmark-htmlsuite-ramp, mm-hr]
- [motionmark-animometer-ramp, mm-ar]
- [motionmark-htmlsuite-ramp, mm-hr]
default:
- ares6
- [assorted-dom, dom]

285
testing/perfdocs/generated/raptor.rst
Просмотреть файл

@ -1690,6 +1690,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-animometer-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-animometer-ramp**
- ✅
- ✅
@ -1743,6 +1748,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-animometer-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-animometer-ramp**
- ✅
- ✅
@ -1838,6 +1848,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-animometer-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-animometer-ramp**
- ✅
- ✅
@ -1970,6 +1985,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-animometer-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-animometer-ramp**
- ✅
- ✅
@ -2023,6 +2043,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-animometer-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-animometer-ramp**
- ✅
- ✅
@ -2118,6 +2143,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-animometer-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-animometer-ramp**
- ✅
- ✅
@ -2821,6 +2851,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-htmlsuite-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-htmlsuite-ramp**
- ✅
- ✅
@ -2874,6 +2909,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-htmlsuite-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-htmlsuite-ramp**
- ✅
- ✅
@ -2969,6 +3009,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-htmlsuite-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-htmlsuite-ramp**
- ✅
- ✅
@ -3102,6 +3147,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-htmlsuite-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-htmlsuite-ramp**
- ✅
- ✅
@ -3155,6 +3205,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-htmlsuite-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-htmlsuite-ramp**
- ✅
- ✅
@ -3250,6 +3305,11 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
- ❌
* - **browsertime-benchmark-custom-car-motionmark-htmlsuite-ramp**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-benchmark-firefox-motionmark-htmlsuite-ramp**
- ✅
- ✅
@ -15556,6 +15616,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-amazon**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-amazon**
- ✅
- ✅
@ -15649,6 +15714,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-amazon**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-amazon**
- ✅
- ✅
@ -15809,6 +15879,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-amazon**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-amazon**
- ✅
- ✅
@ -15948,6 +16023,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-bing-search**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-bing-search**
- ✅
- ✅
@ -16036,6 +16116,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-bing-search**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-bing-search**
- ✅
- ✅
@ -16191,6 +16276,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-bing-search**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-bing-search**
- ✅
- ✅
@ -16676,6 +16766,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-cnn**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-cnn**
- ✅
- ✅
@ -16759,6 +16854,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-cnn**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-cnn**
- ✅
- ✅
@ -16899,6 +16999,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-cnn**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-cnn**
- ✅
- ✅
@ -18341,6 +18446,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-fandom**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-fandom**
- ✅
- ✅
@ -18429,6 +18539,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-fandom**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-fandom**
- ✅
- ✅
@ -18584,6 +18699,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-fandom**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-fandom**
- ✅
- ✅
@ -19692,6 +19812,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-google-slides**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-google-slides**
- ✅
- ✅
@ -19780,6 +19905,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-google-slides**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-google-slides**
- ✅
- ✅
@ -19935,6 +20065,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-google-slides**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-google-slides**
- ✅
- ✅
@ -20901,6 +21036,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-instagram**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-instagram**
- ✅
- ✅
@ -20969,6 +21109,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-instagram**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-instagram**
- ✅
- ✅
@ -21084,6 +21229,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-instagram**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-instagram**
- ✅
- ✅
@ -24740,6 +24890,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-twitter**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-twitter**
- ✅
- ✅
@ -24828,6 +24983,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-twitter**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-twitter**
- ✅
- ✅
@ -24978,6 +25138,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-twitter**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-twitter**
- ✅
- ✅
@ -25463,6 +25628,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-wikipedia**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-wikipedia**
- ✅
- ✅
@ -25551,6 +25721,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-wikipedia**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-wikipedia**
- ✅
- ✅
@ -25706,6 +25881,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-wikipedia**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-wikipedia**
- ✅
- ✅
@ -25845,6 +26025,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-yahoo-mail**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-yahoo-mail**
- ✅
- ✅
@ -25933,6 +26118,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-yahoo-mail**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-yahoo-mail**
- ✅
- ✅
@ -26088,6 +26278,11 @@ Tests for page-load performance. The links direct to the actual websites that ar
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-yahoo-mail**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-yahoo-mail**
- ✅
- ✅
@ -28036,6 +28231,11 @@ A set of test pages that are run as live sites instead of recorded versions. The
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-cnn**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-cnn**
- ✅
- ✅
@ -28119,6 +28319,11 @@ A set of test pages that are run as live sites instead of recorded versions. The
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-cnn**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-cnn**
- ✅
- ✅
@ -28259,6 +28464,11 @@ A set of test pages that are run as live sites instead of recorded versions. The
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-cnn**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-cnn**
- ✅
- ✅
@ -30020,6 +30230,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-amazon**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-amazon**
- ✅
- ✅
@ -30113,6 +30328,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-amazon**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-amazon**
- ✅
- ✅
@ -30273,6 +30493,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-amazon**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-amazon**
- ✅
- ✅
@ -31002,6 +31227,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-cnn**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-cnn**
- ✅
- ✅
@ -31085,6 +31315,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-cnn**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-cnn**
- ✅
- ✅
@ -31225,6 +31460,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-cnn**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-cnn**
- ✅
- ✅
@ -33579,6 +33819,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-instagram**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-instagram**
- ✅
- ✅
@ -33647,6 +33892,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-instagram**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-instagram**
- ✅
- ✅
@ -33762,6 +34012,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-instagram**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-instagram**
- ✅
- ✅
@ -34883,6 +35138,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-wikipedia**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-wikipedia**
- ✅
- ✅
@ -34971,6 +35231,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-wikipedia**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-wikipedia**
- ✅
- ✅
@ -35126,6 +35391,11 @@ Page-load performance test suite on Android. The links direct to the actual webs
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-wikipedia**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-wikipedia**
- ✅
- ✅
@ -35937,6 +36207,11 @@ These tests aren't used in standard testing, they are only used in the Raptor un
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-amazon**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-amazon**
- ✅
- ✅
@ -36030,6 +36305,11 @@ These tests aren't used in standard testing, they are only used in the Raptor un
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-amazon**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-amazon**
- ✅
- ✅
@ -36190,6 +36470,11 @@ These tests aren't used in standard testing, they are only used in the Raptor un
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-custom-car-amazon**
- ❌
- ❌
- ❌
- ❌
* - **browsertime-tp6-essential-firefox-amazon**
- ✅
- ✅