6c2ef85237
Bug 1614282 - [Mozproxy] Add retry and custom error message to download_file_from_url r=whimboo,tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D62201
--HG--
extra : moz-landing-system : lando
2020-02-11 21:04:04 +00:00
d181041431
Bug 1613743 - Merge application.json and jobs.json r=tarek,perftest-reviewers,sparky
...
Raptor was previously generating two artifacts for the `run-visual-metrics.py`
script to consume: `jobs.json` and `application.json`. These artifacts have
been merged.
Differential Revision: https://phabricator.services.mozilla.com/D62363
--HG--
extra : moz-landing-system : lando
2020-02-11 20:03:30 +00:00
d9766a8b08
Bug 1612005 - Do not specifiy individual videos in jobs.json r=tarek,perftest-reviewers,sparky
...
The `run-visual-metrics.py` script was intended to consume a `jobs.json` file
containing one `browsertime.json` per video. However it was not being used as
such and was continuously re-processing the first video specified in the
`browsertime.json` file. If a job were submitted with a `browsertime.json`
containing 15 videos and 15 different videos, only the first would be
processed. This leads to us having incorrect metrics because over all runs all
the metrics will be identical.
Now we only specify the `browsertime.json` in the `jobs.json` file and extract
the paths to videos from there. Also because we are never downloading inputs
this way, we get to remove some dead code and our dependency on `requests`.
Differential Revision: https://phabricator.services.mozilla.com/D62320
--HG--
extra : moz-landing-system : lando
2020-02-11 19:59:58 +00:00
b5685da087
Bug 1614504 - Remove blank lines from profile before passing it to browsertime. r=perftest-reviewers,octavian_negru
...
With this patch, we now remove all blank lines from the profiles we produce which cause Browsertime 8 to break.
Differential Revision: https://phabricator.services.mozilla.com/D62370
--HG--
extra : moz-landing-system : lando
2020-02-11 13:18:33 +00:00
90a395b2ab
Bug 1585022 - Add support to raptor-browsertime for running the ares6 benchmark on Firefox desktop r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D61414
--HG--
extra : moz-landing-system : lando
2020-02-10 08:13:19 +00:00
f4def64e94
Bug 1613938 - Raptor tests hang when downloading conditioned profiles r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D62053
--HG--
extra : moz-landing-system : lando
2020-02-07 16:38:05 +00:00
3161788df7
Bug 1603499 - Report java exceptions from minidump .extra files; r=bc
...
Stop using mozcrash.check_for_java_exception(), which scans logcat for uncaught
exceptions. Instead, check for java_stack in the extra dump information during
normal crash reporting; if java_stack is present, display the java exception and
stack instead of dumping a full native crash report.
Differential Revision: https://phabricator.services.mozilla.com/D61836
--HG--
extra : moz-landing-system : lando
2020-02-07 16:05:52 +00:00
6d2d6bf95e
Bug 1612375 - Run raptor youtube playback tests on fennec68 r=perftest-reviewers,alexandru.irimovici,AlexandruIonescu
...
Differential Revision: https://phabricator.services.mozilla.com/D61313
--HG--
extra : moz-landing-system : lando
2020-02-06 13:58:11 +00:00
3f74e6d794
Bug 1613332 - Add log messages around raptor conditioned profiles r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D61663
--HG--
extra : moz-landing-system : lando
2020-02-05 14:01:50 +00:00
917f420412
Bug 1601009 - Add support to raptor-browsertime for running the speedometer benchmark on Firefox android r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D61305
--HG--
extra : moz-landing-system : lando
2020-02-05 08:59:19 +00:00
16a28d6c06
Backed out changeset c1b764e0ddb3 (bug 1601009) for massive raptor failures.
...
CLOSED TREE
2020-02-04 12:17:59 +02:00
049e26b384
Bug 1601009 - Add support to raptor-browsertime for running the speedometer benchmark on Firefox android r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D61305
--HG--
extra : moz-landing-system : lando
2020-02-03 16:26:21 +00:00
cb537c4f9f
Bug 1611924 - raptor remove_raptor_webext does not reset the webext value after is closed r=perftest-reviewers,octavian_negru
...
Differential Revision: https://phabricator.services.mozilla.com/D61170
--HG--
extra : moz-landing-system : lando
2020-01-28 13:45:28 +00:00
f1633f6bb4
Bug 1611781 - Raptor tests not showing mozproxy error messages r=perftest-reviewers,Marian,octavian_negru
...
Differential Revision: https://phabricator.services.mozilla.com/D61112
--HG--
extra : moz-landing-system : lando
2020-01-27 12:14:35 +00:00
427a899017
Bug 1611110 - [Raptor] Do not check if runnier is started in dektop run_test_teardown r=perftest-reviewers,tarek,AlexandruIonescu
...
Differential Revision: https://phabricator.services.mozilla.com/D60816
--HG--
extra : moz-landing-system : lando
2020-01-27 10:21:34 +00:00
f72ab8b7fe
Bug 1553131 - Run raptor-youtube-playback jobs with profiler enabled in CI r=perftest-reviewers,sparky,tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D42516
--HG--
extra : moz-landing-system : lando
2020-01-21 16:54:29 +00:00
007caefd70
Bug 1609392 - Split Raptor and Browsertime classes from raptor.py r=tarek,AlexandruIonescu,octavian_negru,sparky,perftest-reviewers,alexandru.irimovici
...
Differential Revision: https://phabricator.services.mozilla.com/D60028
--HG--
rename : testing/raptor/raptor/raptor.py => testing/raptor/raptor/webextension/base.py
extra : moz-landing-system : lando
2020-01-21 09:07:19 +00:00
600d034817
Bug 1568092 - don't ignore tooltool_download() exit code and add a retry r=AlexandruIonescu,Bebe
...
This patch will check the exit code of the ProcessHandler() that
calls the tooltool script, and will also retry when it fails.
Differential Revision: https://phabricator.services.mozilla.com/D60397
--HG--
extra : moz-landing-system : lando
2020-01-21 07:17:05 +00:00
583e5855cb
Bug 1609029 - Fix perma [tier2] ytp raptor-manifest Error r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D59875
--HG--
extra : moz-landing-system : lando
2020-01-14 16:04:49 +00:00
46a00cab18
Bug 1608374 - Split perftest from raptor.py r=AlexandruIonescu,rwood
...
Differential Revision: https://phabricator.services.mozilla.com/D59489
--HG--
extra : moz-landing-system : lando
2020-01-14 09:48:21 +00:00
73c92a684c
Bug 1607891 - Add fennec (68) support to raptor-browsertime r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D59737
--HG--
extra : moz-landing-system : lando
2020-01-13 20:08:07 +00:00
15cf5c2afd
Bug 1607722 - Permanent speedometer KeyError: 'measure' r=sparky
...
If we don't have any provided 'measure' value, defaults to all.
Differential Revision: https://phabricator.services.mozilla.com/D59277
--HG--
extra : moz-landing-system : lando
2020-01-09 19:23:40 +00:00
ec66c3e408
Backed out changeset 80f5c59d5df4 (bug 1602657) as requested by sdonner on slack. CLOSED TREE
2020-01-10 01:01:15 +02:00
ea4c73f19d
Bug 1608038 - Split signal_handler from raptor.py r=AlexandruIonescu,perftest-reviewers,octavian_negru,rwood
...
Differential Revision: https://phabricator.services.mozilla.com/D59287
--HG--
extra : moz-landing-system : lando
2020-01-09 13:52:29 +00:00
961808bced
Bug 1592746 - filter measures in perfherder_data r=sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D58963
--HG--
extra : moz-landing-system : lando
2020-01-07 16:47:00 +00:00
2a56033519
Bug 1573200 - Add regex support for alert_on field from raptor tests r=perftest-reviewers,rwood
...
Differential Revision: https://phabricator.services.mozilla.com/D41781
--HG--
extra : moz-landing-system : lando
2020-01-06 18:20:22 +00:00
a9bbb507bf
Bug 1606767 - Fix permafail on reference browser by using non-conditioned profile. r=rwood,perftest-reviewers
...
Differential Revision: https://phabricator.services.mozilla.com/D58592
--HG--
extra : moz-landing-system : lando
2020-01-03 19:25:53 +00:00
78ac064d46
Bug 1606826 - Browsertime android visual metrics job failing to get application name r=perftest-reviewers,stephendonner
...
Differential Revision: https://phabricator.services.mozilla.com/D58627
--HG--
extra : moz-landing-system : lando
2020-01-03 18:05:46 +00:00
dcb41e6ebd
Bug 1602801 - Mark StyleBench as subtest_lower_is_better r=perftest-reviewers,rwood
...
Stylebench is patterned after Speedometer and calculates overall score from subtests in the same way. Stylebench should be marked as subtest_lower_is_better just like Speedometer.
Differential Revision: https://phabricator.services.mozilla.com/D56557
--HG--
extra : moz-landing-system : lando
2020-01-02 20:03:24 +00:00
a1780842dc
Bug 1601414 - Add application info json file to browsertime results for vismet jobs r=perftest-reviewers,stephendonner
...
Differential Revision: https://phabricator.services.mozilla.com/D58404
--HG--
extra : moz-landing-system : lando
2020-01-02 19:53:48 +00:00
1c212cd60a
Bug 1606199 - Fix permafail by using non-conditioned profile for fennec_aurora; r=perftest-reviewers,rwood
...
Differential Revision: https://phabricator.services.mozilla.com/D58473
--HG--
extra : moz-landing-system : lando
2020-01-02 18:20:24 +00:00
decc696b7f
Bug 1602657 - Set appropriate value for post-startup-delay in (and only in) CI, for conditioned-profile runs. r=perftest-reviewers,rwood
...
Differential Revision: https://phabricator.services.mozilla.com/D57550
--HG--
extra : moz-landing-system : lando
2020-01-02 18:15:13 +00:00
a97f8f3531
Bug 1606131 - Have the browsertime test jobs differentiate between app in the perfherder data output r=perftest-reviewers,stephendonner
...
Differential Revision: https://phabricator.services.mozilla.com/D58445
--HG--
extra : moz-landing-system : lando
2019-12-31 18:48:17 +00:00
fba75b1823
Bug 1605880 - Use consistent fission perfherder data naming for talos/raptor/awsy r=sparky,Marian
...
Differential Revision: https://phabricator.services.mozilla.com/D58299
--HG--
extra : moz-landing-system : lando
2019-12-27 17:11:55 +00:00
8f9b171008
Bug 1605758 - Raptor fission data missing 'fission_enabled' from PERFHERDER_DATA extraOptions r=perftest-reviewers,sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D58147
--HG--
extra : moz-landing-system : lando
2019-12-23 18:50:46 +00:00
94a084875c
Bug 1592333 - Improve raptor-browsertime page-load failure messages r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D57712
--HG--
extra : moz-landing-system : lando
2019-12-20 17:24:25 +00:00
977f77cafe
Bug 1605132 - Differentiate fission-enabled vs non-fission enabled talos data in perfherder r=perftest-reviewers,stephendonner
...
Differential Revision: https://phabricator.services.mozilla.com/D57819
--HG--
extra : moz-landing-system : lando
2019-12-19 20:41:44 +00:00
a2b3c15851
Bug 1605070 - no_conditioned_profile not surfaced in raptor_class r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D57774
--HG--
extra : moz-landing-system : lando
2019-12-19 12:09:18 +00:00
af46e7328b
Bug 1597711 - Add conditioned profile support for GV & Fenix in Raptor r=stephendonner,rwood
...
Enable using the 'settled' conditioned profile for geckoview and fenix apps.
Differential Revision: https://phabricator.services.mozilla.com/D55408
--HG--
extra : moz-landing-system : lando
2019-12-18 22:20:18 +00:00
4935beddbd
Bug 1603152 - For cold page-load set browser cycles to 3 when gecko profiling is enabled r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D57533
--HG--
extra : moz-landing-system : lando
2019-12-17 21:21:46 +00:00
9a1d963353
Bug 1592063 - Browsertime failing on fenix with 'browser failed to start' error r=tarek
...
Differential Revision: https://phabricator.services.mozilla.com/D57369
--HG--
extra : moz-landing-system : lando
2019-12-17 15:29:55 +00:00
fccf6da7c0
Bug 1601011 - Add gecko profiling support to raptor-browsertime (android) r=rwood
...
Differential Revision: https://phabricator.services.mozilla.com/D57329
--HG--
extra : moz-landing-system : lando
2019-12-17 08:04:47 +00:00
6e1c7fdb61
Bug 1592048 - Browsertime is intermittently failing to measure 'firstPaint' from Youtube on android r=rwood
...
Differential Revision: https://phabricator.services.mozilla.com/D57341
--HG--
extra : moz-landing-system : lando
2019-12-16 17:00:18 +00:00
20f3a40513
Bug 1601333 - Browsertime conditioning sequence is very different and longer than Raptor web-ext r=acreskey
...
This patch makes the Browsertime conditioning sequence closer to the web-ext one,
Differential Revision: https://phabricator.services.mozilla.com/D56867
--HG--
extra : moz-landing-system : lando
2019-12-12 16:40:40 +00:00
5bbafc894a
Bug 1595775 - Update raptor mobile recordings Q4/2019 r=perftest-reviewers,alexandru.irimovici,AlexandruIonescu
...
Differential Revision: https://phabricator.services.mozilla.com/D53184
--HG--
extra : moz-landing-system : lando
2019-12-12 14:48:29 +00:00
c424194edf
Bug 1600838 - Make conditioned profiles for desktop Firefox the default, in Raptor and Browsertime. r=tarek,perftest-reviewers,sparky
...
Differential Revision: https://phabricator.services.mozilla.com/D55833
--HG--
extra : moz-landing-system : lando
2019-12-11 22:43:03 +00:00
bfe666563e
Bug 1585013 - Add gecko profiling support to raptor-browsertime r=rwood,perftest-reviewers
...
Adds support for --gekoprofile with --browsertime in raptor.
Differential Revision: https://phabricator.services.mozilla.com/D50959
--HG--
extra : moz-landing-system : lando
2019-12-11 20:38:00 +00:00
9696f02d54
Bug 1585456 - [mozproxy] Error out when mozproxy can't bind to port r=tarek,perftest-reviewers,davehunt
...
Differential Revision: https://phabricator.services.mozilla.com/D50594
--HG--
extra : moz-landing-system : lando
2019-12-11 10:07:28 +00:00
920d0d6dbd
Bug 1587080 - Part 4: Add raptor performance test documentation. r=perftest-reviewers,octavian_negru
...
This patch adds in-tree performance documentation for raptor which is verified by perfdocs.
Depends on D53649
Differential Revision: https://phabricator.services.mozilla.com/D53650
--HG--
extra : moz-landing-system : lando
2019-12-10 17:04:29 +00:00
72d7d8ff5e
Backed out 4 changesets (bug 1587080) for causing perfdocs Lint failure CLOSED TREE
...
Backed out changeset 254fd0c3af4a (bug 1587080)
Backed out changeset 34efe80bfd61 (bug 1587080)
Backed out changeset 4a552742b1cd (bug 1587080)
Backed out changeset 22afcb56e8ea (bug 1587080)
2019-12-10 19:01:07 +02:00
Florin Strugariu
Barret Rennie
Gregory Mierzwinski
Geoff Brown
Marian Raiciof
Cosmin Sabou
Tarek Ziadé
Rob Wood
Brindusan Cristian
Stephen Donner
David Major
Noemi Erli