From 5e217b643562a66e46ef8c4e4402e35c64747e3e Mon Sep 17 00:00:00 2001 From: Luca Greco Date: Wed, 17 Jan 2024 14:07:53 +0000 Subject: [PATCH] Bug 1874487 - Add perfdocs to describe browsertime-tp6-webextensions and browsertime-tp6m-webextensions jobs. r=sparky,perftest-reviewers Differential Revision: https://phabricator.services.mozilla.com/D198603 --- testing/perfdocs/generated/browsertime.rst | 52 +++++++++++++++++++ .../raptor/raptor/perfdocs/browsertime.rst | 52 +++++++++++++++++++ 2 files changed, 104 insertions(+) diff --git a/testing/perfdocs/generated/browsertime.rst b/testing/perfdocs/generated/browsertime.rst index aa3968e10b1a..4d1975224f42 100644 --- a/testing/perfdocs/generated/browsertime.rst +++ b/testing/perfdocs/generated/browsertime.rst @@ -124,6 +124,58 @@ Or for Raptor-Browsertime (use ``chrome`` for desktop, and ``chrome-m`` for mobi ./mach raptor --browsertime -t amazon --app chrome --browsertime-chromedriver +Running Page-load tests with third party WebExtensions +------------------------------------------------------ +Page-load tests can also be executed on both Firefox Desktop and Firefox for Android builds with a set of popular +third party extensions installed (similarly to talos-realworld-webextensions, which runs tp5 tests with a set of third +party extensions installed). + +Any of the page-load tests can be executed locally with the pre-selected set of third party extensions installed by just +adding to the base raptor command the additional ``--conditioned-profile settled-webext`` command line option. + +Launch amazon tp6 page-load test on Firefox Desktop: + +:: + + ./mach raptor --browsertime -t amazon --conditioned-profile settled-webext + +Launch amazon tp6 mobile page-load test on Firefox for Android (the apk has to be pre-installed, mach raptor does detect if already installed but +it does not install it): + +:: + + ./mach raptor --browsertime -t amazon --app fenix --binary org.mozilla.fenix --conditioned-profile settled-webext + +To run these jobs on try, make sure to select the tp6 jobs that include the string `webextensions`, as an example (add ``--no-push`` to force try fuzzy to only +list the jobs selected by the try fuzzy query) to run all tp6 page-load webextensons jobs currently defined: + +:: + + ./mach try fuzzy -q "'tp6 'webextensions" + +Similarly for running tp6m on Firefox for Android builds: + +:: + + ./mach try fuzzy --full -q "'tp6m 'webextensions" + +The set of extensions installed are the ones listed in the ``"addons"`` property of the condprof customization file +`webext.json`_ from the ``testing/condprofile/condprof/customization/`` directory. + +All extensions listed in the ``webext.json`` file are expected to have been predownloaded and included in the ``firefox-addons.tar`` archive +defined in the CI fetch config named `firefox-addons`_, but they will be automatically downloaded from the url specified in the ``webext.json`` +file if they are not. + +In a try push we allow to run jobs on new extension xpi files not part of the firefox-addons.tar archive, the new extension needs to be just +added in the `webext.json`_ condprof customization file in a patch part of the same stack of patches being pushed to try. + +On the contrary new extensions added to the `webext.json`_ condprof customization file on mozilla-central patches will require the xpi file to be +added to the ``firefox-addons.tar`` archive and the `firefox-addons`_ CI fetch config updated accordingly (missing to update the archive will +trigger explicit linter errors, :doc:`see condprof-addons linter docs `). + +.. _webext.json: https://searchfox.org/mozilla-central/rev/bc6a50e6f08db0bb371ef7197c472555499e82c0/testing/condprofile/condprof/customization/webext.json +.. _firefox-addons: https://searchfox.org/mozilla-central/rev/bc6a50e6f08db0bb371ef7197c472555499e82c0/taskcluster/ci/fetch/browsertime.yml#169-176 + More Examples ------------- diff --git a/testing/raptor/raptor/perfdocs/browsertime.rst b/testing/raptor/raptor/perfdocs/browsertime.rst index aa3968e10b1a..4d1975224f42 100644 --- a/testing/raptor/raptor/perfdocs/browsertime.rst +++ b/testing/raptor/raptor/perfdocs/browsertime.rst @@ -124,6 +124,58 @@ Or for Raptor-Browsertime (use ``chrome`` for desktop, and ``chrome-m`` for mobi ./mach raptor --browsertime -t amazon --app chrome --browsertime-chromedriver +Running Page-load tests with third party WebExtensions +------------------------------------------------------ +Page-load tests can also be executed on both Firefox Desktop and Firefox for Android builds with a set of popular +third party extensions installed (similarly to talos-realworld-webextensions, which runs tp5 tests with a set of third +party extensions installed). + +Any of the page-load tests can be executed locally with the pre-selected set of third party extensions installed by just +adding to the base raptor command the additional ``--conditioned-profile settled-webext`` command line option. + +Launch amazon tp6 page-load test on Firefox Desktop: + +:: + + ./mach raptor --browsertime -t amazon --conditioned-profile settled-webext + +Launch amazon tp6 mobile page-load test on Firefox for Android (the apk has to be pre-installed, mach raptor does detect if already installed but +it does not install it): + +:: + + ./mach raptor --browsertime -t amazon --app fenix --binary org.mozilla.fenix --conditioned-profile settled-webext + +To run these jobs on try, make sure to select the tp6 jobs that include the string `webextensions`, as an example (add ``--no-push`` to force try fuzzy to only +list the jobs selected by the try fuzzy query) to run all tp6 page-load webextensons jobs currently defined: + +:: + + ./mach try fuzzy -q "'tp6 'webextensions" + +Similarly for running tp6m on Firefox for Android builds: + +:: + + ./mach try fuzzy --full -q "'tp6m 'webextensions" + +The set of extensions installed are the ones listed in the ``"addons"`` property of the condprof customization file +`webext.json`_ from the ``testing/condprofile/condprof/customization/`` directory. + +All extensions listed in the ``webext.json`` file are expected to have been predownloaded and included in the ``firefox-addons.tar`` archive +defined in the CI fetch config named `firefox-addons`_, but they will be automatically downloaded from the url specified in the ``webext.json`` +file if they are not. + +In a try push we allow to run jobs on new extension xpi files not part of the firefox-addons.tar archive, the new extension needs to be just +added in the `webext.json`_ condprof customization file in a patch part of the same stack of patches being pushed to try. + +On the contrary new extensions added to the `webext.json`_ condprof customization file on mozilla-central patches will require the xpi file to be +added to the ``firefox-addons.tar`` archive and the `firefox-addons`_ CI fetch config updated accordingly (missing to update the archive will +trigger explicit linter errors, :doc:`see condprof-addons linter docs `). + +.. _webext.json: https://searchfox.org/mozilla-central/rev/bc6a50e6f08db0bb371ef7197c472555499e82c0/testing/condprofile/condprof/customization/webext.json +.. _firefox-addons: https://searchfox.org/mozilla-central/rev/bc6a50e6f08db0bb371ef7197c472555499e82c0/taskcluster/ci/fetch/browsertime.yml#169-176 + More Examples -------------