This patch adds a new set of tests (essential only) to run in CI at tier 2. A command line argument is added to turn bytecode cache testing on in warm pageloads. This also disables the testing in our primary pageload tests to fix the variance and increases in runtime.
Differential Revision: https://phabricator.services.mozilla.com/D151757
This patch fixes up how fission is handled in raptor. Previously, there were multiple ways in which it was handled, and now, we make all the decisions when we parse the command-line options. It also fixes the issue with fission being enabled on mobile tests.
Differential Revision: https://phabricator.services.mozilla.com/D147527
This patch runs the visual-metrics processing within the test task rather than the (deleted) `-vismet` tasks. It also updates the browsertime version to v15. It requires an update to node16 so new code was added to pull, and install the *-node-16 toolchain artifacts from CI.
This patch is possible because the browsertime update gives us the new `visualmetrics-portable.py` script which doesn't require ImageMagick anymore (they've been converted to Python dependencies). We did this conversion here: https://github.com/sitespeedio/browsertime/pull/1741
Furthermore, some changes are made to handle timeouts better, and increase the timeout for an interactive test.
Depends on D143502
Differential Revision: https://phabricator.services.mozilla.com/D142838
This patch builds off the ability to specify custom browsertime arguments on the command line in raptor to let the user run custom tests in "vanilla" browsertime. In this patch, we create a new test called `browsertime` that can be used to get a "dummy" template for a raptor test. Then, the user will be responsible for providing either a test script, or a URL to test.
To use a custom script we can pass something like `--browsertime-arg test_script=/path/to/script` or `--browsertime-arg url=https://www.sitespeed.io`. Furthermore, we can also use `test_script=pageload` to specify that we want to use the browsertime pageload script.
Differential Revision: https://phabricator.services.mozilla.com/D144168
In the vendored `mock` library, `assert_called()` returns the
`MagicMock` instance again - I'm not sure that it does the actual
assertion, so `assert instance.is_called()` may be a no-op.
Fortunately, in `unittest.mock`, `assert_called()` does the assertion
internally, and returns `None`. Since `assert None` causes a failure,
we need to tweak the invalid usage accordingly.
Differential Revision: https://phabricator.services.mozilla.com/D117454
This patch removes the mitm4 recordings that were being used in the raptor desktop unittests and replaces them with the mitm5 recordings.
Differential Revision: https://phabricator.services.mozilla.com/D114951
This patch rewrites some parts of the GeckoProfiler code to make it clearer and easier to maintain. It also changes how the profiles get organized into separate folders for each type. Furthermore, the archives no longer have the full directory path in them. To do this, we also have to update browsertime.
Differential Revision: https://phabricator.services.mozilla.com/D102043
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
This patch adds a new pageload testing mode called chimera mode. This mode runs the cold and warm pageload tests together within a single browser session (cold first, warm second). Some changes had to be made in the visual metrics task to be able to do this (per-test specification of extraOptions).
Differential Revision: https://phabricator.services.mozilla.com/D83356
This patch adds all desktop pageload tests to browsertime. It adds taskcluster tasks for these but doesn't enable them in production. They can only be run when --full is used.
Differential Revision: https://phabricator.services.mozilla.com/D82432
This patch disables conditioned-profiles in the cpu raptor tests - we have no need for conditioned profiles in this unittest.
Differential Revision: https://phabricator.services.mozilla.com/D73064
Myeongjun Go
Gregory Mierzwinski
Joel Maher
Kimberly Sereduck
Butkovits Atila
Florin Strugariu
Mitchell Hentges
Alex Ionescu
Steve Fink
Alexandru Michis
Dave Hunt
Dorel Luca
Akshat Dixit
Bob Clary
Andrey Bienkowski
Ricky Stewart
Bogdan Tara
Narcis Beleuzu
alexandru.ionescu