This patch upgrade browsertime to v10, enables webdriver-based navigation in browsertime tests, and also disables the Firefox WindowRecorder for a fairer comparison between Chrome and Firefox.
Differential Revision: https://phabricator.services.mozilla.com/D95425
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
Depends on D94045
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
These files were omitted from the original patch because reformatting them required some manual intervention in order to avoid breaking unit tests. Generally the `noqa` lines were already there and just needed to be moved from one line to another (due to the reformatting by `black`), but sometimes `black` saw fit to move a bunch of stuff all onto one line, requiring me to introduce new `noqa` lines.
Besides the autoformat by `black` and some manual fixups, this patch contains no other changes.
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94052
This patch adds all desktop pageload tests to browsertime. It adds taskcluster tasks for these but doesn't enable them in production. They can only be run when --full is used.
Differential Revision: https://phabricator.services.mozilla.com/D82432
This patch adds a small README file to the tools/browsertime folder to give people more information about its purpose and provides a link to the perftest wiki.
Depends on D78044
Differential Revision: https://phabricator.services.mozilla.com/D78045
When a package is installed with "pip --user", the import works right after the virtualenv activation.
Even if we force its install via mozbuild virtualenv it will fail. This patch checks if the installed package is **inside** or
**outside** the virtualenv. If it's outside, it forces its installation **inside**.
Notice that we might want to move that logic in mozbuild/virtualenv.py
Differential Revision: https://phabricator.services.mozilla.com/D67592
--HG--
extra : moz-landing-system : lando
Might as well automate what we can automate. I considered making the
"update package.json" logic part of setup_helper.py, so that it can be
used more broadly, but it's not clear it would be used with the
general push towards vendoring into the tree as part of `mach vendor`.
Depends on D63370
Differential Revision: https://phabricator.services.mozilla.com/D63371
--HG--
extra : moz-landing-system : lando
This patch upgrades the major browsertime version used in-tree from 4 to 8 (including some additional fixes to fix some failing tests on our end).
We also add the node v10 requirement in this patch. Also, there were some changes in the browsertime repo's visualmetrics.py script that made it necessary to change where we find the file.
Differential Revision: https://phabricator.services.mozilla.com/D59235
--HG--
extra : moz-landing-system : lando
Teach `mach browsertime` to tell you to `mach browsertime --setup` if we're not already set up
Differential Revision: https://phabricator.services.mozilla.com/D59501
--HG--
extra : moz-landing-system : lando
Let's stop if the packages could not be installed.
Also, let's skip the call if we find them.
Differential Revision: https://phabricator.services.mozilla.com/D59690
--HG--
extra : moz-landing-system : lando
Use the binary from MOZ_OBJDIR as the default when invoking mach browsertime.
The options --firefox.binaryPath, --firefox.release, --firefox.beta, --firefox.nightly, or -b chrome, will override this default.
Differential Revision: https://phabricator.services.mozilla.com/D34382
--HG--
extra : moz-landing-system : lando
This patch fixes local `./mach browsertime` issues on Windows.
First issue that is fixed is that the archive is extracted into the browsertime folder instead of a separate folder for imagemagick within the browsertime folder. Now, it is extracted into it's own folder whose name depends on the `fetches` path entry. Second issue fixed is that the environment was being contaminated with bad strings because of the `GECKODRIVER_BASE_URL` addition - using `str` on the entries solves this issue.
Finally, convert and compare were not passing in the visualmetrics.py check call. This is because ImageMagick expects to find the path to them within `ProgramFiles` on windows. Since we don't have them there, we have to add an entry in the `PATH` environment variable to point to the imagemagick directory to get around this.
There is also an issue with the macosx imagemagick directory which was being extracted as 7.0.9, but we expected 7.0.8. The imagemagick version used is modified to the correct version with the expected directory.
Differential Revision: https://phabricator.services.mozilla.com/D52751
--HG--
extra : moz-landing-system : lando
This updates our in-tree Browsertime version to support the async composition
recorder API introduced in bug 1581240.
Differential Revision: https://phabricator.services.mozilla.com/D52717
--HG--
extra : moz-landing-system : lando
This patch fixes some issues with binaries not being executable after being extracted. It also fixes the ImageMagick expected extraction path to the correct location. To prevent that from occuring again, a check is done after extraction to be sure that the extraction completed successfully.
Differential Revision: https://phabricator.services.mozilla.com/D51300
--HG--
extra : moz-landing-system : lando
This patch updgrades the current version used in-tree so that browsertime works with chrome, it currently timesout with that browser.
Differential Revision: https://phabricator.services.mozilla.com/D50999
--HG--
extra : moz-landing-system : lando
In automation, we install `ffmpeg` as part of `mach browsertime
--setup` in the browsertime toolchain task. Those tasks run on Linux
64 from within AWS, and most of the hosts we hit (intermittently) deny
AWS traffic. Let's just use github.com in automation (and locally),
for all platforms, which will agree with upcoming fetch tasks.
Differential Revision: https://phabricator.services.mozilla.com/D43656
--HG--
extra : moz-landing-system : lando
This will allow mozharness tooling, which does not run through `mach`,
to fish these paths.
Differential Revision: https://phabricator.services.mozilla.com/D38775
--HG--
extra : moz-landing-system : lando
In browsertime.zip we should have:
browsertime/
package.json
package-lock.json
node_modules/
.bin/
browsertime -> ../browsertime/bin/browsertime.js
browsertime/
...
The idea is that we'll fetch browsertime.zip in a generic-worker
environment and be able to run Node.js from within the top level
browsertime/ directory.
Differential Revision: https://phabricator.services.mozilla.com/D38773
--HG--
extra : moz-landing-system : lando
browsertime depends on a few architecture and OS specific packages:
- sharp (libvips)
- geckodriver
- chromedriver
Our toolchain task packages up `tools/browsertime/node_modules` and
we'd like to use the resulting toolchain archive across all of our
test platforms. Since in automation we don't require sharp (which is
only used for screenshotting), and we provide `geckodriver` and
`chromedriver` at the task level, the simplest way is to make these
`optionalDependencies` at the NPM level and not install them in our
toolchain task.
Differential Revision: https://phabricator.services.mozilla.com/D38772
--HG--
extra : moz-landing-system : lando
Update the browsertime snapshot to 4989d0c22bba3a165078b8d784e8d303a727a119 which uses lodash 4.7.14 and lodash.merge 4.6.2.
Differential Revision: https://phabricator.services.mozilla.com/D37806
--HG--
extra : moz-landing-system : lando
Gregory Mierzwinski
Brindusan Cristian
Sean Feng
Ricky Stewart
Bogdan Tara
Ben Hearsum
Dorel Luca
Andrew Halberstadt
Sylvestre Ledru
Tooru Fujisawa
Myeongjun Go
Tarek Ziadé
Nick Alexander
Barret Rennie
Stephen Donner
Denis Palmeiro