gecko-dev/testing/raptor
Nick Alexander 19d52d9edc Bug 1566174 - Part 2: Add browsertime dependencies to Raptor tasks when --browsertime flag is present. r=rwood,tomprince
This commit prepares the decks for turning specific Raptor tasks into
Raptor + browsertime tasks.  The `--browsertime` flag to `mach try
...` flips the switch; eventually, the Raptor harness will recognize
the `--browsertime` flag and use browsertime to perform the pageload
measurements.

To run browsertime, we need:

1) Node.js
2) the browsertime `node_modules` (provided by the
   `toolchain-browsertime` task)
3) ffmpeg (for producing videos from captured frames)
4) chromedriver (in the future, when targeting Chrome/Chromium)
5) geckodriver (provided by the `toolchain-*-geckodriver` tasks)
6) `PATH` configured

This commit arranges those things.

Since the configuration varies by test platform, and eventually we
expect the changes implemented by the flag to be moved into YAML task
definitions, we elect to use `by-test-platform` conditionals as much
as possible.  The end expression is pleasant, thanks to
`evaluate_keyed_by`.

Handling PATH, however, is a rabbit hole.  At this time, it's not
possible to use `fetch` task repackaging, because `releng-hardware`
doesn't support `zstandard` (Bug 1576244) and there's no appetite to
avoid `zstandard` entirely (Bug 1576698).  Generally PATH is
configured using `mozharness` configuration files, which can execute
arbitrary Python and configure the PATH only for browsertime jobs.
However, the Raptor mozharness script itself runs the Raptor harness
in a stripped down environment, throwing away modifications to PATH.
It's not clear what impacts changing that has, so we leave it alone,
and add a `--browsertime-ffmpeg` flag and custom handling in the
Raptor harness.  This can transition smoothly into a browsertime flag
(so that the PATH doesn't need to be set at all) and into a unified
interface for Raptor and `mach browsertime` to configure the
browsertime execution environment.

Differential Revision: https://phabricator.services.mozilla.com/D38781

--HG--
extra : moz-landing-system : lando
2019-09-06 18:53:49 +00:00
..
constants Bug 1568461 - Run raptor-youtube-playback-geckoview jobs on integration branches with alerting enabled. r=perftest-reviewers,sparky 2019-08-21 14:22:31 +00:00
logger Bug 1565438 - Intermittent AttributeError: RaptorLogger instance has no attribute 'crash' r=perftest-reviewers,sparky 2019-07-19 18:09:53 +00:00
raptor Bug 1566174 - Part 2: Add browsertime dependencies to Raptor tasks when --browsertime flag is present. r=rwood,tomprince 2019-09-06 18:53:49 +00:00
test Bug 1563164 - Remove option to enable/disable upstream certs in raptor r=perftest-reviewers,tarek,AlexandruIonescu 2019-09-04 13:39:19 +00:00
webext/raptor Bug 1562844 - Use correct tabId in Raptor r=robwu,rwood 2019-08-15 20:02:33 +00:00
.eslintrc.js Bug 1551829 - Upgrade to ESLint 6.1.0. r=mossop 2019-08-21 19:08:44 +00:00
MANIFEST.in Bug 1455872 - Raptor fixes for production; r=ahal 2018-05-01 14:02:39 -04:00
mach_commands.py Bug 1566174 - Part 2: Add browsertime dependencies to Raptor tasks when --browsertime flag is present. r=rwood,tomprince 2019-09-06 18:53:49 +00:00
moz.build Bug 1536112 - testing/raptor misses moz.build file. r=ahal 2019-03-19 03:45:24 +00:00
requirements.txt Bug 1573940 - Integrate mozpower into raptor desktop. r=perftest-reviewers,rwood 2019-08-23 18:27:05 +00:00