Граф коммитов

20 Коммитов

Автор SHA1 Сообщение Дата
Nick Alexander f2e38650f0 Bug 1566171 - Part 1: Expose browsertime helpers to Raptor harness. r=barret
This will allow mozharness tooling, which does not run through `mach`,
to fish these paths.

Differential Revision: https://phabricator.services.mozilla.com/D38775

--HG--
extra : moz-landing-system : lando
2019-07-26 21:29:57 +00:00
Nick Alexander 280126dff5 Bug 1564256 - Part 2: Produce browsertime.zip in a toolchain task. r=mshal
In browsertime.zip we should have:

browsertime/
  package.json
  package-lock.json
  node_modules/
    .bin/
      browsertime -> ../browsertime/bin/browsertime.js
    browsertime/
      ...

The idea is that we'll fetch browsertime.zip in a generic-worker
environment and be able to run Node.js from within the top level
browsertime/ directory.

Differential Revision: https://phabricator.services.mozilla.com/D38773

--HG--
extra : moz-landing-system : lando
2019-07-24 20:59:55 +00:00
Nick Alexander ceec08a80d Bug 1564256 - Part 1: Don't install optional browsertime packages in automation. r=barret
browsertime depends on a few architecture and OS specific packages:
- sharp (libvips)
- geckodriver
- chromedriver

Our toolchain task packages up `tools/browsertime/node_modules` and
we'd like to use the resulting toolchain archive across all of our
test platforms.  Since in automation we don't require sharp (which is
only used for screenshotting), and we provide `geckodriver` and
`chromedriver` at the task level, the simplest way is to make these
`optionalDependencies` at the NPM level and not install them in our
toolchain task.

Differential Revision: https://phabricator.services.mozilla.com/D38772

--HG--
extra : moz-landing-system : lando
2019-07-24 20:58:39 +00:00
Denis Palmeiro 25e544d4a8 Bug 1565399 - Update github tarball to 4989d0c22bba3a165078b8d784e8d303a727a119 r=nalexander
Update the browsertime snapshot to 4989d0c22bba3a165078b8d784e8d303a727a119 which uses lodash 4.7.14 and lodash.merge 4.6.2.

Differential Revision: https://phabricator.services.mozilla.com/D37806

--HG--
extra : moz-landing-system : lando
2019-07-16 15:09:17 +00:00
Andrew Halberstadt 3486ba642c Bug 1563797 - Use 'backports.shutil_which' instead of 'which' across the tree r=Callek
Differential Revision: https://phabricator.services.mozilla.com/D37097

--HG--
extra : moz-landing-system : lando
2019-07-11 14:03:39 +00:00
Sean Feng aed178a11b Bug 1563228 - Bump browsertime target hash r=nalexander
Differential Revision: https://phabricator.services.mozilla.com/D36758

--HG--
extra : moz-landing-system : lando
2019-07-03 16:01:52 +00:00
Barret Rennie f1d04044bd Bug 1560193 - Always specify the Python executable to use when running `./mach browsertime` r=nalexander
If `./mach browsertime` runs browsertime with a globally-installed node, due to
an existing bug in [execa][1], the wrong Python will be executed. We now
specify the full path of the Python binary we wish to use (via the `PYTHON`
environment variable that our fork of browsertime supports) and avoid this
issue altogether.

[1]: https://github.com/sindresorhus/execa/issues/153

Differential Revision: https://phabricator.services.mozilla.com/D35374

--HG--
extra : moz-landing-system : lando
2019-07-02 17:29:33 +00:00
Nick Alexander 86801c417d Bug 1556567 - Use custom `geckodriver` with pre-release Android support in `mach browsertime`. r=sefeng
Differential Revision: https://phabricator.services.mozilla.com/D33574

--HG--
extra : moz-landing-system : lando
2019-06-18 22:44:52 +00:00
Denis Palmeiro 7b9e1b6ba5 Bug 1558271 - Update mozilla/browsertime snapshot to b8c1becaee74970a6f6e4222a64d1e2e18f20cd6 r=nalexander
Changes include a fix for visual metrics calculations on desktop, and appending visual metrics information to gecko profiles.

Additionally, add the browsertime-results directory to gitignore.

Differential Revision: https://phabricator.services.mozilla.com/D34412

--HG--
extra : moz-landing-system : lando
2019-06-10 21:03:22 +00:00
Denis Palmeiro 2c01bcf632 Bug 1558271 - Set verbose to false by default in ./mach browsertime r=nalexander
Flip set_log_level to false by default so we don't clutter stdout.  Flip on with --verbose.

Differential Revision: https://phabricator.services.mozilla.com/D34391

--HG--
extra : moz-landing-system : lando
2019-06-10 19:30:00 +00:00
Nick Alexander 9366390a2b Bug 1555158 - Part 4: Default to `-b firefox --skipHar` in `mach browsertime`. r=dpalmeiro
Differential Revision: https://phabricator.services.mozilla.com/D33326

--HG--
extra : moz-landing-system : lando
2019-06-01 02:23:46 +00:00
Nick Alexander 2c85bd96b3 Bug 1555158 - Part 3: Make `mach visualmetrics` calculate contentfulSpeedIndex. r=dpalmeiro
Differential Revision: https://phabricator.services.mozilla.com/D33325

--HG--
extra : moz-landing-system : lando
2019-06-01 02:23:37 +00:00
Nick Alexander 7666bb7d95 Bug 1555158 - Part 2: Accept system ImageMagick on Linux. r=dpalmeiro
Differential Revision: https://phabricator.services.mozilla.com/D32923

--HG--
extra : moz-landing-system : lando
2019-06-01 02:23:22 +00:00
Nick Alexander ae0bdfac53 Bug 1555158 - Part 1: Point `mach browsertime --setup` at mozilla/browsertime github repository. r=dpalmeiro
Key features in this commit:

- support for `contentfulSpeedIndex` visual metric
- support for the Gecko Window Recorder (Desktop only)
- support for the Gecko Profiler (Desktop only)
- partial support for GeckoView on Android

Differential Revision: https://phabricator.services.mozilla.com/D32907

--HG--
extra : moz-landing-system : lando
2019-06-01 02:23:09 +00:00
Nick Alexander 3df6d400a1 Bug 1549781 - Ensure browsertime state path is created before use. r=dpalmeiro
Differential Revision: https://phabricator.services.mozilla.com/D32906

--HG--
extra : moz-landing-system : lando
2019-06-01 02:22:55 +00:00
Nick Alexander 730a0a46f2 Bug 1543247 - Part 2: Install `visualmetrics.py` prerequisites. r=ahal
Under the hood, browsertime invokes a certain `visualmetrics.py`
script.  That script depends on `ffmpeg` and ImageMagick's `convert`,
`compare`, and `mogrify` commands.  It also depends on certain Python
packages.

So this installs those dependencies, and then wires up the evaluation
environment such that `./mach browsertime` can find the dependencies.
It also adds a `./mach visualmetrics` command for processing a
captured MP4 file in the same way that browsertime processes such a
file.

In order to avoid downloading dependencies multiple time, the existing
artifact cache is extracted.  This is a small first step towards [Bug
1526021](https://bugzilla.mozilla.org/show_bug.cgi?id=1526021), which
might want to use this artifact cache as well.

At this time, hashes and filesizes are not verified.  During
development, the upstream files changed multiple times, and it's not
worth being completely locked down while experimenting with this
functionality.  If we start running this code in automation or in more
sensitive environments, we can build fetch tasks and TC indexes to
streamline the artifact gathering process.

It is expected that a future mach command will want to invoke
browsertime without suffering the overhead of invoking Python (and
mach, which is itself bulky) so a nod is given to exposing the
relevant environment pieces.

During testing, it was discovered that [MozillaBuild doesn't ship
git](https://bugzilla.mozilla.org/show_bug.cgi?id=1503028), so that
git repositories can't be used out-of-the-box on Windows.  So instead
we use a [tarball link from github.com/$USER/$REPO/tarball/$COMMIT-LIKE](https://github.blog/2008-03-03-tarball-downloads/).

Differential Revision: https://phabricator.services.mozilla.com/D29442

--HG--
rename : python/mozbuild/mozbuild/test/test_artifacts.py => python/mozbuild/mozbuild/test/test_artifact_cache.py
extra : moz-landing-system : lando
2019-05-06 23:56:59 +00:00
Nick Alexander a7e3d82701 Bug 1543247 - Part 1: Add `mach browsertime` command that installs and invokes browsertime. r=Standard8,ahal
[browsertime](https://github.com/sitespeedio/browsertime) is a harness
for running performance tests, similar to Mozilla's Raptor testing
framework.  The Performance Team is using it locally with some
success, but we're running a heavily modified toolchain that is
challenging to install.  This mach command is intended to be leverage
for getting more folks able to use browsertime easily.

In particular, the version of browsertime that this installs has
nalexander's changes to support testing GeckoView-based vehicles.  If
this approach meets with approval, I'll continue to follow-up with
additional configuration and tooling layers to make it even easier to
drive GeckoView-based vehicles.

I elected to piggy-back install on the eslint installation process,
since this is very similar.  To that end, I generalized what was there
very slightly.  I elected not to try to move the existing code into a
more obvious shared location, although it might be possible, because
it wasn't clear what contexts the existing code would be invoked
from.  In particular I wasn't certain the code could rely on a
complete mozbuild checkout.

I did need to ensure the local Node.js binary is early on the PATH;
this was an issue I ran into with my initial Node/Yarn prototyping
many months ago.  At heart the issue is that package scripts in the
wild invoke a bare `node` or `npm` command; if there was a culture of
invoking $NODE or $NPM, this wouldn't be necessary.  There's no harm
doing it for ESlint, and it will help the next person who wants to
install an NPM package for tooling in this manner.

Differential Revision: https://phabricator.services.mozilla.com/D26820

--HG--
extra : moz-landing-system : lando
2019-05-06 23:56:49 +00:00
Razvan Maries 12bcfbb334 Backed out 2 changesets (bug 1543247) for build bustages. CLOSED TREE
Backed out changeset feb726e4f15d (bug 1543247)
Backed out changeset 4b3619d89abd (bug 1543247)
2019-05-04 03:10:55 +03:00
Nick Alexander 7128d7e528 Bug 1543247 - Part 2: Install `visualmetrics.py` prerequisites. r=ahal
Under the hood, browsertime invokes a certain `visualmetrics.py`
script.  That script depends on `ffmpeg` and ImageMagick's `convert`,
`compare`, and `mogrify` commands.  It also depends on certain Python
packages.

So this installs those dependencies, and then wires up the evaluation
environment such that `./mach browsertime` can find the dependencies.
It also adds a `./mach visualmetrics` command for processing a
captured MP4 file in the same way that browsertime processes such a
file.

In order to avoid downloading dependencies multiple time, the existing
artifact cache is extracted.  This is a small first step towards [Bug
1526021](https://bugzilla.mozilla.org/show_bug.cgi?id=1526021), which
might want to use this artifact cache as well.

At this time, hashes and filesizes are not verified.  During
development, the upstream files changed multiple times, and it's not
worth being completely locked down while experimenting with this
functionality.  If we start running this code in automation or in more
sensitive environments, we can build fetch tasks and TC indexes to
streamline the artifact gathering process.

It is expected that a future mach command will want to invoke
browsertime without suffering the overhead of invoking Python (and
mach, which is itself bulky) so a nod is given to exposing the
relevant environment pieces.

During testing, it was discovered that [MozillaBuild doesn't ship
git](https://bugzilla.mozilla.org/show_bug.cgi?id=1503028), so that
git repositories can't be used out-of-the-box on Windows.  So instead
we use a [tarball link from github.com/$USER/$REPO/tarball/$COMMIT-LIKE](https://github.blog/2008-03-03-tarball-downloads/).

Differential Revision: https://phabricator.services.mozilla.com/D29442

--HG--
extra : moz-landing-system : lando
2019-05-03 22:45:22 +00:00
Nick Alexander 6df6c7ee39 Bug 1543247 - Part 1: Add `mach browsertime` command that installs and invokes browsertime. r=Standard8,ahal
[browsertime](https://github.com/sitespeedio/browsertime) is a harness
for running performance tests, similar to Mozilla's Raptor testing
framework.  The Performance Team is using it locally with some
success, but we're running a heavily modified toolchain that is
challenging to install.  This mach command is intended to be leverage
for getting more folks able to use browsertime easily.

In particular, the version of browsertime that this installs has
nalexander's changes to support testing GeckoView-based vehicles.  If
this approach meets with approval, I'll continue to follow-up with
additional configuration and tooling layers to make it even easier to
drive GeckoView-based vehicles.

I elected to piggy-back install on the eslint installation process,
since this is very similar.  To that end, I generalized what was there
very slightly.  I elected not to try to move the existing code into a
more obvious shared location, although it might be possible, because
it wasn't clear what contexts the existing code would be invoked
from.  In particular I wasn't certain the code could rely on a
complete mozbuild checkout.

I did need to ensure the local Node.js binary is early on the PATH;
this was an issue I ran into with my initial Node/Yarn prototyping
many months ago.  At heart the issue is that package scripts in the
wild invoke a bare `node` or `npm` command; if there was a culture of
invoking $NODE or $NPM, this wouldn't be necessary.  There's no harm
doing it for ESlint, and it will help the next person who wants to
install an NPM package for tooling in this manner.

Differential Revision: https://phabricator.services.mozilla.com/D26820

--HG--
extra : moz-landing-system : lando
2019-05-03 22:44:23 +00:00