When a package is installed with "pip --user", the import works right after the virtualenv activation.
Even if we force its install via mozbuild virtualenv it will fail. This patch checks if the installed package is **inside** or
**outside** the virtualenv. If it's outside, it forces its installation **inside**.
Notice that we might want to move that logic in mozbuild/virtualenv.py
Differential Revision: https://phabricator.services.mozilla.com/D67592
--HG--
extra : moz-landing-system : lando
Might as well automate what we can automate. I considered making the
"update package.json" logic part of setup_helper.py, so that it can be
used more broadly, but it's not clear it would be used with the
general push towards vendoring into the tree as part of `mach vendor`.
Depends on D63370
Differential Revision: https://phabricator.services.mozilla.com/D63371
--HG--
extra : moz-landing-system : lando
This patch upgrades the major browsertime version used in-tree from 4 to 8 (including some additional fixes to fix some failing tests on our end).
We also add the node v10 requirement in this patch. Also, there were some changes in the browsertime repo's visualmetrics.py script that made it necessary to change where we find the file.
Differential Revision: https://phabricator.services.mozilla.com/D59235
--HG--
extra : moz-landing-system : lando
Teach `mach browsertime` to tell you to `mach browsertime --setup` if we're not already set up
Differential Revision: https://phabricator.services.mozilla.com/D59501
--HG--
extra : moz-landing-system : lando
Let's stop if the packages could not be installed.
Also, let's skip the call if we find them.
Differential Revision: https://phabricator.services.mozilla.com/D59690
--HG--
extra : moz-landing-system : lando
Use the binary from MOZ_OBJDIR as the default when invoking mach browsertime.
The options --firefox.binaryPath, --firefox.release, --firefox.beta, --firefox.nightly, or -b chrome, will override this default.
Differential Revision: https://phabricator.services.mozilla.com/D34382
--HG--
extra : moz-landing-system : lando
This patch fixes local `./mach browsertime` issues on Windows.
First issue that is fixed is that the archive is extracted into the browsertime folder instead of a separate folder for imagemagick within the browsertime folder. Now, it is extracted into it's own folder whose name depends on the `fetches` path entry. Second issue fixed is that the environment was being contaminated with bad strings because of the `GECKODRIVER_BASE_URL` addition - using `str` on the entries solves this issue.
Finally, convert and compare were not passing in the visualmetrics.py check call. This is because ImageMagick expects to find the path to them within `ProgramFiles` on windows. Since we don't have them there, we have to add an entry in the `PATH` environment variable to point to the imagemagick directory to get around this.
There is also an issue with the macosx imagemagick directory which was being extracted as 7.0.9, but we expected 7.0.8. The imagemagick version used is modified to the correct version with the expected directory.
Differential Revision: https://phabricator.services.mozilla.com/D52751
--HG--
extra : moz-landing-system : lando
This updates our in-tree Browsertime version to support the async composition
recorder API introduced in bug 1581240.
Differential Revision: https://phabricator.services.mozilla.com/D52717
--HG--
extra : moz-landing-system : lando
This patch fixes some issues with binaries not being executable after being extracted. It also fixes the ImageMagick expected extraction path to the correct location. To prevent that from occuring again, a check is done after extraction to be sure that the extraction completed successfully.
Differential Revision: https://phabricator.services.mozilla.com/D51300
--HG--
extra : moz-landing-system : lando
This patch updgrades the current version used in-tree so that browsertime works with chrome, it currently timesout with that browser.
Differential Revision: https://phabricator.services.mozilla.com/D50999
--HG--
extra : moz-landing-system : lando
In automation, we install `ffmpeg` as part of `mach browsertime
--setup` in the browsertime toolchain task. Those tasks run on Linux
64 from within AWS, and most of the hosts we hit (intermittently) deny
AWS traffic. Let's just use github.com in automation (and locally),
for all platforms, which will agree with upcoming fetch tasks.
Differential Revision: https://phabricator.services.mozilla.com/D43656
--HG--
extra : moz-landing-system : lando
This will allow mozharness tooling, which does not run through `mach`,
to fish these paths.
Differential Revision: https://phabricator.services.mozilla.com/D38775
--HG--
extra : moz-landing-system : lando
In browsertime.zip we should have:
browsertime/
package.json
package-lock.json
node_modules/
.bin/
browsertime -> ../browsertime/bin/browsertime.js
browsertime/
...
The idea is that we'll fetch browsertime.zip in a generic-worker
environment and be able to run Node.js from within the top level
browsertime/ directory.
Differential Revision: https://phabricator.services.mozilla.com/D38773
--HG--
extra : moz-landing-system : lando
browsertime depends on a few architecture and OS specific packages:
- sharp (libvips)
- geckodriver
- chromedriver
Our toolchain task packages up `tools/browsertime/node_modules` and
we'd like to use the resulting toolchain archive across all of our
test platforms. Since in automation we don't require sharp (which is
only used for screenshotting), and we provide `geckodriver` and
`chromedriver` at the task level, the simplest way is to make these
`optionalDependencies` at the NPM level and not install them in our
toolchain task.
Differential Revision: https://phabricator.services.mozilla.com/D38772
--HG--
extra : moz-landing-system : lando
Update the browsertime snapshot to 4989d0c22bba3a165078b8d784e8d303a727a119 which uses lodash 4.7.14 and lodash.merge 4.6.2.
Differential Revision: https://phabricator.services.mozilla.com/D37806
--HG--
extra : moz-landing-system : lando
If `./mach browsertime` runs browsertime with a globally-installed node, due to
an existing bug in [execa][1], the wrong Python will be executed. We now
specify the full path of the Python binary we wish to use (via the `PYTHON`
environment variable that our fork of browsertime supports) and avoid this
issue altogether.
[1]: https://github.com/sindresorhus/execa/issues/153
Differential Revision: https://phabricator.services.mozilla.com/D35374
--HG--
extra : moz-landing-system : lando
Changes include a fix for visual metrics calculations on desktop, and appending visual metrics information to gecko profiles.
Additionally, add the browsertime-results directory to gitignore.
Differential Revision: https://phabricator.services.mozilla.com/D34412
--HG--
extra : moz-landing-system : lando
Flip set_log_level to false by default so we don't clutter stdout. Flip on with --verbose.
Differential Revision: https://phabricator.services.mozilla.com/D34391
--HG--
extra : moz-landing-system : lando
Key features in this commit:
- support for `contentfulSpeedIndex` visual metric
- support for the Gecko Window Recorder (Desktop only)
- support for the Gecko Profiler (Desktop only)
- partial support for GeckoView on Android
Differential Revision: https://phabricator.services.mozilla.com/D32907
--HG--
extra : moz-landing-system : lando
Under the hood, browsertime invokes a certain `visualmetrics.py`
script. That script depends on `ffmpeg` and ImageMagick's `convert`,
`compare`, and `mogrify` commands. It also depends on certain Python
packages.
So this installs those dependencies, and then wires up the evaluation
environment such that `./mach browsertime` can find the dependencies.
It also adds a `./mach visualmetrics` command for processing a
captured MP4 file in the same way that browsertime processes such a
file.
In order to avoid downloading dependencies multiple time, the existing
artifact cache is extracted. This is a small first step towards [Bug
1526021](https://bugzilla.mozilla.org/show_bug.cgi?id=1526021), which
might want to use this artifact cache as well.
At this time, hashes and filesizes are not verified. During
development, the upstream files changed multiple times, and it's not
worth being completely locked down while experimenting with this
functionality. If we start running this code in automation or in more
sensitive environments, we can build fetch tasks and TC indexes to
streamline the artifact gathering process.
It is expected that a future mach command will want to invoke
browsertime without suffering the overhead of invoking Python (and
mach, which is itself bulky) so a nod is given to exposing the
relevant environment pieces.
During testing, it was discovered that [MozillaBuild doesn't ship
git](https://bugzilla.mozilla.org/show_bug.cgi?id=1503028), so that
git repositories can't be used out-of-the-box on Windows. So instead
we use a [tarball link from github.com/$USER/$REPO/tarball/$COMMIT-LIKE](https://github.blog/2008-03-03-tarball-downloads/).
Differential Revision: https://phabricator.services.mozilla.com/D29442
--HG--
rename : python/mozbuild/mozbuild/test/test_artifacts.py => python/mozbuild/mozbuild/test/test_artifact_cache.py
extra : moz-landing-system : lando
[browsertime](https://github.com/sitespeedio/browsertime) is a harness
for running performance tests, similar to Mozilla's Raptor testing
framework. The Performance Team is using it locally with some
success, but we're running a heavily modified toolchain that is
challenging to install. This mach command is intended to be leverage
for getting more folks able to use browsertime easily.
In particular, the version of browsertime that this installs has
nalexander's changes to support testing GeckoView-based vehicles. If
this approach meets with approval, I'll continue to follow-up with
additional configuration and tooling layers to make it even easier to
drive GeckoView-based vehicles.
I elected to piggy-back install on the eslint installation process,
since this is very similar. To that end, I generalized what was there
very slightly. I elected not to try to move the existing code into a
more obvious shared location, although it might be possible, because
it wasn't clear what contexts the existing code would be invoked
from. In particular I wasn't certain the code could rely on a
complete mozbuild checkout.
I did need to ensure the local Node.js binary is early on the PATH;
this was an issue I ran into with my initial Node/Yarn prototyping
many months ago. At heart the issue is that package scripts in the
wild invoke a bare `node` or `npm` command; if there was a culture of
invoking $NODE or $NPM, this wouldn't be necessary. There's no harm
doing it for ESlint, and it will help the next person who wants to
install an NPM package for tooling in this manner.
Differential Revision: https://phabricator.services.mozilla.com/D26820
--HG--
extra : moz-landing-system : lando
Under the hood, browsertime invokes a certain `visualmetrics.py`
script. That script depends on `ffmpeg` and ImageMagick's `convert`,
`compare`, and `mogrify` commands. It also depends on certain Python
packages.
So this installs those dependencies, and then wires up the evaluation
environment such that `./mach browsertime` can find the dependencies.
It also adds a `./mach visualmetrics` command for processing a
captured MP4 file in the same way that browsertime processes such a
file.
In order to avoid downloading dependencies multiple time, the existing
artifact cache is extracted. This is a small first step towards [Bug
1526021](https://bugzilla.mozilla.org/show_bug.cgi?id=1526021), which
might want to use this artifact cache as well.
At this time, hashes and filesizes are not verified. During
development, the upstream files changed multiple times, and it's not
worth being completely locked down while experimenting with this
functionality. If we start running this code in automation or in more
sensitive environments, we can build fetch tasks and TC indexes to
streamline the artifact gathering process.
It is expected that a future mach command will want to invoke
browsertime without suffering the overhead of invoking Python (and
mach, which is itself bulky) so a nod is given to exposing the
relevant environment pieces.
During testing, it was discovered that [MozillaBuild doesn't ship
git](https://bugzilla.mozilla.org/show_bug.cgi?id=1503028), so that
git repositories can't be used out-of-the-box on Windows. So instead
we use a [tarball link from github.com/$USER/$REPO/tarball/$COMMIT-LIKE](https://github.blog/2008-03-03-tarball-downloads/).
Differential Revision: https://phabricator.services.mozilla.com/D29442
--HG--
extra : moz-landing-system : lando
[browsertime](https://github.com/sitespeedio/browsertime) is a harness
for running performance tests, similar to Mozilla's Raptor testing
framework. The Performance Team is using it locally with some
success, but we're running a heavily modified toolchain that is
challenging to install. This mach command is intended to be leverage
for getting more folks able to use browsertime easily.
In particular, the version of browsertime that this installs has
nalexander's changes to support testing GeckoView-based vehicles. If
this approach meets with approval, I'll continue to follow-up with
additional configuration and tooling layers to make it even easier to
drive GeckoView-based vehicles.
I elected to piggy-back install on the eslint installation process,
since this is very similar. To that end, I generalized what was there
very slightly. I elected not to try to move the existing code into a
more obvious shared location, although it might be possible, because
it wasn't clear what contexts the existing code would be invoked
from. In particular I wasn't certain the code could rely on a
complete mozbuild checkout.
I did need to ensure the local Node.js binary is early on the PATH;
this was an issue I ran into with my initial Node/Yarn prototyping
many months ago. At heart the issue is that package scripts in the
wild invoke a bare `node` or `npm` command; if there was a culture of
invoking $NODE or $NPM, this wouldn't be necessary. There's no harm
doing it for ESlint, and it will help the next person who wants to
install an NPM package for tooling in this manner.
Differential Revision: https://phabricator.services.mozilla.com/D26820
--HG--
extra : moz-landing-system : lando
Tooru Fujisawa
Myeongjun Go
Gregory Mierzwinski
Tarek Ziadé
Nick Alexander
Barret Rennie
Stephen Donner
Denis Palmeiro
Andrew Halberstadt
Sean Feng
Razvan Maries