gecko-dev/testing/runtimes
Andrew Halberstadt b483d6326c Bug 1583364 - Generate 'manifest-runtimes.json' and update mochitest harness to use it, r=gbrown
As a side-effect this will also update runtime data for all suites using
'--chunk-by-runtime'.

This change simultaneously:

1. Stores runtime data from all suites
2. Stores runtime data from all tests (no more percentile)
3. Stores distinct data for android, unix (osx/linux) and windows
4. Reduces the size of 'testing/runtimes' from 408k -> 168k

The chunks look more balanced from my unscientific glance (especially on Windows).

Differential Revision: https://phabricator.services.mozilla.com/D53702

--HG--
extra : moz-landing-system : lando
2019-11-22 19:23:17 +00:00
..
README.md Bug 1583364 - Update testing/runtimes/writeruntimes script to write info at the manifest level, r=gbrown 2019-11-20 21:21:17 +00:00
manifest-runtimes-android.json Bug 1583364 - Generate 'manifest-runtimes.json' and update mochitest harness to use it, r=gbrown 2019-11-22 19:23:17 +00:00
manifest-runtimes-unix.json Bug 1583364 - Generate 'manifest-runtimes.json' and update mochitest harness to use it, r=gbrown 2019-11-22 19:23:17 +00:00
manifest-runtimes-windows.json Bug 1583364 - Generate 'manifest-runtimes.json' and update mochitest harness to use it, r=gbrown 2019-11-22 19:23:17 +00:00
moz.build Bug 1583364 - Generate 'manifest-runtimes.json' and update mochitest harness to use it, r=gbrown 2019-11-22 19:23:17 +00:00
writeruntimes Bug 1583364 - Update testing/runtimes/writeruntimes script to write info at the manifest level, r=gbrown 2019-11-20 21:21:17 +00:00

README.md

Test Runtimes

These files contain runtimes for test manifests in the tree. They are of the form:

{ '<path to manifest>': <average runtime in seconds> }

They are being used to normalize chunk durations so all chunks take roughly the same length of time.

Generating a Test Runtime File

The writeruntimes script can be used to generate this file:

$ ./writeruntimes

It will take awhile. You can optionally specify platforms or suites on the command line, but these should only be used for debugging purposes (not for committing an update to the data). For more info, see:

$ ./writeruntimes -- --help