A quick and basic implementation of 'mach test-info report-diff' to provide a
high level comparison of two reports generated by 'mach test-info report'.
Mostly this is trying to give an impression of how many and which tests
have been added and deleted during the time period between the reports.
There is more information in the reports; a more thorough comparison is
possible, but seems lower-value: To be considered in a future bug.
Differential Revision: https://phabricator.services.mozilla.com/D65202
--HG--
extra : moz-landing-system : lando
Minor changes to test-info report's interaction with ActiveData:
- with improved reftest data in ActiveData, several path modifications can
be removed;
- adjust some crashtest query data ranges to more efficiently handle the
new data;
- add a missing path modification for xpcshell, to handle test names that
include manifests.
Differential Revision: https://phabricator.services.mozilla.com/D60551
--HG--
extra : moz-landing-system : lando
When an ActiveData query returns HTTP 200 but no data, treat that as a permanent condition:
Do not retry. I expect that will be the case for some test-info queries for reftests
until reftest test names are normalized.
Differential Revision: https://phabricator.services.mozilla.com/D57892
--HG--
extra : moz-landing-system : lando
Currently 'mach test-info report --show-activedata' makes over a dozen requests to
ActiveData, one at a time. Most of the elapsed time for generating the report is
waiting for a response from ActiveData. A significant run-time savings is achieved
using threads to submit multiple requests concurrently.
Differential Revision: https://phabricator.services.mozilla.com/D57041
--HG--
extra : moz-landing-system : lando
Adjust ActiveData queries so that no query matches more than the ActiveData maximum
of 10000 records (crashtest excepted, until bug 1596567 is fixed). This provides a
dramatic improvement in the quality of run counts and run times reported for wpt and
reftests. Also adjusts some of the update mechanisms to handle duplicate paths better.
Also adds more (verbose) logging of match counts, report run times, etc in anticipation
of further refinements.
Differential Revision: https://phabricator.services.mozilla.com/D56616
--HG--
extra : moz-landing-system : lando
Move almost all test-info code out of mach_commands.py and into its own file
and consolidate logging and imports; no functional changes.
Differential Revision: https://phabricator.services.mozilla.com/D56064
--HG--
extra : moz-landing-system : lando
Geoff Brown
James Graham