Bug 1003417 - Add ability to run subsuites to |mach python-test|, r=ted

This adds the ability to use manifestparser subsuites to |mach python-test|.
Subsuites are based on the premise of a "default" set that gets run when no
subsuites are explicitly specified. When a test is labelled with a subsuite,
that test is removed from the default set and will only run if that subsuite
is explicitly specified. This will allow us to chunk python unittests out of
'make check' piecemeal. The default set will run in 'make check', and
individual tasks (e.g mozbase), will specify a subsuite explicitly.

The |mach python-test| implementation is slightly different. By default,
subsuites are not considered if developers do not pass in --subsuite. This
means running |mach python-test| without arguments will still run the full set
of tests, and similarly, passing in test paths will *just work*.

If for some reason a developer needs to actually run the default set, a special
"default" subsuite has been create, so they can use
|mach python-test --subsuite default|. This default subsuite is also what 'make
check' will explicitly invoke.

MozReview-Commit-ID: FaHb4nvuoK9

--HG--
extra : rebase_source : 2ecbc902bb6bafa7cd78ac0e380a10bad7c14351

python/mach_commands.py

@@ -17,6 +17,7 @@ from concurrent.futures import (
 
 import mozinfo
 from manifestparser import TestManifest
+from manifestparser import filters as mpf
 
 from mozbuild.base import (
     MachCommandBase,
@@ -65,6 +66,10 @@ class MachCommands(MachCommandBase):
         default=1,
         type=int,
         help='Number of concurrent jobs to run. Default is 1.')
+    @CommandArgument('--subsuite',
+        default=None,
+        help=('Python subsuite to run. If not specified, all subsuites are run. '
+             'Use the string `default` to only run tests without a subsuite.'))
     @CommandArgument('tests', nargs='*',
         metavar='TEST',
         help=('Tests to run. Each test can be a single file or a directory. '
@@ -133,7 +138,14 @@ class MachCommands(MachCommandBase):
 
         mp = TestManifest()
         mp.tests.extend(test_objects)
-        tests = mp.active_tests(disabled=False, **mozinfo.info)
+
+        filters = []
+        if subsuite == 'default':
+            filters.append(mpf.subsuite(None))
+        elif subsuite:
+            filters.append(mpf.subsuite(subsuite))
+
+        tests = mp.active_tests(filters=filters, disabled=False, **mozinfo.info)
 
         self.jobs = jobs
         self.terminate = False