Bug 1762837 - [devtols] Update DAMP docs to suggest --suite instead of --activeTests r=nchevobbe

Differential Revision: https://phabricator.services.mozilla.com/D143981

devtools/docs/contributor/tests/performance-tests-damp.md

@@ -5,21 +5,22 @@ DAMP (DevTools At Maximum Performance) is our test suite to track performance.
 ## How to run it locally?
 
 ```bash
-./mach talos-test --activeTests damp
+./mach talos-test --suite damp
 ```
 Note that the first run is slower as it pulls a large tarball with various website copies.
 This will run all DAMP tests, you can filter by test name with:
 ```bash
-./mach talos-test --activeTests damp --subtests console
+./mach talos-test --suite damp --subtests console
 ```
 This command will run all tests which contains "console" in their name.
 
+Note that in continuous integration, DAMP tests are split in smaller tests suites: `damp-inspector`, `damp-other` and `damp-webconsole`. Actually `--suite damp` is only used locally because it contains all possible tests and makes it easier to use. But if needed you can substitute `damp` with any of the other test suites if you want to only run tests associated to a given test suite. You can find the mapping between tests and test suites in [damp-tests.js](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/damp-tests.js).
 ### Command line options
 
 #### Running tests only once
 
 ```bash
-./mach talos-test --activeTests damp --cycles 1 --tppagecycles 1
+./mach talos-test --suite damp --cycles 1 --tppagecycles 1
 ```
 `--cycles` will limit the number of Firefox restart to only one, while
 `--tppagecycles` will limit the number of test re-run in each firefox start to one.
@@ -28,7 +29,7 @@ This is often helpful when debugging one particular subtest.
 #### Taking screenshots
 
 ```bash
-DEBUG_DEVTOOLS_SCREENSHOTS=1 ./mach talos-test --activeTests damp
+DEBUG_DEVTOOLS_SCREENSHOTS=1 ./mach talos-test --suite damp
 ```
 When passing `DEBUG_DEVTOOLS_SCREENSHOTS` env variable, screenshots will be taken after each subtest
 was run. The screenshot will be opened in new tabs and their title
@@ -37,12 +38,12 @@ includes the subtest label. Firefox won't automatically close so that you can vi
 #### Recording a profile
 
 ```bash
-./mach talos-test --activeTests damp --gecko-profile --gecko-profile-entries 100000000
+./mach talos-test --suite damp --gecko-profile --gecko-profile-entries 100000000
 ```
 This will automatically record the tests and open the profile. You may use the following command in order
 to focus on just one subtest run:
 ```bash
-./mach talos-test --activeTests damp --subtests custom.webconsole --cycles 1 --tppagecycles 1 --gecko-profile --gecko-profile-entries 100000000
+./mach talos-test --suite damp --subtests custom.webconsole --cycles 1 --tppagecycles 1 --gecko-profile --gecko-profile-entries 100000000
 ```
 
 ## How to run it on try?

devtools/docs/contributor/tests/writing-perf-tests-example.md

@@ -60,8 +60,10 @@ Finally we add an entry in [damp-tests.js](https://searchfox.org/mozilla-central
   },
 ```
 
+Since this is an inspector test, we add it under `TEST_SUITES.INSPECTOR`, which contains all the tests which will run with the `damp-inspector` test suite in continuous integration. The test is still part of the overall `damp` suite by default, there is no action needed to ensure that.
+
 Then we can run our test with:
 ```
-./mach talos-test --activeTests damp --subtest inspector.click
+./mach talos-test --suite damp --subtest inspector.click
 ```
 

devtools/docs/contributor/tests/writing-perf-tests.md

@@ -113,24 +113,26 @@ For the example, it would be `inspector.click`.
 
 In general, the test name should try to match the path of the test file. As you can see in damp-tests.js this naming convention is not consistently followed. We have discrepancies for simple/complicated/custom tests, as well as for webconsole tests. This is largely for historical reasons.
 
+You will see that tests are split across different subsuites: damp-inspector, damp-other and damp-webconsole. The goal of this split is to run DAMP tests in parallel in CI, so we aim to keep them balanced in terms of number of tests, and mostly running time. Add your test in the suite which makes the most sense. We can add more suites and rearrange tests in the future if needed.
+
 
 # How to run your new test?
 
 You can run any performance test with this command:
 ```
-./mach talos-test --activeTests damp --subtest ${your-test-name}
+./mach talos-test --suite damp --subtest ${your-test-name}
 ```
 
 By default, it will run the test 25 times. In order to run it just once, do:
 ```
-./mach talos-test --activeTests damp --subtest ${your-test-name} --cycles 1 --tppagecycles 1
+./mach talos-test --suite damp --subtest ${your-test-name} --cycles 1 --tppagecycles 1
 ```
 `--cycles` controls the number of times Firefox is restarted
 `--tppagecycles` defines the number of times we repeat the test after each Firefox start
 
 Also, you can record a profile while running the test. To do that, execute:
 ```
-./mach talos-test --activeTests damp --subtest ${your-test-name} --cycles 1 --tppagecycles 1 --gecko-profile --gecko-profile-entries 100000000
+./mach talos-test --suite damp --subtest ${your-test-name} --cycles 1 --tppagecycles 1 --gecko-profile --gecko-profile-entries 100000000
 ```
 `--gecko-profile` enables the profiler
 `--gecko-profile-entries` defines the profiler buffer size, which needs to be large while recording performance tests

testing/talos/talos/tests/devtools/addon/content/damp-tests.js

@@ -7,6 +7,8 @@
 const Services = require("Services");
 const isWindows = Services.appinfo.OS === "WINNT";
 
+// DAMP is split in sub-suites to run the tests faster on continuous integration.
+// See the initial patches in Bug 1749928 if we need to add more suites.
 const TEST_SUITES = {
   INSPECTOR: "inspector",
   WEBCONSOLE: "webconsole",
@@ -14,8 +16,10 @@ const TEST_SUITES = {
 };
 
 /**
- * This is the registry for all DAMP tests. Tests will be run in the order specified by
- * the DAMP_TESTS array.
+ * This is the registry for all DAMP tests. The registry is an object containing
+ * one property for each DAMP sub-suite used in continuous integration. And each
+ * property contains the array of tests which correspond to this suite.
+ * Tests will be run in the order specified by the array.
  *
  * A test is defined with the following properties:
  * - {String} name: the name of the test (should match the path when possible)