docs: add documentation for test subsets (#56)

Author: FritzHeiden

tools/wave/docs/config.md

@@ -20,6 +20,13 @@ parameters and what they are used for.
    10. [Host override](#210-host-override)
    11. [Enable test type selection](#211-enable-test-type-selection)
    12. [Tests base URL](#212-tests-base-url)
+3. [Test Subsets](#3-test-subsets)
+   1. [Define a subset](#31-define-a-subset)
+   2. [Make subset selectable](#32-make-subset-selectable)
+   3. [Default subsets](#33-default-subsets)
+   4. [Tags and filters](#34-tags-and-filters)
+   5. [Default filters](#35-default-filters)
+   6. [Messages](#36-messages)
 
 ## 1. Location and structure
 
@@ -64,7 +71,8 @@ directory:
   }
 }
 ```
-[🠑 top](#configuration---wave-test-runner)
+
+[▲ top](#configuration---wave-test-runner)
 
 ## 2. Parameters
 
@@ -91,7 +99,7 @@ The results parameter sets where results and session information are stored.
 }
 ```
 
-[🠑 top](#configuration---wave-test-runner)
+[▲ top](#configuration---wave-test-runner)
 
 ### 2.2 Test Timeouts
 
@@ -123,7 +131,7 @@ The test timeouts set the default test timeout for different test types.
 }
 ```
 
-[🠑 top](#configuration---wave-test-runner)
+[▲ top](#configuration---wave-test-runner)
 
 ### 2.3 Enable import of results
 
@@ -148,7 +156,7 @@ WAVE Test Runner instances into the current one.
 }
 ```
 
-[🠑 top](#configuration---wave-test-runner)
+[▲ top](#configuration---wave-test-runner)
 
 ### 2.4 Web namespace
 
@@ -174,7 +182,7 @@ parameter.
 }
 ```
 
-[🠑 top](#configuration---wave-test-runner)
+[▲ top](#configuration---wave-test-runner)
 
 ### 2.5 Persisting interval
 
@@ -205,7 +213,7 @@ state.
 }
 ```
 
-[🠑 top](#configuration---wave-test-runner)
+[▲ top](#configuration---wave-test-runner)
 
 ### 2.6 API titles
 
@@ -256,7 +264,7 @@ a name to an API subdirectory.
 }
 ```
 
-[🠑 top](#configuration---wave-test-runner)
+[▲ top](#configuration---wave-test-runner)
 
 ### 2.7 Enable listing all sessions
 
@@ -280,7 +288,7 @@ This parameter enables the [REST API endpoint to list all available sessions](./
 }
 ```
 
-[🠑 top](#configuration---wave-test-runner)
+[▲ top](#configuration---wave-test-runner)
 
 ### 2.8 Event caching duration
 
@@ -306,12 +314,11 @@ accordingly.
 }
 ```
 
-[🠑 top](#configuration---wave-test-runner)
-
+[▲ top](#configuration---wave-test-runner)
 
 ### 2.9 Pre test control page delay
 
-This parameter specifies how long the pre-test control pages is active before 
+This parameter specifies how long the pre-test control pages is active before
 redirecting to the actual test.
 
 **Parameters**:
@@ -332,12 +339,12 @@ redirecting to the actual test.
 }
 ```
 
-[🠑 top](#configuration---wave-test-runner)
+[▲ top](#configuration---wave-test-runner)
 
 ### 2.10 Host override
 
-The `host_override` parameter overrides the hostname of the wpt runner. This 
-can be used to configure the access to the test runner using a static IP 
+The `host_override` parameter overrides the hostname of the wpt runner. This
+can be used to configure the access to the test runner using a static IP
 address. As it is just an override, a proper host file setup as described
 [here](https://web-platform-tests.org/running-tests/from-local-system.html#hosts-file-setup)
 is still required.
@@ -356,7 +363,7 @@ is still required.
 
 None
 
-[🠑 top](#configuration---wave-test-runner)
+[▲ top](#configuration---wave-test-runner)
 
 ### 2.11 Enable test type selection
 
@@ -376,7 +383,7 @@ Sets display of test type configuration UI elements.
 
 False
 
-[🠑 top](#configuration---wave-test-runner)
+[▲ top](#configuration---wave-test-runner)
 
 ### 2.12 Tests base URL
 
@@ -396,4 +403,145 @@ Sets the base URL for links to test implementation URLs in generated reports
 
 Empty String
 
-[🠑 top](#configuration---wave-test-runner)
+[▲ top](#configuration---wave-test-runner)
+
+## 3. Test Subsets
+
+The `test-subsets.json`, in the root of the test runner directory, allows to define subsets of tests. These subsets are used in the session configuration page to quickly select these subsets or add tags to the tests to filter them.
+
+[▲ top](#configuration---wave-test-runner)
+
+### 3.1 Define a subset
+
+To define a subset use the json structure as follows:
+
+```json
+{
+  "subsets": [
+    {
+      "tests": ["/path/to/test.html"]
+    }
+  ]
+}
+```
+
+It is also possible to use wildcards in the path:
+
+```json
+{
+  "subsets": [
+    {
+      "tests": ["/path/to/directory/*", "*-manual.html"]
+    }
+  ]
+}
+```
+
+[▲ top](#configuration---wave-test-runner)
+
+### 3.2 Make subset selectable
+
+To make a defined subset available for selection in the configuration page, you have to assign it a title:
+
+```json
+{
+  "subsets": [
+    {
+      "title": "Selectable subset",
+      "tests": ["/path/to/test.html"]
+    }
+  ]
+}
+```
+
+![Config page subset select](./res/configuration_page_subset_select.png)
+
+[▲ top](#configuration---wave-test-runner)
+
+### 3.3 Default subsets
+
+Defining a subset as default subset will select it automatically when opening the configuration page.
+
+```json
+{
+  "subsets": [
+    {
+      "default_subset": true,
+      "tests": ["/path/to/test.html"]
+    }
+  ]
+}
+```
+
+[▲ top](#configuration---wave-test-runner)
+
+### 3.4 Tags and filters
+
+A subset can be used to apply tags to tests, which can be filtered:
+
+```json
+{
+  "subsets": [
+    {
+      "tag": {
+        "title": "my tag",
+        "color": "warning"
+      },
+      "tests": ["/test-group-a/*"]
+    }
+  ]
+}
+```
+
+![alt text](./res/configuration_page_tags.png)
+
+[▲ top](#configuration---wave-test-runner)
+
+### 3.5 Default filters
+
+Defining a default filter automatically activates it when opening the configuration page:
+
+```json
+{
+  "subsets": [
+    {
+      "tag": {
+        "title": "my tag",
+        "color": "warning",
+        "default_filter": true
+      },
+      "tests": ["/test-group-a/*"]
+    }
+  ]
+}
+```
+
+For possible colors click [here](https://bulma.io/documentation/elements/tag/#colors)
+
+[▲ top](#configuration---wave-test-runner)
+
+### 3.6 Messages
+
+Defining a messages shows a corresponding message box whenever a test from this subset is selected:
+
+```json
+{
+  "subsets": [
+    {
+      "tag": {
+        "title": "my tag",
+        "color": "warning"
+      },
+      "message": {
+        "text": "Text to display in the message box. [Link](https://link.somwhere/)",
+        "color": "info"
+      },
+      "tests": ["/test-group-a/*"]
+    }
+  ]
+}
+```
+
+For possible colors click [here](https://bulma.io/documentation/components/message/#colors)
+
+[▲ top](#configuration---wave-test-runner)