You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: test-runner/README.md
+46-5Lines changed: 46 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -6,17 +6,29 @@ Test Runner is a simple automated testing framework for replicating and validati
6
6
7
7
These steps assume you are using a modern Linux OS like [Ubuntu 22.04](https://www.releases.ubuntu.com/jammy/) with [Python 3.10+](https://www.python.org/downloads/release/python-3100/).
2. Create a `tests.yaml` file in your project's test directory
29
+
2. Create a `tests.yaml`[file](#create-tests-definition-file) in your project's test directory
18
30
3. Add a [test](#test-definition) to the `tests.yaml` file
19
-
4. Run the Test Runner application command line interface to execute the test
31
+
4.[Run the Test Runner application](#run-test-file) command line interface to execute the test
20
32
21
33
### Test Definition
22
34
@@ -47,6 +59,15 @@ A test is defined as a set of commands to be executed along with their associate
47
59
> [!TIP]
48
60
> To find more information about each property, click on the attribute link to see it's OCI spec definition. To see the list of default values used for each property, see the [class definition](utils/test.py#L18).
49
61
62
+
#### Tests Definition Example
63
+
64
+
A sample minimal tests.yaml file will look like this:
65
+
66
+
```yaml
67
+
test_name:
68
+
cmd: something -you <want|the|test_runner> --todo
69
+
```
70
+
50
71
### Advanced Tests
51
72
52
73
There are two more complex properties that can be enabled to provide additional functionality to your tests.
@@ -55,20 +76,40 @@ There are two more complex properties that can be enabled to provide additional
55
76
56
77
Environment variables can be set for a test by providing a dictionary of key-value pairs in the `env` property of the test definition. These environment variables will be set when the test is executed using the [`expandvars`](https://pypi.org/project/expandvars/) library.
57
78
58
-
Here's an example of a test definition with environment variables:
79
+
Here's an example of a test definition using environment variables:
In this instance, the default value of `hello` is overridden first by the value in the `env` variable dictionary, and then by the value passed on the test command line. Thus, the first output will be `Yo!`, and the second output will be `world`.
112
+
72
113
> [!TIP]
73
114
> It is a best practice to set a default value in the case where `VAR1` is not set with `:-<value>`, if the environment variable is not set for whatever reason, the test runner application will fail the test.
74
115
@@ -164,7 +205,7 @@ There are two modes for running the tests: baremetal and container.
164
205
165
206
#### Baremetal
166
207
167
-
In baremetal mode, the test command is executed directly on the host system. This mode is used when the img attribute is not provided in the test definition.
208
+
In baremetal mode, the test command is executed directly on the host system. This mode is implied when the `img` key/value pair is not provided in the test definition.
0 commit comments