Skip to content

Commit 75a2562

Browse files
claynerobisonTyler Titsworth
andauthored
Adapt Test Runner README.md to AI Ref Models Requirements (#107)
Signed-off-by: Clayne Robison <[email protected]> Co-authored-by: Tyler Titsworth <[email protected]>
1 parent e13b13e commit 75a2562

File tree

1 file changed

+46
-5
lines changed

1 file changed

+46
-5
lines changed

test-runner/README.md

Lines changed: 46 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,17 +6,29 @@ Test Runner is a simple automated testing framework for replicating and validati
66

77
These steps assume you are using a modern Linux OS like [Ubuntu 22.04](https://www.releases.ubuntu.com/jammy/) with [Python 3.10+](https://www.python.org/downloads/release/python-3100/).
88

9+
### Install Test Runner
10+
911
```bash
1012
git clone https://github.com/intel/ai-containers
1113
pip install -r ai-containers/test-runner/requirements.txt
14+
```
15+
16+
### Create Tests Definition File
17+
18+
```bash
1219
touch tests.yaml
20+
```
21+
22+
### Run test file
23+
24+
```bash
1325
python ai-containers/test-runner/test_runner.py -f tests.yaml
1426
```
1527

1628
1. Install the python requirements file
17-
2. Create a `tests.yaml` file in your project's test directory
29+
2. Create a `tests.yaml` [file](#create-tests-definition-file) in your project's test directory
1830
3. Add a [test](#test-definition) to the `tests.yaml` file
19-
4. Run the Test Runner application command line interface to execute the test
31+
4. [Run the Test Runner application](#run-test-file) command line interface to execute the test
2032

2133
### Test Definition
2234

@@ -47,6 +59,15 @@ A test is defined as a set of commands to be executed along with their associate
4759
> [!TIP]
4860
> To find more information about each property, click on the attribute link to see it's OCI spec definition. To see the list of default values used for each property, see the [class definition](utils/test.py#L18).
4961
62+
#### Tests Definition Example
63+
64+
A sample minimal tests.yaml file will look like this:
65+
66+
```yaml
67+
test_name:
68+
cmd: something -you <want|the|test_runner> --todo
69+
```
70+
5071
### Advanced Tests
5172
5273
There are two more complex properties that can be enabled to provide additional functionality to your tests.
@@ -55,20 +76,40 @@ There are two more complex properties that can be enabled to provide additional
5576
5677
Environment variables can be set for a test by providing a dictionary of key-value pairs in the `env` property of the test definition. These environment variables will be set when the test is executed using the [`expandvars`](https://pypi.org/project/expandvars/) library.
5778

58-
Here's an example of a test definition with environment variables:
79+
Here's an example of a test definition using environment variables:
5980

6081
```yaml
6182
test:
6283
cmd: echo "${VAR1:-hello}"
6384
```
6485

86+
Execution:
87+
6588
```bash
6689
python test-runner/test_runner.py -f path/to/tests.yaml
67-
VAR=world python test-runner/test_runner.py -f path/to/tests.yaml
90+
VAR1=world python test-runner/test_runner.py -f path/to/tests.yaml
6891
```
6992

7093
In the example above, the first output will be `hello`, and the second output will be `world`.
7194

95+
Modifying the above example to use the `env` dictionary option:
96+
97+
```yaml
98+
test:
99+
cmd: echo "${VAR1:-hello}"
100+
env:
101+
VAR1: "Yo!"
102+
```
103+
104+
Execution:
105+
106+
```bash
107+
python test-runner/test_runner.py -f path/to/tests.yaml
108+
VAR1=world python test-runner/test_runner.py -f path/to/tests.yaml
109+
```
110+
111+
In this instance, the default value of `hello` is overridden first by the value in the `env` variable dictionary, and then by the value passed on the test command line. Thus, the first output will be `Yo!`, and the second output will be `world`.
112+
72113
> [!TIP]
73114
> It is a best practice to set a default value in the case where `VAR1` is not set with `:-<value>`, if the environment variable is not set for whatever reason, the test runner application will fail the test.
74115

@@ -164,7 +205,7 @@ There are two modes for running the tests: baremetal and container.
164205

165206
#### Baremetal
166207

167-
In baremetal mode, the test command is executed directly on the host system. This mode is used when the img attribute is not provided in the test definition.
208+
In baremetal mode, the test command is executed directly on the host system. This mode is implied when the `img` key/value pair is not provided in the test definition.
168209

169210
Example of a test definition for baremetal mode:
170211

0 commit comments

Comments
 (0)