Skip to content

Commit a58b27d

Browse files
committed
get_all_cases is now exported in init. Updated documentation. Fixes #16
1 parent 9ed34d4 commit a58b27d

File tree

5 files changed

+241
-62
lines changed

5 files changed

+241
-62
lines changed

docs/api_reference.md

Lines changed: 65 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -97,9 +97,59 @@ You may wish to use this type hint instead of `CaseData` when your case function
9797

9898
## 2 - On test functions side
9999

100+
### `@cases_fixture`
101+
102+
`@cases_fixture(cases=None, module=None, case_data_argname='case_data', has_tag=None, filter=None)`
103+
104+
Decorates a function so that it becomes a parametrized fixture.
105+
106+
The fixture will be automatically parametrized with all cases listed in module `module`, or with
107+
all cases listed explicitly in `cases`.
108+
109+
Using it with a non-None `module` argument is equivalent to
110+
* extracting all cases from `module`
111+
* then decorating your function with @pytest.fixture(params=cases) with all the cases
112+
113+
So
114+
115+
```python
116+
from pytest_cases import cases_fixture, CaseData
117+
118+
# import the module containing the test cases
119+
import test_foo_cases
120+
121+
@cases_fixture(module=test_foo_cases)
122+
def foo_fixture(case_data: CaseData):
123+
...
124+
```
125+
126+
is equivalent to:
127+
128+
```python
129+
import pytest
130+
from pytest_cases import get_all_cases
131+
132+
# import the module containing the test cases
133+
import test_foo_cases
134+
135+
# manually list the available cases
136+
cases = get_all_cases(module=test_foo_cases)
137+
138+
# parametrize the fixture manually
139+
@pytest.fixture(params=cases)
140+
def foo_fixture(request):
141+
case_data = request.param # type: CaseData
142+
...
143+
```
144+
145+
**Parameters**
146+
147+
- `case_data_argname`: the optional name of the function parameter that should receive the `CaseDataGetter` object. Default is `case_data`.
148+
- Other parameters (cases, module, has_tag, filter) can be used to perform explicit listing, or filtering, of cases to include. See `get_all_cases()` for details about them.
149+
100150
### `@cases_data`
101151

102-
`@cases_data(cases=None, module=None, case_data_argname: str= 'case_data', has_tag: Any=None, filter: Callable[[List[Any]], bool]=None`
152+
`@cases_data(cases=None, module=None, case_data_argname='case_data', has_tag=None, filter=None)`
103153

104154
Decorates a test function so as to automatically parametrize it with all cases listed in module `module`, or with all cases listed explicitly in `cases`.
105155

@@ -139,13 +189,10 @@ def test_foo(case_data: CaseData):
139189
...
140190
```
141191

142-
**Parameters:**
192+
**Parameters**
143193

144-
- `cases`: a single case or a hardcoded list of cases to use. Only one of `cases` and `module` should be set.
145-
- `module`: a module or a hardcoded list of modules to use. You may use `THIS_MODULE` to indicate that the module is the current one. Only one of `cases` and `module` should be set.
146194
- `case_data_argname`: the optional name of the function parameter that should receive the `CaseDataGetter` object. Default is `case_data`.
147-
- `has_tag`: an optional tag used to filter the cases in the `module`. Only cases with the given tag will be selected.
148-
- `filter`: an optional filtering function taking as an input a list of tags associated with a case, and returning a boolean indicating if the case should be selected. It will be used to filter the cases in the `module`. It both `has_tag` and `filter` are set, both will be applied in sequence.
195+
- Other parameters (cases, module, has_tag, filter) can be used to perform explicit listing, or filtering, of cases to include. See `get_all_cases()` for details about them.
149196

150197
### `CaseDataGetter`
151198

@@ -172,8 +219,17 @@ If `expected_e` is an exception validation function, returns `Exception, None, e
172219
- `expected_e`: an `ExpectedError`, that is, either an exception type, an exception instance, or an exception validation function
173220

174221

175-
### `extract_cases_from_module`
222+
### `get_all_cases`
176223

177-
`extract_cases_from_module(module, has_tag: Any=None, filter: Callable[[List[Any]], bool]=None) -> List[CaseDataGetter]`
224+
`get_all_cases(cases=None, module=None, this_module_object=None, has_tag=None, filter=None) -> List[CaseDataGetter]`
178225

179-
Internal method used to create a list of `CaseDataGetter` for all cases available from the given module. See `@cases_data` for parameters usage.
226+
Lists all desired cases for a given user query. This function may be convenient for debugging purposes.
227+
228+
229+
**Parameters:**
230+
231+
- `cases`: a single case or a hardcoded list of cases to use. Only one of `cases` and `module` should be set.
232+
- `module`: a module or a hardcoded list of modules to use. You may use `THIS_MODULE` to indicate that the module is the current one. Only one of `cases` and `module` should be set.
233+
- `this_module_object`: any variable defined in the module of interest, for example a function. It is used to find "this module", when `module` contains `THIS_MODULE`.
234+
- `has_tag`: an optional tag used to filter the cases in the `module`. Only cases with the given tag will be selected.
235+
- `filter`: an optional filtering function taking as an input a list of tags associated with a case, and returning a boolean indicating if the case should be selected. It will be used to filter the cases in the `module`. It both `has_tag` and `filter` are set, both will be applied in sequence.

docs/index.md

Lines changed: 100 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44

55
[![Build Status](https://travis-ci.org/smarie/python-pytest-cases.svg?branch=master)](https://travis-ci.org/smarie/python-pytest-cases) [![Tests Status](https://smarie.github.io/python-pytest-cases/junit/junit-badge.svg?dummy=8484744)](https://smarie.github.io/python-pytest-cases/junit/report.html) [![codecov](https://codecov.io/gh/smarie/python-pytest-cases/branch/master/graph/badge.svg)](https://codecov.io/gh/smarie/python-pytest-cases) [![Documentation](https://img.shields.io/badge/docs-latest-blue.svg)](https://smarie.github.io/python-pytest-cases/) [![PyPI](https://img.shields.io/badge/PyPI-pytest_cases-blue.svg)](https://pypi.python.org/pypi/pytest_cases/)
66

7+
!!! success "New `@cases_fixture` decorator is there, [check it out](#d-case-fixtures) !"
8+
79
Did you ever thought that most of your test functions were actually *the same test code*, but with *different data inputs* and expected results/exceptions ?
810

911
`pytest-cases` leverages `pytest` and its great `@pytest.mark.parametrize` decorator, so that you can **separate your test cases from your test functions**. For example with `pytest-cases` you can now write your tests with the following pattern:
@@ -20,7 +22,7 @@ Did you ever thought that most of your test functions were actually *the same te
2022
> pip install pytest_cases
2123
```
2224

23-
## Usage
25+
## Usage - 'Data' cases
2426

2527
### a- Some code to test
2628

@@ -36,69 +38,49 @@ def foo(a, b):
3638
First we create a `test_foo_cases.py` file. This file will contain *test cases generator* functions, that we will call **case functions** for brevity:
3739

3840
```python
39-
from pytest_cases import CaseData
40-
41-
def case_two_positive_ints() -> CaseData:
41+
def case_two_positive_ints():
4242
""" Inputs are two positive integers """
43+
return dict(a=1, b=2)
4344

44-
ins = dict(a=1, b=2)
45-
outs = 2, 3
46-
47-
return ins, outs, None
48-
49-
def case_two_negative_ints() -> CaseData:
45+
def case_two_negative_ints():
5046
""" Inputs are two negative integers """
51-
52-
ins = dict(a=-1, b=-2)
53-
outs = 0, -1
54-
55-
return ins, outs, None
47+
return dict(a=-1, b=-2)
5648
```
5749

5850
In these functions, you will typically either parse some test data files, or generate some simulated test data and expected results.
5951

6052
Case functions **do not have any particular requirement**, apart from their names starting with `case_`. They can return anything that is considered useful to run the associated test.
6153

62-
However, as shown in the example above, `pytest_cases` proposes to adopt a convention where the functions always returns a tuple of inputs/outputs/errors. A handy `CaseData` PEP484 type hint can be used to denote that.
63-
64-
!!! note "A case function can return **anything**"
65-
Even if in all examples in this documentation we chose to return a tuple (inputs/outputs/errors) (type hint `CaseData`), you can decide to return anything: a single variable, a dictionary, a tuple of a different length, etc. Whatever you return will be available through `case_data.get()` (see below).
6654

6755
### c- Test functions
6856

69-
Finally, as usual we write our `pytest` functions starting with `test_`, in a `test_foo.py` file:
57+
Then, as usual we write our `pytest` functions starting with `test_`, in a `test_foo.py` file:
7058

7159
```python
72-
from pytest_cases import cases_data, CaseDataGetter
60+
from pytest_cases import cases_data
7361
from example import foo
7462

7563
# import the module containing the test cases
7664
import test_foo_cases
7765

78-
7966
@cases_data(module=test_foo_cases)
80-
def test_foo(case_data: CaseDataGetter):
67+
def test_foo(case_data):
8168
""" Example unit test that is automatically parametrized with @cases_data """
8269

8370
# 1- Grab the test case data
84-
i, expected_o, expected_e = case_data.get()
71+
inputs = case_data.get()
8572

8673
# 2- Use it
87-
if expected_e is None:
88-
# **** Nominal test ****
89-
outs = foo(**i)
90-
assert outs == expected_o
91-
92-
else:
93-
# **** Error tests: see <Usage> page to fill this ****
94-
pass
74+
foo(**inputs)
9575
```
9676

97-
As you can see above there are three things that are needed to bind a test function with associated case functions:
77+
*Note: as explained [here](https://smarie.github.io/python-pytest-cases/usage/basics/#cases-in-the-same-file-than-tests), cases can also be located inside the test file.*
78+
79+
As you can see above there are three things that are needed to parametrize a test function with associated case functions:
9880

9981
* decorate your test function with `@cases_data`, indicating which module contains the cases functions
10082
* add an input argument to your test function, named `case_data` with optional type hint `CaseData`
101-
* use that input argument at the beginning of the test function, to retrieve the test data: `i, expected_o, expected_e = case_data.get()`
83+
* use that input argument at the beginning of the test function, to retrieve the test data: `inputs = case_data.get()`
10284

10385

10486
Once you have done these three steps, executing `pytest` will run your test function **once for every case function**:
@@ -113,15 +95,95 @@ Once you have done these three steps, executing `pytest` will run your test func
11395
========================== 2 passed in 0.24 seconds ==========================
11496
```
11597

98+
### d- Case fixtures
99+
100+
You might be concerned that case data is gathered inside test execution. Indeed gathering case data is not part of the test *per se*. Besides if you use for example [pytest-harvest](https://smarie.github.io/python-pytest-harvest/) to benchmark your tests durations, you may want the test duration to be computed without acccounting for the data retrieval time (especially if you decide to add some caching mechanism as explained [here](https://smarie.github.io/python-pytest-cases/usage/advanced/#caching)).
101+
102+
The answer is simple: instead of parametrizing your test function, rather create a parametrized fixture:
103+
104+
```python
105+
from pytest_cases import cases_fixture
106+
from example import foo
107+
108+
# import the module containing the test cases
109+
import test_foo_cases
110+
111+
@cases_fixture(module=test_foo_cases)
112+
def inputs(case_data):
113+
""" Example fixture that is automatically parametrized with @cases_data """
114+
# retrieve case data
115+
return case_data.get()
116+
117+
def test_foo(inputs):
118+
# Use case data
119+
foo(**inputs)
120+
```
121+
122+
Note: you can still use `request` in your fixture's signature if you wish to.
123+
124+
## Usage - 'True' test cases
125+
126+
#### a- Case functions update
127+
128+
In the above example the cases were only containing inputs for the function to test. In real-world applications we often need more: we need both inputs **and an expected outcome**.
129+
130+
For this, `pytest_cases` proposes to adopt a convention where the case functions returns a tuple of inputs/outputs/errors. A handy `CaseData` PEP484 type hint can be used to denote that. But of course this is only a proposal, which is not mandatory as we saw above.
131+
132+
!!! note "A case function can return **anything**"
133+
Even if in most examples in this documentation we chose to return a tuple (inputs/outputs/errors) (type hint `CaseData`), you can decide to return anything: a single variable, a dictionary, a tuple of a different length, etc. Whatever you return will be available through `case_data.get()`.
134+
135+
Here is how we can rewrite our case functions with an expected outcome:
136+
137+
```python
138+
def case_two_positive_ints() -> CaseData:
139+
""" Inputs are two positive integers """
140+
141+
ins = dict(a=1, b=2)
142+
outs = 2, 3
143+
144+
return ins, outs, None
145+
146+
def case_two_negative_ints() -> CaseData:
147+
""" Inputs are two negative integers """
148+
149+
ins = dict(a=-1, b=-2)
150+
outs = 0, -1
151+
152+
return ins, outs, None
153+
```
154+
155+
We propose that the "expected error" (`None` above) may contain exception type, exception instances, or callables. If you follow this convention, you will be able to write your test more easily with the provided utility function `unfold_expected_err`. See [here for details](https://smarie.github.io/python-pytest-cases/usage/basics/#handling-exceptions).
156+
157+
### b- Test body update
158+
159+
With our new case functions, a case will be made of three items. So `case_data.get()` will return a tuple. Here is how we can update our test function body to retrieve it correctly, and check that the outcome is as expected:
160+
161+
```python
162+
@cases_data(module=test_foo_cases)
163+
def test_foo(case_data: CaseDataGetter):
164+
""" Example unit test that is automatically parametrized with @cases_data """
165+
166+
# 1- Grab the test case data: now a tuple !
167+
i, expected_o, expected_e = case_data.get()
168+
169+
# 2- Use it: we can now do some asserts !
170+
if expected_e is None:
171+
# **** Nominal test ****
172+
outs = foo(**i)
173+
assert outs == expected_o
174+
else:
175+
# **** Error tests: see <Usage> page to fill this ****
176+
pass
177+
```
116178

117-
See [Usage](./usage) for a complete example with custom case names, case generators, exceptions handling, and more.
179+
See [Usage](./usage) for complete examples with custom case names, case generators, exceptions handling, and more.
118180

119181

120182
## Main features / benefits
121183

122184
* **Separation of concerns**: test code on one hand, test cases data on the other hand. This is particularly relevant for data science projects where a lot of test datasets are used on the same block of test code.
123185

124-
* **Everything in the test**, not outside. A side-effect of `@pytest.mark.parametrize` is that users tend to create or parse their datasets outside of the test function. `pytest_cases` suggests a model where the potentially time and memory consuming step of case data generation/retrieval is performed *inside* the test case, thus keeping every test case run more independent. It is also easy to put debug breakpoints on specific test cases.
186+
* **Everything in the test or in the fixture**, not outside. A side-effect of `@pytest.mark.parametrize` is that users tend to create or parse their datasets outside of the test function. `pytest_cases` suggests a model where the potentially time and memory consuming step of case data generation/retrieval is performed *inside* the test node or the required fixture, thus keeping every test case run more independent. It is also easy to put debug breakpoints on specific test cases.
125187

126188
* **Easier iterable-based test case generation**. If you wish to generate several test cases using the same function, `@cases_generator` makes it very intuitive to do so. See [here](./usage#case-generators) for details.
127189

@@ -130,7 +192,9 @@ See [Usage](./usage) for a complete example with custom case names, case generat
130192
## See Also
131193

132194
- [pytest documentation on parametrize](https://docs.pytest.org/en/latest/parametrize.html)
195+
- [pytest documentation on fixtures](https://docs.pytest.org/en/latest/fixture.html#fixture-parametrize)
133196
- [pytest-steps](https://smarie.github.io/python-pytest-steps/)
197+
- [pytest-harvest](https://smarie.github.io/python-pytest-harvest/)
134198

135199
### Others
136200

docs/usage/advanced.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ This tutorial assumes that you are already familiar with [pytest-steps](https://
159159
160160
### 1- If steps can run with the same data
161161
162-
If all of the test steps require the same data to execute, it is very easy:
162+
If all of the test steps require the same data to execute, it is straightforward, both in parametrizer mode (shown below) or in the new pytest steps generator mode (not shown):
163163
164164
165165
```python
@@ -234,7 +234,7 @@ This is actually quite straightforward: simply adapt your custom case data forma
234234
235235
For example you can choose the format proposed by the `MultipleStepsCaseData` type hint, where each item in the returned inputs/outputs/errors tuple can either be a single element, or a dictionary of name -> element. This allows your case functions to return alternate contents depending on the test step being executed.
236236
237-
The example below shows a test suite where the inputs of the steps are the same, but the outputs and expected errors are different:
237+
The example below shows a test suite where the inputs of the steps are the same, but the outputs and expected errors are different. Note that once again the example relies on the legacy "parametrizer" mode of pytest-steps, but it would be similar with the new "generator" mode.
238238
239239
```python
240240
from pytest_cases import cases_data, CaseDataGetter, THIS_MODULE, \

pytest_cases/__init__.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,14 @@
66
pass
77

88
from pytest_cases.main import cases_data, CaseDataGetter, cases_fixture, \
9-
unfold_expected_err, extract_cases_from_module, THIS_MODULE
9+
unfold_expected_err, get_all_cases, extract_cases_from_module, THIS_MODULE
1010

1111
__all__ = [
1212
# the 2 submodules
1313
'main', 'case_funcs',
1414
# all symbols imported above
15-
'cases_data', 'CaseData', 'CaseDataGetter', 'cases_fixture', 'unfold_expected_err', 'extract_cases_from_module',
15+
'cases_data', 'CaseData', 'CaseDataGetter', 'cases_fixture', 'unfold_expected_err', 'get_all_cases',
16+
'extract_cases_from_module',
1617
'case_name', 'Given', 'ExpectedNormal', 'ExpectedError',
1718
'test_target', 'case_tags', 'THIS_MODULE', 'cases_generator', 'MultipleStepsCaseData'
1819
]

0 commit comments

Comments
 (0)