|
| 1 | +# `pytest` Goodies |
| 2 | + |
| 3 | +Many `pytest` features were missing to make `pytest_cases` work with such a "no-boilerplate" experience. Many of these can be of interest to the general `pytest` audience, so they are exposed in the public API. |
| 4 | + |
| 5 | + |
| 6 | +## `@fixture` |
| 7 | + |
| 8 | +`@fixture` is similar to `pytest.fixture` but without its `param` and `ids` arguments. Instead, it is able to pick the parametrization from `@pytest.mark.parametrize` marks applied on fixtures. This makes it very intuitive for users to parametrize both their tests and fixtures. As a bonus, its `name` argument works even in old versions of pytest (which is not the case for `fixture`). |
| 9 | + |
| 10 | +Finally it now supports unpacking, see [unpacking feature](#unpack_fixture-unpack_into). |
| 11 | + |
| 12 | +!!! note "`@fixture` deprecation if/when `@pytest.fixture` supports `@pytest.mark.parametrize`" |
| 13 | + The ability for pytest fixtures to support the `@pytest.mark.parametrize` annotation is a feature that clearly belongs to `pytest` scope, and has been [requested already](https://github.com/pytest-dev/pytest/issues/3960). It is therefore expected that `@fixture` will be deprecated in favor of `@pytest_fixture` if/when the `pytest` team decides to add the proposed feature. As always, deprecation will happen slowly across versions (at least two minor, or one major version update) so as for users to have the time to update their code bases. |
| 14 | + |
| 15 | +## `unpack_fixture` / `unpack_into` |
| 16 | + |
| 17 | +In some cases fixtures return a tuple or a list of items. It is not easy to refer to a single of these items in a test or another fixture. With `unpack_fixture` you can easily do it: |
| 18 | + |
| 19 | +```python |
| 20 | +import pytest |
| 21 | +from pytest_cases import unpack_fixture, fixture |
| 22 | + |
| 23 | +@fixture |
| 24 | +@pytest.mark.parametrize("o", ['hello', 'world']) |
| 25 | +def c(o): |
| 26 | + return o, o[0] |
| 27 | + |
| 28 | +a, b = unpack_fixture("a,b", c) |
| 29 | + |
| 30 | +def test_function(a, b): |
| 31 | + assert a[0] == b |
| 32 | +``` |
| 33 | + |
| 34 | +Note that you can also use the `unpack_into=` argument of `@fixture` to do the same thing: |
| 35 | + |
| 36 | +```python |
| 37 | +import pytest |
| 38 | +from pytest_cases import fixture |
| 39 | + |
| 40 | +@fixture(unpack_into="a,b") |
| 41 | +@pytest.mark.parametrize("o", ['hello', 'world']) |
| 42 | +def c(o): |
| 43 | + return o, o[0] |
| 44 | + |
| 45 | +def test_function(a, b): |
| 46 | + assert a[0] == b |
| 47 | +``` |
| 48 | + |
| 49 | +And it is also available in `fixture_union`: |
| 50 | + |
| 51 | +```python |
| 52 | +import pytest |
| 53 | +from pytest_cases import fixture, fixture_union |
| 54 | + |
| 55 | +@fixture |
| 56 | +@pytest.mark.parametrize("o", ['hello', 'world']) |
| 57 | +def c(o): |
| 58 | + return o, o[0] |
| 59 | + |
| 60 | +@fixture |
| 61 | +@pytest.mark.parametrize("o", ['yeepee', 'yay']) |
| 62 | +def d(o): |
| 63 | + return o, o[0] |
| 64 | + |
| 65 | +fixture_union("c_or_d", [c, d], unpack_into="a, b") |
| 66 | + |
| 67 | +def test_function(a, b): |
| 68 | + assert a[0] == b |
| 69 | +``` |
| 70 | + |
| 71 | +## `param_fixture[s]` |
| 72 | + |
| 73 | +If you wish to share some parameters across several fixtures and tests, it might be convenient to have a fixture representing this parameter. This is relatively easy for single parameters, but a bit harder for parameter tuples. |
| 74 | + |
| 75 | +The two utilities functions `param_fixture` (for a single parameter name) and `param_fixtures` (for a tuple of parameter names) handle the difficulty for you: |
| 76 | + |
| 77 | +```python |
| 78 | +import pytest |
| 79 | +from pytest_cases import param_fixtures, param_fixture |
| 80 | + |
| 81 | +# create a single parameter fixture |
| 82 | +my_parameter = param_fixture("my_parameter", [1, 2, 3, 4]) |
| 83 | + |
| 84 | +@pytest.fixture |
| 85 | +def fixture_uses_param(my_parameter): |
| 86 | + ... |
| 87 | + |
| 88 | +def test_uses_param(my_parameter, fixture_uses_param): |
| 89 | + ... |
| 90 | + |
| 91 | +# ----- |
| 92 | +# create a 2-tuple parameter fixture |
| 93 | +arg1, arg2 = param_fixtures("arg1, arg2", [(1, 2), (3, 4)]) |
| 94 | + |
| 95 | +@pytest.fixture |
| 96 | +def fixture_uses_param2(arg2): |
| 97 | + ... |
| 98 | + |
| 99 | +def test_uses_param2(arg1, arg2, fixture_uses_param2): |
| 100 | + ... |
| 101 | +``` |
| 102 | + |
| 103 | +You can mark any of the argvalues with `pytest.mark` to pass a custom id or a custom "skip" or "fail" mark, just as you do in `pytest`. See [pytest documentation](https://docs.pytest.org/en/stable/example/parametrize.html#set-marks-or-test-id-for-individual-parametrized-test). |
| 104 | + |
| 105 | +## `fixture_union` |
| 106 | + |
| 107 | +As of `pytest` 5, it is not possible to create a "union" fixture, i.e. a parametrized fixture that would first take all the possible values of fixture A, then all possible values of fixture B, etc. Indeed all fixture dependencies (a.k.a. "closure") of each test node are grouped together, and if they have parameters a big "cross-product" of the parameters is done by `pytest`. |
| 108 | + |
| 109 | +The topic has been largely discussed in [pytest-dev#349](https://github.com/pytest-dev/pytest/issues/349) and a [request for proposal](https://docs.pytest.org/en/latest/proposals/parametrize_with_fixtures.html) has been finally made. |
| 110 | + |
| 111 | +`fixture_union` is an implementation of this proposal. It is also used by `parametrize` to support `fixture_ref` in parameter values, see [below](#parametrize). |
| 112 | + |
| 113 | +```python |
| 114 | +from pytest_cases import fixture, fixture_union |
| 115 | + |
| 116 | +@fixture |
| 117 | +def first(): |
| 118 | + return 'hello' |
| 119 | + |
| 120 | +@fixture(params=['a', 'b']) |
| 121 | +def second(request): |
| 122 | + return request.param |
| 123 | + |
| 124 | +# c will first take all the values of 'first', then all of 'second' |
| 125 | +c = fixture_union('c', [first, second]) |
| 126 | + |
| 127 | +def test_basic_union(c): |
| 128 | + print(c) |
| 129 | +``` |
| 130 | + |
| 131 | +yields |
| 132 | + |
| 133 | +``` |
| 134 | +<...>::test_basic_union[c_is_first] hello PASSED |
| 135 | +<...>::test_basic_union[c_is_second-a] a PASSED |
| 136 | +<...>::test_basic_union[c_is_second-b] b PASSED |
| 137 | +``` |
| 138 | + |
| 139 | +As you can see the ids of union fixtures are slightly different from standard ids, so that you can easily understand what is going on. You can change this feature with `ìdstyle`, see [API documentation](./api_reference.md#fixture_union) for details. |
| 140 | + |
| 141 | +You can mark any of the alternatives with `pytest.mark` to pass a custom id or a custom "skip" or "fail" mark, just as you do in `pytest`. See [pytest documentation](https://docs.pytest.org/en/stable/example/parametrize.html#set-marks-or-test-id-for-individual-parametrized-test). |
| 142 | + |
| 143 | +Fixture unions also support unpacking with the `unpack_into` argument, see [unpacking feature](#unpack_fixture-unpack_into). |
| 144 | + |
| 145 | +Fixture unions are a **major change** in the internal pytest engine, as fixture closures (the ordered set of all fixtures required by a test node to run - directly or indirectly) now become trees where branches correspond to alternative paths taken in the "unions", and leafs are the alternative fixture closures. This feature has been tested in very complex cases (several union fixtures, fixtures that are not selected by a given union but that is requested by the test function, etc.). But if you find some strange behaviour don't hesitate to report it in the [issues](https://github.com/smarie/python-pytest-cases/issues) page ! |
| 146 | + |
| 147 | +**IMPORTANT** if you do not use `@fixture` but only `@pytest.fixture`, then you will see that your fixtures are called even when they are not used, with a parameter `NOT_USED`. This symbol is automatically ignored if you use `@fixture`, otherwise you have to handle it. Alternatively you can use `@ignore_unused` on your fixture function. |
| 148 | + |
| 149 | +!!! note "fixture unions vs. cases" |
| 150 | + If you're familiar with `pytest-cases` already, you might note that `@cases_data` is not so different than a fixture union: we do a union of all case functions. If one day union fixtures are directly supported by `pytest`, we will probably refactor this lib to align all the concepts. |
| 151 | + |
| 152 | + |
| 153 | +## `@parametrize` |
| 154 | + |
| 155 | +`@parametrize` is a replacement for `@pytest.mark.parametrize` with many additional features to make the most of parametrization. See [API reference](./api_reference.md#parametrize) for details about all the new features. In particular it allows you to include references to fixtures and to value-generating functions in the parameter values. |
| 156 | + |
| 157 | + - Simply use `fixture_ref(<fixture>)` in the parameter values, where `<fixture>` can be the fixture name or fixture function. |
| 158 | + - if you do not wish to create a fixture, you can also use `lazy_value(<function>)` |
| 159 | + - Note that when parametrizing several argnames, both `fixture_ref` and `lazy_value` can be used *as* the tuple, or *in* the tuple. Several `fixture_ref` and/or `lazy_value` can be used in the same tuple, too. |
| 160 | + |
| 161 | +For example, with a single argument: |
| 162 | + |
| 163 | +```python |
| 164 | +import pytest |
| 165 | +from pytest_cases import parametrize, fixture, fixture_ref, lazy_value |
| 166 | + |
| 167 | +@pytest.fixture |
| 168 | +def world_str(): |
| 169 | + return 'world' |
| 170 | + |
| 171 | +def whatfun(): |
| 172 | + return 'what' |
| 173 | + |
| 174 | +@fixture |
| 175 | +@parametrize('who', [fixture_ref(world_str), |
| 176 | + 'you']) |
| 177 | +def greetings(who): |
| 178 | + return 'hello ' + who |
| 179 | + |
| 180 | +@parametrize('main_msg', ['nothing', |
| 181 | + fixture_ref(world_str), |
| 182 | + lazy_value(whatfun), |
| 183 | + fixture_ref(greetings)]) |
| 184 | +@pytest.mark.parametrize('ending', ['?', '!']) |
| 185 | +def test_prints(main_msg, ending): |
| 186 | + print(main_msg + ending) |
| 187 | +``` |
| 188 | + |
| 189 | +yields the following |
| 190 | + |
| 191 | +```bash |
| 192 | +> pytest -s -v |
| 193 | +collected 10 items |
| 194 | +test_prints[main_msg_is_nothing-?] PASSED [ 10%]nothing? |
| 195 | +test_prints[main_msg_is_nothing-!] PASSED [ 20%]nothing! |
| 196 | +test_prints[main_msg_is_world_str-?] PASSED [ 30%]world? |
| 197 | +test_prints[main_msg_is_world_str-!] PASSED [ 40%]world! |
| 198 | +test_prints[main_msg_is_whatfun-?] PASSED [ 50%]what? |
| 199 | +test_prints[main_msg_is_whatfun-!] PASSED [ 60%]what! |
| 200 | +test_prints[main_msg_is_greetings-who_is_world_str-?] PASSED [ 70%]hello world? |
| 201 | +test_prints[main_msg_is_greetings-who_is_world_str-!] PASSED [ 80%]hello world! |
| 202 | +test_prints[main_msg_is_greetings-who_is_you-?] PASSED [ 90%]hello you? |
| 203 | +test_prints[main_msg_is_greetings-who_is_you-!] PASSED [100%]hello you! |
| 204 | +``` |
| 205 | + |
| 206 | +You can also mark any of the argvalues with `pytest.mark` to pass a custom id or a custom "skip" or "fail" mark, just as you do in `pytest`. See [pytest documentation](https://docs.pytest.org/en/stable/example/parametrize.html#set-marks-or-test-id-for-individual-parametrized-test). |
| 207 | + |
| 208 | +As you can see in the example above, the default ids are a bit more explicit than usual when you use at least one `fixture_ref`. This is because the parameters need to be replaced with a fixture union that will "switch" between alternative groups of parameters, and the appropriate fixtures referenced. As opposed to `fixture_union`, the style of these ids is not configurable for now, but feel free to propose alternatives in the [issues page](https://github.com/smarie/python-pytest-cases/issues). Note that this does not happen if you only use `lazy_value`s, as they do not require to create a fixture union behind the scenes. |
| 209 | + |
| 210 | +Another consequence of using `fixture_ref` is that the priority order of the parameters, relative to other standard `pytest.mark.parametrize` parameters that you would place on the same function, will get impacted. You may solve this by replacing your other `@pytest.mark.parametrize` calls with `param_fixture`s so that all the parameters are fixtures (see [above](#param_fixtures).) |
| 211 | + |
| 212 | +## passing a `hook` |
| 213 | + |
| 214 | +As per version `1.14`, all the above functions now support passing a `hook` argument. This argument should be a callable. It will be called everytime a fixture is about to be created by `pytest_cases` on your behalf. The fixture function is passed as the argument of the hook, and the hook should return it as the result. |
| 215 | + |
| 216 | +You can use this fixture to better understand which fixtures are created behind the scenes, and also to decorate the fixture functions before they are created. For example you can use `hook=saved_fixture` (from [`pytest-harvest`](https://smarie.github.io/python-pytest-harvest/)) in order to save the created fixtures in the fixture store. |
| 217 | + |
| 218 | +## `assert_exception` |
| 219 | + |
| 220 | +`assert_exception` context manager is an alternative to `pytest.raises` to check exceptions in your tests. You can either check type, instance equality, repr string pattern, or use custom validation functions. See [API reference](./api_reference.md). |
| 221 | + |
| 222 | +## `--with-reorder` |
| 223 | + |
| 224 | +`pytest` postprocesses the order of the collected items in order to optimize setup/teardown of session, module and class fixtures. This optimization algorithm happens at the `pytest_collection_modifyitems` stage, and is still under improvement, as can be seen in [pytest#3551](https://github.com/pytest-dev/pytest/pull/3551), [pytest#3393](https://github.com/pytest-dev/pytest/issues/3393), [#2846](https://github.com/pytest-dev/pytest/issues/2846)... |
| 225 | + |
| 226 | +Besides other plugins such as [pytest-reorder](https://github.com/not-raspberry/pytest_reorder) can modify the order as well. |
| 227 | + |
| 228 | +This new commandline is a goodie to change the reordering: |
| 229 | + |
| 230 | + * `--with-reorder normal` is the default behaviour: it lets pytest and all the plugins execute their reordering in each of their `pytest_collection_modifyitems` hooks, and simply does not interact |
| 231 | + |
| 232 | + * `--with-reorder skip` allows you to restore the original order that was active before `pytest_collection_modifyitems` was initially called, thus not taking into account any reordering done by pytest or by any of its plugins. |
0 commit comments