Skip to content

Commit d9694a2

Browse files
author
Sylvain MARIE
committed
Updated documentation
1 parent f25102d commit d9694a2

File tree

1 file changed

+338
-20
lines changed

1 file changed

+338
-20
lines changed

‎docs/index.md‎

Lines changed: 338 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -7,17 +7,27 @@
77
[![Documentation](https://img.shields.io/badge/doc-latest-blue.svg)](https://smarie.github.io/python-pyfields/) [![PyPI](https://img.shields.io/pypi/v/pyfields.svg)](https://pypi.python.org/pypi/pyfields/) [![Downloads](https://pepy.tech/badge/pyfields)](https://pepy.tech/project/pyfields) [![Downloads per week](https://pepy.tech/badge/pyfields/week)](https://pepy.tech/project/pyfields) [![GitHub stars](https://img.shields.io/github/stars/smarie/python-pyfields.svg)](https://github.com/smarie/python-pyfields/stargazers)
88

99

10-
`pyfields` provides a simple, elegant, extensible and fast way to **define fields in python classes**:
10+
`pyfields` provides a simple and elegant way to define fields in python classes. With `pyfields` you explicitly define all aspects of a field (default value, type, documentation...) in a single place, and can refer to it from other places.
1111

12-
- with **as little impact on the class as possible** so that it can be used in *any* python class including mix-in classes. In particular not fiddling with `__init__` nor `__setattr__`, not requiring a class decorator, and nor requiring classes to inherit from a specific `BaseModel` or similar class,
13-
14-
- with a **fast native implementation** by default (same speed as usual python after the first attribute access),
12+
It is designed with **development freedom** as primary target:
13+
14+
- *code segregation*. Everything is in the field, not in `__init__`, not in `__setattr__`.
15+
16+
- *absolutely no constraints*. Your class does not need to use type hints. You can use python 2 and 3.5. Your class is not modified behind your back: `__init__` and `__setattr__` are untouched. You do not need to decorate your class. You do not need your class to inherit from anything. This is particularly convenient for mix-in classes, and in general for users wishing to stay in control of their class design.
1517

16-
- with support for **default values**, **default values factories**, **type hints** and **docstring**,
18+
- *no performance loss by default*. If you use `pyfields` to declare fields without adding validators nor converters, instance attributes will be replaced with a native python attribute on first access, preserving the same level of performance than what you are used to.
19+
20+
It provides **many optional features** that will make your object-oriented developments easier:
21+
22+
- all field declarations support *type hints* and *docstring*,
23+
24+
- optional fields can have *default values* but also *default values factories* (such as *"if no value is provided, copy this other field"*)
1725

18-
- with optional support for **validation** and **conversion**. This makes field access obviously slower than the default native implementation but it is done field by field and not on the whole class at once, so fast native fields can coexist with slower validated ones.
26+
- adding *validators* and *converters* to a field does not require you to write complex logic nor many lines of code. This makes field access obviously slower than the default native implementation but it is done field by field and not on the whole class at once, so fast native fields can coexist with slower validated ones (segregation principle).
27+
28+
- initializing fields in your *constructor* is very easy and highly customizable
1929

20-
If your first reaction is "what about attrs/dataclasses/traitlets/autoclass/pydantic", please have a look [here](why.md).
30+
If your first reaction is "what about `attrs` / `dataclasses` / `pydantic` / `characteristic` / `traits` / `traitlets` / `autoclass` / ...", please have a look [here](why.md).
2131

2232

2333
## Installing
@@ -28,31 +38,339 @@ If your first reaction is "what about attrs/dataclasses/traitlets/autoclass/pyda
2838

2939
## Usage
3040

31-
### a - basics
41+
### 1. Defining a field
42+
43+
A field is defined as a class member using the `field()` method. The idea (not new) is that you declare in a single place all aspects related to each field. For mandatory fields you do not need to provide any argument. For optional fields, you will typically provide a `default` value or a `default_factory` (we will see that later).
44+
45+
For example let's create a `Wall` class with one *mandatory* `height` and one *optional* `color` field:
46+
47+
```python
48+
from pyfields import field
49+
50+
class Wall:
51+
height: int = field(doc="Height of the wall in mm.")
52+
color: str = field(default='white', doc="Color of the wall.")
53+
```
54+
55+
!!! info "Compliance with python < 3.6"
56+
If you use python < `3.6` you know that PEP484 type hints can not be declared as shown above. However you can provide them as [type comments](https://www.python.org/dev/peps/pep-0484/#type-comments), or using the `type_hint` argument.
57+
58+
#### Field vs. Python attribute
59+
60+
By default when you use `field()`, nothing more than a "lazy field" is created on your class. This field will only be activated when you access it on an instance. That means that you are free to implement `__init__` as you wish, or even to rely on the default `object` constructor to create instances:
61+
62+
```python
63+
# instantiate using the default `object` constructor
64+
w = Wall()
65+
```
66+
67+
No exception here even if we did not provide any value for the mandatory field `height` ! Although this default behaviour can look surprising, you will find that this feature is quite handy to define mix-in classes *with* attributes but *without* constructor. See [mixture](https://smarie.github.io/python-mixture/) for discussion. Of course if you do not like this behaviour you can very easily [add a constructor](#2-adding-a-constructor).
68+
69+
Until it is accessed for the first time, a field is visible on an instance with `dir()` (because its definition is inherited from the class) but not with `vars()` (because it has not been initialized on the object):
70+
71+
```python
72+
>>> dir(w)[-2:]
73+
['color', 'height']
74+
>>> vars(w)
75+
{}
76+
```
77+
78+
As soon as you access it, a field is replaced with a standard native python attribute, visible in `vars`:
79+
80+
```python
81+
>>> w.color # optional field: tdefault value is used
82+
'white'
83+
84+
>>> vars(w)
85+
{'color': 'white'}
86+
```
87+
88+
Of course mandatory fields must be initialized:
89+
90+
```python
91+
>>> w.height
92+
pyfields.core.MandatoryFieldInitError: \
93+
Mandatory field 'height' has not been initialized yet on instance <...>.
94+
95+
>>> w.height = 12
96+
>>> vars(w)
97+
{'color': 'white', 'height': 12}
98+
```
99+
100+
Your IDE (e.g. PyCharm) should recognize the name and type of the field, so you can already refer to it easily in other code using autocompletion:
101+
102+
![pycharm_autocomplete1.png](imgs/autocomplete1.png)
103+
104+
#### Type validation
105+
106+
You can add type validation to a field by setting `check_type=True`.
107+
108+
```python
109+
class Wall(object):
110+
height: int = field(check_type=True, doc="Height of the wall in mm.")
111+
color: str = field(check_type=True, default='white', doc="Color of the wall.")
112+
```
113+
114+
yields
115+
116+
```
117+
>>> w = Wall()
118+
>>> w.height = 1
119+
>>> w.height = '1'
120+
TypeError: Invalid value type provided for 'Wall.height'. \
121+
Value should be of type 'int'. Instead, received a 'str': '1'
122+
```
123+
124+
!!! info "Compliance with python < 3.6"
125+
If you use python < `3.6` and require type validation you should not use [type comments](https://www.python.org/dev/peps/pep-0484/#type-comments) but rather use the `type_hint` argument in `field`. Indeed it is not possible for python code to access type comments without source code inspection.
126+
127+
128+
#### Value validation
129+
130+
You can add value (and type) validation to a field by providing `validators`. `pyfields` relies on `valid8` for validation, so the supported syntax is the same:
131+
132+
- For a single validator, either provide a `<callable>` or a tuple `(<callable>, <error_msg>)`, `(<callable>, <failure_type>)` or `(<callable>, <error_msg>, <failure_type>)`. See [here](https://smarie.github.io/python-valid8/validation_funcs/c_simple_syntax/#1-one-validation-function) for details.
133+
134+
- For several validators, either provide a list or a dictionary. See [here](https://smarie.github.io/python-valid8/validation_funcs/c_simple_syntax/#2-several-validation-functions) for details.
135+
136+
For example:
137+
138+
```python
139+
from mini_lambda import x
140+
from valid8.validation_lib import is_in
141+
142+
colors = {'white', 'blue', 'red'}
143+
144+
class Wall(object):
145+
height: int = field(validators={'should be a positive number': x > 0,
146+
'should be a multiple of 100': x % 100 == 0},
147+
doc="Height of the wall in mm.")
148+
color: str = field(validators=is_in(colors),
149+
default='white',
150+
doc="Color of the wall.")
151+
```
152+
153+
yields
154+
155+
```
156+
>>> w = Wall()
157+
>>> w.height = 100
158+
>>> w.height = 1
159+
valid8.entry_points.ValidationError[ValueError]:
160+
Error validating [<...>.Wall.height=1].
161+
At least one validation function failed for value 1.
162+
Successes: ['x > 0'] / Failures: {
163+
'x % 100 == 0': 'InvalidValue: should be a multiple of 100. Returned False.'
164+
}.
165+
>>> w.color = 'magenta'
166+
valid8.entry_points.ValidationError[ValueError]:
167+
Error validating [<...>.Wall.color=magenta].
168+
NotInAllowedValues: x in {'blue', 'red', 'white'} does not hold for x=magenta.
169+
Wrong value: 'magenta'.
170+
```
171+
172+
See `valid8` documentation for details about the [syntax](https://smarie.github.io/python-valid8/validation_funcs/c_simple_syntax/) and available [validation lib](https://smarie.github.io/python-valid8/validation_funcs/b_base_validation_lib/).
173+
174+
In addition to the above syntax, `pyfields` support that you add validators to a field after creation, using the `@field.validator` decorator:
175+
176+
*todo*
177+
178+
Finally, for advanced validation scenarios you might with your validation callables to receive a bit of context. `pyfields` supports that the callables accept one, two or three arguments for this (where `valid8` supports only 1): `f(val)`, `f(obj, val)`, and `f(obj, field, val)`.
179+
180+
For example we can define walls where the width is a multiple of the length:
181+
182+
*todo*
183+
184+
#### Converters
185+
186+
*todo*
32187

33-
TODO
188+
#### Native vs. Descriptor fields
189+
190+
`field()` by default creates a so-called **native field**. This special construct is designed to be as fast as a normal python attribute after the first access, so that performance is not impacted. This high level of performance has a drawback: validation and conversion are not possible on a native field.
191+
192+
So when you add type or value validation, or conversion, to a field, `field()` will automatically create a **descriptor field** instead of a native field. This is an object relying on the [python descriptor protocol](https://docs.python.org/howto/descriptor.html). Such objects have slower access time than native python attributes but provide convenient hooks necessary to perform validation and conversion.
193+
194+
For experiments, you can force a field to be a descriptor by setting `native=False`:
34195

35196
```python
36197
from pyfields import field
37198

38-
class TweeterMixin:
39-
afraid = field(default=False,
40-
doc="Status of the tweeter. When this is `True`,"
41-
"tweets will be less aggressive.")
199+
class Foo:
200+
a = field() # a native field
201+
b = field(native=False) # a descriptor field
202+
```
203+
204+
We can easily see the difference (note: direct class access `Foo.a` is currently forbidden because of [this issue](https://github.com/smarie/python-pyfields/issues/12)):
205+
206+
```python
207+
>>> Foo.__dict__['a']
208+
<NativeField: <...>.Foo.a>
209+
210+
>>> Foo.__dict__['b']
211+
<DescriptorField: <...>.Foo.a>
212+
```
213+
214+
And measure the difference in access time:
215+
216+
```python
217+
import timeit
218+
219+
f = Foo()
220+
221+
def set_a(): f.a = 12
222+
223+
def set_b(): f.b = 12
224+
225+
def set_c(): f.c = 12
226+
227+
ta = timeit.Timer(set_a).timeit()
228+
tb = timeit.Timer(set_b).timeit()
229+
tc = timeit.Timer(set_c).timeit()
230+
231+
print("Average time (ns) setting the field:")
232+
print("%0.2f (normal python) ; %0.2f (native field) ;"
233+
" %0.2f (descriptor field)" % (tc, ta, tb))
234+
```
235+
236+
yields (results depend on your machine):
237+
238+
```
239+
Average time (ns) setting the field:
240+
0.09 (normal python) ; 0.09 (native field) ; 0.44 (descriptor field)
241+
```
242+
243+
!!! info "Why are native fields so fast ?"
244+
Native fields are implemented as a ["non-data" python descriptor](https://docs.python.org/3.7/howto/descriptor.html) that overrides itself on first access. So the first time the attribute is read, a small python method call extra cost is paid but the attribute is immediately replaced with a normal attribute inside the object `__dict__`. That way, subsequent calls use native python attribute access without overhead. This trick was inspired by [werkzeug's @cached_property](https://tedboy.github.io/flask/generated/generated/werkzeug.cached_property.html).
245+
246+
247+
### 2. Adding a constructor
248+
249+
`pyfields` provides you with several alternatives to add a constructor to a class equipped with fields. The reason why we do not follow the [Zen of python](https://www.python.org/dev/peps/pep-0020/#the-zen-of-python) here (*"There should be one-- and preferably only one --obvious way to do it."*) is to recognize that different developers may have different coding style or philosophies, and to be as much as possible agnostic in front of these.
250+
251+
#### a - `make_init`
252+
253+
`make_init` is the **most compact** way to add a constructor to a class with fields. With it you create your `__init__` method in one line:
254+
255+
```python hl_lines="6"
256+
from pyfields import field, make_init
257+
258+
class Wall:
259+
height: int = field(doc="Height of the wall in mm.")
260+
color: str = field(default='white', doc="Color of the wall.")
261+
__init__ = make_init()
262+
```
263+
264+
By default, all fields will appear in the constructor, in the order of appearance in the class and its parents, following the `mro` (method resolution order, the order in which python looks for a method in the hierarchy of classes). Since it is not possible for mandatory fields to appear *after* optional fields in the signature, all mandatory fields will appear first, and then all optional fields will follow.
265+
266+
The easiest way to see the result is probably to look at the help on your class:
42267

43-
def tweet(self):
44-
how = "lightly" if self.afraid else "loudly"
45-
print("tweeting %s" % how)
268+
``` hl_lines="5"
269+
>>> help(Wall)
270+
Help on class Wall in module <...>:
271+
272+
class Wall(builtins.object)
273+
| Wall(height, color='white')
274+
| (...)
275+
```
276+
277+
or you can inspect the method:
278+
279+
``` hl_lines="4"
280+
>>> help(Wall.__init__)
281+
Help on function __init__ in module <...>:
282+
283+
__init__(self, height, color='white')
284+
The `__init__` method generated for you when you use `make_init`
285+
```
286+
287+
You can check that your constructor works as expected:
288+
289+
```python
290+
>>> w = Wall(2)
291+
>>> vars(w)
292+
{'color': 'white', 'height': 2}
293+
294+
>>> w = Wall(color='blue', height=12)
295+
>>> vars(w)
296+
{'color': 'blue', 'height': 12}
297+
298+
>>> Wall(color='blue')
299+
TypeError: __init__() missing 1 required positional argument: 'height'
300+
```
301+
302+
If you do not wish the generated constructor to expose all fields, you can customize it by providing an **explicit** ordered list of fields. For example below only `height` will be in the constructor:
303+
304+
```python hl_lines="8"
305+
from pyfields import field, make_init
306+
307+
class Wall:
308+
height: int = field(doc="Height of the wall in mm.")
309+
color: str = field(default='white', doc="Color of the wall.")
310+
311+
# only `height` will be in the constructor
312+
__init__ = make_init(height)
313+
```
314+
315+
The list can contain fields defined in another class, typically a parent class:
316+
317+
```python hl_lines="9"
318+
from pyfields import field, make_init
319+
320+
class Wall:
321+
height: int = field(doc="Height of the wall in mm.")
322+
323+
class ColoredWall(Wall):
324+
color: str = field(default='white', doc="Color of the wall.")
325+
__init__ = make_init(Wall.height)
46326
```
47327

48-
!!! success "No performance overhead"
49-
`field` by default returns a ["non-data" python descriptor](https://docs.python.org/3.7/howto/descriptor.html). So the first time the attribute is read, a small python method call extra cost is paid. *But* afterwards the attribute is replaced with a native attribute inside the object `__dict__`, so subsequent calls use native access without overhead. This was inspired by [werkzeug's @cached_property](https://tedboy.github.io/flask/generated/generated/werkzeug.cached_property.html).
328+
Note: a pending [issue](https://github.com/smarie/python-pyfields/issues/12) prevents the above example to work, you have to use `Wall.__dict__['height']` instead of `Wall.height` to reference the field from the other class.
329+
330+
Finally, you can customize the created constructor by declaring a post-init method as the `post_init_fun` argument. This is roughly equivalent to `@init_fields` so we do not present it here, see [documentation](api_reference.md#make_init).
331+
332+
333+
#### b - `@init_fields`
334+
335+
If you prefer to write an init function as usual, you can use the `@init_fields` decorator to augment this init function's signature with all or some fields.
336+
337+
```python hl_lines="7"
338+
from pyfields import field, init_fields
50339

340+
class Wall:
341+
height = field(doc="Height of the wall in mm.") # type: int
342+
color = field(default='white', doc="Color of the wall.") # type: str
51343

52-
### b - advanced
344+
@init_fields
345+
def __init__(self, msg='hello'):
346+
"""
347+
Constructor. After initialization, some print message is done
53348
54-
TODO
349+
:param msg: the message details to add
350+
"""
351+
print("post init ! height=%s, color=%s, msg=%s" % (self.height, self.color, msg))
352+
self.non_field_attr = msg
353+
```
354+
355+
Note: as you can see in this example, you can of course create other attributes in this init function (done in the last line here with `self.non_field_attr = msg`). Indeed, declaring fields in a class do not "pollute" the class, so you can do anything you like as usual.
356+
357+
You can check that the resulting constructor works as expected:
358+
359+
```
360+
>>> help(Wall)
361+
Help on class Wall in module <...>:
362+
class Wall(builtins.object)
363+
| Wall(height, msg='hello', color='white')
364+
...
365+
366+
>>> w = Wall(1, 'hey')
367+
post init ! height=1, color=white, msg=hey
368+
369+
>>> vars(w)
370+
{'height': 1, 'color': 'white', 'non_field_attr': 'hey'}
371+
```
55372

373+
Note on the order of arguments in the resulting `__init__` signature: as you can see, `msg` appears between `height` and `color` in the signature. This corresponds to the
56374

57375

58376
## Main features / benefits

0 commit comments

Comments
 (0)