Skip to content
This repository was archived by the owner on Oct 8, 2021. It is now read-only.

Commit 7afbf29

Browse files
authored
Generic types and 3.4 compatibility (#16)
* Add generic type support. - Refactors the annotation support for test_attrs - Refactors errors and types into their own modules. * Mistake in changes to config.yml * Fix broken import for typeddict. * Make sure poetry >= 1.00 * Make SoftMod handle a SyntaxError * Make sure poetry is the release version. * Fix some syntax issues for python3.4 * Prepare for 2.2.0 release * Add tests for generic types, improve supporting functions. - Correct some issues with get_generic_origin. * Ran black. * Ensure get_origin works with non-parameterized generic types. * Make tests skip if `slots=True` doesn't work with Generic base. * get_origin falls back to the original type * Add workaround for change in behavior of typ.__parameters__ * Improved testing advice. - Remove black and dephell as dev dependencies. - Add dephell config to easily generate docker - Require a newer version of attrs. * Tidying. * Cleanup
1 parent b410744 commit 7afbf29

33 files changed

+1355
-693
lines changed

.circleci/config.yml

Lines changed: 21 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,67 +1,76 @@
11
version: 2.1
22
jobs:
3-
test-35:
3+
test-34:
44
docker:
5-
- image: circleci/python:3.5
5+
- image: circleci/python:3.4
66
environment:
77
&std_env
88
TERM: xterm
99
LANG: en_US.UTF-8
10+
PIP_DISABLE_PIP_VERSION_CHECK: 1
1011
working_directory: ~/json-syntax
1112
steps:
13+
&steps34
1214
- checkout
1315
- run:
1416
name: Set up virtualenv
1517
command: |
16-
pip install poetry
17-
poetry install
18+
pip install --user 'poetry>=1'
19+
python -m poetry install
1820
1921
- run:
2022
name: Run tests
2123
command: |
22-
poetry run pytest tests/
24+
python -m poetry run pytest tests/
2325
2426
- store_artifacts: # If a property test fails, this contains the example that failed.
2527
path: ".hypothesis"
2628
destination: ".hypothesis"
29+
test-35:
30+
docker:
31+
- image: circleci/python:3.5
32+
environment: *std_env
33+
steps: *steps34
34+
working_directory: ~/json-syntax
2735
test-36:
2836
docker:
2937
- image: circleci/python:3.6
3038
environment: *std_env
39+
working_directory: ~/json-syntax
3140
steps:
32-
&std_steps
41+
&steps36
3342
- checkout
3443
- run:
3544
name: Set up virtualenv
3645
command: |
37-
pip install poetry
38-
poetry install
46+
pip install --user 'poetry>=1'
47+
python -m poetry install
3948
4049
- run:
4150
name: Run tests
4251
command: |
43-
poetry run pytest --doctest-modules json_syntax/ tests/
52+
python -m poetry run pytest --doctest-modules json_syntax/ tests/
4453
4554
- store_artifacts: # If a property test fails, this contains the example that failed.
4655
path: ".hypothesis"
4756
destination: ".hypothesis"
48-
working_directory: ~/json-syntax
4957
test-37:
5058
docker:
5159
- image: circleci/python:3.7
5260
environment: *std_env
53-
steps: *std_steps
61+
steps: *steps36
5462
working_directory: ~/json-syntax
5563
test-38:
5664
docker:
5765
- image: circleci/python:3.8
5866
environment: *std_env
59-
steps: *std_steps
67+
steps: *steps36
6068
working_directory: ~/json-syntax
6169

6270
workflows:
6371
test:
6472
jobs:
73+
- test-34
6574
- test-35
6675
- test-36
6776
- test-37

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,4 @@
99
setup.py
1010
requirements.txt
1111
.tox/
12+
README.rst

README.md

Lines changed: 30 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -168,8 +168,8 @@ Thus we have:
168168
* `dict` and `Dict[K, V]`
169169

170170
Tuple is a special case. In Python, they're often used to mean "frozenlist", so
171-
`Tuple[E, ...]` (the `...` is [the Ellipsis object][ellipsis]) indicates all elements have the type
172-
`E`.
171+
`Tuple[E, ...]` (the `...` is [the Ellipsis object][ellipsis]) indicates all elements have
172+
the type `E`.
173173

174174
They're also used to represent an unnamed record. In this case, you can use
175175
`Tuple[A, B, C, D]` or however many types. It's generally better to use a `dataclass`.
@@ -180,6 +180,24 @@ The standard rules don't support:
180180
2. Using type variables.
181181
3. Any kind of callable, coroutine, file handle, etc.
182182

183+
#### Support for deriving from Generic
184+
185+
There is experimental support for deriving from `typing.Generic`. An `attrs` or `dataclass`
186+
may declare itself a generic class. If another class invokes it as `YourGeneric[Param,
187+
Param]`, those `Param` types will be substituted into the fields during encoding. This is
188+
useful to construct parameterized container types. Example:
189+
190+
@attr.s(auto_attribs=True)
191+
class Wrapper(Generic[T, M]):
192+
body: T
193+
count: int
194+
messages: List[M]
195+
196+
@attr.s(auto_attribs=True)
197+
class Message:
198+
first: Wrapper[str, str]
199+
second: Wrapper[Dict[str, str], int]
200+
183201
#### Unions
184202

185203
A union type lets you present alternate types that the converters will attempt in
@@ -347,28 +365,19 @@ This package is maintained via the [poetry][] tool. Some useful commands:
347365

348366
1. Setup: `poetry install`
349367
2. Run tests: `poetry run pytest tests/`
350-
3. Reformat: `poetry run black json_syntax/ tests/`
351-
352-
### Setting up tox
353-
354-
You'll want pyenv, then install the pythons:
355-
356-
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
357-
pyenv install --list | egrep '^ *3\.[4567]|^ *pypy3.5'
358-
# figure out what versions you want
359-
for v in 3.4.9 3.5.10 ...; do
360-
pyenv install $v
361-
PYENV_VERSION=$v python get-pip.py
362-
done
368+
3. Reformat: `black json_syntax/ tests/`
369+
4. Generate setup.py: `dephell deps convert -e setup`
370+
5. Generate requirements.txt: `dephell deps convert -e req`
363371

364-
Once you install `tox` in your preferred python, running it is just `tox`. (Note: this is
365-
largely redundant as the build is configured to all the different pythons on Circle.)
372+
### Running tests via docker
366373

367-
### Contributor roll call
374+
The environments for 3.4 through 3.9 are in `pyproject.toml`, so just run:
368375

369-
* @bsamuel-ui -- Ben Samuel
370-
* @dschep
371-
* @rugheid
376+
dephell deps convert -e req # Create requirements.txt
377+
dephell docker run -e test34 pip install -r requirements.txt
378+
dephell docker run -e test34 pytest tests/
379+
dephell docker shell -e test34 pytest tests/
380+
dephell docker destroy -e test34
372381

373382
### Notes
374383

json_syntax/__init__.py

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
"""
2-
The JSON syntax library is a combinatorial parser / generator library for managing conversion of Python objects to and
3-
from common JSON types.
2+
The JSON syntax library is a combinatorial parser / generator library for managing
3+
conversion of Python objects to and from common JSON types.
44
55
It's not strictly limited to JSON, but that's the major use case.
66
"""
@@ -39,9 +39,11 @@ def std_ruleset(
3939
cache=None,
4040
):
4141
"""
42-
Constructs a RuleSet with the provided rules. The arguments here are to make it easy to override.
42+
Constructs a RuleSet with the provided rules. The arguments here are to make it easy to
43+
override.
4344
44-
For example, to replace ``decimals`` with ``decimals_as_str`` just call ``std_ruleset(decimals=decimals_as_str)``
45+
For example, to replace ``decimals`` with ``decimals_as_str`` just call
46+
``std_ruleset(decimals=decimals_as_str)``
4547
"""
4648
return custom(
4749
enums,
@@ -59,5 +61,5 @@ def std_ruleset(
5961
stringify_keys,
6062
unions,
6163
*extras,
62-
cache=cache,
64+
cache=cache
6365
)

json_syntax/attrs.py

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
)
1111
from . import pattern as pat
1212
from .product import build_attribute_map, build_named_tuple_map, build_typed_dict_map
13+
from .types import is_generic, get_origin, get_argument_map
1314

1415
from functools import partial
1516

@@ -47,7 +48,13 @@ def attrs_classes(
4748
"""
4849
if verb not in _SUPPORTED_VERBS:
4950
return
50-
inner_map = build_attribute_map(verb, typ, ctx)
51+
if is_generic(typ):
52+
typ_args = get_argument_map(typ)
53+
typ = get_origin(typ)
54+
else:
55+
typ_args = None
56+
57+
inner_map = build_attribute_map(verb, typ, ctx, typ_args)
5158
if inner_map is None:
5259
return
5360

@@ -115,11 +122,11 @@ def named_tuples(verb, typ, ctx):
115122

116123
def typed_dicts(verb, typ, ctx):
117124
"""
118-
Handle the TypedDict product type. This allows you to construct a dict with specific (string) keys, which
119-
is often how people really use dicts.
125+
Handle the TypedDict product type. This allows you to construct a dict with specific
126+
(string) keys, which is often how people really use dicts.
120127
121-
Both the class form and the functional form, ``TypedDict('Name', {'field': type, 'field': type})`` are
122-
supported.
128+
Both the class form and the functional form,
129+
``TypedDict('Name', {'field': type, 'field': type})`` are supported.
123130
"""
124131
if verb not in _SUPPORTED_VERBS:
125132
return

json_syntax/cache.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,8 @@ def complete(self, verb, typ, action):
103103

104104
class ThreadLocalCache(SimpleCache):
105105
"""
106-
Avoids threads conflicting while looking up rules by keeping the cache in thread local storage.
106+
Avoids threads conflicting while looking up rules by keeping the cache in thread local
107+
storage.
107108
108109
You can also prevent this by looking up rules during module loading.
109110
"""

json_syntax/errors.py

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
class _Context:
2+
"""
3+
Stash contextual information in an exception. As we don't know exactly when an exception
4+
is displayed to a user, this class tries to keep it always up to date.
5+
6+
This class subclasses string (to be compatible) and tracks an insertion point.
7+
"""
8+
9+
__slots__ = ("original", "context", "lead")
10+
11+
def __init__(self, original, lead, context):
12+
self.original = original
13+
self.lead = lead
14+
self.context = [context]
15+
16+
def __str__(self):
17+
return "{}{}{}".format(
18+
self.original, self.lead, "".join(map(str, reversed(self.context)))
19+
)
20+
21+
def __repr__(self):
22+
return repr(self.__str__())
23+
24+
@classmethod
25+
def add(cls, exc, context):
26+
args = exc.args
27+
if args and isinstance(args[0], cls):
28+
args[0].context.append(context)
29+
return
30+
args = list(exc.args)
31+
if args:
32+
args[0] = cls(args[0], "; at ", context)
33+
else:
34+
args.append(cls("", "At ", context))
35+
exc.args = tuple(args)
36+
37+
38+
class ErrorContext:
39+
"""
40+
Inject contextual information into an exception message. This won't work for some
41+
exceptions like OSError that ignore changes to `args`; likely not an issue for this
42+
library. There is a neglible performance hit if there is no exception.
43+
44+
>>> with ErrorContext('.foo'):
45+
... with ErrorContext('[0]'):
46+
... with ErrorContext('.qux'):
47+
... 1 / 0
48+
Traceback (most recent call last):
49+
ZeroDivisionError: division by zero; at .foo[0].qux
50+
51+
The `__exit__` method will catch the exception and look for a `_context` attribute
52+
assigned to it. If none exists, it appends `; at ` and the context string to the first
53+
string argument.
54+
55+
As the exception walks up the stack, outer ErrorContexts will be called. They will see
56+
the `_context` attribute and insert their context immediately after `; at ` and before
57+
the existing context.
58+
59+
Thus, in the example above:
60+
61+
('division by zero',) -- the original message
62+
('division by zero; at .qux',) -- the innermost context
63+
('division by zero; at [0].qux',)
64+
('division by zero; at .foo[0].qux',) -- the outermost context
65+
66+
For simplicity, the method doesn't attempt to inject whitespace. To represent names,
67+
consider surrounding them with angle brackets, e.g. `<Class>`
68+
"""
69+
70+
def __init__(self, *context):
71+
self.context = context
72+
73+
def __enter__(self):
74+
pass
75+
76+
def __exit__(self, exc_type, exc_value, traceback):
77+
if exc_value is not None:
78+
_Context.add(exc_value, "".join(self.context))
79+
80+
81+
def err_ctx(context, func):
82+
"""
83+
Execute a callable, decorating exceptions raised with error context.
84+
85+
``err_ctx(context, func)`` has the same effect as:
86+
87+
with ErrorContext(context):
88+
return func()
89+
"""
90+
try:
91+
return func()
92+
except Exception as exc:
93+
_Context.add(exc, context)
94+
raise

0 commit comments

Comments
 (0)