Merge pull request #6 from UnitedIncome/property-tests

Property tests

Author: bsamuel-ui

.gitignore

@@ -3,6 +3,7 @@
 /dist/
 /.python-version
 /.coverage
+/.hypothesis
 /htmlcov
 /pip-wheel-metadata
 setup.py

README.md

@@ -26,8 +26,8 @@ structure using libraries like [attrs][].
  * The library has no dependencies of its own
     * It does not actually read or write JSON
 
-At the time of writing, the library is in **alpha** and the API may move around or be
-renamed.
+At the time of writing, the library is in **beta** and the API is relatively stable but
+may change.
 
 ### Supported types
 
@@ -219,7 +219,7 @@ During encoding, the reverse sequence takes place:
 
 #### JSON type check hook
 
-Type checks are only used in `json-syntax` to support `typing.Union`; in a nutshell, the
+Type checks are only used in _json-syntax_ to support `typing.Union`; in a nutshell, the
 `unions` rule will inspect some JSON to see which variant is present.
 
 If a type-check hook is not defined, `__json_pre_decode__` will be called before the
@@ -249,9 +249,40 @@ encode_account = rules.lookup(typ=Union[AccountA, AccountB, AccountC],
 
 See [the examples][] for details on custom rules.
 
+### Debugging amibguous structures
+
+(May need more docs and some test cases.)
+
+As _json-syntax_ tries to directly translate your Python types to JSON, it is possible
+to write ambiguous structures. To avoid this, there is a handy `is_ambiguous` method:
+
+```python
+# This is true because both are represented as an array of numbers in JSON.
+rules.is_ambiguous(typ=Union[List[int], Set[int]])
+
+@dataclass
+class Account:
+    user: str
+    address: str
+
+# This is true because such a dictionary would always match the contents of the account.
+rules.is_ambiguous(typ=Union[Dict[str, str], Account])
+```
+
+The aim of this is to let you put a check in your unit tests to make sure data can be
+reliably expressed given your particular case.
+
+Internally, this is using the `PATTERN` verb to represent the JSON pattern, so this may
+be helpful in understanding how _json-syntax_ is trying to represent your data:
+
+```python
+print(rules.lookup(typ=MyAmbiguousClass, verb='show_pattern'))
+```
+
 ### Sharp edges
 
-_Alpha release status._ This API may change, there are probably bugs!
+_Beta release status._ This API may change, there are probably bugs! In particular, the
+status of rules accepting subclasses is likely to change.
 
 _The RuleSet caches encoders._ Construct a new ruleset if you want to change settings.
 
@@ -265,28 +296,8 @@ _Everything to do with typing._ It's a bit magical and sort of wasn't designed f
 [We have a guide to it to try and help][types].
 
 _Union types._ You can use `typing.Union` to allow a member to be one of some number of
-alternates, but there are some caveats. These are documented in code in `test_unions`,
-but in plain English:
-
-When encoding Python to JSON:
-
- * `Union[Super, Sub]` will never match Sub when converting from Python to JSON.
-
-When decoding JSON to Python:
-
- * `Union[str, Stringly]` will never construct an instance that is represented as a
-   string in JSON.
-    * This includes enums, dates and special float values (`Nan`, `-inf`, etc.) may be
-      represented as strings.
- * `Union[datetime, date]` will never construct a date because `YYYY-MM-DD` is a valid
-   datetime according to ISO8601.
- * `Union[Dict[str, Value], MyAttrs]` will never construct `MyAttrs` if all its
-   attributes are `Value`.
- * `Union[List[X], Set[X], FrozenSet[X], Tuple[X, ...]]` will only ever construct
-   `List[X]` because all the others are also represented as JSON lists.
- * `Union[MyClassA, MyClassB, MyClassC]` can be ambiguous if these classes all share
-   common fields. Consider using the `__json_check__` hook to differentiate. Simply
-   adding a field named `class` or something can be unambiguous and fast.
+alternates, but there are some caveats. You should use the `.is_ambiguous()` method of
+RuleSet to warn you of these.
 
 _Rules accept subclasses._ If you subclass `int`, the atoms rule will match it, and then
 the converter will call `int` against your instance. I haven't taken the time to examine
@@ -306,7 +317,7 @@ This package is maintained via the [poetry][] tool. Some useful commands:
 
  1. Setup: `poetry install`
  2. Run tests: `poetry run pytest tests/`
- 3. Reformat: `poetry run black -N json_syntax/ tests/`
+ 3. Reformat: `poetry run black json_syntax/ tests/`
 
 ### Setting up tox
 
@@ -322,6 +333,10 @@ You'll want pyenv, then install the pythons:
 
 Once you install `tox` in your preferred python, running it is just `tox`.
 
+(Caveat: `poetry install` is now breaking in `tox` because `pip` has changed: it now
+tries to create a dist in _pip-wheel-metadata_ each time. I'm nuking that directory, but
+most likely there's some new config variable to hunt down.)
+
 ### Notes
 
 <b id="f1">1</b>: Writing the encoder is deceptively easy because the instances in

json_syntax/__init__.py

@@ -22,7 +22,7 @@
 )
 from .attrs import attrs_classes, named_tuples, tuples
 from .unions import unions
-from .helpers import JSON2PY, PY2JSON, INSP_PY, INSP_JSON  # noqa
+from .helpers import JSON2PY, PY2JSON, INSP_PY, INSP_JSON, PATTERN  # noqa
 
 
 def std_ruleset(
@@ -43,18 +43,18 @@ def std_ruleset(
     For example, to replace ``decimals`` with ``decimals_as_str`` just call ``std_ruleset(decimals=decimals_as_str)``
     """
     return custom(
+        enums,
         atoms,
         floats,
         decimals,
         dates,
         optional,
-        enums,
         lists,
         attrs_classes,
         sets,
-        dicts,
         named_tuples,
         tuples,
+        dicts,
         unions,
         *extras,
         cache=cache,

json_syntax/action_v1.py

@@ -1,7 +1,9 @@
 from .helpers import ErrorContext, err_ctx
 
-from datetime import date, datetime, time
+from datetime import date, datetime, time, timedelta
+from decimal import InvalidOperation
 import math
+import re
 
 
 def check_parse_error(value, parser, error):
@@ -21,6 +23,13 @@ def check_has_type(value, typ):
     return type(value) == typ
 
 
+def convert_decimal_str(value):
+    result = str(value)
+    if result == "sNaN":
+        raise InvalidOperation("Won't save signalling NaN")
+    return result
+
+
 def convert_float(value):
     value = float(value)
     if math.isfinite(value):
@@ -74,6 +83,56 @@ def convert_str_enum(value, mapping):
     del instance
 
 
+def convert_timedelta_str(dur):
+    "Barebones support for storing a timedelta as an ISO8601 duration."
+    micro = ".{:06d}".format(dur.microseconds) if dur.microseconds else ""
+    return "P{:d}DT{:d}{}S".format(dur.days, dur.seconds, micro)
+
+
+_iso8601_duration = re.compile(
+    r"^P(?!$)([-+]?\d+(?:[.,]\d+)?Y)?"
+    r"([-+]?\d+(?:[.,]\d+)?M)?"
+    r"([-+]?\d+(?:[.,]\d+)?W)?"
+    r"([-+]?\d+(?:[.,]\d+)?D)?"
+    r"(?:(T)(?=[0-9+-])"
+    r"([-+]?\d+(?:[.,]\d+)?H)?"
+    r"([-+]?\d+(?:[.,]\d+)?M)?"
+    r"([-+]?\d+(?:[.,]\d+)?S)?)?$"
+)
+_duration_args = {
+    "PW": "weeks",
+    "PD": "days",
+    "TH": "hours",
+    "TM": "minutes",
+    "TS": "seconds",
+}
+
+
+def convert_str_timedelta(dur):
+    if not isinstance(dur, str):
+        raise ValueError("Value was not a string.")
+    match = _iso8601_duration.match(dur.upper().replace(",", "."))
+    section = "P"
+    if not match:
+        raise ValueError("Value was not an ISO8601 duration.")
+    args = {}
+    for elem in match.groups():
+        if elem is None:
+            continue
+        if elem == "T":
+            section = "T"
+            continue
+        part = section + elem[-1]
+        value = float(elem[:-1])
+        if not value:
+            continue
+
+        if part in ("PY", "PM"):
+            raise ValueError("Year and month durations not supported")
+        args[_duration_args[part]] = value
+    return timedelta(**args)
+
+
 def convert_optional(value, inner):
     if value is None:
         return None

json_syntax/attrs.py

@@ -3,6 +3,7 @@
     PY2JSON,
     INSP_JSON,
     INSP_PY,
+    PATTERN,
     SENTINEL,
     has_origin,
     identity,
@@ -18,6 +19,7 @@
     convert_dict_to_attrs,
     convert_tuple_as_list,
 )
+from . import pattern as pat
 
 from functools import partial
 
@@ -33,7 +35,7 @@ def attrs_classes(
     """
     Handle an ``@attr.s`` or ``@dataclass`` decorated class.
     """
-    if verb not in (JSON2PY, PY2JSON, INSP_PY, INSP_JSON):
+    if verb not in (JSON2PY, PY2JSON, INSP_PY, INSP_JSON, PATTERN):
         return
     try:
         fields = typ.__attrs_attrs__
@@ -59,7 +61,7 @@ def attrs_classes(
             )
             if verb == PY2JSON:
                 tup += (field.default,)
-            elif verb == INSP_JSON:
+            elif verb in (INSP_JSON, PATTERN):
                 tup += (is_attrs_field_required(field),)
             inner_map.append(tup)
 
@@ -82,6 +84,12 @@ def attrs_classes(
             return check
         pre_hook_method = getattr(typ, pre_hook, identity)
         return partial(check_dict, inner_map=inner_map, pre_hook=pre_hook_method)
+    elif verb == PATTERN:
+        return pat.Object.exact(
+            (pat.String.exact(name), inner or pat.Unkown)
+            for name, inner, req in inner_map
+            if req
+        )
 
 
 def named_tuples(verb, typ, ctx):
@@ -90,7 +98,9 @@ def named_tuples(verb, typ, ctx):
 
     Also handles a ``collections.namedtuple`` if you have a fallback handler.
     """
-    if verb not in (JSON2PY, PY2JSON, INSP_PY, INSP_JSON) or not issub_safe(typ, tuple):
+    if verb not in (JSON2PY, PY2JSON, INSP_PY, INSP_JSON, PATTERN) or not issub_safe(
+        typ, tuple
+    ):
         return
     try:
         fields = typ._field_types
@@ -116,7 +126,7 @@ def named_tuples(verb, typ, ctx):
         )
         if verb == PY2JSON:
             tup += (defaults.get(name, SENTINEL),)
-        elif verb == INSP_JSON:
+        elif verb in (INSP_JSON, PATTERN):
             tup += (name not in defaults,)
         inner_map.append(tup)
 
@@ -133,14 +143,20 @@ def named_tuples(verb, typ, ctx):
         )
     elif verb == INSP_JSON:
         return partial(check_dict, pre_hook=identity, inner_map=tuple(inner_map))
+    elif verb == PATTERN:
+        return pat.Object.exact(
+            (pat.String.exact(name), inner) for name, inner, req in inner_map if req
+        )
 
 
 def tuples(verb, typ, ctx):
     """
     Handle a ``Tuple[type, type, type]`` product type. Use a ``NamedTuple`` if you don't
     want a list.
     """
-    if verb not in (JSON2PY, PY2JSON, INSP_PY, INSP_JSON) or not has_origin(typ, tuple):
+    if verb not in (JSON2PY, PY2JSON, INSP_PY, INSP_JSON, PATTERN) or not has_origin(
+        typ, tuple
+    ):
         return
     args = typ.__args__
     if Ellipsis in args:
@@ -155,3 +171,5 @@ def tuples(verb, typ, ctx):
         return partial(check_tuple_as_list, inner=inner, con=tuple)
     elif verb == INSP_JSON:
         return partial(check_tuple_as_list, inner=inner, con=list)
+    elif verb == PATTERN:
+        return pat.Array.exact(inner)

json_syntax/helpers.py

@@ -9,6 +9,7 @@
 PY2JSON = "python_to_json"
 INSP_JSON = "inspect_json"
 INSP_PY = "inspect_python"
+PATTERN = "show_pattern"
 NoneType = type(None)
 SENTINEL = object()
 python_minor = sys.version_info[:2]