Skip to content

Attempt to fix distribute Tuple issue #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 30, 2025
Merged

Conversation

VallinZ
Copy link
Owner

@VallinZ VallinZ commented Apr 24, 2025

This PR is an attempt to fix issue python#18922 from the myPy repo.
Issue link: python#18922 (comment)

The approach addresses the bug report by explicitly distributing union types across tuple elements when a TupleType is compared against a UnionType. It first checks whether any elements of the tuple are themselves union types, and if so, computes the Cartesian product of all combinations of those element types. For each combination, it constructs a new TupleType representing one possible instantiation of the original tuple. These are then combined into a single UnionType using make_simplified_union, effectively transforming something like Tuple[float, Optional[float]] into Union[Tuple[float, float], Tuple[float, None]]. Finally, this expanded union is compared to the right-hand side type for subtyping.

This approaches successful identify there is no issue with:

from typing import Union, Tuple, Optional

def id(x: Tuple[float, Optional[float]]) -> Union[Tuple[float, float], Tuple[float, None]]:
    return x

and identify there is an issue with:

from typing import Union, Tuple, Optional

def id(x: Tuple[float, Optional[float]]) -> Union[Tuple[float, str], Tuple[float, None]]:
    return x

However, this approach leads to an maximum recursive depth error when running on recursive data structure, more specifically on the test testRecursiveDoubleUnionNoCrash in check-recursive-types.test.

This comment has been minimized.

This comment has been minimized.

@VallinZ VallinZ changed the title Initial approach for distribute Tuple issue Attempt to fix distribute Tuple issue Apr 24, 2025

This comment has been minimized.

This comment has been minimized.

Copy link

Diff from mypy_primer, showing the effect of this PR on open source code:

pydantic (https://github.com/pydantic/pydantic)
+ pydantic/json_schema.py:2478: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
+ https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
+ Please report a bug at https://github.com/python/mypy/issues
+ version: 1.16.0+dev.b59e45bc5a965f6365c79026a0f80a1708e2ec59
+ pydantic/json_schema.py:2478: : note: use --pdb to drop into pdb
- pydantic/errors.py:128: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/_internal/_utils.py:182: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/_internal/_validators.py:134: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/_internal/_mock_val_ser.py:137: error: Incompatible types in assignment (expression has type "MockValSer[SchemaValidator | PluggableSchemaValidator | SchemaSerializer]", variable has type "SchemaValidator | PluggableSchemaValidator")  [assignment]
- pydantic/_internal/_mock_val_ser.py:143: error: Incompatible types in assignment (expression has type "MockValSer[SchemaSerializer]", variable has type "SchemaSerializer")  [assignment]
- pydantic/_internal/_generics.py:63: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/_internal/_generics.py:223: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/_internal/_decorators.py:300: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/_internal/_decorators.py:753: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/_internal/_config.py:142: error: TypedDict key must be a string literal; expected one of ("title", "model_title_generator", "field_title_generator", "str_to_lower", "str_to_upper", ...)  [literal-required]
- pydantic/json_schema.py:451: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/json_schema.py:1961: error: Unused "type: ignore" comment  [unused-ignore]
+ Traceback (most recent call last):
+   File "", line 8, in <module>
+     sys.exit(console_entry())
+   File "/__main__.py", line 15, in console_entry
+     main()
+   File "/main.py", line 127, in main
+     res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
+   File "/main.py", line 211, in run_build
+     res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
+   File "/build.py", line 191, in build
+     result = _build(
+   File "/build.py", line 267, in _build
+     graph = dispatch(sources, manager, stdout)
+   File "/build.py", line 2939, in dispatch
+     process_graph(graph, manager)
+   File "/build.py", line 3337, in process_graph
+     process_stale_scc(graph, scc, manager)
+   File "/build.py", line 3438, in process_stale_scc
+     graph[id].type_check_first_pass()
+   File "/build.py", line 2311, in type_check_first_pass
+     self.type_checker().check_first_pass()
+   File "/checker.py", line 472, in check_first_pass
+     self.accept(d)
+   File "/checker.py", line 579, in accept
+     stmt.accept(self)
+   File "/nodes.py", line 813, in accept
+     return visitor.visit_func_def(self)
+   File "/checker.py", line 1129, in visit_func_def
+     self._visit_func_def(defn)
+   File "/checker.py", line 1133, in _visit_func_def
+     self.check_func_item(defn, name=defn.name)
+   File "/checker.py", line 1168, in check_func_item
+     self.check_func_def(defn, typ, name, allow_empty)
+   File "/checker.py", line 1455, in check_func_def
+     self.accept(item.body)
+   File "/checker.py", line 579, in accept
+     stmt.accept(self)
+   File "/nodes.py", line 1281, in accept
+     return visitor.visit_block(self)
+   File "/checker.py", line 3025, in visit_block
+     self.accept(s)
+   File "/checker.py", line 579, in accept
+     stmt.accept(self)
+   File "/nodes.py", line 1549, in accept
+     return visitor.visit_if_stmt(self)
+   File "/checker.py", line 4846, in visit_if_stmt
+     self.accept(b)
+   File "/checker.py", line 579, in accept
+     stmt.accept(self)
+   File "/nodes.py", line 1281, in accept
+     return visitor.visit_block(self)
+   File "/checker.py", line 3025, in visit_block
+     self.accept(s)
+   File "/checker.py", line 579, in accept
+     stmt.accept(self)
+   File "/nodes.py", line 1477, in accept
+     return visitor.visit_return_stmt(self)
+   File "/checker.py", line 4724, in visit_return_stmt
+     self.check_return_stmt(s)
+   File "/checker.py", line 4760, in check_return_stmt
+     self.expr_checker.accept(
+   File "/checkexpr.py", line 5970, in accept
+     typ = node.accept(self)
+   File "/nodes.py", line 1990, in accept
+     return visitor.visit_call_expr(self)
+            ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
+   File "/checkexpr.py", line 488, in visit_call_expr
+     return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
+            ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+   File "/checkexpr.py", line 622, in visit_call_expr_inner
+     ret_type = self.check_call_expr_with_callee_type(
+         callee_type, e, fullname, object_type, member
+     )
+   File "/checkexpr.py", line 1478, in check_call_expr_with_callee_type
+     ret_type, callee_type = self.check_call(
+                             ~~~~~~~~~~~~~~~^
+         callee_type,
+         ^^^^^^^^^^^^
+     ...<6 lines>...
+         object_type=object_type,
+         ^^^^^^^^^^^^^^^^^^^^^^^^
+     )
+     ^
+   File "/checkexpr.py", line 1580, in check_call
+     return self.check_callable_call(
+            ~~~~~~~~~~~~~~~~~~~~~~~~^
+         callee,
+         ^^^^^^^
+     ...<6 lines>...
+         object_type,
+         ^^^^^^^^^^^^
+     )
+     ^
+   File "/checkexpr.py", line 1752, in check_callable_call
+     callee = self.infer_function_type_arguments_using_context(callee, context)
+   File "/checkexpr.py", line 2042, in infer_function_type_arguments_using_context
+     args = infer_type_arguments(
+         callable.variables, ret_type, erased_ctx, skip_unsatisfied=True
+     )
+   File "/infer.py", line 75, in infer_type_arguments
+     constraints = infer_constraints(template, actual, SUPERTYPE_OF if is_supertype else SUBTYPE_OF)
+   File "/constraints.py", line 317, in infer_constraints
+     res = _infer_constraints(template, actual, direction, skip_neg_op)
+   File "/constraints.py", line 336, in _infer_constraints
+     actual = mypy.typeops.make_simplified_union(actual.items, keep_erased=True)
+   File "/typeops.py", line 622, in make_simplified_union
+     simplified_set: Sequence[Type] = _remove_redundant_union_items(items, keep_erased)
+                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
+   File "/typeops.py", line 708, in _remove_redundant_union_items
+     if is_proper_subtype(
+        ~~~~~~~~~~~~~~~~~^
+         ti, tj, keep_erased_types=keep_erased, ignore_promotions=True
+         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+     ):
+     ^
+   File "/subtypes.py", line 227, in is_proper_subtype
+     return _is_subtype(left, right, subtype_context, proper_subtype=True)
+   File "/subtypes.py", line 395, in _is_subtype
+     return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
+            ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+   File "/types.py", line 1475, in accept
+     return visitor.visit_instance(self)
+            ~~~~~~~~~~~~~~~~~~~~~~^^^^^^
+   File "/subtypes.py", line 622, in visit_instance
+     if not check_type_parameter(
+            ~~~~~~~~~~~~~~~~~~~~^
+         lefta, righta, variance, self.proper_subtype, self.subtype_context
+         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+     ):
+     ^
+   File "/subtypes.py", line 412, in check_type_parameter
+     return is_proper_subtype(left, right, subtype_context=subtype_context)
+   File "/subtypes.py", line 227, in is_proper_subtype
+     return _is_subtype(left, right, subtype_context, proper_subtype=True)
+   File "/subtypes.py", line 342, in _is_subtype
+     _is_subtype(t, right, subtype_context, proper_subtype=False)
+     ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+   File "/subtypes.py", line 307, in _is_subtype
+     subtype_context.check_context(proper_subtype)
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
+   File "/subtypes.py", line 120, in check_context
+     assert not self.erase_instances and not self.keep_erased_types
+            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+ AssertionError: 
- pydantic/json_schema.py:2056: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/json_schema.py:2698: error: Missing return statement  [return]
- pydantic/_internal/_fields.py:68: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/_internal/_fields.py:153: error: Incompatible types in assignment (expression has type "tuple[str]", variable has type "tuple[()]")  [assignment]
- pydantic/_internal/_fields.py:156: error: Incompatible types in assignment (expression has type "tuple[str]", variable has type "tuple[()]")  [assignment]
- pydantic/types.py:230: error: Incompatible return value type (got "<typing special form>", expected "type[int]")  [return-value]
- pydantic/types.py:491: error: Incompatible return value type (got "<typing special form>", expected "type[float]")  [return-value]
- pydantic/types.py:679: error: Incompatible return value type (got "<typing special form>", expected "type[bytes]")  [return-value]
- pydantic/types.py:817: error: Incompatible return value type (got "<typing special form>", expected "type[str]")  [return-value]
- pydantic/types.py:852: error: Incompatible return value type (got "<typing special form>", expected "type[set[HashableItemType]]")  [return-value]
- pydantic/types.py:868: error: Incompatible return value type (got "<typing special form>", expected "type[frozenset[HashableItemType]]")  [return-value]
- pydantic/types.py:903: error: Incompatible return value type (got "<typing special form>", expected "type[list[AnyItemType]]")  [return-value]
- pydantic/types.py:1125: error: Incompatible return value type (got "<typing special form>", expected "type[Decimal]")  [return-value]
- pydantic/types.py:1704: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/types.py:1736: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/types.py:2262: error: Incompatible return value type (got "<typing special form>", expected "type[date]")  [return-value]
- pydantic/types.py:2974: error: "Mapping[str, Any]" has no attribute "setdefault"  [attr-defined]
- pydantic/color.py:104: error: Need type annotation for "field_schema" (hint: "field_schema: dict[<type>, <type>] = ...")  [var-annotated]
- pydantic/_internal/_known_annotated_metadata.py:191: error: "Mapping[str, Any]" has no attribute "copy"  [attr-defined]
- pydantic/_internal/_known_annotated_metadata.py:222: error: Unsupported target for indexed assignment ("Mapping[str, Any]")  [index]
- pydantic/_internal/_known_annotated_metadata.py:222: error: Unsupported target for indexed assignment ("Mapping[str, Any] | AfterValidatorFunctionSchema")  [index]
- pydantic/_internal/_known_annotated_metadata.py:222: error: TypedDict key must be a string literal; expected one of ("function", "schema", "ref", "metadata", "serialization", ...)  [literal-required]
- pydantic/_internal/_known_annotated_metadata.py:255: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/_internal/_known_annotated_metadata.py:309: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/_internal/_known_annotated_metadata.py:315: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/fields.py:492: error: Unused "type: ignore" comment  [unused-ignore]
- pydantic/fields.py:560: error: No overload variant of "Field" matches argument types "Any", "Any", "bool", "dict[Any, Any]"  [call-overload]
- pydantic/fields.py:560: note: Possible overload variants:
- pydantic/fields.py:560: note:     def Field(default, default: EllipsisType, *, alias: str | None = ..., alias_priority: int | None = ..., validation_alias: str | AliasPath | AliasChoices | None = ..., serialization_alias: str | None = ..., title: str | None = ..., field_title_generator: Callable[[str, FieldInfo], str] | None = ..., description: str | None = ..., examples: list[Any] | None = ..., exclude: bool | None = ..., discriminator: str | Discriminator | None = ..., deprecated: deprecated | str | bool | None = ..., json_schema_extra: JsonDict | Callable[[JsonDict], None] | None = ..., frozen: bool | None = ..., validate_default: bool | None = ..., repr: bool = ..., init: bool | None = ..., init_var: bool | None = ..., kw_only: bool | None = ..., pattern: str | Pattern[str] | None = ..., strict: bool | None = ..., coerce_numbers_to_str: bool | None = ..., gt: SupportsGt | None = ..., ge: SupportsGe | None = ..., lt: SupportsLt | None = ..., le: SupportsLe | None = ..., multiple_of: float | None = ..., allow_inf_nan: bool | None = ..., max_digits: int | None = ..., decimal_places: int | None = ..., min_length: int | None = ..., max_length: int | None = ..., union_mode: Literal['smart', 'left_to_right'] = ..., fail_fast: bool | None = ...) -> Any
- pydantic/fields.py:560: note:     def [_T] Field(default, default: _T, *, alias: str | None = ..., alias_priority: int | None = ..., validation_alias: str | AliasPath | AliasChoices | None = ..., serialization_alias: str | None = ..., title: str | None = ..., field_title_generator: Callable[[str, FieldInfo], str] | None = ..., description: str | None = ..., examples: list[Any] | None = ..., exclude: bool | None = ..., discriminator: str | Discriminator | None = ..., deprecated: deprecated | str | bool | None = ..., json_schema_extra: JsonDict | Callable[[JsonDict], None] | None = ..., frozen: bool | None = ..., validate_default: bool | None = ..., repr: bool = ..., init: bool | None = ..., init_var: bool | None = ..., kw_only: bool | None = ..., pattern: str | Pattern[str] | None = ..., strict: bool | None = ..., coerce_numbers_to_str: bool | None = ..., gt: SupportsGt | None = ..., ge: SupportsGe | None = ..., lt: SupportsLt | None = ..., le: SupportsLe | None = ..., multiple_of: float | None = ..., allow_inf_nan: bool | None = ..., max_digits: int | None = ..., decimal_places: int | None = ..., min_length: int | None = ..., max_length: int | None = ..., union_mode: Literal['smart', 'left_to_right'] = ..., fail_fast: bool | None = ...) -> _T
- pydantic/fields.py:560: note:     def [_T] Field(default_factory, *, default_factory: Callable[[], _T] | Callable[[dict[str, Any]], _T], alias: str | None = ..., alias_priority: int | None = ..., validation_alias: str | AliasPath | AliasChoices | None = ..., serialization_alias: str | None = ..., title: str | None = ..., field_title_generator: Callable[[str, FieldInfo], str] | None = ..., description: str | None = ..., examples: list[Any] | None = ..., exclude: bool | None = ..., discriminator: str | Discriminator | None = ..., deprecated: deprecated | str | bool | None = ..., json_schema_extra: JsonDict | Callable[[JsonDict], None] | None = ..., frozen: bool | None = ..., validate_default: bool | None = ..., repr: bool = ..., init: bool | None = ..., init_var: bool | None = ..., kw_only: bool | None = ..., pattern: str | Pattern[str] | None = ..., strict: bool | None = ..., coerce_numbers_to_str: bool | None = ..., gt: SupportsGt | None = ..., ge: SupportsGe | None = ..., lt: SupportsLt | None = ..., le: SupportsLe | None = ..., multiple_of: float | None = ..., allow_inf_nan: bool | None = ..., max_digits: int | None = ..., decimal_places: int | None = ..., min_length: int | None = ..., max_length: int | None = ..., union_mode: Literal['smart', 'left_to_right'] = ..., fail_fast: bool | None = ...) -> _T

... (truncated 186 lines) ...

Copy link
Collaborator

@ChristinaTrinh ChristinaTrinh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe create a new issue for the maximum recursive depth error so that someone could work on it in the future

@ChristinaTrinh ChristinaTrinh merged commit 442d8c5 into master Apr 30, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants