TypeForm: Enable by default - #21262
Conversation
...removing the need to use --enable-incomplete-feature=TypeForm
This comment has been minimized.
This comment has been minimized.
|
|
||
| [case testRecognizesParameterizedTypeFormInAnnotation] | ||
| # flags: --python-version 3.14 --enable-incomplete-feature=TypeForm | ||
| # flags: --python-version 3.15 |
There was a problem hiding this comment.
hm do we already support 3.15 enough for this to make sense? The test uses typing_extensions anyway so I'm not sure why it needs to pin a version at all.
There was a problem hiding this comment.
The test uses typing_extensions anyway so I'm not sure why it needs to pin a version at all.
If it's OK for the test to (A) continue to use typing_extensions, then I agree that the --python-version 3.15 doesn't make a lot of sense.
OTOH, if it would be more conventional in tests to (B) use the typing version, then I think --python-version 3.15 becomes necessary.
I'll take a look in the next few days to see what other tests are doing to choose whether to do (A) or (B).
There was a problem hiding this comment.
We don't run 3.15 in CI, and may potentially not run it for another few months, so we should use 3.14 for now. It will be easy to mass update these tests to typing in the future (as we did with other type system features).
This comment has been minimized.
This comment has been minimized.
|
According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
ilevkivskyi
left a comment
There was a problem hiding this comment.
Thanks! It is probably better to put this in v2.1 to be safe. I will leave this up to @JukkaL
There was a problem hiding this comment.
In #20946, I kept the --python-version 3.14 flags, maybe it makes sense to keep them?
|
We'll likely have mypy 2.1 out pretty soon afterwards, so we can wait for 2.1. |
|
Hm, our daily benchmark shows this PR caused 2% slow-down on self-check. Taking into account mypy itself doesn't use I guess the slow-down is because we call I think we should only interpret an expression as type whenever a |
|
I’ll take a look at the performance on selfcheck in the next few days. The TypeForm-related code already has “cache-hit” / “cache-miss” style performance counters plus a script to interpret them, which should be useful for investigation. |
|
Btw, I briefly looked at the current I don't know how hard it is, but I think the optimal way would be to fix the
For the last items you don't need to do an extra visit, simply record the presence of |
|
Don't know how much it will help but I noticed an easy opportunity to get rid of one of the two regexes: #21459. Also looking into the second one a bit to see if we can get a better heuristic. |
|
Rearranged the code a bit more based on some observations about strings appearing in mypy itself. If the regression is really driven by the string matching in |
To be clear, I don't think it is the main factor, but likely a significant factor. I think it may be necessary to do most of the things I mentioned above to make the regression go away ~completely. |
IIRC the only reason there’s logic in the semantic analyzer (and not just only in the TypeChecker) is because string literals are sometimes type annotations and TypeChecker doesn’t have enough info to resolve all references that could occur in a string context. In particular locally defined classes are thrown out by the time TypeChecker gets to look at an expression. I’m still tracking the task of optimizing performance related to this thread. Last week I was finishing up items for work as I was transitioning out into parental leave. Hopefully I should have some more bandwidth over the next few days. |
Btw if someone can benchmark this PR on something unrelated, like |
|
Update: Detailed profiling on the slowdown reveals some insights:
Therefore I'm adding some more-aggressive quick-reject heuristics. I've already recovered about 50% of the slowdown so far - when measured on the mypy codebase. Next steps:
|
|
Update: I've done about as much optimizing for the mypy codebase as I can. Overall I've reduced the +2.07% in runtime to +1.10% when using In the meantime I'll press on to see what level of runtime increase I see on other non-trivial Python codebases (black and pytorch) to see if it is less, as suggested by @ilevkivskyi. If non-mypy slowdown is notably less then the change is more likely to be acceptable. I am considering adding a flag to mypy that disables support for TypeForms that include a string literal ( |
OK, as I mentioned above +1% is acceptable.
I don't think this is needed. Mypy has already too many flags. Some performance penalty is OK, this is a non-trivial feature after all. Performance penalty will be even smaller when we will make native parser the default parser (it is almost twice faster than default Python parser, which is still used for |
|
Benchmarks are hard. I've had to revisit a lot of the performance numbers I've been computing because of large variance in results. Consequently I've been spending the last several days extending perf_compare.py locally with additional statistical methods† to get higher-confidence numbers to ensure my performance optimizations are valid. I'll keep working at it... † Hello pairwise analysis (to cancel common mode noise); CPU time vs. wall-clock time (to minimize CPU throttling and core migration effects); worker count 1 (to avoid adding variance of CPU noise together); etc |
|
PR posted to address the regression: #21585 |
## Mypy 2.2
### Support for Closed TypedDicts (PEP 728)
Mypy now supports closed TypedDicts as specified in PEP 728. A closed TypedDict cannot have extra
keys beyond those explicitly defined. This allows the type checker to determine that certain
operations are safe when they otherwise wouldn't be due to the potential presence of unknown keys.
You can use the `closed` keyword argument with `TypedDict`:
```python
HasName = TypedDict("HasName", {"name": str})
HasOnlyName = TypedDict("HasOnlyName", {"name": str}, closed=True)
Movie = TypedDict("Movie", {"name": str, "year": int})
movie: Movie = {"name": "Nimona", "year": 2023}
has_name: HasName = movie # OK: HasName is open (default)
has_only_name: HasOnlyName = movie # Error: HasOnlyName is closed and Movie has extra "year" key
```
Closed TypedDicts enable more precise type checking because the type checker knows exactly which
keys are present. This is particularly useful when working with TypedDict unions or when you want
to ensure that a TypedDict conforms to an exact shape.
The `closed` keyword also enables safe type narrowing with `in` checks:
```python
Book = TypedDict('Book', {'book': str}, closed=True)
DVD = TypedDict('DVD', {'dvd': str}, closed=True)
type Inventory = Book | DVD
def print_type(inventory: Inventory) -> None:
if "book" in inventory:
# Type is narrowed to Book here - safe because DVD is closed
print(inventory["book"])
else:
# Type is narrowed to DVD here
print(inventory["dvd"])
```
The `closed` keyword is also supported in class-based syntax:
```python
class HasOnlyName(TypedDict, closed=True):
name: str
```
Note that closed TypedDicts are structural types, so a closed TypedDict is assignable to an open
TypedDict with the same keys, but not vice versa.
Contributed by Alice (PR [21382](python/mypy#21382)).
### Complete Support for Type Variable Defaults (PEP 696)
Mypy now has complete support for type variable defaults as specified in PEP 696. This allows you to
specify default values for type parameters in generic classes, functions, and type aliases.
Traditional syntax (Python 3.11 and earlier):
```python
T = TypeVar("T", default=int) # This means that if no type is specified T = int
@DataClass
class Box(Generic[T]):
value: T | None = None
reveal_type(Box()) # type is Box[int]
reveal_type(Box(value="Hello World!")) # type is Box[str]
```
New syntax (Python 3.12+):
```python
class Box[T = int]:
def __init__(self, value: T) -> None:
self.value = value
reveal_type(Box()) # type is Box[int]
reveal_type(Box(value="Hello World!")) # type is Box[str]
```
Type variable defaults work with all forms of generics, including classes, functions, and type aliases.
This release completes the implementation by fixing various edge cases involving recursive defaults,
dependencies between type variables, and interactions with variadic generics.
Contributed by Ivan Levkivskyi (PRs [21491](python/mypy#21491),
[21526](python/mypy#21526), [21544](python/mypy#21544)).
### Respect Explicit Return Type of `__new__()`
Mypy now respects explicitly annotated return types in `__new__()` methods. Previously, mypy would
always assume that `__new__()` returns an instance of the current class, ignoring explicit annotations.
With this change, if you explicitly annotate a return type that differs from the implicit type, mypy
will use the explicit annotation:
```python
class Factory:
def __new__(cls) -> Product:
return Product()
reveal_type(Factory()) # type is Product, not Factory
```
Note that mypy still gives an error at the definition site if the explicit annotation is not a
subtype of the current class, since this is technically not type-safe.
For backwards compatibility, there are two exceptions:
- If the return type is `Any`, mypy will still use the current class as the return type.
- If the explicit return type comes from a superclass and is a supertype of the implicit return type,
mypy will use the implicit (more specific) type:
```python
class A:
def __new__(cls) -> A: ...
reveal_type(A()) # type is A
class B:
def __new__(cls) -> B:
return cls()
class C(B): ...
reveal_type(C()) # type is C
```
This fixes several long-standing issues where explicit `__new__()` return types were ignored.
Contributed by Ivan Levkivskyi (PR [21441](python/mypy#21441)).
### TypeForm Support No Longer Experimental
Support for `TypeForm` is no longer experimental. `TypeForm` (introduced in Python 3.14) allows you
to annotate parameters that accept type expressions, providing better type checking for functions
that work with types as values.
```python
from typing import TypeForm
def make_list(tp: TypeForm[T]) -> list[T]:
...
# Correctly typed as list[int]
int_list = make_list(int)
```
`TypeForm` support was previously reverted from mypy 2.1 due to a performance regression, but this
has now been mitigated.
Contributed by Ivan Levkivskyi and Jelle Zijlstra (PRs [21262](python/mypy#21262), [21591](python/mypy#21591),
[21459](python/mypy#21459)).
### Experimental WASM Wheel for Python 3.14
Mypy now ships an experimental WebAssembly (WASM) wheel for Python 3.14. This allows mypy to run
in WASM environments such as Pyodide and browser-based Python implementations.
The WASM wheel is considered experimental and may have limitations compared to native builds. Please
report any issues you encounter when using mypy in WASM environments.
Contributed by Ivan Levkivskyi (PR [21671](python/mypy#21671)).
### Mypyc Free-threading Improvements
- Make function wrappers thread-safe on free-threaded builds (Jukka Lehtosalo, PR [21620](python/mypy#21620))
- Make list remove and index thread-safe on free-threaded builds (Jukka Lehtosalo, PR [21614](python/mypy#21614))
- Fix dict iteration memory safety on free-threaded builds (Jukka Lehtosalo, PR [21617](python/mypy#21617))
- Make some dict primitives thread-safe on free-threading builds (Jukka Lehtosalo, PR [21616](python/mypy#21616))
- Fix free-threading race condition in argument parsing (Jukka Lehtosalo, PR [21613](python/mypy#21613))
- Document free threading and other doc updates (Jukka Lehtosalo, PR [21494](python/mypy#21494))
### `librt.strings` Updates
- Add `librt.strings.toupper` and `librt.strings.tolower` codepoint primitives (Vaggelis Danias, PR [21553](python/mypy#21553))
- Add `librt.strings.isidentifier` codepoint primitive (Vaggelis Danias, PR [21522](python/mypy#21522))
- Add `librt.strings.isalpha` codepoint primitive (Vaggelis Danias, PR [21521](python/mypy#21521))
- Add `librt.strings.isalnum` codepoint primitive (Vaggelis Danias, PR [21509](python/mypy#21509))
- Add `librt.strings.isdigit` codepoint primitive (Vaggelis Danias, PR [21504](python/mypy#21504))
- Add `librt.strings.isspace` char primitive (Vaggelis Danias, PR [21462](python/mypy#21462))
### Mypyc Improvements
- Fix name lookup when class var and module var have the same name (Jukka Lehtosalo, PR [21594](python/mypy#21594))
- Report file and line number on uncaught exceptions (Jukka Lehtosalo, PR [21584](python/mypy#21584))
- Use `other` arg instead of `self` for RHS type (Ryan Heard, PR [21569](python/mypy#21569))
- Use `method_sig` to get the method signature (Ryan Heard, PR [21567](python/mypy#21567))
- Preserve inherited attribute defaults under `separate=True` (Jo, PR [21547](python/mypy#21547))
- Fix missing cross-group header deps in incremental builds (Jo, PR [21490](python/mypy#21490))
- Fix cross-group call to inherited `__mypyc_defaults_setup` (Jo, PR [21481](python/mypy#21481))
- Fix non-deterministic class struct layout under `separate=True` (Vaggelis Danias, PR [21530](python/mypy#21530))
- Specialize `s[i] == 'x'` to a codepoint int compare (Vaggelis Danias, PR [21579](python/mypy#21579))
- Fix reference leak in mypyc bytes concatenation (Colinxu2020, PR [21469](python/mypy#21469))
### Fixes to Crashes
- Fix crash on invalid recursive variadic alias (Ivan Levkivskyi, PR [21572](python/mypy#21572))
- Fix crashes on variadic unpacking in synthetic types (Ivan Levkivskyi, PR [21555](python/mypy#21555))
- Fix crash on unhandled meet variadic tuple vs instance (Ivan Levkivskyi, PR [21558](python/mypy#21558))
- Fix crash on deferred generic class nested in function (Ivan Levkivskyi, PR [21557](python/mypy#21557))
- Fix crash in new-style type alias with variadic unpack (Ivan Levkivskyi, PR [21551](python/mypy#21551))
- Fix various crashes on recursive type variable defaults (Ivan Levkivskyi, PR [21491](python/mypy#21491))
- Fix crash for empty `Annotated` type application (Rayan Salhab, PR [21503](python/mypy#21503))
- Fix crash on `Unpack` used without arguments in class bases (Sai Asish Y, PR [21470](python/mypy#21470))
### Performance Improvements
- Memoize the options snapshot (Kevin Kannammalil, PR [21354](python/mypy#21354))
- Don't include `not_ready_deps` tracking as relating to mypy internals (Kevin Kannammalil, PR [21389](python/mypy#21389))
- Speed up transitive dependency hash for singleton SCCs (Kevin Kannammalil, PR [21390](python/mypy#21390))
- Optimize typeform checks (Jelle Zijlstra, PR [21459](python/mypy#21459))
### Improvements to the Native Parser
- Support `--shadow-file` with `--native-parser` (Jukka Lehtosalo, PR [21623](python/mypy#21623))
- Add Python version checks to native parser (Kevin Kannammalil, PR [21539](python/mypy#21539))
- Allow nativeparse to parse source code directly (bzoracler, PR [21260](python/mypy#21260))
### Other Notable Fixes and Improvements
- Add function definition notes for `too many positional arguments` errors (Kevin Kannammalil, PR [21410](python/mypy#21410))
- Fix the exportjson tool (.ff cache to .json conversion) (Jukka Lehtosalo, PR [21628](python/mypy#21628))
- Support floats in JSON in fixed-format cache (Ivan Levkivskyi, PR [21603](python/mypy#21603))
- Update `TypedDictType.__init__` signature to preserve backward compat (Jukka Lehtosalo, PR [21590](python/mypy#21590))
- Fix constructor calls for union-bounded `TypeVar`s (Jingchen Ye, PR [21571](python/mypy#21571))
- Fix `TypedDict` indexing with literal keys in comprehensions (Jingchen Ye, PR [21556](python/mypy#21556))
- Correctly handle empty tuple index when unpacked (Ivan Levkivskyi, PR [21545](python/mypy#21545))
- Support protocol checks for self-types in tuple types (Ivan Levkivskyi, PR [21535](python/mypy#21535))
- Fix edge cases in variadic tuple subclasses (Ivan Levkivskyi, PR [21518](python/mypy#21518))
- Special-case constructor for tuple types (Ivan Levkivskyi, PR [21502](python/mypy#21502))
- Fix false positive "Expected TypedDict key to be string literal" for `Union[TypedDict, dict[K, V]]` (Zakir Jiwani, PR [21511](python/mypy#21511))
- Use explicit `Never` for type inference (Ivan Levkivskyi, PR [21497](python/mypy#21497))
- Narrow membership in statically known containers (Shantanu, PR [21461](python/mypy#21461))
- Improve negative narrowing for membership checks on tuples (Shantanu, PR [21456](python/mypy#21456))
- Analyze `TypedDict` decorators (Pranav Manglik, PR [21267](python/mypy#21267))
- Start testing Python 3.15 (Marc Mueller, PR [21439](python/mypy#21439))
- Improved handling of `NamedTuple`, `TypedDict`, `Enum`, and regular classes nested in functions (Ivan Levkivskyi, PR [21478](python/mypy#21478))
References #21262. Replaces #21585 and obsoletes #21596. ## Summary Enabling `TypeForm` by default (referenced #21262) made `SemanticAnalyzer.try_parse_as_type_expression` run eagerly on every expression in certain syntactic positions. The cost is concentrated in the expensive full-parse block (`expr_to_analyzed_type` + `isolated_error_analysis`), which **fails ~87% of the time** - pure wasted work. This branch adds early-reject filters that eliminate **74% of full parses (2570 → 666)** on mypy's self-check, recovering **~46% of the regression**: **+1.57% → +0.84%** CPU time. **No new regexes** - per review feedback on replaced #21585. Every filter here is plain string/`isinstance` work, and the two shape tests that were regexes are now helper functions. <details> <summary>Why not do a type-context check?</summary> Review of replaced #21585 suggested skipping the call to `SemanticAnalyzer.try_parse_as_type_expression` entirely when the type context cannot be a `TypeForm`. That optimization already exists, but in the other *type checker* pass at `ExpressionChecker.try_parse_as_type_expression`. The same skip cannot be used in the *semantic analyzer* pass's function because the type context is not yet known. So cheaply filtering the inputs to `SA.try_parse_as_type_expression` is the only remaining (obvious) lever to reduce its runtime contribution. </details> ## Optimization Results CPU time, single worker, paired per-round deltas, n=300: ``` python misc/perf_compare.py --warmup-runs 3 --num-runs 300 -j 3 \ --metric cpu --workers1 \ <TypeForm-disabled-commit> 5bb72b7 <tip-of-this-pr-branch> ``` | Commit | Mean | Median | Δ vs baseline (paired median ±95% CI) | |---|---:|---:|---:| | baseline, `<TypeForm-disabled-commit>` | 2.679 s | 2.676 s | - | | master, `5bb72b788` | 2.720 s | 2.720 s | **+41.9 ms ±2.9 (+1.57%)** | | all filters, `<tip-of-this-pr-branch>` | 2.703 s | 2.699 s | **+22.4 ms ±2.9 (+0.84%)** | The feature branch recovers **19.5 ms of the 41.9 ms regression (~46% by paired median)** - leaving **+22.4 ms (~54%)**. Derivation: - +41.9 ms - +22.4 ms == 19.5 ms recovered - 19.5 ms / +41.9 ms == 46.5% (~46%) recovered - +22.4 ms / +41.9 ms == 53.5% (~54%) left A separate 2-way run of master vs `<tip-of-this-pr-branch>` measured **−21.1 ms ±3.2**, consistent with the 19.5 ms recovered that was derived above. <details> <summary>Notes on the measurement</summary> The baseline (`<TypeForm-disabled-commit>`) is current master (`5bb72b788`) with referenced #21262 (SHA: `dd851f559`) reverted, so all three arms share today's code and differ only in `TypeForm`. Measuring against the original pre-#21262 master commit instead of today's master would have conflated optimizations made during the following ~80 commits, including notably `c0cced35c`, which optimised `SA.try_parse_as_type_expression` specifically. Thus runtime regression measured here (+41.9 ms) is smaller than the +50.2 ms reported in replaced #21585: part of the original regression has already been absorbed upstream. </details> Full parses per self-check, identical corpus: | | master | branch | | |---|---:|---:|---| | full parses | 2570 | 666 | **−74.1%** | | - succeeded (produced a type) | 345 | 345 | **±0** ✅ | | - failed (wasted work) | 2225 | 321 | **−85.6%** | The successful-parse count is unchanged at every commit on the branch, as expected: No expression that previously parsed as a type stopped doing so. ## Overview of changes - Most changes are made to the `SemanticAnalyzer.try_parse_as_type_expression` function. All other changes occur within the same file. - 5 commits, each individually profiled: - 4 commits add a filter - 1 commit reorders existing filters - Any added filter can be dropped (if needed) without disturbing the other filters ### The filter commits Bare-identifier strings (`"Foo"`): 1. Reject a `Var` whose declared type is a concrete `Instance` - a value, not a type. 2. Reject `FuncDef` / `OverloadedFuncDef` / `MypyFile` - functions and modules are never types. 3. Reorder the mutually-exclusive checks by measured rejection frequency. Other strings: 4. Reject strings containing a character or boundary pattern that never appears in a type expression - leading/trailing `.`, or one of ``!:/<>@%$^?;&~`\``, or a `-` that is not a `Literal[...]` unary minus. Catches `"utf-8"`, `".pyi"`, `"error:"`, `"pkg/mod.py"`. 5. Dotted-name strings (`"builtins.tuple"`, `"typing.Mapping"`): look up the leftmost component and reject when it does not resolve, or resolves to a placeholder or a value `Var`. Filters 4 and 5 replace `_NONTYPE_PATTERN_RE` and `_DOTTED_IDENTIFIER_RE` from replaced #21585 with the helpers `has_nontype_char()` and `dotted_identifier_leftmost()`. Each was verified to agree with the regex it replaces on all 1171 distinct strings the full-parse profiler observes during a self-check. <details> <summary>Two specific hazards, and how they are handled</summary> `var_is_typing_special_form` was extended to recognize `typing.Self` / `typing_extensions.Self`, so filter 1 does not reject a stringified `'Self'` annotation (otherwise `testSelfRecognizedInOtherSyntacticLocations` regresses). In filter 4, `-` is treated as a unary minus wherever the preceding non-space character is `[` or `,`, so `"Literal[-1, -2]"` and `"Literal[1, -2]"` are still recognized. (`_NONTYPE_PATTERN_RE` in replaced #21585 used `(?<!\[)-`, which rejected those.) On the strings observed during a self-check the two rules reject identical sets, so the (improved) soundness costs nothing. </details> ## Notes - I don't think it's worth trying to recover the remaining +22.4 ms: - The 321 surviving failed parses are spread across four categories with no common cheap/obvious shape left - I experimented with adding some fancy `OpExpr` filters that actually gave a net *slowdown* of 1.9ms. - The profiling instrumentation and the `misc/perf_compare.py` improvements used to produce these numbers are in a separate PR: #21832. Happy to fold them in here instead if that is easier to review. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
...removing the need to use --enable-incomplete-feature=TypeForm
REVIEWER NOTES: