Fix Python PEP 695 generic functions being reported with the name ] - #492
Merged
Merged
Conversation
PythonStates._function assumed exactly one token between `def` and `(`, so every token before `(` called restart_new_function and the last one won. For `def soma[T](a, b)` the tokens are `soma`, `[`, `T`, `]`, leaving the function named `]`. Add _state_type_parameters, entered from _function on `[`, which consumes the PEP 695 type parameter list without restarting the function. Bracket depth is tracked so nested bounds such as `def f[T: list[int]](x)` work. The state recovers on `def`/`class`, so an unbalanced `[` in truncated or invalid source does not swallow the rest of the file. Measured on home-assistant/core@73898c29e2a: 242 of 4235 functions in the 175 files using `def name[...]` were reported as `]`. Add 14 regression tests covering generic functions, methods, async, multiple type parameters, nested bounds, constraints, PEP 696 defaults, TypeVarTuple, ParamSpec, multi-line lists, bracket-bearing string literals in bounds, long_name integrity and unbalanced-bracket recovery.
Owner
|
thanks. Looks great! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Functions declared with a PEP 695 type parameter list (Python 3.12+) are reported with the
name
]instead of their real name:python -m lizard repro.pyreports this as]@1-2@repro.py. Through the API,function_list[0].nameis']'andlong_nameis']( a : T , b : T )'.nloc,cyclomatic_complexityandparameter_countare correct, so only the name breaks.Measured on
home-assistant/core@73898c29e2a: 175.pyfiles usedef name[...], and242 of the 4,235 functions (5.7%) in those files come out as
]. Everything that keys onthe name sees
]:-woutput, CSV/XML/HTML reports, whitelists, duplicate detection.Cause
PythonStates._functionassumes exactly one token sits betweendefand(:For
def soma[T](...)the tokens before(aresoma,[,T,]. Each one restarts thefunction, so the last one wins.
The reader already skips
[...]groups, but only inside the parameter list(
_state_parameterized_type_annotation, entered from_dec). A PEP 695 list sits before the(, so it never reaches that state.Fix
Enter a dedicated state when
_functionsees[, and consume the type parameter list withoutrestarting the function. Bracket depth is tracked so nested bounds work.
The state also recovers on
defandclass. Neither keyword can appear inside a valid typeparameter list, so an unbalanced
[resumes at the next definition instead of swallowing therest of the file.
All of these keep the correct name:
Notes
long_name, which readssoma( a : T , b : T ). Including themmeans deferring the
(that is currently appended as soon as the name is read, which changeslong_nameconstruction for every function, not just generic ones. This PR restores the namewith the smallest edit. Adding type parameters to
long_nameis an easy follow-up.3.10 interpreters in CI without a
SyntaxError.Tests
14 tests in
test/test_languages/testPython.py::Test_Python_pep695_type_parameters. They covergeneric functions, methods,
async def, multiple type parameters, nested bounds, constraints,PEP 696 defaults,
TypeVarTuple,ParamSpec, multi-line lists, string literals with bracketsinside a bound,
long_nameintegrity, and recovery from an unbalanced[.Full suite on the base commit:
8 failed, 1364 passed, 7 skipped.With this change:
8 failed, 1378 passed, 7 skipped.The 8 failures also fail on the base commit, 2 in
testFilesFilter.pyand 6 in the HTML outputtests. They do not touch Python function parsing.