-
-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
JuliaSyntax parser-based REPL completions overhaul #57767
base: master
Are you sure you want to change the base?
Conversation
Adds another permitted return type for complete_line, where the second element of the tuple is a Region (a Pair{Int, Int}) describing the region of text to be replaced. This is useful for making completions work consistently when the closing delimiter may or may not be present: the cursor can be made to "jump" out of the delimiters regardless of whether it is there already. "exam| =TAB=> "example.jl"| "exam|" =TAB=> "example.jl"|
This commit replaces the heuristic parsing done by REPLCompletions.completions with a new approach that parses the entire input buffer once with JuliaSyntax. In addition to fixing bugs, the more precise parsing should allow for new features in the future. Some features now work in more situations "for free", like dictionary key completion (the expression evaluated to find the keys is now more precise) and method suggestions (arguments beyond the cursor can be used to narrow the list). The tests have been updated to reflect slightly differing behaviour for string and Cmd-string completion: the new code returns a character range encompassing the entire string when completing paths (not observable by the user), and the behaviour of '~'-expansion has be tweaked to be consistent across all places where paths can be completed. Some escaping issues have also been fixed. Fixes: JuliaLang#55420, JuliaLang#55518, JuliaLang#55520, JuliaLang#55842, JuliaLang#56389, JuliaLang#57611
This sounds great! I think we should backport it to at least 1.12 given it fixes so many issues. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
stdlib/REPL/test/replcompletions.jl
Outdated
# #55518 | ||
let s = "CompletionFoo.@barfoo kwtest" | ||
c, r = test_complete(s) | ||
@test isempty(c) | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure this quite cover all the broken cases reported in #55518 (there are a few scattered in the discussion)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've added a few more from the thread
Bravo. Truly excellent quality-of-life PR 👏🏻 |
Best part is that this PR has a negative diff, depsite the fact it added many tests. |
@@ -158,6 +158,10 @@ let ex = | |||
export exported_symbol | |||
exported_symbol(::WeirdNames) = nothing | |||
|
|||
macro ignoremacro(e...) | |||
:nothing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be
:nothing | |
nothing |
?
julia> macro ignoremacro(e...)
:nothing
end
@ignoremacro (macro with 1 method)
julia> @macroexpand @ignoremacro
:(Main.nothing)
julia> macro ignoremacro(e...)
nothing
end
@ignoremacro (macro with 1 method)
julia> @macroexpand @ignoremacro
julia>
Edit: anyway this is for a test, so not particularly important to get perfectly right.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was actually unaware that so many things were self-quoting!
…JuliaLang#57307, JuliaLang#57624 Also fix test failing for silly reason
Since we parse the entire input now, we need to avoid triggering method completion when the cursor is on the function name of a valid call.
let s = "begin\n using Linear" | ||
c, r = test_complete(s) | ||
@test "LinearAlgebra" in c | ||
@test r == 15:20 | ||
@test s[r] == "Linear" | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't pass when run via Pkg.test("REPL")
because Base.runtests
puts all stdlibs on the load path, whereas Pkg.test
only does REPL deps and test deps.
let s = "begin\n using Linear" | |
c, r = test_complete(s) | |
@test "LinearAlgebra" in c | |
@test r == 15:20 | |
@test s[r] == "Linear" | |
end | |
let s = "begin\n using Ran" | |
c, r = test_complete(s) | |
@test "Random" in c | |
@test r == 15:17 | |
@test s[r] == "Ran" | |
end |
|
||
# Expand '~' if the user hits TAB after exhausting completions (either | ||
# because we have found an existing file, or there is no such file). | ||
full_path = ispath(path) || isempty(paths) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I found two cases that can cause ispath
to error, so I think we should make a safe_ispath
-
ArgumentError: embedded NULs are not allowed in C strings: "invalid\0"
forlet name = \"valid.txt\", name⁺ = \"invalid\\0.txt\"
-
IOError name too long (ENAMETOOLONG) for strings that are too long.
Something like this? Or just never throw and just return false?
function safe_ispath(path::AbstractString)
try
return ispath(path)
catch err
if err isa Base.IOError # i.e. name too long (ENAMETOOLONG)
return false
elseif err isa Base.ArgumentError && occursin("embedded NULs", err.msg)
return false
else
rethrow()
end
end
end
Overview
As we add REPL features, bugs related to the ad-hoc parsing done by
REPLCompletions.completions
have crept in. This pull request replaces most ofthe manual parsing (regex,
find_start_brace
) with a new approach that parsesthe entire input buffer once, before and after the cursor, using JuliaSyntax.
We then query the parsed syntax tree to determine the kind of completion
to be done.
Changes
New, JuliaSyntax-based completions mechanism.
The
complete_line
interface now has the option of replacing arbitraryregions of text in the input buffer by returning a
Region
(Pair{Int, Int}
for consistency with the convention in LineEdit, and
pos
being a 0-basedbyte offset).
This is used to improve the handling of auto-inserted closing. Leaving this unchanged for now.delimiters.
Fixes parsing-related bugs:
ModuleName.function
call is broken #55420@time
calls involving another macro #55429ModuleName.@macro_name
is broken #55518@benchmark A .= A setup=(A=
is broken #55520begin-end
block #56389v = Module.Submodule.<TAB>
is broken #57611Fixes some bugs that exist on 28d3bd5 that were found by fuzzing:
x \"
+TAB
throws aFieldError
exceptionThe duplicate code for path completion in strings,
Cmd
-strings, and theshell has been removed, causing paths to complete the same way for all three.
Now,
~
is expanded in two situations:If
foo
exists, or iffoo
does not exist but there are nopossible completions:
If the current path ends with a
/
and you hit TAB again:Future work
Method completions could be changed to look for methods with exactly the given
number of arguments if the closing
)
is present, and search for signatureswith the right prefix otherwise.
by putting
::T
in place of arguments).Other REPL features could benefit from JuliaSyntax, so it might be worth
sharing the parse tree between completions and other features:
C-M-f
/C-M-b
/C-M-u
, etc.It would be nice if hints worked even when the cursor is between text.
CursorNode
is a slightly tweaked copy ofSyntaxNode
from JuliaSyntax thattracks the parent node but includes all trivia. It is used with
seek_pos
,which navigates to the innermost node at a given position so we can examine
nearby nodes and the parent. This could probably duplicate less code from
JuliaSyntax.