Search scrollback

[mitchellh]

A major missing feature is the ability to search scrollback, i.e. cmd+f on Mac, ctrl+F on Linux. This issue can be implemented in multiple steps, not just one giant PR:

• Core search functionality in src/terminal (Naive search internals (core only) #2885)
• Make core search work simultaneously to PageList read/write to allow multi-threading
• Implement multi-threaded search, including data modeling for notifications of matches
• Search UI, glue in apprt

We can also support search only in certain modes as long as the apprt glue is all there for someone to come along and finish it up. For example, only macOS, Linux+GTK, etc.

[amadeus]

Would be cool to fake what terminal emulator in vim does -- where you can pause the existing session ctrl+w N, and move around the buffer with vim commands for searching and highlighting, then press i to return back into the session. Here's an example video (not saying the commands should be the same to pull this off, just using that as an example):

CleanShot.2023-07-07.at.00.09.42.mp4

[andrewrk]

Looking forward to this feature - I missed it yesterday when I had a lot of output from a command and wanted to search for the word "fail".

[vancluever]

Kitty has a a pretty novel approach here that I've actually really enjoyed - ctrl+shift+h actually sends the scrollback to a pager (e.g. less). I've found this much less confusing in comparison to others (like iTerm's).

This also lets you take advantage of the pager's features and can be tailored to a certain user's preference without minimal load on our side.

[gpanders]

Kitty has a a pretty novel approach here that I've actually really enjoyed - ctrl+shift+h actually sends the scrollback to a pager (e.g. less). I've found this much less confusing in comparison to others (like iTerm's).

FYI Ghostty has this too, bound to Cmd-Shift-J on macOS and Ctrl-Shift-J on Linux.

It works slightly differently than Kitty in that it puts the temp file path to the captured scrollback on the command line rather than opening it in a pager directly.

[benwaffle]

Once we have search, searching via regex would be really nice to have. I've been finding myself search for exit [^0] a lot recently.

[mattpolzin]

Kitty has a a pretty novel approach here that I've actually really enjoyed - ctrl+shift+h actually sends the scrollback to a pager (e.g. less). I've found this much less confusing in comparison to others (like iTerm's).

FYI Ghostty has this too, bound to Cmd-Shift-J on macOS and Ctrl-Shift-J on Linux.

It works slightly differently than Kitty in that it puts the temp file path to the captured scrollback on the command line rather than opening it in a pager directly.

One nice thing about Kitty's implementation is you can even use it as a program is actively running and potentially still outputting text; new text doesn't automatically get fed into your pager but you don't need to wait until a command finishes to use the pager functionality.

I'm not immediately sure if that would be equivalent in behavior to

1. Pause current execution
2. Send to background
3. Open ghostty's scrollback file in pager
4. On quit, bring task to foreground if still running

[pjz]

I'm used to screen/tmux behavior here: ctrl-A,[ puts you into 'scrollback mode' where you can then use the keyboard (vi-keys) to scroll up and down, and then even hilight some text and put it into a clipboard where you can then paste it with ctrl-A,].

I don't think it updates the buffer while scrolling, though... but I don't see why it couldn't just extend it downward.

[yankcrime]

Personally I'd rather this stuck to OS-native conventions for search functionality as well as look and feel. I don't think implementation-specific idiosyncrasies are a good idea since a lot of folks will have differing preferences and expectations.

The implementation along the lines of what's provided by Terminal.app or iTerm would be fine since that easily lets you search the scrollback buffer in a way that doesn't affect any additional output.

[andrewrk]

Some prior art:

It... suffices.

[peteschaffner]

@qwerasd205 Are you working on this now? I'd be happy to help out on the macOS UI front.

(I mentioned this in Discord but just surfacing it here too since I rarely check over there.)

[qwerasd205]

I am working on laying some groundwork for this with a basic implementation of the search functionality - focused mainly on proper threading for the search and creating a good interface for the UI code to talk to with minimal coupling, so that in the future we can improve the performance of the search (if necessary) by changing the algorithm and/or data structures used for the searching. I'll open a draft PR once I've got the first "technically functioning" version of the code.

Right now it's in too early of a state to really create UI for.

[ferdinandyb]

Personally, "native" in a terminal means "vi-like" to me :) I also think kitty's approach is pretty flexible and can probably be used along side a more classic ctrl-f as well. I also quite like contour's vim-like normal mode tbh.

But why I actually wanted to comment is related to generic search, but a bit more specific: sometimes you actually see what you need on screen, but want to quickly copy/paste/execute it without a mouse. Back when I was using kitty I used hints a lot, and in tmux I use tmux-butler. Behind the scenes these use cases could be regex-searches constrained to the currently visible text, although obviously the UI would be quite different. Imho especially quickly opening links in the browser and copying paths to the prompt without a mouse is really useful.

[sluongng]

I wanted to file a feature request for something like Kitty Hints like @ferdinandyb mentioned.
https://sw.kovidgoyal.net/kitty/kittens/hints/

However, after a bit of searching, I figured this issue is a prerequisite for such a hinting system.

[andrewrk]

More prior art:

Konsole has a pretty decent implementation if you want something to draw inspiration from.

[rmacklin]

Re:

I wanted to file a feature request for something like Kitty Hints like @ferdinandyb mentioned. https://sw.kovidgoyal.net/kitty/kittens/hints/

However, after a bit of searching, I figured this issue is a prerequisite for such a hinting system.

Mitchell confirmed on Discord that this issue isn't a prerequisite for a "hints" feature - so I've gone ahead and created a separate issue to specifically discuss that feature request:

• Feature Request: "Hints" mode to select and act on arbitrary text patterns visible in the terminal #2012

[qwerasd205]

I am working on laying some groundwork for this with a basic implementation of the search functionality - focused mainly on proper threading for the search and creating a good interface for the UI code to talk to with minimal coupling, so that in the future we can improve the performance of the search (if necessary) by changing the algorithm and/or data structures used for the searching. I'll open a draft PR once I've got the first "technically functioning" version of the code.

Right now it's in too early of a state to really create UI for.

I got sidetracked from this shortly after starting work on it, but to summarize the architecture I landed on for the search thread:

Search Thread

Data

• Search results: Doubly linked list of spans represented by pairs of pins
• Active area search results: Separate results list for content in the active area, since it needs to be repopulated whenever the active area's content changes (dirty tracking logic could be added in the future if necessary)
• Search pattern: Struct containing info used to perform the search (in my prototype it's just a string and a flag for whether to be case insensitive)

Messages

• Search pattern updated; if new pattern is different from old pattern, reset search results, and if currently mid-search, reset search back to start (either top or bottom of scrollback, depending on which direction the search is performed in)
• Begin search; start searching, intermittently yielding to the event loop. In my prototype I use a line iterator. If no iterator exists, create a new one, take a line from it, apply the pattern, add any results to the results list, notify the surface - if the iterator is out, we're done searching, notify the surface of this fact - then check the mailbox again.
• Stop search; self explanatory.
• Active area updated; if not currently searching, do nothing - otherwise throw out current active area results list, apply the search pattern to all lines in the active area, update the results list, notify the surface.

Whenever the surface is notified of new search results, it updates its display accordingly, reading from the results list should be thread safe.

[AdrielVelazquez]

This is the PR--which should definitely not be used--of how I'm getting around it for now.

I essentially added a new option to WriteScreenAction which will open up the scrollback in your defined $EDITOR so that it stays within the terminal.

This is doesn't fully leverage the way that ghostty or termio expect you to launch this command, but it was more me getting familiar with the codebase in general.

PR New Write Screen Action

This allows the new keybind of ctrl + shift + f to open the scrollback in whatever you use as $EDITOR

[jamesharr]

Specific designs aside, the big 2 use cases I have:

1. searching for something in my scrollback.
2. highlighting all instances of a specific field. As mentioned above, searching for "FAIL" or "ERROR" is probably a common, but it depends on what I'm doing.
   • Sometimes I might want to highlight a specific log term, so using a static configuration might not be appropriate.
   • I also wind up glancing at the scrollbar to look for when terms may have scrolled by

It's possible that these are better as separate actions/features, but that's how I use it today. Sometimes I want to find go something, sometimes I just want to draw my attention to certain strings while I run other stuff.

[Syphdias]

Please stop suggesting workarounds. There are enough of it in this thread. I believe there are situations where theses workarounds (|less, tee/>, etc.) don'te work but I won't mention them to not trigger a response that there is in fact a workaround.

The only things I — and potentially other subscribers to this issue — am interested in this issue are contributions regarding the resolution.

I would suggest closing the issue to discussion.

/rant over, sorry

[jcollie]

Sorry, but I'm locking this conversation. There's hasn't been any productive conversation on this issue for a very long time. We won't forget about this feature, it's just not on the immediate roadmap.

[mitchellh]

To be completely clear, it’s not on the “IMMEDIATE” roadmap (as noted in the prior comment). It’s absolutely on the roadmap and I even already started some it in a branch. But as a passion project, we prioritize working on whatever we want and this isn’t currently the priority. It’s high on the list but not like.. next release (“immediate”) priority at the time of this comment.

This issue was closed because the comments haven’t been productive towards completing this.

I outlined some suggestions for how this can be incrementally contributed. If someone comes along and implements this I’d be happy to take a look and probably speed up merging.

Otherwise, if you aren’t doing the work, please respect the other volunteers on this project and be patient. We’ll get it it when we want. If this is a blocker for you, you can and should use another terminal. There are fantastic choices out there!