feat: add interactive chat example for cactus crate

[twill-hq]

Summary

• Adds a simple interactive CLI chat example at crates/cactus/examples/chat.rs
• Takes a model path as the first argument, loads it, and starts a multi-turn streaming conversation loop
• Follows the same API patterns used in crates/cactus/tests/llm.rs

Usage

cargo run -p cactus --example chat -- /path/to/model.gguf

Test plan

• cargo check -p cactus --example chat compiles without errors
• Run with a GGUF model, send messages, verify streamed output and multi-turn context
• Type "exit" or "quit" to verify clean shutdown

---

View on Twill

You can tag @twill anywhere in this PR for follow-ups.

---

Note

Low Risk
Adds a standalone example binary only; no library/API behavior changes, so risk is limited to example compilation/runtime issues.

Overview
Adds a new cactus example (examples/chat.rs) that runs an interactive, multi-turn CLI chat loop using Model::complete_streaming, streaming tokens to stdout and persisting conversation history until the user exits.

^{Written by Cursor Bugbot for commit 1ce30e5. This will update automatically on new commits. Configure here.}

[netlify]

✅ Deploy Preview for char-cli-web canceled.

Name	Link
🔨 Latest commit	9ebd110
🔍 Latest deploy log	https://app.netlify.com/projects/char-cli-web/deploys/69cf42f22d766c0008782019

[netlify]

✅ Deploy Preview for hyprnote canceled.

Name	Link
🔨 Latest commit	9ebd110
🔍 Latest deploy log	https://app.netlify.com/projects/hyprnote/deploys/69cf42f23c7bb500086f8929

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: Empty input exits chat instead of reprompting
  • The loop now breaks only on EOF (read_line returning 0) while blank trimmed input uses continue to reprompt.
• ✅ Fixed: Failed completion leaves orphaned user message in history
  • On completion errors the just-appended user message is popped from messages, preventing malformed history for subsequent turns.

Or push these changes by commenting:

@cursor push aeeb1a6acc

Preview (aeeb1a6acc)

diff --git a/crates/cactus/examples/chat.rs b/crates/cactus/examples/chat.rs
--- a/crates/cactus/examples/chat.rs
+++ b/crates/cactus/examples/chat.rs
@@ -22,10 +22,17 @@
         let _ = io::stdout().flush();
 
         let mut input = String::new();
-        io::stdin().lock().read_line(&mut input).unwrap();
+        let bytes_read = io::stdin().lock().read_line(&mut input).unwrap();
+        if bytes_read == 0 {
+            break;
+        }
         let input = input.trim();
 
-        if input.is_empty() || input == "exit" || input == "quit" {
+        if input.is_empty() {
+            continue;
+        }
+
+        if input == "exit" || input == "quit" {
             break;
         }
 
@@ -44,7 +51,10 @@
 
         match result {
             Ok(_) => messages.push(Message::assistant(&response_text)),
-            Err(e) => eprintln!("Error: {e}"),
+            Err(e) => {
+                let _ = messages.pop();
+                eprintln!("Error: {e}");
+            }
         }
     }
 }

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

[cursor]

Empty input exits chat instead of reprompting

Medium Severity

The input.is_empty() check in the break condition causes the chat loop to exit when the user simply presses Enter without typing anything. Most interactive CLI chat applications skip blank lines and re-display the prompt. This makes accidental exits too easy and contradicts the PR description which states only "exit" or "quit" trigger clean shutdown. The is_empty() condition conflates EOF handling with blank-line handling — EOF from read_line (returning Ok(0)) could be detected via the return value instead.

 

[cursor]

Failed completion leaves orphaned user message in history

Medium Severity

The user message is pushed into messages on line 32 before complete_streaming is called. If complete_streaming returns Err, the error branch only prints the error — it never removes the user message that was already appended. On subsequent turns, the conversation history contains a user message with no corresponding assistant reply, breaking the expected alternating user/assistant pattern. This corrupts multi-turn context for all future turns in the session.

 

[cursor]

You have used all of your free Bugbot PR reviews.

To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

[yujonglee]

Applied locally in /Users/yujonglee/dev/char with a smaller example-focused pass and local cleanup.