Skip to content

feat: screenshot capture for image input - #31

Merged
quiet-node merged 8 commits into
mainfrom
worktree-mighty-giggling-manatee
Apr 5, 2026
Merged

feat: screenshot capture for image input#31
quiet-node merged 8 commits into
mainfrom
worktree-mighty-giggling-manatee

Conversation

@quiet-node

@quiet-node quiet-node commented Apr 5, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds a screenshot capture button to the ask bar, letting users grab a screen region and attach it as an image without leaving Thuki
  • New Rust backend command (capture_screenshot_command) hides the window, runs screencapture -i -x, re-shows via show_and_make_key() to restore NSPanel keyboard focus, then returns the captured region as base64 PNG
  • Frontend decodes the base64 and feeds it into the existing handleImagesAttached pipeline (identical path to paste and drag-drop)
  • Camera button uses a viewfinder/frame icon; shows a "Take a screenshot" tooltip on hover; switches to a "Maximum 3 images attached" tooltip with hover effects removed when at the image limit
  • Tooltip component gains a window focus listener so it dismisses itself when the Thuki window reappears after a screenshot

Code quality

  • process_screenshot_result() extracted as a pure, testable function from the command wrapper; 3 dedicated Rust tests cover the cancelled path, happy path (reads + deletes temp file), and read-error path
  • show_and_make_key() used on re-show (mirrors lib.rs) so the NSPanel becomes the key window and the textarea reliably gets keyboard focus
  • MAX_IMAGES exported from AskBarView and imported in App.tsx; defensive guard added to handleScreenshot with the matching dep in useCallback
  • 100% line/statement/branch/function coverage maintained across both frontend (Vitest/V8) and backend (llvm-cov)

Test plan

  • Double-tap Cmd to open Thuki, click the viewfinder button, drag a region, confirm image appears as a thumbnail
  • Press Escape during region select, confirm no image is attached and Thuki reappears normally
  • Confirm textarea receives keyboard focus after a screenshot (can type immediately without clicking)
  • Attach 3 images, hover the camera button, confirm "Maximum 3 images attached" tooltip and no hover glow
  • Hover the camera button with fewer than 3 images, confirm "Take a screenshot" tooltip
  • Take a screenshot, confirm the tooltip dismisses when the window reappears

🤖 Generated with Claude Code

quiet-node and others added 8 commits April 5, 2026 09:31
@quiet-node
feat: add capture_screenshot Tauri command with macOS screencapture -i
aed0137 
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node @claude
feat: add screenshot capture button to AskBarView
b336e81 
Adds a camera icon button to the AskBarView input bar that triggers
onScreenshot. The button is disabled while generating, submit is pending,
or the max image count is already reached. Includes a temporary no-op
stub in App.tsx to satisfy TypeScript until Task 4 wires up the handler.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node
feat: wire screenshot capture into App — handleScreenshot feeds handl…
6bd8b01 
…eImagesAttached

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node @claude
fix: apply Prettier and cargo fmt formatting to screenshot feature files
2229a0d 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node
fix: correct capture_screenshot IPC command name to capture_screensho…
4d34be4 
…t_command

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node @claude
feat: polish screenshot button UX — viewfinder icon, hover tooltip, m…
128a7ef 
…ax-images guard

- Replace camera lens icon with a 4-corner viewfinder/frame icon
- Add "Take a screenshot" tooltip on hover for discoverability
- Remove hover styles on camera button when max images are attached;
  show "Maximum 3 images attached" tooltip instead
- Dismiss any open tooltip on window focus so it doesn't linger
  after the Thuki window hides and reappears mid-screenshot

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node @claude
fix: three code-review issues in screenshot feature
aaa4677 
- Extract process_screenshot_result() pure fn from capture_screenshot_command
  so the cancellation-detection and file-read logic is testable independently;
  adds three new backend tests (missing file, happy path + deletion, read error).
  Satisfies CLAUDE.md: coverage(off) exclusion is now a genuine thin wrapper.

- Use show_and_make_key() (via tauri-nspanel ManagerExt) when re-showing the
  window after screencapture, mirroring lib.rs show_overlay(). Without this
  the NSPanel may not become the key window and the textarea loses keyboard focus.

- Add MAX_IMAGES guard + attachedImages dep to handleScreenshot in App.tsx.
  Exports MAX_IMAGES from AskBarView so the constant stays DRY. Guard is
  defensive: the disabled button is the primary protection, but the logic layer
  should not rely solely on UI state.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node @claude
fix: restore 100% coverage after MAX_IMAGES guard addition
5f629d1 
Mark the defensive guard in handleScreenshot with v8 ignore start/stop —
the button is always disabled at max images so this branch is unreachable
through normal UI interaction. Follows the existing project convention
(start/stop pairs, not next) used throughout App.tsx and other components.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node
quiet-node merged commit dba91cd into main Apr 5, 2026
0 of 3 checks passed
@quiet-node
quiet-node deleted the worktree-mighty-giggling-manatee branch April 5, 2026 15:07
This was referenced Apr 5, 2026
Closed
Closed
quiet-node added a commit that referenced this pull request Apr 5, 2026
@quiet-node
docs: update spec to reflect existing screenshot.rs from PR #31
40efa7a 
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@github-actions github-actions Bot mentioned this pull request Apr 5, 2026
Merged
quiet-node added a commit that referenced this pull request Apr 5, 2026
@quiet-node
docs: update spec to reflect existing screenshot.rs from PR #31
81814ac 
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
quiet-node added a commit that referenced this pull request Apr 5, 2026
@quiet-node @claude
feat: add /screen slash command with tab-completion and screen capture (
354403a 
#35)

* docs: add design spec for /screen command and slash command system

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* docs: update spec to reflect existing screenshot.rs from PR #31

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: add /screen slash command with tab-completion system

Implements the full /screen command feature from the 2026-04-05 design spec:

- `src-tauri/src/screenshot.rs`: adds `capture_full_screen` using CoreGraphics
  CGWindowListCreateImageFromArray, filtering out Thuki's own PID so no window
  hide or flicker is needed. Wrapped by `capture_full_screen_command` Tauri command.
- `src-tauri/src/images.rs`: bumps MAX_IMAGES_PER_MESSAGE from 3 to 4 (3 manual + 1 /screen).
- `src-tauri/src/lib.rs`: registers `capture_full_screen_command` in the invoke handler.
- `src/config/commands.ts`: new command registry (single source of truth for slash commands).
- `src/components/CommandSuggestion.tsx`: presentational popover that renders above the
  ask bar when the user types a / prefix.
- `src/view/AskBarView.tsx`: wires up command suggestion state (show/hide, ArrowDown/Up,
  Tab completion, Escape dismiss).
- `src/App.tsx`: detects /screen at submit time, invokes capture_full_screen_command,
  merges screenshot path with attached images, and calls ask() with clean message.
- `docs/commands.md`: user-facing reference doc for all slash commands.

All tests pass (407 frontend, 110 backend). Frontend and backend coverage at 100% lines.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: command suggestion visibility, screenshot crash, and capture error handling

- Render CommandSuggestion in normal DOM flow (not portal) so the
  ResizeObserver grows the native window to reveal the popover in
  ask-bar mode. Wrap in AnimatePresence + motion.div for smooth
  height transition.

- Fix interactive screenshot crash: wrap window.hide() and
  panel.show_and_make_key() in run_on_main_thread() since AppKit
  APIs must only be called from the macOS main thread (was being
  called from tokio pool thread).

- Rewrite capture_full_screen to use CGWindowListCreateImage with
  CGDisplayBounds (concrete screen coordinates) instead of
  CGWindowListCreateImageFromArray with abstract CGRectNull. Use
  kCGWindowListOptionOnScreenBelowWindow to exclude Thuki's own
  NSPanel from the capture.

- Surface Rust backend errors directly in a capture error banner
  instead of swallowing them silently.

- Fix Enter key requiring two presses on exact trigger match:
  fall through to submit instead of completing with trailing space.

- Add outline-none to history button to prevent native macOS focus
  ring after screenshot capture.

- Smooth ask-bar history panel animation with delayed opacity.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: run CoreGraphics capture on main thread to prevent deadlock

CoreGraphics APIs (CGWindowListCreateImage, CGWindowListCopyWindowInfo,
CGDisplayBounds) internally dispatch work to the macOS main thread.
When called from tokio::task::spawn_blocking (a background thread),
they deadlock because the main thread event loop is not processing
their dispatched calls.

Split capture_full_screen_command into two phases:
1. Capture raw RGBA pixels on the main thread via run_on_main_thread()
   and a oneshot channel to return the result
2. Encode PNG and save to disk on a blocking thread (CPU-bound, no
   CoreGraphics dependency)

This matches the pattern used by capture_screenshot_command, which
already dispatches AppKit calls to the main thread.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: show pending chat bubble immediately when /screen capture is in flight

When the user submits a /screen message, the chat view now shows the
user bubble instantly (with a clean loading placeholder thumbnail)
instead of waiting silently for capture to complete. This prevents
double-submit spam and gives immediate visual feedback.

- handleScreenSubmit sets isSubmitPending + pendingUserMessage before
  the invoke call, then clears/commits on success or restores state on
  failure (query and context are preserved so the user can retry)
- ImageThumbnails gains a placeholder prop: renders a translucent
  bg-black/15 tile with a white spinner instead of a broken image
- ChatBubble detects the SCREEN_CAPTURE_PLACEHOLDER sentinel path and
  sets placeholder: true on the thumbnail item
- SCREEN_CAPTURE_PLACEHOLDER constant extracted to config/commands.ts
  so App.tsx and ChatBubble.tsx share a single source of truth

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: address all code review issues before PR

- Replace all em dashes in newly written code with colons or semicolons
  per the global writing style rule (docs, Rust module comments, JSX
  comments, test comments across 7 files)
- Fix non-macOS Rust test: capture_full_screen_returns_err_on_non_macos
  called the nonexistent capture_full_screen(); correct stub is
  capture_full_screen_pixels() (no arguments)
- Clear captureError in replayEntranceAnimation so stale error banners
  do not persist across overlay hide/show cycles
- Fix handleCancel to handle the /screen capture in-flight case: add
  case 2 that signals cancellation via screenCapturePendingRef and
  clears pending UI; handleScreenSubmit checks the ref after await to
  skip ask() when the user cancelled mid-capture
- Fix MAX_IMAGES comment: the constant is the manual attachment limit (3),
  not a mirror of MAX_IMAGES_PER_MESSAGE (4, which includes /screen)
- Add test: cancelling during in-flight capture prevents ask from firing

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: restore input state and reset capture refs on cancel/close/new

- handleCancel case 2 now restores query and selectedContext from
  screenCaptureInputSnapshotRef so the ask bar returns to its pre-capture
  state when the user aborts an in-flight /screen capture
- requestHideOverlay resets screenCapturePendingRef and snapshot ref so
  closing the overlay during a capture does not trigger a ghost ask()
- replayEntranceAnimation and resetForNewConversation do the same, so
  starting a new session mid-capture is also safe
- Remove superpowers design spec from worktree (development artifact only)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: restore input state and reset capture refs on cancel/close/new

- handleCancel case 2 now restores query and selectedContext from
  screenCaptureInputSnapshotRef so the ask bar returns to its pre-capture
  state when the user aborts an in-flight /screen capture
- requestHideOverlay resets screenCapturePendingRef and snapshot ref so
  closing the overlay during a capture does not trigger a ghost ask()
- replayEntranceAnimation and resetForNewConversation do the same, so
  starting a new session mid-capture is also safe

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* docs: update README and CLAUDE.md to reflect /screen slash command

- README features list: add /screen screen capture bullet; update image
  input bullet to clarify it covers both images and screenshots
- README roadmap: mark /screen as shipped in the slash commands item;
  update auto-capture screen context item to note /screen delivers
  full-screen capture today
- CLAUDE.md architecture: add config/commands.ts to Frontend section;
  add screenshot.rs to Backend section
- CLAUDE.md: fix "Command key" -> "Control key" in Key Design Constraints
  to match the rest of the document (activator.rs watches Control, not Command)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix(ci): run build validation on macos-latest

cargo clippy pulls in Tauri's GTK/WebKit dependencies which require
system packages (libwebkit2gtk, libgtk-3-dev) that ubuntu-latest does
not have. Since Thuki is macOS-only and backend tests already run on
macos-latest, switching build validation to the same runner is the
clean fix.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

---------

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
quiet-node added a commit that referenced this pull request Apr 10, 2026
@quiet-node
feat: screenshot capture for image input (#31)
deed66c 
* feat: add capture_screenshot Tauri command with macOS screencapture -i

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: add screenshot capture button to AskBarView

Adds a camera icon button to the AskBarView input bar that triggers
onScreenshot. The button is disabled while generating, submit is pending,
or the max image count is already reached. Includes a temporary no-op
stub in App.tsx to satisfy TypeScript until Task 4 wires up the handler.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: wire screenshot capture into App — handleScreenshot feeds handleImagesAttached

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: apply Prettier and cargo fmt formatting to screenshot feature files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: correct capture_screenshot IPC command name to capture_screenshot_command

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: polish screenshot button UX — viewfinder icon, hover tooltip, max-images guard

- Replace camera lens icon with a 4-corner viewfinder/frame icon
- Add "Take a screenshot" tooltip on hover for discoverability
- Remove hover styles on camera button when max images are attached;
  show "Maximum 3 images attached" tooltip instead
- Dismiss any open tooltip on window focus so it doesn't linger
  after the Thuki window hides and reappears mid-screenshot

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: three code-review issues in screenshot feature

- Extract process_screenshot_result() pure fn from capture_screenshot_command
  so the cancellation-detection and file-read logic is testable independently;
  adds three new backend tests (missing file, happy path + deletion, read error).
  Satisfies CLAUDE.md: coverage(off) exclusion is now a genuine thin wrapper.

- Use show_and_make_key() (via tauri-nspanel ManagerExt) when re-showing the
  window after screencapture, mirroring lib.rs show_overlay(). Without this
  the NSPanel may not become the key window and the textarea loses keyboard focus.

- Add MAX_IMAGES guard + attachedImages dep to handleScreenshot in App.tsx.
  Exports MAX_IMAGES from AskBarView so the constant stays DRY. Guard is
  defensive: the disabled button is the primary protection, but the logic layer
  should not rely solely on UI state.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: restore 100% coverage after MAX_IMAGES guard addition

Mark the defensive guard in handleScreenshot with v8 ignore start/stop —
the button is always disabled at max images so this branch is unreachable
through normal UI interaction. Follows the existing project convention
(start/stop pairs, not next) used throughout App.tsx and other components.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

---------

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
quiet-node added a commit that referenced this pull request Apr 10, 2026
@quiet-node
feat: add /screen slash command with tab-completion and screen capture (
f0a5948 
#35)

* docs: add design spec for /screen command and slash command system

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* docs: update spec to reflect existing screenshot.rs from PR #31

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: add /screen slash command with tab-completion system

Implements the full /screen command feature from the 2026-04-05 design spec:

- `src-tauri/src/screenshot.rs`: adds `capture_full_screen` using CoreGraphics
  CGWindowListCreateImageFromArray, filtering out Thuki's own PID so no window
  hide or flicker is needed. Wrapped by `capture_full_screen_command` Tauri command.
- `src-tauri/src/images.rs`: bumps MAX_IMAGES_PER_MESSAGE from 3 to 4 (3 manual + 1 /screen).
- `src-tauri/src/lib.rs`: registers `capture_full_screen_command` in the invoke handler.
- `src/config/commands.ts`: new command registry (single source of truth for slash commands).
- `src/components/CommandSuggestion.tsx`: presentational popover that renders above the
  ask bar when the user types a / prefix.
- `src/view/AskBarView.tsx`: wires up command suggestion state (show/hide, ArrowDown/Up,
  Tab completion, Escape dismiss).
- `src/App.tsx`: detects /screen at submit time, invokes capture_full_screen_command,
  merges screenshot path with attached images, and calls ask() with clean message.
- `docs/commands.md`: user-facing reference doc for all slash commands.

All tests pass (407 frontend, 110 backend). Frontend and backend coverage at 100% lines.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: command suggestion visibility, screenshot crash, and capture error handling

- Render CommandSuggestion in normal DOM flow (not portal) so the
  ResizeObserver grows the native window to reveal the popover in
  ask-bar mode. Wrap in AnimatePresence + motion.div for smooth
  height transition.

- Fix interactive screenshot crash: wrap window.hide() and
  panel.show_and_make_key() in run_on_main_thread() since AppKit
  APIs must only be called from the macOS main thread (was being
  called from tokio pool thread).

- Rewrite capture_full_screen to use CGWindowListCreateImage with
  CGDisplayBounds (concrete screen coordinates) instead of
  CGWindowListCreateImageFromArray with abstract CGRectNull. Use
  kCGWindowListOptionOnScreenBelowWindow to exclude Thuki's own
  NSPanel from the capture.

- Surface Rust backend errors directly in a capture error banner
  instead of swallowing them silently.

- Fix Enter key requiring two presses on exact trigger match:
  fall through to submit instead of completing with trailing space.

- Add outline-none to history button to prevent native macOS focus
  ring after screenshot capture.

- Smooth ask-bar history panel animation with delayed opacity.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: run CoreGraphics capture on main thread to prevent deadlock

CoreGraphics APIs (CGWindowListCreateImage, CGWindowListCopyWindowInfo,
CGDisplayBounds) internally dispatch work to the macOS main thread.
When called from tokio::task::spawn_blocking (a background thread),
they deadlock because the main thread event loop is not processing
their dispatched calls.

Split capture_full_screen_command into two phases:
1. Capture raw RGBA pixels on the main thread via run_on_main_thread()
   and a oneshot channel to return the result
2. Encode PNG and save to disk on a blocking thread (CPU-bound, no
   CoreGraphics dependency)

This matches the pattern used by capture_screenshot_command, which
already dispatches AppKit calls to the main thread.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: show pending chat bubble immediately when /screen capture is in flight

When the user submits a /screen message, the chat view now shows the
user bubble instantly (with a clean loading placeholder thumbnail)
instead of waiting silently for capture to complete. This prevents
double-submit spam and gives immediate visual feedback.

- handleScreenSubmit sets isSubmitPending + pendingUserMessage before
  the invoke call, then clears/commits on success or restores state on
  failure (query and context are preserved so the user can retry)
- ImageThumbnails gains a placeholder prop: renders a translucent
  bg-black/15 tile with a white spinner instead of a broken image
- ChatBubble detects the SCREEN_CAPTURE_PLACEHOLDER sentinel path and
  sets placeholder: true on the thumbnail item
- SCREEN_CAPTURE_PLACEHOLDER constant extracted to config/commands.ts
  so App.tsx and ChatBubble.tsx share a single source of truth

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: address all code review issues before PR

- Replace all em dashes in newly written code with colons or semicolons
  per the global writing style rule (docs, Rust module comments, JSX
  comments, test comments across 7 files)
- Fix non-macOS Rust test: capture_full_screen_returns_err_on_non_macos
  called the nonexistent capture_full_screen(); correct stub is
  capture_full_screen_pixels() (no arguments)
- Clear captureError in replayEntranceAnimation so stale error banners
  do not persist across overlay hide/show cycles
- Fix handleCancel to handle the /screen capture in-flight case: add
  case 2 that signals cancellation via screenCapturePendingRef and
  clears pending UI; handleScreenSubmit checks the ref after await to
  skip ask() when the user cancelled mid-capture
- Fix MAX_IMAGES comment: the constant is the manual attachment limit (3),
  not a mirror of MAX_IMAGES_PER_MESSAGE (4, which includes /screen)
- Add test: cancelling during in-flight capture prevents ask from firing

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: restore input state and reset capture refs on cancel/close/new

- handleCancel case 2 now restores query and selectedContext from
  screenCaptureInputSnapshotRef so the ask bar returns to its pre-capture
  state when the user aborts an in-flight /screen capture
- requestHideOverlay resets screenCapturePendingRef and snapshot ref so
  closing the overlay during a capture does not trigger a ghost ask()
- replayEntranceAnimation and resetForNewConversation do the same, so
  starting a new session mid-capture is also safe
- Remove superpowers design spec from worktree (development artifact only)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: restore input state and reset capture refs on cancel/close/new

- handleCancel case 2 now restores query and selectedContext from
  screenCaptureInputSnapshotRef so the ask bar returns to its pre-capture
  state when the user aborts an in-flight /screen capture
- requestHideOverlay resets screenCapturePendingRef and snapshot ref so
  closing the overlay during a capture does not trigger a ghost ask()
- replayEntranceAnimation and resetForNewConversation do the same, so
  starting a new session mid-capture is also safe

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* docs: update README and CLAUDE.md to reflect /screen slash command

- README features list: add /screen screen capture bullet; update image
  input bullet to clarify it covers both images and screenshots
- README roadmap: mark /screen as shipped in the slash commands item;
  update auto-capture screen context item to note /screen delivers
  full-screen capture today
- CLAUDE.md architecture: add config/commands.ts to Frontend section;
  add screenshot.rs to Backend section
- CLAUDE.md: fix "Command key" -> "Control key" in Key Design Constraints
  to match the rest of the document (activator.rs watches Control, not Command)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix(ci): run build validation on macos-latest

cargo clippy pulls in Tauri's GTK/WebKit dependencies which require
system packages (libwebkit2gtk, libgtk-3-dev) that ubuntu-latest does
not have. Since Thuki is macOS-only and backend tests already run on
macos-latest, switching build validation to the same runner is the
clean fix.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

---------

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@github-actions github-actions Bot mentioned this pull request Apr 10, 2026
Closed
quiet-node added a commit that referenced this pull request Apr 10, 2026
@quiet-node
feat: screenshot capture for image input (#31)
88f0fef 
* feat: add capture_screenshot Tauri command with macOS screencapture -i

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: add screenshot capture button to AskBarView

Adds a camera icon button to the AskBarView input bar that triggers
onScreenshot. The button is disabled while generating, submit is pending,
or the max image count is already reached. Includes a temporary no-op
stub in App.tsx to satisfy TypeScript until Task 4 wires up the handler.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: wire screenshot capture into App — handleScreenshot feeds handleImagesAttached

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: apply Prettier and cargo fmt formatting to screenshot feature files

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: correct capture_screenshot IPC command name to capture_screenshot_command

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: polish screenshot button UX — viewfinder icon, hover tooltip, max-images guard

- Replace camera lens icon with a 4-corner viewfinder/frame icon
- Add "Take a screenshot" tooltip on hover for discoverability
- Remove hover styles on camera button when max images are attached;
  show "Maximum 3 images attached" tooltip instead
- Dismiss any open tooltip on window focus so it doesn't linger
  after the Thuki window hides and reappears mid-screenshot

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: three code-review issues in screenshot feature

- Extract process_screenshot_result() pure fn from capture_screenshot_command
  so the cancellation-detection and file-read logic is testable independently;
  adds three new backend tests (missing file, happy path + deletion, read error).
  Satisfies CLAUDE.md: coverage(off) exclusion is now a genuine thin wrapper.

- Use show_and_make_key() (via tauri-nspanel ManagerExt) when re-showing the
  window after screencapture, mirroring lib.rs show_overlay(). Without this
  the NSPanel may not become the key window and the textarea loses keyboard focus.

- Add MAX_IMAGES guard + attachedImages dep to handleScreenshot in App.tsx.
  Exports MAX_IMAGES from AskBarView so the constant stays DRY. Guard is
  defensive: the disabled button is the primary protection, but the logic layer
  should not rely solely on UI state.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: restore 100% coverage after MAX_IMAGES guard addition

Mark the defensive guard in handleScreenshot with v8 ignore start/stop —
the button is always disabled at max images so this branch is unreachable
through normal UI interaction. Follows the existing project convention
(start/stop pairs, not next) used throughout App.tsx and other components.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

---------

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
quiet-node added a commit that referenced this pull request Apr 10, 2026
@quiet-node
feat: add /screen slash command with tab-completion and screen capture (
072de70 
#35)

* docs: add design spec for /screen command and slash command system

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* docs: update spec to reflect existing screenshot.rs from PR #31

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: add /screen slash command with tab-completion system

Implements the full /screen command feature from the 2026-04-05 design spec:

- `src-tauri/src/screenshot.rs`: adds `capture_full_screen` using CoreGraphics
  CGWindowListCreateImageFromArray, filtering out Thuki's own PID so no window
  hide or flicker is needed. Wrapped by `capture_full_screen_command` Tauri command.
- `src-tauri/src/images.rs`: bumps MAX_IMAGES_PER_MESSAGE from 3 to 4 (3 manual + 1 /screen).
- `src-tauri/src/lib.rs`: registers `capture_full_screen_command` in the invoke handler.
- `src/config/commands.ts`: new command registry (single source of truth for slash commands).
- `src/components/CommandSuggestion.tsx`: presentational popover that renders above the
  ask bar when the user types a / prefix.
- `src/view/AskBarView.tsx`: wires up command suggestion state (show/hide, ArrowDown/Up,
  Tab completion, Escape dismiss).
- `src/App.tsx`: detects /screen at submit time, invokes capture_full_screen_command,
  merges screenshot path with attached images, and calls ask() with clean message.
- `docs/commands.md`: user-facing reference doc for all slash commands.

All tests pass (407 frontend, 110 backend). Frontend and backend coverage at 100% lines.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: command suggestion visibility, screenshot crash, and capture error handling

- Render CommandSuggestion in normal DOM flow (not portal) so the
  ResizeObserver grows the native window to reveal the popover in
  ask-bar mode. Wrap in AnimatePresence + motion.div for smooth
  height transition.

- Fix interactive screenshot crash: wrap window.hide() and
  panel.show_and_make_key() in run_on_main_thread() since AppKit
  APIs must only be called from the macOS main thread (was being
  called from tokio pool thread).

- Rewrite capture_full_screen to use CGWindowListCreateImage with
  CGDisplayBounds (concrete screen coordinates) instead of
  CGWindowListCreateImageFromArray with abstract CGRectNull. Use
  kCGWindowListOptionOnScreenBelowWindow to exclude Thuki's own
  NSPanel from the capture.

- Surface Rust backend errors directly in a capture error banner
  instead of swallowing them silently.

- Fix Enter key requiring two presses on exact trigger match:
  fall through to submit instead of completing with trailing space.

- Add outline-none to history button to prevent native macOS focus
  ring after screenshot capture.

- Smooth ask-bar history panel animation with delayed opacity.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: run CoreGraphics capture on main thread to prevent deadlock

CoreGraphics APIs (CGWindowListCreateImage, CGWindowListCopyWindowInfo,
CGDisplayBounds) internally dispatch work to the macOS main thread.
When called from tokio::task::spawn_blocking (a background thread),
they deadlock because the main thread event loop is not processing
their dispatched calls.

Split capture_full_screen_command into two phases:
1. Capture raw RGBA pixels on the main thread via run_on_main_thread()
   and a oneshot channel to return the result
2. Encode PNG and save to disk on a blocking thread (CPU-bound, no
   CoreGraphics dependency)

This matches the pattern used by capture_screenshot_command, which
already dispatches AppKit calls to the main thread.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: show pending chat bubble immediately when /screen capture is in flight

When the user submits a /screen message, the chat view now shows the
user bubble instantly (with a clean loading placeholder thumbnail)
instead of waiting silently for capture to complete. This prevents
double-submit spam and gives immediate visual feedback.

- handleScreenSubmit sets isSubmitPending + pendingUserMessage before
  the invoke call, then clears/commits on success or restores state on
  failure (query and context are preserved so the user can retry)
- ImageThumbnails gains a placeholder prop: renders a translucent
  bg-black/15 tile with a white spinner instead of a broken image
- ChatBubble detects the SCREEN_CAPTURE_PLACEHOLDER sentinel path and
  sets placeholder: true on the thumbnail item
- SCREEN_CAPTURE_PLACEHOLDER constant extracted to config/commands.ts
  so App.tsx and ChatBubble.tsx share a single source of truth

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: address all code review issues before PR

- Replace all em dashes in newly written code with colons or semicolons
  per the global writing style rule (docs, Rust module comments, JSX
  comments, test comments across 7 files)
- Fix non-macOS Rust test: capture_full_screen_returns_err_on_non_macos
  called the nonexistent capture_full_screen(); correct stub is
  capture_full_screen_pixels() (no arguments)
- Clear captureError in replayEntranceAnimation so stale error banners
  do not persist across overlay hide/show cycles
- Fix handleCancel to handle the /screen capture in-flight case: add
  case 2 that signals cancellation via screenCapturePendingRef and
  clears pending UI; handleScreenSubmit checks the ref after await to
  skip ask() when the user cancelled mid-capture
- Fix MAX_IMAGES comment: the constant is the manual attachment limit (3),
  not a mirror of MAX_IMAGES_PER_MESSAGE (4, which includes /screen)
- Add test: cancelling during in-flight capture prevents ask from firing

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: restore input state and reset capture refs on cancel/close/new

- handleCancel case 2 now restores query and selectedContext from
  screenCaptureInputSnapshotRef so the ask bar returns to its pre-capture
  state when the user aborts an in-flight /screen capture
- requestHideOverlay resets screenCapturePendingRef and snapshot ref so
  closing the overlay during a capture does not trigger a ghost ask()
- replayEntranceAnimation and resetForNewConversation do the same, so
  starting a new session mid-capture is also safe
- Remove superpowers design spec from worktree (development artifact only)

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: restore input state and reset capture refs on cancel/close/new

- handleCancel case 2 now restores query and selectedContext from
  screenCaptureInputSnapshotRef so the ask bar returns to its pre-capture
  state when the user aborts an in-flight /screen capture
- requestHideOverlay resets screenCapturePendingRef and snapshot ref so
  closing the overlay during a capture does not trigger a ghost ask()
- replayEntranceAnimation and resetForNewConversation do the same, so
  starting a new session mid-capture is also safe

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* docs: update README and CLAUDE.md to reflect /screen slash command

- README features list: add /screen screen capture bullet; update image
  input bullet to clarify it covers both images and screenshots
- README roadmap: mark /screen as shipped in the slash commands item;
  update auto-capture screen context item to note /screen delivers
  full-screen capture today
- CLAUDE.md architecture: add config/commands.ts to Frontend section;
  add screenshot.rs to Backend section
- CLAUDE.md: fix "Command key" -> "Control key" in Key Design Constraints
  to match the rest of the document (activator.rs watches Control, not Command)

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix(ci): run build validation on macos-latest

cargo clippy pulls in Tauri's GTK/WebKit dependencies which require
system packages (libwebkit2gtk, libgtk-3-dev) that ubuntu-latest does
not have. Since Thuki is macOS-only and backend tests already run on
macos-latest, switching build validation to the same runner is the
clean fix.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

---------

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
quiet-node added a commit that referenced this pull request Apr 11, 2026
@quiet-node
feat: screenshot capture for image input (#31)
78f28e6 
* feat: add capture_screenshot Tauri command with macOS screencapture -i

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: add screenshot capture button to AskBarView

Adds a camera icon button to the AskBarView input bar that triggers
onScreenshot. The button is disabled while generating, submit is pending,
or the max image count is already reached. Includes a temporary no-op
stub in App.tsx to satisfy TypeScript until Task 4 wires up the handler.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: wire screenshot capture into App — handleScreenshot feeds handleImagesAttached

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: apply Prettier and cargo fmt formatting to screenshot feature files

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: correct capture_screenshot IPC command name to capture_screenshot_command

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: polish screenshot button UX — viewfinder icon, hover tooltip, max-images guard

- Replace camera lens icon with a 4-corner viewfinder/frame icon
- Add "Take a screenshot" tooltip on hover for discoverability
- Remove hover styles on camera button when max images are attached;
  show "Maximum 3 images attached" tooltip instead
- Dismiss any open tooltip on window focus so it doesn't linger
  after the Thuki window hides and reappears mid-screenshot

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: three code-review issues in screenshot feature

- Extract process_screenshot_result() pure fn from capture_screenshot_command
  so the cancellation-detection and file-read logic is testable independently;
  adds three new backend tests (missing file, happy path + deletion, read error).
  Satisfies CLAUDE.md: coverage(off) exclusion is now a genuine thin wrapper.

- Use show_and_make_key() (via tauri-nspanel ManagerExt) when re-showing the
  window after screencapture, mirroring lib.rs show_overlay(). Without this
  the NSPanel may not become the key window and the textarea loses keyboard focus.

- Add MAX_IMAGES guard + attachedImages dep to handleScreenshot in App.tsx.
  Exports MAX_IMAGES from AskBarView so the constant stays DRY. Guard is
  defensive: the disabled button is the primary protection, but the logic layer
  should not rely solely on UI state.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: restore 100% coverage after MAX_IMAGES guard addition

Mark the defensive guard in handleScreenshot with v8 ignore start/stop —
the button is always disabled at max images so this branch is unreachable
through normal UI interaction. Follows the existing project convention
(start/stop pairs, not next) used throughout App.tsx and other components.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

---------

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
quiet-node added a commit that referenced this pull request Apr 11, 2026
@quiet-node
feat: add /screen slash command with tab-completion and screen capture (
d6ff460 
#35)

* docs: add design spec for /screen command and slash command system

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* docs: update spec to reflect existing screenshot.rs from PR #31

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: add /screen slash command with tab-completion system

Implements the full /screen command feature from the 2026-04-05 design spec:

- `src-tauri/src/screenshot.rs`: adds `capture_full_screen` using CoreGraphics
  CGWindowListCreateImageFromArray, filtering out Thuki's own PID so no window
  hide or flicker is needed. Wrapped by `capture_full_screen_command` Tauri command.
- `src-tauri/src/images.rs`: bumps MAX_IMAGES_PER_MESSAGE from 3 to 4 (3 manual + 1 /screen).
- `src-tauri/src/lib.rs`: registers `capture_full_screen_command` in the invoke handler.
- `src/config/commands.ts`: new command registry (single source of truth for slash commands).
- `src/components/CommandSuggestion.tsx`: presentational popover that renders above the
  ask bar when the user types a / prefix.
- `src/view/AskBarView.tsx`: wires up command suggestion state (show/hide, ArrowDown/Up,
  Tab completion, Escape dismiss).
- `src/App.tsx`: detects /screen at submit time, invokes capture_full_screen_command,
  merges screenshot path with attached images, and calls ask() with clean message.
- `docs/commands.md`: user-facing reference doc for all slash commands.

All tests pass (407 frontend, 110 backend). Frontend and backend coverage at 100% lines.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: command suggestion visibility, screenshot crash, and capture error handling

- Render CommandSuggestion in normal DOM flow (not portal) so the
  ResizeObserver grows the native window to reveal the popover in
  ask-bar mode. Wrap in AnimatePresence + motion.div for smooth
  height transition.

- Fix interactive screenshot crash: wrap window.hide() and
  panel.show_and_make_key() in run_on_main_thread() since AppKit
  APIs must only be called from the macOS main thread (was being
  called from tokio pool thread).

- Rewrite capture_full_screen to use CGWindowListCreateImage with
  CGDisplayBounds (concrete screen coordinates) instead of
  CGWindowListCreateImageFromArray with abstract CGRectNull. Use
  kCGWindowListOptionOnScreenBelowWindow to exclude Thuki's own
  NSPanel from the capture.

- Surface Rust backend errors directly in a capture error banner
  instead of swallowing them silently.

- Fix Enter key requiring two presses on exact trigger match:
  fall through to submit instead of completing with trailing space.

- Add outline-none to history button to prevent native macOS focus
  ring after screenshot capture.

- Smooth ask-bar history panel animation with delayed opacity.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: run CoreGraphics capture on main thread to prevent deadlock

CoreGraphics APIs (CGWindowListCreateImage, CGWindowListCopyWindowInfo,
CGDisplayBounds) internally dispatch work to the macOS main thread.
When called from tokio::task::spawn_blocking (a background thread),
they deadlock because the main thread event loop is not processing
their dispatched calls.

Split capture_full_screen_command into two phases:
1. Capture raw RGBA pixels on the main thread via run_on_main_thread()
   and a oneshot channel to return the result
2. Encode PNG and save to disk on a blocking thread (CPU-bound, no
   CoreGraphics dependency)

This matches the pattern used by capture_screenshot_command, which
already dispatches AppKit calls to the main thread.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* feat: show pending chat bubble immediately when /screen capture is in flight

When the user submits a /screen message, the chat view now shows the
user bubble instantly (with a clean loading placeholder thumbnail)
instead of waiting silently for capture to complete. This prevents
double-submit spam and gives immediate visual feedback.

- handleScreenSubmit sets isSubmitPending + pendingUserMessage before
  the invoke call, then clears/commits on success or restores state on
  failure (query and context are preserved so the user can retry)
- ImageThumbnails gains a placeholder prop: renders a translucent
  bg-black/15 tile with a white spinner instead of a broken image
- ChatBubble detects the SCREEN_CAPTURE_PLACEHOLDER sentinel path and
  sets placeholder: true on the thumbnail item
- SCREEN_CAPTURE_PLACEHOLDER constant extracted to config/commands.ts
  so App.tsx and ChatBubble.tsx share a single source of truth

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: address all code review issues before PR

- Replace all em dashes in newly written code with colons or semicolons
  per the global writing style rule (docs, Rust module comments, JSX
  comments, test comments across 7 files)
- Fix non-macOS Rust test: capture_full_screen_returns_err_on_non_macos
  called the nonexistent capture_full_screen(); correct stub is
  capture_full_screen_pixels() (no arguments)
- Clear captureError in replayEntranceAnimation so stale error banners
  do not persist across overlay hide/show cycles
- Fix handleCancel to handle the /screen capture in-flight case: add
  case 2 that signals cancellation via screenCapturePendingRef and
  clears pending UI; handleScreenSubmit checks the ref after await to
  skip ask() when the user cancelled mid-capture
- Fix MAX_IMAGES comment: the constant is the manual attachment limit (3),
  not a mirror of MAX_IMAGES_PER_MESSAGE (4, which includes /screen)
- Add test: cancelling during in-flight capture prevents ask from firing

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: restore input state and reset capture refs on cancel/close/new

- handleCancel case 2 now restores query and selectedContext from
  screenCaptureInputSnapshotRef so the ask bar returns to its pre-capture
  state when the user aborts an in-flight /screen capture
- requestHideOverlay resets screenCapturePendingRef and snapshot ref so
  closing the overlay during a capture does not trigger a ghost ask()
- replayEntranceAnimation and resetForNewConversation do the same, so
  starting a new session mid-capture is also safe
- Remove superpowers design spec from worktree (development artifact only)

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix: restore input state and reset capture refs on cancel/close/new

- handleCancel case 2 now restores query and selectedContext from
  screenCaptureInputSnapshotRef so the ask bar returns to its pre-capture
  state when the user aborts an in-flight /screen capture
- requestHideOverlay resets screenCapturePendingRef and snapshot ref so
  closing the overlay during a capture does not trigger a ghost ask()
- replayEntranceAnimation and resetForNewConversation do the same, so
  starting a new session mid-capture is also safe

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* docs: update README and CLAUDE.md to reflect /screen slash command

- README features list: add /screen screen capture bullet; update image
  input bullet to clarify it covers both images and screenshots
- README roadmap: mark /screen as shipped in the slash commands item;
  update auto-capture screen context item to note /screen delivers
  full-screen capture today
- CLAUDE.md architecture: add config/commands.ts to Frontend section;
  add screenshot.rs to Backend section
- CLAUDE.md: fix "Command key" -> "Control key" in Key Design Constraints
  to match the rest of the document (activator.rs watches Control, not Command)

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

* fix(ci): run build validation on macos-latest

cargo clippy pulls in Tauri's GTK/WebKit dependencies which require
system packages (libwebkit2gtk, libgtk-3-dev) that ubuntu-latest does
not have. Since Thuki is macOS-only and backend tests already run on
macos-latest, switching build validation to the same runner is the
clean fix.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>

---------

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
This was referenced Apr 11, 2026
Closed
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@quiet-node