Chameleon — dictation fits where you drop it. (Implementation of advanced adaptive context-aware smart spacing, capitalization, punctuation, and text insertion logic)

[pianoandglass]

Chameleon — context-aware dictation that fits where you drop it

Dictated text now adapts to its surroundings — capitalization, spacing, and punctuation — instead of being pasted in raw at the cursor. It reads the text around the cursor without ever disturbing it, and the cleanup model uses that context as a hint too. The new post-LLM step happens entirely locally and instantly.

Closes #200.

---

What it does, in plain terms

Today, when the user dictates, the cleaned‑up text is dropped at the cursor exactly as the model produced it: the first word capitalized, a space tacked onto the end, and nothing else adjusted for what is actually around the cursor. That is fine when it is starting a clean new line, but at the moment we are anywhere else (mid‑sentence, right before a question mark, inside parentheses, in the middle of a list, or replacing a selection) the result comes out messy: a stray capital letter, missing or doubled spaces, and punctuation that clashes with what was already there.

This change reads the few characters around the cursor and reshapes the inserted text so it reads naturally in that exact spot: it fixes the first-letter case, adds or removes the space on each side, cleans up clashing punctuation, recognizes real paragraph breaks, and keeps your custom spellings.

It works in normal Mac apps and inside web / Electron apps (chat apps, editors, browsers), where the cursor information is normally unreliable.

---

See it in action

In each pair, │ is the cursor. "Before" is how it works today (first word capitalized, a trailing space, nothing else touched); "After" is with this change.

Mid-sentence — at I think we should │, you say "go home now"

• Before: I think we should Go home now. (stray capital G, plus a leftover space at the end)
• After: I think we should go home now.

Right before a question mark — at Did you finish│?, you say "the report yesterday"

• Before: Did you finishThe report yesterday. ? (glued to the word, capital T, a period, a space, then the ?)
• After: Did you finish the report yesterday?

In the middle of a list — at I bought apples│, oranges, and pears., you say "bananas"

• Before: I bought applesBananas. , oranges, and pears. (glued, capital B, a period and a space dumped before the comma)
• After: I bought apples bananas, oranges, and pears.

Right before an existing comma — at we met on Tuesday│, then left, you say "early" and the model adds its own comma/period

• Before: we met on Tuesday Early., then left (the model's trailing mark collides with the comma already there)
• After: we met on Tuesday early, then left (the document's comma wins; the model's trailing mark is dropped)

Two sentences in a row — at I finished the task.│, you say "let's review it tomorrow"

• Before: I finished the task.Let's review it tomorrow. (no space after the first period, leftover space at the end)
• After: I finished the task. Let's review it tomorrow.

Inside a line of code — at print(│), you say "hello world"

• Before: print(Hello world. ) (capital H, a period, and a space before the closing parenthesis)
• After: print(hello world)

Replacing a selection — in the old replace, you select old and say "seamless"

• Before: the Seamless. replace (capital S, a stray period, and a double space)
• After: the seamless replace

Custom vocabulary awareness, mid-sentence — vocabulary GitHub, API; at we use │ daily, you say "github and the api"

• Before: we use Github and the api. daily (wrong casing, a period mid-sentence, a double space)
• After: we use GitHub and the API daily

---

What the Run Log shows

This feature also turns the FreeFlow's setting tab Run Log into a full, readable trace of every dictation, with a completely new stage 4 of the pipeline, logging exactly how a piece of text travelled from your voice to the cursor. Most of this is new:

• The new fourth stage: deterministic, on-device formatting. After the model, the text runs through the rule-based formatter this change adds. The Run Log shows its formatted output and a plain-English summary of every rule that fired — no network, no model, just deterministic local rules.
• What was read around the cursor. The kind of app (a standard Mac app or a web/Electron view), how the surrounding text was read (the exact method/rung that succeeded), whether any text was found at all, and the actual text before and after the cursor — the exact context the insertion was based on.
• What the post-processing model received and returned. The raw transcript that went into the cleanup model and the cleaned text it gave back, shown together with the bounded surrounding context that informed it.
• How it was finally inserted. The resulting text shown in place (text before + inserted text + text after), exactly as it landed at the cursor.

The labels are faithful: the "how it was read" line reflects the method that actually produced the context (so if the non-destructive start-of-recording snapshot was used, that is what it says — not a stale value).

That summary of the fourth stage reads like this:

• "Capitalized the first letter" — at the start of an empty field.
• "Made the first letter lowercase, added a space before" — continuing a sentence mid-line.
• "Capitalized the first letter, added a space before" — right after a period.
• "Added spaces on both sides" — dropped between two words.
• "Added a space after" — inserted just before existing text.
• "Removed an extra punctuation mark" — a period/comma/semicolon clashing with a following ), ?, or comma.
• "Removed an extra punctuation mark, made the first letter lowercase" — a mid-sentence period before lowercase text.
• "Restored a custom-word spelling" — a vocabulary word corrected (github to GitHub).
• "Replaced the selected text" — dictating over a selection.
• "No changes" — the text already fit perfectly.

---

What's new

1. Context-aware capitalization. The first letter follows the sentence: uppercase at the start of a field, after ./?/!/…, after a line/paragraph break, after a bullet, or when opening a quote/parenthesis that begins a sentence — lowercase when you are continuing one. Abbreviations (Dr., i.e., etc.-shaped tokens), the pronoun I and its contractions (I'm, I'll…), and all-caps acronyms (API, NASA) are never mangled, and a period that ends an abbreviation is not treated as a sentence boundary.

2. Smart spacing at the seams. Today a single space is always tacked onto the end of every dictated sentence, no matter the context. This change drops that blanket trailing space and instead adds or omits a space on each side based on what is actually there — words and numbers get a space, opening/closing punctuation does not — and double spaces are never produced.

3. Punctuation cleanup. Removes punctuation that clashes with what follows: a period before a closing )/,/?, a duplicated terminal mark (!!, ,,), a mid-sentence period before lowercase text, a stray ellipsis dropped into the middle of a sentence, and — when you are inserting right before an existing comma — any terminal mark the model appended (, . ; : ! ? …), so the comma already in the document stays the single separator (abbreviation periods are preserved).

4. Correct replacement of selected text. When you dictate over a selection (ordinary dictation, not the AI "edit selected text" command mode), the spaces that belonged to the selection are restored, so the replacement does not glue itself to the neighboring words. The selected text is read untrimmed so those edge spaces survive.

5. Custom-vocabulary casing. Any word you defined (a name, a brand, an acronym) is restored to your exact spelling wherever it appears in the dictation — not just the first word — matched as a whole word so it never corrupts substrings (MacOS stays MacOS).

6. Reliable, purely observational context reading in web / Electron apps. Uses the WebKit/Chromium text-position API (TextMarkers) as the primary read: it addresses the real document position, so it is immune to the lying integer cursor (Chromium reports a fraudulent (0,0)) and it preserves the paragraph breaks that the plain value read flattens to a space — including a caret sitting on a blank line, which is reconstructed as a real line start. A self-scoped form-control read, a tree search, and the start-of-recording snapshot are the fallbacks. The accessibility tree is woken just-in-time so the very first dictation in an app still works. Reading context never types, selects, or moves the caret — on any path.

7. The cleanup model sees a bounded slice of the surrounding text. The post-processing model now also receives the text immediately before and after the cursor as a reference — so it can match the spelling, casing, and tone of what is already there (for example, a recipient's name visible just before the cursor). That slice is bounded (a couple of sentences before, one after, capped by length, flattened to one line) and the prompt explicitly instructs the model never to copy the surrounding text into its output — together this prevents the model from "restoring" already-typed text.

8. Recognizes real paragraph / page breaks. Line, paragraph, and page breaks are normalized to a canonical newline before the formatter looks at them, so capitalization after a true paragraph break works regardless of how the app encodes the break (CR, CRLF, Unicode line/paragraph separators, next-line, or form-feed).

9. Full pipeline visibility in the Run Log. Each dictation is now traced end to end — what was read around the cursor (and by which method), what the cleanup model received and returned, what the new deterministic formatting stage did (in plain English), and the final inserted text — plus a per-session metric of how often the reads succeed versus come back blind.

---

How it works (the short version)

• Reads up to a few hundred characters before and after the cursor through the macOS Accessibility API, without ever typing, selecting, or moving the cursor — the read is purely observational on every path.
• Native controls use the standard API. Web / Electron views are read with the WebKit/Chromium TextMarker attributes first — they follow the real document position, so they keep the line breaks the plain value read flattens and they ignore the unreliable integer cursor. A real form field (<textarea>/<input>) whose leaf doesn't host TextMarkers is read from its own self-scoped value instead, a breadth-first search finds the field when the app hands over the wrong element, and a snapshot taken when recording started covers a blind stop-time read.
• A small, bounded slice of that context goes to the cleanup model as a hint (never copied into the output); the full context feeds the deterministic formatter.
• The deterministic formatter then runs four ordered phases — normalize, punctuation, capitalization, spacing — plus the vocabulary-casing pass.
• None of this changes what you said; it only changes how the already-cleaned text is fitted into place.

---

How it works (the technical version)

1. Reading the surrounding text — an explicit extraction ladder

Context is read by AccessibilityTextReader, which tries methods in priority order and stops at the first that returns usable text. Every rung is purely observational (nothing is ever typed, selected, or scrolled), and the ladder branches on whether the focused element is a native control or a web/Electron view (detected by AXDOM*/AXWeb* attribute names on the element, or — since Chromium does not guarantee those prefixes on every focused leaf — by an AXWebArea ancestor):

• Native (AppKit): read kAXValue + kAXSelectedTextRange and slice around the selection. Reliable; nothing else is needed; the destructive fallback is never reached for native apps.

• Web / Electron:

  1. TextMarkers (primary) — the WebKit/Chromium AXTextMarker* parameterized attributes, with the underlying C functions (AXTextMarkerRangeCopyStartMarker / …EndMarker / AXTextMarkerRangeCreate) resolved at runtime via dlsym. TextMarkers address positions by opaque markers instead of integer offsets, so they are immune to the fraudulent (0,0) cursor Chromium reports through kAXSelectedTextRange, and they preserve structural paragraph breaks — a web app often renders a line break as a DOM boundary with no literal newline in the plain value, so an offset read cannot see it and would treat a fresh line as mid-sentence. When the caret sits at the start of its line — including a blank line, whose degenerate line-head range is recognized as a line start — the missing newline is reconstructed, so capitalization after a break comes out right. The read is scoped to the focused field (AXTextMarkerRangeForUIElement, asked for the original leaf even when the markers are hosted on the AXWebArea ancestor; if field-scoping is unavailable it clamps to the caret's current line) so it does not reach back into unrelated page content (e.g. a live-region "response ready" status just above a chat input).
  2. Self-scoped form-control read — a genuine HTML <textarea>/<input> is exposed by Safari/Chromium as an accessibility leaf (AXTextField/AXTextArea) whose text is its own kAXValue + kAXSelectedTextRange — the same contract a native macOS field uses, and one that can never leak neighbouring page text. It covers leaves that don't host TextMarkers. The gate is site-agnostic — it keys on the element's accessibility contract (plain leaf text control + honest caret), never on per-website markup — and bails for anything that is not a plain control or that reports the fraudulent (0,0) cursor.
  3. Offset AX with an explicit fraudulent-(0,0) guard — in both the async and the synchronous pre-paste read, so a lying caret can never fabricate a confident field-start.
  4. Breadth-first search for the focused element inside an AXWebArea (when the app hands us the wrong element from its root).
  5. Start-of-recording snapshot — a read taken when recording began (the text before the cursor does not change while you dictate, so it is still valid at paste time). If the stop-time read came back blind, a just-in-time re-read runs right before pasting — up to 3 observational attempts under a 2-second wall-clock budget, so a hung target app can never hold the paste hostage.
  6. Blind — return no context; the formatter degrades gracefully (it preserves the model's own casing rather than forcing a sentence start, and keeps the upstream trailing space after sentence punctuation so consecutive dictations don't jam together).

A built-in cross-check keeps the web reads honest: when the focused box exposes its own trustworthy text, a marker read is only used if it matches that text right next to the cursor — if it doesn't (say, a status banner from elsewhere on the page), the box's own text wins. This was added after a real-world catch on Google's AI-mode search box, whose content is invisible to the marker system: without the check, a page message ("the AI response is ready") was silently mistaken for the text before the cursor. The comparison ignores whitespace on purpose, because a faithful marker read differs from the plain value only in how line breaks are represented — which is exactly why markers stay the primary read.

The accessibility tree is woken just-in-time by setting the private AXManualAccessibility attribute (Chromium keeps its tree asleep for performance, and builds it lazily). This replaced an older trick that nudged the tree awake by synthesizing arrow keys — which is exactly what could shift the caret — so that key-based wake and its dead parameters were removed.

Robustness when the machine is busy. Reading another app's text over the Accessibility API is a cross-process request, so a CPU-starved or unresponsive target app could otherwise stall the read. Three safeguards bound that: (a) a per-request timeout (AXUIElementSetMessagingTimeout, 2 s) so a single Accessibility call can't block for the ~6 s system default — a slow app degrades to a fallback instead of hanging; (b) the heavy read runs off the main thread, so the app's own UI never freezes while waiting; and (c) the just-in-time Accessibility wake is held for a grace period and until the in-flight paste pipeline finishes (teardown is skipped entirely if a newer dictation re-armed the wake, and the private AXManualAccessibility attribute is switched back off at teardown), so a slow paste can't have the tree torn down underneath it. These bound latency, not success: a non-empty read is never guaranteed for any cross-process Accessibility read, but the pipeline always completes and the dictation is never dropped — at worst it falls back to a plain insertion.

2. Bounding the context sent to the cleanup model (SurroundingTextLimiter)

Feeding the model the full surrounding text invites it to echo that text back into its answer (duplicating already-typed content). A new, self-contained SurroundingTextLimiter produces a small, bounded slice for the prompt only:

• keep at most the last 2 sentences before the cursor and the first sentence after it;
• enforce a character budget with a tolerance band, reducing the sentence count before truncating;
• if a single sentence is still over budget, truncate at a word boundary (never mid-word; an over-long single token is kept whole rather than split);
• flatten whitespace to single spaces for the prompt.

This affects the prompt only — the deterministic formatter still receives the full, unflattened text with its real line breaks. The user-facing prompt was also strengthened: the surrounding text is framed as reference-only, with a positive role description plus an explicit "never copy, repeat, continue, or complete the surrounding text" instruction.

Prompt hygiene. The prompt labels its sections with uppercase tokens (PRECEDING_TEXT, FOLLOWING_TEXT, …) and the model is explicitly told never to copy or emit them. The surrounding-text values are sentinel-fenced exactly like the transcript itself (<<<PRECEDING_TEXT … PRECEDING_TEXT), so a quote character in the on-screen text cannot break the framing and screen content gets the same injection hardening as dictated audio. Because a model can otherwise narrate an empty labeled section, empty PRECEDING_TEXT/FOLLOWING_TEXT sections are omitted from the prompt entirely (no blank labeled field to comment on). Leaks are prevented at the prompt level rather than scrubbed from the output.

3. The deterministic formatter (ContextualFormattingService), four ordered phases

1. Normalize — trim, standardize ellipses, and canonicalize every line/paragraph/page break (\r, \r\n, U+2028, U+2029, U+0085, U+000C) to \n so the later phases recognize a real break regardless of encoding.
2. Punctuation — drop a stray mid-sentence ellipsis; collapse a duplicated terminal mark; remove a trailing period that collides with a following closing/terminal mark or with lowercase continuation (abbreviations exempt); and, before an existing comma, drop any terminal mark the model appended.
3. Capitalization — decide the first letter from the preceding context (field start, after .?!…, after a line/paragraph break, after a bullet, after a sentence-opening quote/parenthesis → uppercase; mid-sentence → lowercase), while preserving abbreviations, the pronoun I/contractions, and all-caps acronyms, and treating an abbreviation-ending period as not a boundary.
4. Spacing — add/omit a leading and trailing space from what is actually on each side, restore edge spaces consumed when replacing a selection, and collapse any accidental double space. Quotes are handled by role: curly and guillemet quotes (“ ” « ») are unambiguous by Unicode, and an ambiguous straight quote (" ') is classified opening/closing by its neighbouring character — English and Portuguese conventions both come out right. Slices are snapped to composed-character boundaries, so a capped read can never split an emoji or accent into a replacement character.

The cursor-position semantics were also tightened: when only one side of the cursor can be read, the position is reported as middle/unknown rather than incorrectly claiming end.

4. Faithful diagnostics

ContextDebugLabels is the single source of labels so the live debug panel and the saved history can never disagree, and ContextReadMetrics tallies per session how often each read method won. The recorded extraction method always reflects the rung that actually produced the context, every read is purpose-tagged in the logs ([start]/[stop]/[jit]/[paste]) so a trace attributes each line to its pipeline step, and a missing cursor position is treated as "unknown" so the pre-paste rescue is never silently skipped.

5. Merge-safety / additivity

The shared backbone files are touched additively so this branch stays merge-compatible with other in-flight work regardless of merge order: the cleanup-prompt change is new defaulted parameters added as new lines inside otherwise-untouched call sites (contested lines stay byte-identical to upstream); the surrounding-text capture lives in the feature's own region of collectContext; and the heavy logic lives in new, feature-owned files. The two largest files (AccessibilityTextReader, ContextualFormattingService) and SurroundingTextLimiter exist only on this branch.

---

Files

New

File	What it adds
ContextualFormattingService.swift	The deterministic formatter: four ordered phases (normalize, punctuation, capitalization, spacing) plus vocabulary-casing, and line-break normalization. The brain of the feature.
AccessibilityTextReader.swift	Reads the text around the cursor via the extraction ladder: WebKit/Chromium TextMarkers first for web/Electron (resolved with dlsym, field-scoped, structural line breaks preserved), the self-scoped read for real <textarea>/<input> leaves, the offset API for native controls, and a web-area tree search. Purely observational on every path.
SurroundingTextLimiter.swift	Bounds the surrounding text sent to the cleanup model (sentence cap + character budget with tolerance + word-safe truncation + whitespace flatten) so the model can use it as a hint without echoing it back.
AccessibilityWakeManager.swift	Wakes the accessibility tree just-in-time via AXManualAccessibility (and observes focus/selection) so the first read in an app is reliable — without synthesizing any keystrokes.
ContextDebugLabels.swift	Single source of plain-English labels for the debug panels, so the live and saved views can never disagree.
ContextReadMetrics.swift	Session tally of how the surrounding text was read (per method, vs blind), logged per dictation.

Modified

File	What changed and why
AppState.swift	Wires the formatter into the paste pipeline; starts the gentle context read when recording begins and resolves the final context at stop; runs the observational just-in-time re-read (3 attempts, 2-second budget) when the stop read was blind; normalizes the resolved context's line breaks once; passes the custom vocabulary; logs the read method/metric; and holds the Accessibility wake until the in-flight paste finishes.
AppContextService.swift	Collects the surrounding text around the cursor (non-destructively) alongside the rest of the context, and reads the selected text untrimmed for correct space restoration. Screenshot and activity-summary behavior left identical to upstream.
PostProcessingService.swift	Passes a bounded slice of the surrounding text into the cleanup prompt as a reference (via SurroundingTextLimiter), with a strengthened never-copy instruction — added through new defaulted parameters so the change is purely additive and stays merge-compatible with other work on this file.
PipelineHistoryItem.swift	New optional fields (surrounding text, cursor position, applied rule, read method, app kind) so history entries are backward-compatible.
PipelineHistoryStore.swift	Lightweight Core Data migration that adds the new fields to old stores without data loss.
PipelineDebugContentView.swift	The live debug panel shows the smart-paste result in plain English.
PipelineDebugPanelView.swift	Passes the new context fields through to the panel.
SettingsView.swift	The saved-history view shows the same plain-English smart-paste summary.

---

Notes

• Never disturbs the document. Reading context never types, selects, or moves the cursor — every rung of the ladder is purely observational, and the clipboard is never touched.
• Local retention, disclosed. The captured surrounding text is persisted (locally, in the rotating pipeline history) alongside the transcript — the same store that already persists the selected text, context summary, and window screenshot upstream, and it rotates on the same schedule.
• Backward compatible. New history fields are optional and the Core Data store migrates in place, so existing history keeps working.
• Graceful degradation. Every read path is guarded; if an app exposes nothing, the feature backs off (blind) and the dictation behaves exactly as before.
• Self-contained. No third-party libraries are added.
• Additive & merge-safe. Backbone files change only additively (defaulted parameters, feature-owned regions); the heavy logic is in new files.
• As little "magic" as the platform allows. macOS exposes no public API for the text around the cursor in web/Electron views, so the unavoidable bits — the private AXManualAccessibility wake and the WebKit/Chromium TextMarker SPIs — are isolated to clearly-labeled rungs of one explicit extraction ladder, documented, bounded by a per-request timeout, and used only when the clean native path doesn't apply. The deterministic formatter is plain, testable string logic with no platform tricks, and every path degrades gracefully. The goal was exactly the "clean implementation without too much voodoo magic" raised in Spacing, capitalization, and punctuation are context-blind — no awareness of text surrounding the cursor #200: the cleverness is fenced off, layered, and reversible rather than spread through the codebase.

---

Reviewer notes — known non-issues

An earlier automated review raised a few items that we investigated and found to be intentional, pre-existing, or cosmetic. Noting them here so a re-review doesn't re-raise them:

• The Smart-Paste step on legacy history rows is transient. Rows recorded before this feature shipped briefly render the new step until they age out; new dictations always carry the fields. Not a defect.
• The window screenshot in collectContext is unchanged upstream code. It is not added or modified by this PR (it sits just below a new Accessibility-read block). It feeds the activity-inference model, which is orthogonal to the surrounding-text read, so it is intentionally not gated on whether surrounding text was found.
• History persistence posture is unchanged. The new optional surrounding-text fields are strictly less sensitive than what the history store already persists by default upstream (selected text, context summary, a full-window screenshot data URL). They don't widen the trust boundary; an at-rest-privacy policy would be a store-wide change for its own PR.
• Sentinels in the Run Log's prompt view are input scaffolding. The prompt fences each context field's value with sentinels (<<<PRECEDING_TEXT …), the same way the dictated transcript is fenced, and the model is told to return text without surrounding quotes. What shows in the prompt view is the input framing, not quoted output.

Summary by CodeRabbit

• New Features
  • Added context-aware Smart Paste formatting with rule tracing and improved capitalization/punctuation/spacing.
  • Added surrounding-text extraction support (preceding/following + cursor position) surfaced across debug panels and expanded history.
  • Added accessibility “wake” manager and new limiting/metrics utilities to improve surrounding-text reliability.
• Bug Fixes
  • Improved caret/surrounding-text extraction across native, web, and Electron-style inputs with safer fallbacks and reduced stale/incorrect context during paste flows.
  • Enhanced post-processing prompts to include optional surrounding text and made clipboard paste behavior preserve transcript formatting.
• Chores
  • Updated test build wiring to compile the new accessibility text reader source.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 14d25151-3e67-4fc8-bbf6-f5a029a16eb6

📥 Commits

Reviewing files that changed from the base of the PR and between c618e44 and 03f612c.

📒 Files selected for processing (1)

• Sources/ContextualFormattingService.swift

🚧 Files skipped from review as they are similar to previous changes (1)

• Sources/ContextualFormattingService.swift

---

📝 Walkthrough

Walkthrough

This PR adds macOS Accessibility-based surrounding-text capture, integrates context-aware formatting and prompt construction into dictation, persists the resulting metadata, and expands Smart Paste diagnostics and history.

Changes

Context-aware dictation formatting and persistence

Layer / File(s)	Summary
Accessibility reader and wake manager Sources/AccessibilityWakeManager.swift, Sources/AccessibilityTextReader.swift	Adds AX tree wake/sleep management and native, TextMarker, web, and Electron-focused surrounding-text extraction paths.
Context service and dictation lifecycle Sources/AppContextService.swift, Sources/AppState.swift	Captures context at recording and shortcut-release stages, resolves fallback reads, threads context through paste handling, and stores diagnostic state.
Formatting and prompt bounds Sources/ContextualFormattingService.swift, Sources/SurroundingTextLimiter.swift, Sources/PostProcessingService.swift	Adds deterministic punctuation, capitalization, vocabulary, and spacing rules; bounds surrounding text; and injects it into processing prompts.
History schema and persistence Sources/PipelineHistoryItem.swift, Sources/PipelineHistoryStore.swift, Sources/AppState.swift	Adds context, formatting, extraction, and app metadata to history models and Core Data persistence.
Smart Paste debug UI Sources/ContextDebugLabels.swift, Sources/ContextReadMetrics.swift, Sources/PipelineDebugContentView.swift, Sources/PipelineDebugPanelView.swift, Sources/SettingsView.swift	Adds context labels, read metrics, Smart Paste debug output, and expanded run-history diagnostics.
Test-runner build wiring Makefile	Makes the test runner compile from its complete prerequisite list, including the accessibility reader.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant AppState
  participant AccessibilityWakeManager
  participant AccessibilityTextReader
  participant AppContextService
  participant PostProcessingService
  participant ContextualFormattingService
  participant PipelineHistoryStore
  participant SettingsView

  AppState->>AccessibilityWakeManager: wakeUpCurrentApp()
  AppState->>AccessibilityTextReader: readSurroundingTextWithSync()
  AccessibilityTextReader-->>AppState: SurroundingTextSnapshot
  AppState->>AppContextService: collectContext(selectionSnapshot:)
  AppContextService-->>AppState: AppContext with surrounding text
  AppState->>PostProcessingService: process with bounded context
  PostProcessingService->>ContextualFormattingService: format transcript with context
  ContextualFormattingService-->>AppState: formattedText and ruleApplied
  AppState->>PipelineHistoryStore: persist resolved context and formatted transcript
  PipelineHistoryStore-->>SettingsView: stored history metadata
  AppState->>AccessibilityWakeManager: sleepCurrentApp()

Loading

Possibly related PRs

• zachlatta/freeflow#90: Shares selection snapshot and raw selected-text context plumbing.
• zachlatta/freeflow#99: Also extends pipeline history models and persistence metadata.
• zachlatta/freeflow#103: Also modifies the dictation recording start-stop flow in Sources/AppState.swift.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 71.08% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the PR as context-aware dictation and formatting work.
Linked Issues check	✅ Passed	The PR implements surrounding-text reads, deterministic formatting, LLM context hints, and Electron/Web fallbacks required by the linked issue.
Out of Scope Changes check	✅ Passed	The additions are all support for context-aware dictation, history, debugging, or persistence; no unrelated changes stand out.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 8

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)

Sources/AppState.swift (2)

2733-2741: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

Resolve fallback context before post-processing.

processTranscript receives appContext before the start-of-recording fallback is applied, so the LLM can still see blind surrounding text even when startSnap later supplies usable context for formatting/history. Settle the start/JIT context first, then pass that resolved context into post-processing.

Also applies to: 2835-2855

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Sources/AppState.swift` around lines 2733 - 2741, Resolve the context passed
into processTranscript after the start-of-recording fallback is applied, since
appContext is currently forwarded before startSnap/JIT fallback can supply
usable surrounding text. Update the call site in AppState around
processTranscript so the resolved context is chosen first, then passed into
post-processing, and make the same adjustment in the other matching block
mentioned by the review.

---

1952-1970: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Avoid waking AX before early-return validation.

wakeUpCurrentApp() runs before prepareRecordingStart and ensureMicrophoneAccess; if either returns false, there is no matching sleep, so AXManualAccessibility can stay enabled. Move the wake/start-read after the guards succeed, or explicitly sleep on every early return after waking.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Sources/AppState.swift` around lines 1952 - 1970, The current start sequence
in AppState’s recording flow wakes accessibility before the early-return checks,
which can leave AX enabled if prepareRecordingStart or ensureMicrophoneAccess
fails. Move the AccessibilityWakeManager.shared.wakeUpCurrentApp call, and the
related start-read behavior, to after both guards succeed in the recording-start
path, or add matching sleep/cleanup on every failure path after waking. Use the
surrounding prepareRecordingStart, ensureMicrophoneAccess, beginRecording, and
startGentleStartContextRead flow to place the change correctly.

🧹 Nitpick comments (1)

Sources/AppState.swift (1)

2772-2775: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Don’t await the start read unless it is needed.

Line 2772 waits for startSurroundingHandle even when stop-time context already has real surrounding text. For short dictations or a slow target app, this can delay paste by the AX read timeout; only await it when the stop read is blind/unknown.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Sources/AppState.swift` around lines 2772 - 2775, The start-time AX read in
the surrounding text flow is being awaited unconditionally even when stop-time
context already provides real text. Update the logic around
startSurroundingHandle in AppState so the await only happens when it is actually
needed, using finalCursorPos, stop-time context, and the existing hasRealText
check to decide whether to skip the start read. Keep the change localized to the
start/stop surrounding-text handling path and preserve the current fallback
behavior when the stop read is blind or unknown.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Sources/AccessibilityTextReader.swift`:
- Around line 151-158: The readSurroundingText(sync) and related cursor handling
treat every (0,0) selection as fraudulent, which breaks legitimate native “caret
at start” cases. Update the fraud guard to only apply to web or otherwise
unreliable elements using the existing kind/appKind checks, and for normal
native controls return an empty precedingText, the computed followingText slice,
and cursorPosition .start instead of .unknown.
- Around line 578-589: The index-based fallback in markerReadViaIndex currently
treats the selection start as the boundary for followingText, which can leak
selected text into the “following” context during replacements. Update
markerReadViaIndex to derive both start and end markers from the selection,
compare their AXIndexForTextMarker values to determine the true range, and use
the selection end index as the starting point for the following slice while
keeping the preceding slice anchored at the selection start. Keep the existing
nil/empty checks intact when adjusting the slicing logic.

In `@Sources/AppContextService.swift`:
- Around line 143-146: Validate the selectionSnapshot before using its
selectedText in AppContextService so it is only reused when it still matches the
current frontmost app/window context. In the logic around windowTitle and
selectedText, compare the snapshot’s bundle/window identity against the current
appElement context, and if it does not match, fall back to
rawSelectedText(from:) instead of combining stale snapshot text with a new
window.

In `@Sources/ContextualFormattingService.swift`:
- Around line 117-122: Remove the blind trailing-space insertion in
ContextualFormattingService’s spacing logic: when both precedingText and
followingText are nil, do not append a space after sentence-ending punctuation
in the `spaced` result. Update the conditional around the blind-apps case so
smart paste never adds an ungrounded trailing space at the end of a field, while
keeping the normal context-aware spacing behavior in the rest of the formatter.
- Around line 199-215: In ContextualFormattingService, the punctuation-collision
logic only removes identical duplicates and trailing periods, so mixed terminal
pairs like “?.” can still be emitted. Update the punctuation handling around the
existing P2/P3 rules to detect conflicting terminal punctuation combinations
between the accumulated result and the incoming character f, and normalize them
by keeping the appropriate terminal mark while dropping the redundant one. Keep
the abbreviation safeguard via isLikelyAbbreviation, and preserve the existing
duplicate-mark behavior while extending it to cover all terminal punctuation
pairs.

In `@Sources/PipelineHistoryStore.swift`:
- Around line 306-314: The read path in PipelineHistoryStore is forcing
isBlindApp to false, which breaks the legacy nil fallback used by
PipelineHistoryItem and RunLogEntryView. Update the mapping around
entity.isBlindApp so unset values stay nil instead of being coerced, while still
converting explicit stored values to Bool. Keep the fix localized to the
PipelineHistoryStore decode/build logic that populates isBlindApp from the
persisted entity.

In `@Sources/PostProcessingService.swift`:
- Around line 438-453: The prompt assembly in PostProcessingService is still
using static inline delimiters and quoted interpolation for data fields, which
makes it vulnerable to boundary collisions if user text contains the same
markers or quotes. Update the prompt-building logic around the userMessage
construction to use the same collision-resistant helper for RAW_TRANSCRIPTION,
PRECEDING_TEXT, FOLLOWING_TEXT, VOICE_COMMAND, and SELECTED_TEXT instead of
embedding those values directly. Keep the helper centralized so all prompt
fields are wrapped with unique, generated boundaries and can’t be escaped by app
content.

In `@Sources/SettingsView.swift`:
- Around line 2335-2421: The “Smart Paste Formatting” section in SettingsView’s
PipelineStepView is shown for all history items, even when smart paste doesn’t
apply. Update this block to render only for dictation entries by checking
item.intent before showing the Step 4 content, matching the gating used in
PipelineDebugContentView. Keep the existing transcript/context fields and
ContextDebugLabels usage inside that conditional so non-dictation history items
don’t display misleading nil state.

---

Outside diff comments:
In `@Sources/AppState.swift`:
- Around line 2733-2741: Resolve the context passed into processTranscript after
the start-of-recording fallback is applied, since appContext is currently
forwarded before startSnap/JIT fallback can supply usable surrounding text.
Update the call site in AppState around processTranscript so the resolved
context is chosen first, then passed into post-processing, and make the same
adjustment in the other matching block mentioned by the review.
- Around line 1952-1970: The current start sequence in AppState’s recording flow
wakes accessibility before the early-return checks, which can leave AX enabled
if prepareRecordingStart or ensureMicrophoneAccess fails. Move the
AccessibilityWakeManager.shared.wakeUpCurrentApp call, and the related
start-read behavior, to after both guards succeed in the recording-start path,
or add matching sleep/cleanup on every failure path after waking. Use the
surrounding prepareRecordingStart, ensureMicrophoneAccess, beginRecording, and
startGentleStartContextRead flow to place the change correctly.

---

Nitpick comments:
In `@Sources/AppState.swift`:
- Around line 2772-2775: The start-time AX read in the surrounding text flow is
being awaited unconditionally even when stop-time context already provides real
text. Update the logic around startSurroundingHandle in AppState so the await
only happens when it is actually needed, using finalCursorPos, stop-time
context, and the existing hasRealText check to decide whether to skip the start
read. Keep the change localized to the start/stop surrounding-text handling path
and preserve the current fallback behavior when the stop read is blind or
unknown.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: aeebf4ad-d2e0-41e6-ac8f-19a0e171e30f

📥 Commits

Reviewing files that changed from the base of the PR and between 13e2788 and bdc5677.

📒 Files selected for processing (14)

• Sources/AccessibilityTextReader.swift
• Sources/AccessibilityWakeManager.swift
• Sources/AppContextService.swift
• Sources/AppState.swift
• Sources/ContextDebugLabels.swift
• Sources/ContextReadMetrics.swift
• Sources/ContextualFormattingService.swift
• Sources/PipelineDebugContentView.swift
• Sources/PipelineDebugPanelView.swift
• Sources/PipelineHistoryItem.swift
• Sources/PipelineHistoryStore.swift
• Sources/PostProcessingService.swift
• Sources/SettingsView.swift
• Sources/SurroundingTextLimiter.swift

[pianoandglass]

@coderabbitai Addressed in 0f19433:

• markerReadViaIndex now starts the following slice at the selection end (no selected-text leak on replacements).
• The start-time selection snapshot is validated against the current app's bundle before reuse.
• isBlindApp keeps nil (legacy rows) distinct from false through the store, on write and read.
• The Smart Paste step is hidden for entries that never ran it (command mode / legacy rows).
• The Accessibility wake now runs only after the start guards succeed (no wake leak on a failed start).
• The start snapshot is awaited only when the stop read came back unknown (no needless paste delay).

Not changed, by design:

• (0,0)-caret guard: mirrors the async path's long-standing policy. A (0,0) report is unverifiable — Chromium and some SwiftUI fields return it mid-document — so it degrades to unknown context (model casing kept; the pre-paste re-read covers the rare honest case) rather than fabricating a confident field-start.
• Blind trailing space: restores upstream's current behavior exactly, and only in the no-context case — without it, consecutive blind dictations jam together ("one.Two").
• what?.: deliberate — the formatter never deletes meaning-bearing marks the user dictated (? !) nor text already in the field; period collisions are handled.
• Prompt boundaries: PRECEDING/FOLLOWING use the same sentinel convention as RAW_TRANSCRIPTION in this file; collision-resistant boundaries for every field (including pre-existing ones) is a file-wide hardening better done separately.
• Resolving fallback context before post-processing: the surrounding-text hint to the cleanup model is reference-only; blocking the LLM call on the start read would add latency to every dictation, while the formatter and history already receive the fully resolved context.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
gh pr diff 251 --repo zachlatta/freeflow > /tmp/pr251.diff || true
wc -l /tmp/pr251.diff
grep -n "markerReadViaIndex" /tmp/pr251.diff | head -20

Length of output: 839

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail
git fetch origin pull/251/head:pr251 2>&1 | tail -5
git checkout pr251 2>&1 | tail -5
echo "---markerReadViaIndex---"
sed -n '570,650p' Sources/AccessibilityTextReader.swift
echo "---isBlindApp store---"
grep -n "isBlindApp" Sources/PipelineHistoryStore.swift
echo "---Smart Paste hidden condition---"
grep -n "Smart Paste\|hasText\|readFailed" Sources/SettingsView.swift | head -30
echo "---snapshot bundle validation---"
grep -n "bundleIdentifier" Sources/AppState.swift | head -20
echo "---wake after guards---"
grep -n "wakeUpCurrentApp" Sources/AppState.swift

Length of output: 7539

---

@pianoandglass Thanks for the detailed follow-up. I spot-checked the branch at 0f19433 and can confirm the key fixes:

• markerReadViaIndex now resolves an endMarker and uses max(idx, endIdx) as the start of the following slice — confirmed this prevents selected text from leaking into followingText on replacements.
• PipelineHistoryStore writes entity.isBlindApp = item.isBlindApp.map { NSNumber(value: $0) } (both insert and update paths) and reads it back via entity.isBlindApp?.boolValue, so nil (legacy rows) stays distinct from false end-to-end.
• The Smart Paste step in SettingsView.swift computes hasText/readFailed from precedingText/followingText/isBlindApp, consistent with hiding the section for entries that never ran the formatter.
• AccessibilityWakeManager.shared.wakeUpCurrentApp() has a single call site, consistent with your description of it only firing after start guards pass.

I didn't find anything to dispute in the design-rationale items — the (0,0)-caret degrade-to-unknown policy, the blind-trailing-space compatibility behavior, preserving user-dictated ?/!, the sentinel-boundary scope decision, and keeping the fallback-context resolution off the LLM's critical path all seem like reasonable, well-justified tradeoffs given the constraints you outlined. No further concerns from me on these points.

(๑˃̵ᴗ˂̵)و good work

[coderabbitai]

🧹 Nitpick comments (2)

Makefile (1)

78-85: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consolidate target prerequisites.

While defining the same target multiple times to append prerequisites is valid in Make (and correctly captured by $^ on line 85), scattering them across the file for a simple compilation rule can reduce readability.

Consider consolidating Sources/AccessibilityTextReader.swift into this main target definition and removing lines 18-19.

💡 Proposed refactor

-$(TEST_RUNNER): Sources/AppContextService.swift Sources/LLMAPITransport.swift Sources/ModelConfiguration.swift Tests/AppContextServiceTests.swift
+$(TEST_RUNNER): Sources/AccessibilityTextReader.swift Sources/AppContextService.swift Sources/LLMAPITransport.swift Sources/ModelConfiguration.swift Tests/AppContextServiceTests.swift
 	`@mkdir` -p "$(BUILD_DIR)"
 	swiftc \
 		-parse-as-library \
 		-o "$(TEST_RUNNER)" \
 		-sdk $(shell xcrun --show-sdk-path) \
 		-target $(ARCH)-apple-macosx13.0 \
 		$^

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Makefile` around lines 78 - 85, Consolidate the $(TEST_RUNNER) prerequisites
by adding Sources/AccessibilityTextReader.swift to the main target definition
shown before the swiftc recipe, then remove the separate prerequisite-only
definition on lines 18–19. Keep the existing $^ compilation behavior and recipe
unchanged.

Source: Linters/SAST tools

Sources/PostProcessingService.swift (1)

820-849: 🩺 Stability & Availability | 🔵 Trivial | ⚖️ Poor tradeoff

Verbatim path bypasses the LLM circuit breaker. translateVerbatimWithFallback never calls LLMCooldownManager.shared.effectivePrimary(...) before dispatching, and translateVerbatim’s 429 branch (Lines 913-916) surfaces requestFailed without registering a cooldown. Both processWithFallback and processCommandTransformWithFallback do both. As a result this path sends requests to a model that is already cooling down and doesn’t feed the breaker on its own 429s. Consider aligning it with the other two fallback paths for consistent rate-limit behavior.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Sources/PostProcessingService.swift` around lines 820 - 849, Update
translateVerbatimWithFallback to resolve the primary model through
LLMCooldownManager.shared.effectivePrimary(...) before dispatch and record
cooldown state when translateVerbatim receives a 429 requestFailed result. Align
both fallback decisions with processWithFallback and
processCommandTransformWithFallback so cooling models are skipped and this
path’s rate limits feed the circuit breaker.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@Makefile`:
- Around line 78-85: Consolidate the $(TEST_RUNNER) prerequisites by adding
Sources/AccessibilityTextReader.swift to the main target definition shown before
the swiftc recipe, then remove the separate prerequisite-only definition on
lines 18–19. Keep the existing $^ compilation behavior and recipe unchanged.

In `@Sources/PostProcessingService.swift`:
- Around line 820-849: Update translateVerbatimWithFallback to resolve the
primary model through LLMCooldownManager.shared.effectivePrimary(...) before
dispatch and record cooldown state when translateVerbatim receives a 429
requestFailed result. Align both fallback decisions with processWithFallback and
processCommandTransformWithFallback so cooling models are skipped and this
path’s rate limits feed the circuit breaker.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4875f75b-783a-4d2d-9203-42e65a280860

📥 Commits

Reviewing files that changed from the base of the PR and between 5f25f81 and c618e44.

📒 Files selected for processing (15)

• Makefile
• Sources/AccessibilityTextReader.swift
• Sources/AccessibilityWakeManager.swift
• Sources/AppContextService.swift
• Sources/AppState.swift
• Sources/ContextDebugLabels.swift
• Sources/ContextReadMetrics.swift
• Sources/ContextualFormattingService.swift
• Sources/PipelineDebugContentView.swift
• Sources/PipelineDebugPanelView.swift
• Sources/PipelineHistoryItem.swift
• Sources/PipelineHistoryStore.swift
• Sources/PostProcessingService.swift
• Sources/SettingsView.swift
• Sources/SurroundingTextLimiter.swift

🚧 Files skipped from review as they are similar to previous changes (12)

• Sources/ContextReadMetrics.swift
• Sources/ContextDebugLabels.swift
• Sources/PipelineHistoryItem.swift
• Sources/PipelineHistoryStore.swift
• Sources/AccessibilityWakeManager.swift
• Sources/SurroundingTextLimiter.swift
• Sources/SettingsView.swift
• Sources/PipelineDebugPanelView.swift
• Sources/PipelineDebugContentView.swift
• Sources/AppContextService.swift
• Sources/AccessibilityTextReader.swift
• Sources/AppState.swift