Skip to content

fix: strip reasoning blocks from AI outputs#317

Merged
ruzin merged 2 commits into
ruzin:mainfrom
Vassista:feature/fix-reasoning-leak
Jul 6, 2026
Merged

fix: strip reasoning blocks from AI outputs#317
ruzin merged 2 commits into
ruzin:mainfrom
Vassista:feature/fix-reasoning-leak

Conversation

@Vassista

@Vassista Vassista commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

This PR hardens the output sanitization layer to ensure that reasoning blocks (e.g., <think>...</think>) from models never leak into user-facing views or exports. It expands support for additional model tags, preserves original Markdown formatting more cleanly, and applies the sanitizer universally across all UI components and features.

Changes:

  • Sanitizer Improvements: stripReasoning now supports <thinking>, <thought>, and <reasoning> tags. It preserves legitimate Markdown indentation and cleanly removes blank-line residue after a block is stripped.
  • Backend Parity: Updated the Markdown normalizer (simple_recorder.py) to handle the expanded set of reasoning tags, keeping frontend and backend parsing logic in sync.
  • Universal Application: Reasoning blocks are now consistently stripped from all user-facing paths. This resolves previous leaks in custom report rendering, shareable note uploads, clipboard copying, search indexing, and list previews.
  • Testing: Expanded unit tests to cover the newly supported tags, formatting preservation, blank-line cleanup, and relevant streaming edge cases.

@Vassista Vassista requested review from Optic00 and ruzin as code owners July 6, 2026 08:36

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Re-trigger cubic

@Optic00 Optic00 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Good PR - the two closed gaps are real (reports rendered activeReport.content raw, and composeShareBody uploaded the raw summary), and I checked the new regex against the tricky cases (case-insensitive backreference, the $(?![\s\S]) end-of-string anchor under /m, the unclosed streaming block, backtracking behaviour on large unmatched input) - all sound. Two remaining gaps of the same class the PR is fixing, plus two smaller notes:

1. copyNotes still copies the raw summary to the clipboard

MeetingDetail.tsx:483 - const summary = meeting.summary?.trim() goes into navigator.clipboard.writeText unfiltered. Same pattern this PR fixes in composeShareBody, one screen up. With meeting.summary = "<think>private chain</think>\nPublic summary", the rendered note and the share upload are clean, but "Copy notes" puts the reasoning block on the clipboard. One-line fix, ideally in the same PR.

2. List preview and search snippets render the raw summary

  • PreviousRow.tsx (previewText, ~line 169) shows meeting.summary?.trim() as the row preview on Home.
  • CommandPalette.tsx:184 passes raw m.summary into snippet() (noteSearch.ts:33), which renders at line 206. This one is double: the search itself also matches text inside reasoning blocks, so a query hit inside <thinking>...</thinking> surfaces reasoning text as the result snippet.

A summary that starts with a reasoning block shows that text in the meeting list and palette even though the detail view is now clean. Wrapping both sources in stripReasoning fixes display; whether search should also match stripped text only is a judgement call (I'd say yes, same input for index and snippet).

3. Backend tag list now lags the renderer

simple_recorder.py:79 - _REASONING_TAG_HEADER_PATTERN normalizes headers after think|thought only. The renderer now also knows thinking|reasoning, so <thinking>...</thinking>### Action Items inline would skip the parser normalization. Not a blocker, but worth syncing the lists (or extracting one source of truth) so they don't drift further.

4. Correction on the PR description (doesn't change the value of the PR)

The regex on main has no /m flag - /<(think|thought)>[\s\S]*?(?:<\/\1>|$)/gi - so $ already matched end-of-string, and it strips closed multi-paragraph blocks and unclosed streaming blocks correctly (I ran both cases against the old code to be sure). What the new version actually improves: the two extra tag names, no more blank-line residue between stripped blocks, and preserved indentation instead of trimStart(). Worth adjusting the description so the changelog doesn't record a bug that wasn't there.

Nits

  • The "handles nested tags" test covers <think>outer<thinking>inner</thinking></think> (different tags) but not same-tag nesting: <think>a<think>b</think>c</think> leaks c</think> with old and new regex alike. Pre-existing limitation and probably fine for real model output - just don't let the test name suggest it's covered.
  • Dropping trimStart() is a deliberate contract change (the tests pin it), looks right to me - flagging it only so it's a conscious decision.

Happy to take 1-3 as a follow-up if you'd rather keep this PR small.

@Vassista

Vassista commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

Review

Good PR - the two closed gaps are real (reports rendered activeReport.content raw, and composeShareBody uploaded the raw summary), and I checked the new regex against the tricky cases (case-insensitive backreference, the $(?![\s\S]) end-of-string anchor under /m, the unclosed streaming block, backtracking behaviour on large unmatched input) - all sound. Two remaining gaps of the same class the PR is fixing, plus two smaller notes:

1. copyNotes still copies the raw summary to the clipboard

MeetingDetail.tsx:483 - const summary = meeting.summary?.trim() goes into navigator.clipboard.writeText unfiltered. Same pattern this PR fixes in composeShareBody, one screen up. With meeting.summary = "<think>private chain</think>\nPublic summary", the rendered note and the share upload are clean, but "Copy notes" puts the reasoning block on the clipboard. One-line fix, ideally in the same PR.

2. List preview and search snippets render the raw summary

  • PreviousRow.tsx (previewText, ~line 169) shows meeting.summary?.trim() as the row preview on Home.
  • CommandPalette.tsx:184 passes raw m.summary into snippet() (noteSearch.ts:33), which renders at line 206. This one is double: the search itself also matches text inside reasoning blocks, so a query hit inside <thinking>...</thinking> surfaces reasoning text as the result snippet.

A summary that starts with a reasoning block shows that text in the meeting list and palette even though the detail view is now clean. Wrapping both sources in stripReasoning fixes display; whether search should also match stripped text only is a judgement call (I'd say yes, same input for index and snippet).

3. Backend tag list now lags the renderer

simple_recorder.py:79 - _REASONING_TAG_HEADER_PATTERN normalizes headers after think|thought only. The renderer now also knows thinking|reasoning, so <thinking>...</thinking>### Action Items inline would skip the parser normalization. Not a blocker, but worth syncing the lists (or extracting one source of truth) so they don't drift further.

4. Correction on the PR description (doesn't change the value of the PR)

The regex on main has no /m flag - /<(think|thought)>[\s\S]*?(?:<\/\1>|$)/gi - so $ already matched end-of-string, and it strips closed multi-paragraph blocks and unclosed streaming blocks correctly (I ran both cases against the old code to be sure). What the new version actually improves: the two extra tag names, no more blank-line residue between stripped blocks, and preserved indentation instead of trimStart(). Worth adjusting the description so the changelog doesn't record a bug that wasn't there.

Nits

  • The "handles nested tags" test covers <think>outer<thinking>inner</thinking></think> (different tags) but not same-tag nesting: <think>a<think>b</think>c</think> leaks c</think> with old and new regex alike. Pre-existing limitation and probably fine for real model output - just don't let the test name suggest it's covered.
  • Dropping trimStart() is a deliberate contract change (the tests pin it), looks right to me - flagging it only so it's a conscious decision.

Happy to take 1-3 as a follow-up if you'd rather keep this PR small.

Thanks for the thorough review and for checking the regex edge cases.

I'm working on fixes for the remaining issues you called out:

  • copyNotes copying the raw summary
  • list preview / command palette using the raw summary (including search/snippet behavior)
  • syncing the backend reasoning tag list with the renderer

I'll also update the PR description to accurately reflect what the regex change improves, and I'll clarify the nested-tag test name so it doesn't imply same-tag nesting is supported.

@Vassista

Vassista commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author
  • copyNotes now strips reasoning blocks before copying.
  • Meeting list previews and search indexing/snippets now use the sanitized summary, so reasoning content is no longer surfaced there.
  • Synced the backend reasoning tag list with the frontend to keep them aligned.
  • I also updated the PR description to accurately reflect the final behavior and renamed the nested-tags test to clarify that it covers different-tag nesting rather than same-tag nesting.

Let me know if you spot something else.

@Vassista Vassista requested a review from Optic00 July 6, 2026 11:52
Optic00 added a commit to Optic00/stenoai that referenced this pull request Jul 6, 2026
Align the copy paths with the rendered views (and with ruzin#317, which makes
the report view render stripReasoning'd content): both the Standard
summary and an open report's markdown are stripped before hitting the
clipboard. Seeded T1 report now carries a leading <think> block and the
spec asserts it never reaches the clipboard.
@Optic00

Optic00 commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Re-checked the follow-up commits - all four points and the nits are addressed:

  1. copyNotes strips the summary before it hits the clipboard.
  2. PreviousRow.previewText and noteSearch (both searchNotes and snippet) now work on stripped text, so search matching and result snippets agree - the judgement call landed where I'd hoped.
  3. Backend _REASONING_TAG_HEADER_PATTERN is synced to the four-tag list.
  4. Description + test name corrected.

I also re-ran the new regex shape ((?:^[ \t]*)?...\n? with the startsWithReasoning guard) against the edge cases from the first pass - indentation preservation, blank-line residue, unclosed streaming block, uppercase tags, cross-tag nesting - all behave as the tests pin them. LGTM.

Coordination note: #318 (just opened) rewrites the same copyNotes block - it extracts the payload builder to lib/notesCopy.ts and fixes "Copy notes" to copy the open template report instead of the Standard note. It already applies stripReasoning to both copy sources (summary + report content), matching this PR's direction. So whichever merges second has a small conflict in MeetingDetail.tsx with a safe resolution: take #318's copyNotes (the strip behaviour from this PR is preserved there). Everything else in the two PRs is disjoint.

@ruzin ruzin merged commit 5a4e065 into ruzin:main Jul 6, 2026
10 checks passed
Optic00 added a commit to Optic00/stenoai that referenced this pull request Jul 6, 2026
Align the copy paths with the rendered views (and with ruzin#317, which makes
the report view render stripReasoning'd content): both the Standard
summary and an open report's markdown are stripped before hitting the
clipboard. Seeded T1 report now carries a leading <think> block and the
spec asserts it never reaches the clipboard.
ruzin pushed a commit that referenced this pull request Jul 8, 2026
…andard note (#318)

* fix(meetings): Copy notes copies the open template report, not always the Standard note

copyNotes built the clipboard payload from the Standard structured fields
unconditionally, so with a generated template report on screen the copy
silently disagreed with the displayed note. Extract the builder into
lib/notesCopy.ts (pure, unit-tested) and route it through the component's
activeReport: report open -> title + meta + the report's markdown,
otherwise the Standard sections as before.

Coverage: vitest unit tests for the builder + a T1 e2e spec driving the
real pill switch and Copy notes action against a seeded meeting carrying
a report (new STENOAI_E2E_SEED_REPORT seam in the mock IPC, gated so
transcript-export.t1 keeps its report-less meeting).

* fix(meetings): review hardening for Copy notes

- Disable Copy notes while a summary/report stream is on screen (all
  non-idle phases are transient in-flux states) so the clipboard can't
  get the old note while the body shows streamed text (Codex finding).
- Make the mock IPC's set-active-report stateful for the seeded report
  so the pill switch survives the post-mutation refetch like the real
  sidecar does, keeping the T1 spec faithful rather than relying on the
  re-sync effect's unchanged-dep skip.

* fix(meetings): strip reasoning blocks in the Copy notes payload

Align the copy paths with the rendered views (and with #317, which makes
the report view render stripReasoning'd content): both the Standard
summary and an open report's markdown are stripped before hitting the
clipboard. Seeded T1 report now carries a leading <think> block and the
spec asserts it never reaches the clipboard.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants