feat(stella-cli): the Claude Code transcript adapter — local-only, option (b), honest about what the corpus cannot supply (#2304)

[macanderson]

Refs #2304. Implements PR 5 of doc:trace-replay-learning-harness §9 — the Claude Code transcript adapter — and completes spec §13 with the adapter's half (§13.3), which #2350 left open.

Based on main, which now carries the harness — #2350 merged while this was in flight.

(Replaces #2357, which GitHub auto-closed when #2350 merged and its base branch feat/2304-trace-replay-harness was auto-deleted. Same content, rebased onto the merged harness.)

The adapter's honest limit is the design

Claude Code transcripts contain no Stella reflection JSON. There is nothing in them to script a lessons array from, so §7.2 puts the choice plainly: derive lessons from the transcript, or decline to. Deriving them means the harness measures the adapter's lesson-invention heuristic instead of Stella's learning — it fabricates the exact signal under test.

This implements option (b), which the spec recommends: shell history, session and turn boundaries, and timing only. It lights up the tool foundry against thousands of real commands and launders nothing.

That needed a fourth ScriptedReflection arm

Expressing "the source carried no reflection" with the three existing arms is impossible without asserting something the source does not support:

Spelling	The claim it makes
Lessons { lessons: [] }	the model had nothing to say
Unreadable { .. }	the model said something unparseable
ModelError { .. }	the call failed

The second is the dangerous one: it would fabricate starvation, and assertion 7 (an_unreadable_corpus_builds_nothing_and_says_why) counts exactly those turns. An adapter that emitted Unreadable for 16,000 turns would corrupt the metric its sibling test depends on.

So NotRecorded is its own arm. The replayer skips the model boundary entirely for it, and the summary counts turns_not_recorded separately from empty reflections — a metric that folded them together would report the learner as idle when it was never asked.

Provenance::Derived already exists on the Lessons arm from #2350, and every_lesson_is_labelled asserts nothing reaches a trace unlabelled — so if option (a) is ever built, it cannot ship un-stamped quietly.

The privacy gate (§7.1), applied where the risk actually is

The spec asks for "a secret-shaped string is quarantined rather than stored". Under option (b) the adapter derives no statements, so there is no proposal to run through gate_proposal/quarantine_for. The highest-risk field it does touch is the shell command — export ANTHROPIC_API_KEY=sk-ant-… and curl -H "Authorization: Bearer …" are ordinary things to have typed.

So the gate runs there, and it is stricter than quarantine: redact_secrets runs on every command, and a command whose redaction fired is dropped, not kept with a [redacted] hole in it. Two reasons:

• The redactor's prefix list is a good filter, not a complete one. A token shape it has not seen would survive in a command it had partly redacted, and the partial redaction is what would make that look safe.
• The foundry loses nothing. A command carrying a credential is a one-off by nature, so it was never going to be a recurring shape worth minting a tool from.

Beyond that: local-only and opt-in (STELLA_REPLAY_CC_CORPUS, reachable only from a test — no shipped command, and a plain cargo test never touches the corpus); nothing derived is committed (the CI corpus stays synthetic, permanently, and no-scratch fails the gate on a committed derivative on its own); and the transcript stub is left empty, because nothing reads it under option (b) and carrying user text into a trace we did not need would be gratuitous retention.

Measured against the real corpus

corpus: 496 project director(ies)
adapted 20 project(s): 16664 turn(s), 17343 command(s)

Every adapted trace was round-tripped through Trace::parse — the loader's own contract, applied to a real uncontrolled source rather than to a synthetic fixture the adapter was written against. That is the assertion that catches a transcript shape the adapter mishandles.

A correction to §7's measurement, recorded in §13.3. The spec measured 485 project directories on 2026-08-08; it is 496 today. Consistent with the rolling window the spec already flags, and the reason a derived trace can never be a committed fixture.

The spec now records what was built (§13)

The document said status: proposed. Rather than leave a plan describing something that now exists, §13 records the divergences and — more usefully — the four things replaying the real loop measured, none of which is visible from reading a single module: the shared dedup/clustering threshold, the foundry's value-like-argument rule, the workspace-derived lineage id, and lexical starts_with. A future reader planning fixtures needs all four.

Verification

• cargo test -p stella-cli — 1531 passed, 0 failed (12 new), on top of the merged harness
• STELLA_REPLAY_CC_CORPUS=1 cargo test … the_real_corpus_adapts — output above
• cargo clippy -p stella-cli --all-targets -- -D warnings — clean
• make guards-fast — clean (main is red: shellcheck fails on scripts/test-arena-scripts.sh, reddening every open PR #2355's shellcheck failure is fixed on main by fix(arenabench): emit the launch stub with a quoted heredoc, so shellcheck and the gate are green again (#2355) #2363)

No tests deleted.

One guard caught a real mistake, on the first run

paths::tests::nothing_else_in_this_crate_reads_a_home_out_of_the_environment failed on my std::env::var_os("HOME"). Fixed to crate::paths::home(), which is what lets a test redirect the anchor without mutating process-global state (#1139). Worth naming because it is exactly the guard working as designed.

Not in this PR

• Option (a) — synthesizing lessons. The labelling machinery is in place if it is ever wanted; the spec's recommendation against it stands.
• Committing any derived trace. Permanently out of scope.

Summary by Sourcery

Add a local-only, opt-in Claude Code transcript adapter that feeds real shell history into the trace replay harness without inventing reflections, and extend the harness to account for turns where the source provided no reflection.

New Features:

• Introduce a Claude Code transcript adapter that converts local transcripts into trace sessions for the replay harness under an explicit environment-based opt-in.
• Add support for a ScriptedReflection::NotRecorded arm and corresponding replay summary metric to represent turns where the source carried no reflection JSON.

Enhancements:

• Update the replayer to skip the reflection model boundary entirely for turns marked as not recorded while still driving the tool foundry and timeline.
• Refine the trace replay harness documentation to describe the built Claude Code adapter, its privacy guarantees, and its measured corpus characteristics.

Documentation:

• Document the Claude Code adapter design, its option (b) behavior, privacy gate, and real corpus measurements in the trace replay harness spec.

Tests:

• Add adapter and privacy-gate tests over synthetic transcripts, including opt-in behavior and timestamp handling, and a guarded test that exercises the real local corpus when enabled.

[sourcery-ai]

Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
stella-cli-docs	Ignored		Aug 8, 2026 10:45pm

[sourcery-ai]

Reviewer's Guide

Implements the Claude Code transcript adapter for the trace replay learning harness using option (b): adapt shell history, session boundaries, and timing only, introduce a NotRecorded reflection arm so turns with no reflection skip the model boundary, wire adapter privacy gates and opt‑in behaviour, and update the harness summary/spec to account for these new metrics and behaviour.

Sequence diagram for replaying turns with NotRecorded reflections

sequenceDiagram
    participant Replayer
    participant Turn
    participant ScriptedReflection
    participant Memory

    Replayer->>Turn: get reflection
    Turn->>ScriptedReflection: reaches_the_model()
    alt reflection reaches model
        ScriptedReflection-->>Replayer: true
        Replayer->>Memory: reflect_and_record(provider, "replay", transcript, true, succeeded, None)
        Memory-->>Replayer: ReflectionReport
    else reflection NotRecorded
        ScriptedReflection-->>Replayer: false
        Replayer->>Replayer: ReflectionReport::default()
    end
    Replayer->>Replayer: run foundry and summary over accumulated history

Loading

File-Level Changes

Change	Details	Files
Add Claude Code transcript adapter that converts local jsonl transcripts into replayable traces while deliberately omitting lessons/reflection content.	Introduce cc_adapter module with functions to adapt a single transcript into a TraceSession and a directory into a Trace, using ScriptedReflection::NotRecorded for all turns. Implement parsing of Claude Code jsonl lines (Line, Message) and extraction of user text and Bash tool_use commands. Apply privacy gate to shell commands via safe_command, redacting secrets and dropping any command where redaction fires or command is empty. Implement RFC3339 timestamp parsing and clamping to ensure non‑decreasing turn.at values within and across sessions. Add opt‑in environment variable STELLA_REPLAY_CC_CORPUS and corpus discovery helpers (opted_in_corpus_root, project_directories). Add every_lesson_is_labelled and derived_message helpers to enforce/provide Provenance::Derived for any future synthesized lessons.	crates/stella-cli/src/memory/replay/cc_adapter.rs
Extend scripted reflection model to represent turns where the source carried no reflection and gate replay accordingly.	Add ScriptedReflection::NotRecorded enum arm with documentation explaining its semantics for non‑reflecting corpora. Update ScriptedReflection::text to return None for NotRecorded just like ModelError. Add ScriptedReflection::reaches_the_model helper that is false for NotRecorded and used to decide whether to call the model during replay.	crates/stella-cli/src/memory/replay/trace.rs
Modify replayer to skip the model boundary for turns marked as not recorded, and ensure downstream metrics reflect this.	Change Replayer::run turn loop to conditionally call reflect_and_record only when turn.reflection.reaches_the_model() is true. Return a default ReflectionReport when the reflection does not reach the model, keeping downstream behaviour deterministic but with zero model activity for such turns.	crates/stella-cli/src/memory/replay.rs
Extend replay summary metrics and textual output to count non‑recorded turns separately from other reflection outcomes.	Add turns_not_recorded field to ReplaySummary and increment it when ScriptedReflection::NotRecorded is seen in ReplaySummary::from. Update textual Display/string rendering to include not recorded by the source count in the reflections line.	crates/stella-cli/src/memory/replay/summary.rs
Add comprehensive tests for the adapter, privacy gate, opt‑in behaviour, and integration with replay and foundry, including an optional real‑corpus smoke test.	Create synthetic transcript helpers (user_line, assistant_bash, transcript) for building Claude Code‑shaped jsonl fixtures. Add privacy tests that assert secret‑shaped strings in commands are redacted/dropped and never reach adapted traces, and that commands whose redaction fires are dropped entirely. Test opt‑in behaviour of opted_in_corpus_root ensuring no corpus is read without STELLA_REPLAY_CC_CORPUS set. Test turn boundary semantics, dropping of shell‑less turns, handling of unreadable lines, timestamp clamping, and RFC3339 parsing. Add integration tests that adapt directories into traces, round‑trip through Trace::parse, replay via Replayer::run, and assert turns_not_recorded and memories counts. Add an opt‑in the_real_corpus_adapts_when_opted_in test that scans the actual local Claude Code corpus when available, prints summary counts, enforces every_lesson_is_labelled, and validates loader contract.	crates/stella-cli/src/memory/replay/cc_adapter/tests.rs
Update trace replay harness spec to document the adapter as built and to correct corpus measurement figures.	Replace §13.3 placeholder with a description of option (b) implementation, NotRecorded arm semantics, and separate counting of turns_not_recorded. Document privacy gate behaviour focused on shell commands, strict dropping of redaction‑triggering commands, and rationale for non‑committed derived traces. Record updated corpus measurement (496 project directories, 16,664 turns, 17,343 commands) and explain rolling‑window nature and implications for fixtures. Adjust surrounding spec text, marking §13.3 as complete and leaving other subsections open as before.	docs/spec/trace-replay-learning-harness.md

Possibly linked issues

• #: The PR delivers the Claude Code transcript adapter and NotRecorded handling exactly as specified in the trace-replay harness epic.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[macanderson]

Heads-up on a spec collision with #2371. Both PRs edit docs/spec/trace-replay-learning-harness.md §13:

• this PR inserts §13.3 "The adapter, as built" and renumbers "Still open" to §13.4;
• fix(stella-cli): mine the recurrence the store deduplicates, and measure the clustering threshold in its own token space (#2358) #2371 (which fixes cli: reflection dedup runs before the mining log, and shares its threshold with the clustering predicate — skills and rules can only be mined inside an accidental band #2358) rewrites §13.1 item 1 to describe the fix rather than the workaround.

Different subsections, so a textual conflict is likely but the resolution is a straight union — take both. Whichever lands second should rebase and confirm §13 still reads 13.1 → 13.2 → 13.3 → 13.4 with no duplicate heading. Flagging it rather than racing, because a same-day rewrite of one file by two PRs is how additions get silently dropped in this repo.

No code overlap: #2371 touches learning.rs, skills.rs, rules.rs, replay/tests.rs and the committed corpus; this touches replay.rs, replay/trace.rs, replay/summary.rs and the new replay/cc_adapter*.

Also worth saying plainly: #2358 came out of this harness, and #2371 shipping a measured fix within hours is the loop working — the corpus in #2371 can now be worded naturally, which is exactly the workaround §13.1 said should not outlive the bug.