Skip to content

fix(optimize): exclude sidechains from behavioral signals - #994

Merged
iamtoruk merged 6 commits into
getagentseal:mainfrom
avs-io:codex/issue-974-sidechains
Aug 18, 2026
Merged

fix(optimize): exclude sidechains from behavioral signals#994
iamtoruk merged 6 commits into
getagentseal:mainfrom
avs-io:codex/issue-974-sidechains

Conversation

@avs-io

@avs-io avs-io commented Aug 13, 2026

Copy link
Copy Markdown
Member

Problem

Claude subagent sidechain transcripts were treated as user-started sessions by codeburn optimize. They inflated the displayed session population and generated low-worth, context-heavy, outlier, capability, coaching, and model-default advice that sidechains fail by construction.

Root cause

The session-level isSidechain marker was parsed but not consistently applied across optimize consumers. Some raw transcript detectors classified each line independently, the compact parser for large JSONL lines dropped the marker, and model/coaching paths consumed sidechain-inclusive project populations.

Change

  • Use one user-started-session predicate for optimize session counts and behavioral projections.
  • Exclude sidechains from per-session findings, raw read/edit detectors, capability reliability, coaching, churn, and model-default recommendations.
  • Keep original projects for aggregate spend, calls, tokens, cost-rate, and configuration-overhead findings.
  • Make raw Claude transcript classification sticky across the whole sidechain file, including earlier unmarked calls.
  • Preserve isSidechain in the compact parser for JSONL entries larger than 32 KiB and through warm-cache/range rebuilds.
  • Include sidechain classification in optimize cache identity.

User impact

codeburn optimize reports user-started session counts and no longer recommends behavioral or actionable changes based solely on subagent transcripts. Delegated token usage remains fully represented in spend, call, token, and configuration-overhead totals.

Preservation and out of scope

  • Sidechain spend is not deleted or attributed to the parent.
  • Overview, general model-efficiency, and global compare populations are unchanged.
  • Unknown or explicitly false markers remain user-started under the existing contract.

Testing

  • Credible composition RED: sidechain-only projects previously emitted an actionable model downgrade and correction coaching despite reporting zero optimize sessions.
  • Large/sticky classification RED: 3 expected failures proved large lines lost the marker and earlier calls escaped late transcript classification.
  • Focused matrix: 116/116.
  • Full root suite: 2,632 passed, 5 skipped.
  • Serial lock suite: 26/26.
  • Full desktop suite: 468/468.
  • Root, dashboard, and desktop TypeScript checks passed.
  • CLI, dashboard, and desktop production builds passed.
  • Exact-SHA independent spec and conservation reviews approved.
  • Isolated real Claude discovery QA: 0 sessions, 67 calls, $0.210 retained, no model recommendation, no coaching, and no sidechain-only behavioral findings.
  • Real TUI and rebuilt Electron checks preserved aggregate spend while Optimize showed only an unrelated configuration finding.

Fixes #974.

avs-io and others added 6 commits August 12, 2026 20:59
…gnals

Only duplicate-reads has a structural reason to skip them: a subagent
starts on a fresh context, so re-reading what its parent read is a
necessary read, not a repeat. Reading node_modules or editing without
reading is the same waste whoever does it, and the CLAUDE.md rule both
findings suggest binds subagents too - filtering them there discarded
most of the evidence on a subagent-heavy corpus.
@iamtoruk
iamtoruk marked this pull request as ready for review August 18, 2026 16:44
@iamtoruk
iamtoruk merged commit 215f501 into getagentseal:main Aug 18, 2026
9 checks passed
timothybrush pushed a commit to timothybrush/codeburn that referenced this pull request Aug 18, 2026
parseLargeJsonl dropped promptSource for exactly the lines SDK-generated
prompts live on, so the recurring-context detector regex-scanned the ends of
the raw line for it. Add the field to LARGE_ROOT_FIELDS (tiny scalar,
add-only, isSidechain already there) and delete the workaround: it read only
2 KB from each end, so a flag further in was missed.

No cache change: optimize scans the raw JSONL each run, so promptSource never
has to persist on CachedFile.

Fixes getagentseal#1030. With getagentseal#994 this closes getagentseal#1023.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

optimize: subagent transcripts (isSidechain=true) are counted as sessions, inflating session counts and the low-worth / context-heavy detectors

2 participants