Skip to content

fix(cost): emit opencode multi-step usage as deltas (#1036) - #1040

Open
vishnujayvel wants to merge 3 commits into
mksglu:mainfrom
vishnujayvel:fix/1036-cost-overcount
Open

fix(cost): emit opencode multi-step usage as deltas (#1036)#1040
vishnujayvel wants to merge 3 commits into
mksglu:mainfrom
vishnujayvel:fix/1036-cost-overcount

Conversation

@vishnujayvel

Copy link
Copy Markdown

What / Why / How

What: Stop the opencode adapter from over-counting multi-step turn cost under additive aggregation.

Why: message.updated fires once per step-finish with a cumulative turn cost, while db.insertEvent always appends a new agent_usage row. A 2-step turn (cost 0.02 then 0.05) was stored as 0.02 + 0.05 = 0.07 instead of the true turn cost 0.05. The old comment claimed re-emitting was "idempotent at the cost column," but there is no cost-column upsert.

Reported by @EvolveAegis in #1036; dynamic PoC + citation corrections also by @EvolveAegis.

How: Convert the cumulative figure to a per-step delta (same incremental shape as the codex/kimi adapters) via toOpencodeUsageStepDelta, keyed by assistant message id in the opencode plugin. Skip no-op refreshes when the cumulative cost does not advance so last-step token snapshots are not double-counted. Bound the in-memory high-water map with insertion-order FIFO (~1000 keys) so long-lived plugin processes cannot unbounded-grow memory. Update the misleading comment.

Fixes #1036

Limitations

  1. Plugin-process restart vs durable DB. The cumulative high-water map is in-memory only. A plugin-process restart clears it while prior agent_usage rows remain in the session DB. The first message.updated after restart for an in-flight multi-step turn re-emits the full cumulative cost as if it were a first observation → a one-time double-count on that narrow edge. Still strictly better than status quo, which over-counted every multi-step turn.

  2. Missing info.id key collision. When info.id is absent the map key falls back to bare sessionId. Two assistant messages in the same session that both lack id share one high-water entry; the second message's first fire is treated as a continuation of the first. If its cumulative cost is not greater, the delta <= 0 guard silently drops it (fails toward under-counting; no signal/log).

  3. FIFO eviction does not refresh insertion position. Re-setting an existing key does not move it to the newest slot. A message still in flight after 1000 newer keys arrive is evicted; its next fire sees no prior cumulative and re-emits the full figure — the same one-time double-count as the restart edge. Practically unreachable at cap 1000 within one turn.

Affected platforms

  • OpenCode
  • KiloCode (shares the opencode plugin entry)

Test plan

  • Regression test: 2-step turn (0.02 → 0.05) produces deltas that sum to 0.05; naive sum still documents the pre-fix 0.07 over-count
  • No-op refresh (same cumulative cost) emits nothing on the second fire
  • Absent native cost: catalog-priced row emitted once per message
  • Map eviction: insertion-order FIFO drops oldest key past cap; update-in-place does not evict
  • npx vitest run tests/session/parse-opencode-usage.test.ts — 20 passed
  • npx vitest run tests/session/parse-opencode-usage.test.ts tests/opencode-plugin.test.ts (+ related)
  • npx tsc --noEmit clean

Checklist

  • Tests added/updated (TDD: red → green)
  • Touched suite + related tests pass
  • npm run typecheck / tsc --noEmit passes
  • Docs updated if needed (README, platform-support.md) — N/A (behavior fix; comment-only docs in code)
  • No Windows path regressions (forward slashes only)
  • Targets next branch (unless hotfix) — staged off main at base 3bad0f4 per work-order; retarget if maintainers prefer next
Cross-platform notes

Our CI runs on Ubuntu, macOS, and Windows.

  • If touching file paths, verify forward-slash normalization on Windows
  • If touching hook paths, verify no backslash separators
  • Use path.join() / path.resolve(), never hardcode / separators
  • Use event-based stdin reading — readFileSync(0) breaks on Windows
  • Use os.tmpdir(), never hardcode /tmp

message.updated fires once per step-finish with a CUMULATIVE turn cost, but
db.insertEvent always appends. Multi-step turns therefore over-counted under
additive aggregation (0.02 then 0.05 stored as 0.07 instead of 0.05).

Convert the cumulative figure to a per-step delta (same incremental shape as
codex/kimi) keyed by assistant message id, and drop no-op refreshes so last-
step token snapshots are not double-counted. Update the misleading
"idempotent at the cost column" comment.

Fixes mksglu#1036
…glu#1036)

Long-lived plugin processes could unbounded-grow lastCumulativeCostByMessage.
No message-completion bus event is consumed by the plugin lifecycle, so cap
the Map at 1000 keys via insertion-order FIFO (rememberOpencodeCumulativeCost).
Updating an existing key does not grow the map. Add unit tests for eviction.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

opencode adapter appends cumulative turn cost once per step, over-counting multi-step turns

1 participant