[scanner] feat: add process-level LLM usage accumulator (refs #13) - #202
Open
kubestellar-hive[bot] wants to merge 1 commit into
Open
[scanner] feat: add process-level LLM usage accumulator (refs #13)#202kubestellar-hive[bot] wants to merge 1 commit into
kubestellar-hive[bot] wants to merge 1 commit into
Conversation
Introduces module-level counters and three helpers in scripts/llm.py:
* get_usage_totals() → snapshot {prompt_tokens, completion_tokens, calls}
* reset_usage_totals() → zero the counters
* _record_usage(usage) → called from _one_shot() on every response
Every successful call now feeds the accumulator with the SDK's
resp.usage counts (missing / non-numeric fields are coerced to 0 so a
weird provider response can never fail an otherwise-successful call).
This is the first, additive slice toward wiring token accounting into
the runs table (issue #13). It changes NO existing signatures — every
caller of call_llm() continues to work unmodified — so follow-up PRs
can consume the totals from rank.py / write.py / publish.py without
another round of refactoring.
Test coverage: 8 new cases in test_llm_helpers.py exercising
reset/accumulate semantics, defensive coercion, snapshot immutability,
and the _one_shot → accumulator hand-off.
Refs #13
Signed-off-by: scanner-agent <scanner@hive.local>
kubestellar-hive
Bot
force-pushed
the
scanner/llm-usage-accumulator
branch
from
July 21, 2026 17:05
02f3b84 to
f0c9d77
Compare
Coverage report
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix
First, additive slice toward wiring token accounting into the
runstable.scripts/llm.pyalready logged usage from each response but discarded the counts. This PR adds three module-level helpers:get_usage_totals()— snapshot{prompt_tokens, completion_tokens, calls}reset_usage_totals()— zero the counters (used in tests + between runs)_record_usage(usage)— called from_one_shot()on every responseEvery successful call now feeds the accumulator. Missing / non-numeric fields are coerced to
0, so a weird provider response can never fail an otherwise-successful call.Why this shape
call_llm()still returns just the parsed dict, so every existing caller (rank.py,write.py,backfill.py,replay_writer.py) keeps working without change.get_usage_totals()frompublish.main()and fillruns.tokens_in/runs.tokens_out— that PR becomes a ~10-line diff instead of a cross-cutting refactor.Test coverage
Adds 8 cases to
tests/test_llm_helpers.py:reset_usage_totals()re-zeros after activity_record_usage()calls accumulate_record_usage(None)is a no-opprompt_tokens/completion_tokenscount as 0 (no crash)get_usage_totals()returns a copy (mutation-safe)_one_shot()feeds the accumulator on a real happy-pathAll 29 tests in
tests/test_llm_helpers.pypass locally.Refs #13
Filed by scanner agent (ACMM L5 — hold-gated mode). Hold-gated: human review required.