Skip to content

Measure Search and inference usage in the Groq agent - #62

Draft
georgeatparallel wants to merge 2 commits into
parallel-web:mainfrom
georgeatparallel:feat/groq-search-economics
Draft

Measure Search and inference usage in the Groq agent#62
georgeatparallel wants to merge 2 commits into
parallel-web:mainfrom
georgeatparallel:feat/groq-search-economics

Conversation

@georgeatparallel

@georgeatparallel georgeatparallel commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

Our Groq agent shows searches and answers, but it doesn't show how much retrieval and model work a run uses. We add that information to the existing stream and UI so developers can inspect their own runs without a separate benchmark harness.

We move the tool from beta Search to GA Search, keep basic as the default, and make the Search mode and price assumptions configurable. A small tracker measures successful Search calls, latency, sources, excerpt characters and result bytes, then adds provider-reported model tokens and elapsed time to the final event. If a model step omits a token count, we leave that count unavailable for the whole run instead of showing a partial total.

We label dollar amounts as estimates, not measured spend. Characters and bytes are not tokens, and model and total cost stay unavailable until both model rates and complete input/output usage are present. The estimates exclude failed requests, cache or reasoning-specific rates, taxes, credits and discounts. They are not billing receipts or benchmark results.

Testing: Previously recorded validation includes five passing tests on Node 20, 22 and 24, strict economics-module and full Worker typechecks, and a Wrangler dry-run bundle without deployment. The current diff also adds tests for missing usage across steps and Worker streaming with mocked network responses. Recorded live validation covers one bounded Turbo Search request returning one result and one Search SKU, not a full live Groq run or deployment.

Tracking: DEV-261.

@georgeatparallel

Copy link
Copy Markdown
Collaborator Author

Tracking this in DEV-261, assigned to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant