feat(providers): [copilot.ts]Add OTel cache token parsing including tests and regression checks. - #477
Conversation
…ests and regression checks. Signed-off-by: steelp02 <pieter.steel@pfizer.com>
|
Otel information: https://code.visualstudio.com/docs/agents/guides/monitoring-agents |
is there a corresponding copilot-cli setting to enable otel? |
export COPILOT_OTEL_ENABLED=true |
Signed-off-by: steelp02 <pieter.steel@pfizer.com>
feat(copilot): OTel cache-token parsing (#477) + maintainer review fixes
|
Merged via #498, which preserves your commits and adds maintainer review fixes on top (debug-log cleanup, parameterized the spans IN() query, folded an N+1, epochToISO guard, dead-code removal, and docs for the Node 22+ requirement + the durable-cache monotonicity behavior). Thanks for the contribution. Closing in favor of #498. |
…#477 Maintainer review fixes on top of the OTel cache-token work: - Remove all DEBUG_OTEL console.warn scaffolding from parser.ts and copilot.ts (gated but unlike the rest of the codebase). - Parameterize the spans IN (...) query instead of string-interpolating trace IDs. - Fold the per-chat-span metadata query into the trace-span query to drop the N+1 (one query per chat span -> one per conversation). - Guard epochToISO against null/NaN/0 so a malformed start_time_ms row no longer throws on new Date(NaN).toISOString(). - Remove dead code: parseSpanAttributes, OTelSpanRow, and the unreachable `if (!db) return`; drop unused catch bindings. - Note in parseProviderSources that the non-durable append path assumes unique source paths. - Document the OTel source in docs/providers/copilot.md: Node 22+ requirement, durable-cache monotonic totals, and the one-time parse-version cache reset on upgrade.
Signed-off-by: steelp02 pieter.steel@pfizer.com
Summary
Testing
npm testpassesnpm run buildsucceedsFor new providers only:
npm run dev -- todayshows correct costs and session counts for this providernpm run dev -- models --provider <name>shows correct model names and pricing[ Today ] 7 Days 30 Days This Month 6 Months
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ CodeBurn Today │
│ $55.58 cost 806 calls 131 sessions 46.0% cache hit │
│ 16.1M in 256.6K out 14.8M cached 1.3M written │
│ Custom (all): $59.30 API-equivalent vs $1000.00 plan ▓░░░░░░░░░ 5.9% │
│ Well within plan. Projected month: $76.13 (reset in 21 days). │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────────╮╭────────────────────────────────────────────────────────────────╮
│ Daily Activity ││ By Project │
│ cost calls ││ cost avg/s sess │
│ 06-10 ██████████ $55.58 806 ││ ██████████ copilot/chat $55.58 $0.431 129 │
│ ││ ░░░░░░░░░░ codeburn $0.00 $0.00 1 │
│ ││ ░░░░░░░░░░ xxxxx $0.00 $0.00 1 │
╰────────────────────────────────────────────────────────────────╯╰────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────────╮╭────────────────────────────────────────────────────────────────╮
│ By Activity ││ By Model │
│ cost turns 1-shot ││ cost cache calls 1-shot │
│ ██████████ Exploration $52.13 303 - ││ ██████████ Sonnet 4.6 $52.93 46.0% 147 - │
│ █░░░░░░░░░ Conversation $3.46 496 - ││ ░░░░░░░░░░ Haiku 4.5 $2.60 46.1% 37 - │
│ ░░░░░░░░░░ Debugging $0.00 4 - ││ ░░░░░░░░░░ GPT-4o Mini $0.047 45.7% 115 - │
│ ░░░░░░░░░░ Feature Dev $0.00 1 - ││ ░░░░░░░░░░ copilot-sugges $0.00 0.0% 4 - │
│ ░░░░░░░░░░ Brainstorming $0.00 2 - ││ ░░░░░░░░░░ copilot-nes-oc $0.00 0.0% 4 - │
│ ││ ░░░░░░░░░░ Copilot (Anthr $0.00 - 499 - │
╰────────────────────────────────────────────────────────────────╯╰────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────────╮╭────────────────────────────────────────────────────────────────╮
│ Core Tools ││ Shell Commands │
│ calls ││ No shell commands │
│ ██████████ Shell 5358 ││ │
│ █████░░░░░ Read 2883 ││ │
│ ███░░░░░░░ replace_string_in_file 1585 ││ │
│ ██░░░░░░░░ manage_todo_list 1126 ││ │
│ █░░░░░░░░░ grep_search 609 ││ │
│ █░░░░░░░░░ file_search 269 ││ │
│ ░░░░░░░░░░ create_file 191 ││ │
│ ░░░░░░░░░░ list_dir 57 ││ │
│ ░░░░░░░░░░ kill_terminal 38 ││ │
│ ░░░░░░░░░░ get_terminal_output 16 ││ │
╰────────────────────────────────────────────────────────────────╯╰────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────────╮╭────────────────────────────────────────────────────────────────╮
│ Skills & Agents ││ MCP Servers │
│ No skill/agent usage ││ No MCP usage │
╰────────────────────────────────────────────────────────────────╯╰────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ <> switch q quit 1 today 2 week 3 30 days 4 month 5 6 months d yesterday o optimize c compare │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯