Skip to content

feat(providers): [copilot.ts]Add OTel cache token parsing including tests and regression checks. - #477

Merged
iamtoruk merged 2 commits into
getagentseal:mainfrom
steelp02:copilot-otel
Jun 18, 2026
Merged

feat(providers): [copilot.ts]Add OTel cache token parsing including tests and regression checks.#477
iamtoruk merged 2 commits into
getagentseal:mainfrom
steelp02:copilot-otel

Conversation

@steelp02

Copy link
Copy Markdown
Contributor

Signed-off-by: steelp02 pieter.steel@pfizer.com

Summary

Testing

  • I have tested this locally against real data (not just unit tests). The costs codeburn displays align with github copilot costs online
  • npm test passes
  • npm run build succeeds

For new providers only:

  • I installed the tool and generated real sessions by using it
  • npm run dev -- today shows correct costs and session counts for this provider
  • npm run dev -- models --provider <name> shows correct model names and pricing
  • Screenshot or terminal output attached below proving it works with real data

[ Today ] 7 Days 30 Days This Month 6 Months
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ CodeBurn Today │
│ $55.58 cost 806 calls 131 sessions 46.0% cache hit │
│ 16.1M in 256.6K out 14.8M cached 1.3M written │
│ Custom (all): $59.30 API-equivalent vs $1000.00 plan ▓░░░░░░░░░ 5.9% │
│ Well within plan. Projected month: $76.13 (reset in 21 days). │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────────╮╭────────────────────────────────────────────────────────────────╮
│ Daily Activity ││ By Project │
│ cost calls ││ cost avg/s sess │
│ 06-10 ██████████ $55.58 806 ││ ██████████ copilot/chat $55.58 $0.431 129 │
│ ││ ░░░░░░░░░░ codeburn $0.00 $0.00 1 │
│ ││ ░░░░░░░░░░ xxxxx $0.00 $0.00 1 │
╰────────────────────────────────────────────────────────────────╯╰────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────────╮╭────────────────────────────────────────────────────────────────╮
│ By Activity ││ By Model │
│ cost turns 1-shot ││ cost cache calls 1-shot │
│ ██████████ Exploration $52.13 303 - ││ ██████████ Sonnet 4.6 $52.93 46.0% 147 - │
│ █░░░░░░░░░ Conversation $3.46 496 - ││ ░░░░░░░░░░ Haiku 4.5 $2.60 46.1% 37 - │
│ ░░░░░░░░░░ Debugging $0.00 4 - ││ ░░░░░░░░░░ GPT-4o Mini $0.047 45.7% 115 - │
│ ░░░░░░░░░░ Feature Dev $0.00 1 - ││ ░░░░░░░░░░ copilot-sugges $0.00 0.0% 4 - │
│ ░░░░░░░░░░ Brainstorming $0.00 2 - ││ ░░░░░░░░░░ copilot-nes-oc $0.00 0.0% 4 - │
│ ││ ░░░░░░░░░░ Copilot (Anthr $0.00 - 499 - │
╰────────────────────────────────────────────────────────────────╯╰────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────────╮╭────────────────────────────────────────────────────────────────╮
│ Core Tools ││ Shell Commands │
│ calls ││ No shell commands │
│ ██████████ Shell 5358 ││ │
│ █████░░░░░ Read 2883 ││ │
│ ███░░░░░░░ replace_string_in_file 1585 ││ │
│ ██░░░░░░░░ manage_todo_list 1126 ││ │
│ █░░░░░░░░░ grep_search 609 ││ │
│ █░░░░░░░░░ file_search 269 ││ │
│ ░░░░░░░░░░ create_file 191 ││ │
│ ░░░░░░░░░░ list_dir 57 ││ │
│ ░░░░░░░░░░ kill_terminal 38 ││ │
│ ░░░░░░░░░░ get_terminal_output 16 ││ │
╰────────────────────────────────────────────────────────────────╯╰────────────────────────────────────────────────────────────────╯
╭────────────────────────────────────────────────────────────────╮╭────────────────────────────────────────────────────────────────╮
│ Skills & Agents ││ MCP Servers │
│ No skill/agent usage ││ No MCP usage │
╰────────────────────────────────────────────────────────────────╯╰────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ <> switch q quit 1 today 2 week 3 30 days 4 month 5 6 months d yesterday o optimize c compare │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

…ests and regression checks.

Signed-off-by: steelp02 <pieter.steel@pfizer.com>
@steelp02

Copy link
Copy Markdown
Contributor Author

Otel information: https://code.visualstudio.com/docs/agents/guides/monitoring-agents
Additional instructions to get it working: enable otel by adding these two lines to settings.json in vscode:
"github.copilot.chat.otel.enabled": true, "github.copilot.chat.otel.dbSpanExporter.enabled": true

@dat-intellis

Copy link
Copy Markdown

Otel information: https://code.visualstudio.com/docs/agents/guides/monitoring-agents Additional instructions to get it working: enable otel by adding these two lines to settings.json in vscode: "github.copilot.chat.otel.enabled": true, "github.copilot.chat.otel.dbSpanExporter.enabled": true

is there a corresponding copilot-cli setting to enable otel?

@steelp02

Copy link
Copy Markdown
Contributor Author

Otel information: https://code.visualstudio.com/docs/agents/guides/monitoring-agents Additional instructions to get it working: enable otel by adding these two lines to settings.json in vscode: "github.copilot.chat.otel.enabled": true, "github.copilot.chat.otel.dbSpanExporter.enabled": true

is there a corresponding copilot-cli setting to enable otel?

export COPILOT_OTEL_ENABLED=true
export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318"

Signed-off-by: steelp02 <pieter.steel@pfizer.com>
iamtoruk added a commit that referenced this pull request Jun 18, 2026
feat(copilot): OTel cache-token parsing (#477) + maintainer review fixes
@iamtoruk
iamtoruk merged commit d68a214 into getagentseal:main Jun 18, 2026
@iamtoruk

Copy link
Copy Markdown
Member

Merged via #498, which preserves your commits and adds maintainer review fixes on top (debug-log cleanup, parameterized the spans IN() query, folded an N+1, epochToISO guard, dead-code removal, and docs for the Node 22+ requirement + the durable-cache monotonicity behavior). Thanks for the contribution. Closing in favor of #498.

timothybrush pushed a commit to timothybrush/codeburn that referenced this pull request Jun 18, 2026
…#477

Maintainer review fixes on top of the OTel cache-token work:

- Remove all DEBUG_OTEL console.warn scaffolding from parser.ts and
  copilot.ts (gated but unlike the rest of the codebase).
- Parameterize the spans IN (...) query instead of string-interpolating
  trace IDs.
- Fold the per-chat-span metadata query into the trace-span query to drop
  the N+1 (one query per chat span -> one per conversation).
- Guard epochToISO against null/NaN/0 so a malformed start_time_ms row no
  longer throws on new Date(NaN).toISOString().
- Remove dead code: parseSpanAttributes, OTelSpanRow, and the unreachable
  `if (!db) return`; drop unused catch bindings.
- Note in parseProviderSources that the non-durable append path assumes
  unique source paths.
- Document the OTel source in docs/providers/copilot.md: Node 22+
  requirement, durable-cache monotonic totals, and the one-time
  parse-version cache reset on upgrade.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants