Epic: public, tracked-over-time benchmarks for gittensory's own review engine (speed, accuracy trend, efficiency)

[JSONbored]

Motivation

We've spent the last week firefighting AI-spend incidents (token bleed from paused repos still spending, repeat-spend on unchanged PRs, silent misrouting to a billed provider instead of the free local one). Every one of those was only caught by hand-querying audit_events live in production. We have no durable, always-on way to see — and show the community — how the review engine is actually performing over time.

This epic is about turning gittensory's own operational behavior into three honest, trackable, public metrics: how fast we review, how accurate our decisions are, and how efficiently we avoid redundant work. Not a benchmark of the underlying LLMs (that's not something we're building), and not raw GPU hardware health (the host is rented infrastructure, not our product) — this is about the engine's own competence.

Non-goals (explicit)

• No GPU hardware utilization/VRAM/temp dashboard. That's tracked separately under feat(selfhost): GPU + AI-provider observability (exporter, Prometheus, Grafana dashboard) #4367 (part of the Epic: migrate self-hosted review-engine deployment to a new dedicated GPU host #4325 host-migration epic) for internal ops purposes; it is not a community-facing product metric and this epic does not duplicate it.
• No "our tokens are cheap" framing. Cost/token-rate is not something we built — it's not part of this epic's public story.

What already exists (don't rebuild)

• Review-accuracy infrastructure is already real and shipped: computeGateEval (src/review/parity.ts:88-169) joins every gate decision to its realized pr_outcome, reversal-discounted via recordReversalSignals (src/review/outcomes-wire.ts:402-471), already live on the homepage as "Decision accuracy %" (feat(stats): public proof-of-power — lifetime PRs-handled + accuracy counter on homepage hero #1059, feat(review): persist every AI review for accuracy-over-time tracking #1567, both shipped). This epic's accuracy sub-issue extends this into a tracked-over-time trend — it does not rebuild the underlying computation.
• #4367 (open, assigned) already covers GPU exporter + Grafana + a new gittensory_ai_provider_request_duration_seconds histogram for raw AI-provider call latency. That's internal ops observability; this epic's speed sub-issue is a different, public, end-to-end metric (webhook received → comment posted) and should reuse/reference that histogram where useful rather than duplicate it.
• #110 (closed) shipped reviewVelocityPerDay — a queue-throughput/PR-resolution-rate trend, not a per-review latency measurement. Different metric, cited for context only.
• #4261/#2352 track the miner's own self-review calibration (a different subsystem, packages/gittensory-engine/src/phase7-calibration-loop.ts) — not the maintainer-facing review engine this epic is about.

Sub-issues (priority order)

• feat(stats): review turnaround-time tracking (webhook received → comment posted), public + historical #4446 — Review turnaround-time tracking (webhook received → comment posted), public, tracked over time — priority 1, cleanest gap, nothing else measures this today.
• feat(stats): review-accuracy as a historical trend, not just a live snapshot #4447 — Review-accuracy as a historical trend, not just a live snapshot — priority 2, builds directly on existing computeGateEval/pr_outcome infrastructure.
• feat(stats): AI-feature cache/reuse-efficiency rate, tracked over time, public #4448 — AI-feature cache/reuse-efficiency rate, tracked over time, public — priority 3, widest scope (extends instrumentation to 6 currently-uninstrumented AI features).

Expected outcome

Three durable, historical, publicly-visible metrics — review speed, review accuracy, and resource efficiency — each backed by a real rollup table (not a live-computed snapshot), each shown on the public stats page/homepage with a trend, and each documented honestly enough that the methodology itself could be published.

Effort

Epic — 3 independently shippable sub-issues.

[linear-code]

JSO-3327