No instrumentation for any of the Build Plan target metrics

[NathanTarbert]

The Build Plan (internal) opens with a target-metrics table that defines what "Outpost is working" means. None of it is measured today, and none of it appeared on the roadmap until now.

Metric	Outpost target
First response time	under 10 minutes
Resolution time	under 24 hours
SLA breaches / month	under 20
AI accuracy	over 80% helpful

Baselines for each of these are tracked internally.

Why this matters now

We are about to cut over from Orca (#59) on the strength of these numbers, and we cannot currently produce any of them.

One is also actively misleading: the SLA breach count is computed against a global per-priority map tuned for paid support, so SLA_CHECK flags essentially every open ticket — the dashboard's breach count has been running at roughly the full open-ticket count. Compared against a target of "under 20 per month", that number is not just wrong, it inverts the conclusion. #203 covers the fix.

"Over 80% helpful" is the one that should gate the cutover, and it is the one with no instrumentation at all. The feedback signal needed to compute it exists on both channels after #101 and the GITHUB_REACTION_POLL job — Discord buttons and GitHub reactions both write feedback — but nothing aggregates it into a rate.

What to build

• First response + resolution time: derivable from Ticket and Message timestamps today. No new data needed.
• Helpful rate: aggregate Message.feedback (POSITIVE / NEGATIVE / null) over a window. Blocked in practice by Withheld drafts are indistinguishable from delivered replies (replayed as assistant history, no DB marker) #148 — if the DB records the draft rather than what was posted, a helpfulness rate is measuring the wrong text.
• Cost tracking: nothing anywhere today. Model spend plus hosting spend, at minimum.
• Somewhere to see them. The dashboard already has an SLA Health card; this is the honest version of it.

Related

• Per-account contractual SLAs + a config UI — the global map flags every open ticket #203 — per-account SLAs, which is what makes the breach number meaningful
• Surface AI response confidence in dashboard + real feedback loop on both channels #100 — surface AI confidence in the dashboard; the same surface should carry the helpful rate
• Withheld drafts are indistinguishable from delivered replies (replayed as assistant history, no DB marker) #148 — record what was posted, so a helpfulness rate measures the delivered answer