Skip to content

fix(ops): hide percentiles in Live Ops Console when sample size is too small - #40

Merged
yatesjalex merged 1 commit into
mainfrom
yatesclaude/dazzling-ishizaka-fbb824
Apr 23, 2026
Merged

fix(ops): hide percentiles in Live Ops Console when sample size is too small#40
yatesjalex merged 1 commit into
mainfrom
yatesclaude/dazzling-ishizaka-fbb824

Conversation

@yatesjalex

@yatesjalex yatesjalex commented Apr 23, 2026

Copy link
Copy Markdown
Owner

Summary

  • The Latency tab in the Live Ops Console unconditionally rendered p50/p95/p99, so with n=1 it stamped the same number three times (e.g. p50 1182 ms p95 1182 ms p99 1182 ms). Mathematically correct (one sample → all percentiles equal that sample), but visually indistinguishable from a broken measurement.
  • Switched LatencyRow to n=N last X ms when n < 5, and kept the percentile breakdown once there are enough samples for the numbers to differ meaningfully.

Why this looked "broken"

  • deals.getActivityOfferPlacements gets a client-side sample only from the page-header Refresh button, a refetchOnWindowFocus, or a staleTime expiry — the SSR prefetch at page.tsx:28 bypasses the latencyLink.
  • Clicking "Re-check attribution" fires the deals.simulatePartnerWebhook mutation, and its onSuccess writes events straight into the React Query cache (useApplyServerEvents.ts:32) — it never triggers a refetch of getActivityOfferPlacements, so that row stayed at n=1.
  • With one sample, percentile() returns sorted[0] for every p — so the three percentiles were identical by construction.

Test plan

  • pnpm typecheck
  • pnpm test (327/327 pass; added cases cover both the n<5 and n≥5 branches)
  • Open the Live Ops Console → Latency tab, confirm a single-sample row now reads n=1 last X ms instead of three identical percentiles, and that rows with ≥5 samples still show p50/p95/p99.

Note

Low Risk
Low risk UI-only change in the Live Ops Console latency display; behavior is gated by a simple n < 5 condition and covered by updated tests.

Overview
Updates the Live Ops Console Latency tab to avoid misleading percentile output for tiny sample sizes: when n < 5, rows now render n=N last X ms instead of p50/p95/p99, while keeping percentile breakdown for n ≥ 5.

Adds/adjusts tests in LiveOpsConsole.test.tsx to assert the new last formatting for a single sample and that p50/p95/p99 appear once five samples are recorded.

Reviewed by Cursor Bugbot for commit fbf07af. Bugbot is set up for automated code reviews on this repo. Configure here.

LatencyRow unconditionally stamped p50, p95, and p99, so with n=1 it
rendered the same number three times (e.g. "p50 1182 p95 1182 p99 1182")
— mathematically correct but visually indistinguishable from a broken
measurement. Switch to `n=N  last X ms` when n<5, and only show the
percentile breakdown once there are enough samples for those numbers to
differ meaningfully.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Apr 23, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
super Ready Ready Preview, Comment Apr 23, 2026 7:12am

@yatesjalex
yatesjalex merged commit f06c25c into main Apr 23, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant