feat(prd-142): Wave 0 measurement backend (real dashboard metrics)#400
Merged
Conversation
The dashboard reported a hardcoded 85.0% success rate, 2.5s avg execution time, and 0 tokens for agents — none backed by a real source. Drop these placeholders from _get_agent_metrics and get_agent_analytics so the API returns only real, DB-derived fields. track_agent_execution only logs to Redis, so there is no persisted per-agent source to derive from yet; mission-level success rate (real) lives in _get_workflow_metrics. No consumer reads the removed fields (verified frontend + backend). Adds tests asserting the fakes are gone and success rate is computed, not 85.0.
Tailored autonomous build harness for the 5 backend stories (US-001/002/004/005/006): PROMPT_build_prd142.md scopes the run, uses the prd-142-wave0.json `passes` field as source of truth, enforces TDD, the reuse map, workspace-scoping + tenant tests, and commits branch-local only (never push/merge). loop-prd142.sh drives it with usage-limit backoff and a RALPH_COMPLETE/RALPH_BLOCKED stop. US-007 (frontend) and US-008 (live gate) are explicitly out of scope for the loop. US-003 marked passes:true.
…s sink Adds the error_events table (alembic + ORM model + best-effort writer) that backs Wave 0's "error rate by subsystem" tile. Mirrors the PRD-008-A widget_event_log pattern: single table, JSONB payload, fire-and-forget writer that never propagates. record_error's signature and never-raises contract are unchanged; the automatos.errors logger emit still happens first so a DB outage cannot blind-spot a failure. Story: scripts/ralph/prd-142-wave0.json US-001 PRD: docs/PRDS/PRD-142-WAVE0-MEASUREMENT.md
Add GET /api/analytics/errors/by-subsystem?window=24h to the existing
api/analytics_real.py router (no new router file). Workspace-scoped via
ctx.workspace_id; aggregates the error_events sink (US-001) by subsystem
over a rolling window. Returns {window,total,by_subsystem,generated_at}
with per-row rate=count/total and zero-safety. Indexes from US-001 carry
the (workspace_id, created_at) + GROUP BY subsystem path.
Story: scripts/ralph/prd-142-wave0.json US-002
PRD: docs/PRDS/PRD-142-WAVE0-MEASUREMENT.md
Adds read-only GET /api/analytics/widget-engagement to existing api/analytics_real.py router (no new file). Reuses widget_event_log (PRD-008-A) — workspace scoped via Site.workspace_id → site_id.in_(), restricted to WIDGET_EVENT_TYPES so idx_widget_event_log_type_created is eligible alongside the created_at window. Sessions counted as distinct session_id. Endpoint constructs zero WidgetEventLog rows; the telemetry writer remains the single source of truth. Story: scripts/ralph/prd-142-wave0.json US-004 PRD: docs/PRDS/PRD-142-WAVE0-MEASUREMENT.md
Adds GET /api/analytics/activation to the existing api/analytics_real.py
router (no new router file). Reuses the OrchestrationRun union the rest
of analytics_real already queries (api/analytics_real.py:69) and the
canonical RunState.COMPLETED.value enum from core/models/orchestration_enums.py
— never the literal "completed" string. Intentionally platform-level
(per US-005 notes): auth gated via get_request_context_hybrid but NO
workspace_id filter on the data. Returns {activated, total_workspaces,
rate, generated_at}; rate=0 when total_workspaces=0 with no divide-by-zero
and no fake fallback.
Story: scripts/ralph/prd-142-wave0.json US-005
PRD: docs/PRDS/PRD-142-WAVE0-MEASUREMENT.md
Contributor
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (13)
📝 WalkthroughWalkthroughThis PR implements PRD-142 Wave 0 "Measurement First," adding error event persistence, three analytics endpoints (errors by subsystem, widget engagement, platform activation rate), and cleaning up hardcoded metrics placeholders. All changes include test coverage and Ralph automation tooling to track completion. ChangesError Events & Analytics Endpoints
Ralph Build Automation & Wave 0 Completion
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PRD-142 Wave 0 — the "Is it working?" measurement backend. Replaces fake/placeholder analytics with real, workspace-scoped metrics feeding the existing dashboard. Backend + instrumentation only — no new page, no new route, no frontend in this PR.
record_error()to a queryableerror_eventssink (new table + model + best-effort write path that never raises and rolls back cleanly).GET /api/analytics/errors/by-subsystem: real error-rate-by-subsystem, workspace-scoped, windowed (?window=24h), backed by an index on(workspace_id, created_at).analytics_engine.py.GET /api/analytics/widget-engagement: real widget engagement, tenant-isolated via Site→workspace resolution.GET /api/analytics/activation: platform-level activation (distinct workspaces with ≥1 completed run / total workspaces).scripts/ralph/) — prompt, loop script, story spec.Scope / deferrals
agent_health,checklist, …), none mapped to the 8 product primitives. Building now would ship a hollow all-"unknown" tile. Contract + sourcing options recorded for Wave 3.Test plan
alembic upgrade heads); confirmerror_eventstable created.Summary by CodeRabbit
Release Notes
New Features
Tests