Real-Time Resource Consumption Anomaly Detector Using Sliding Window Z-Score and Adaptive Thresholding

[elizabetheonoja-art]

Problem Statement / Feature Objective

Operator dashboards display live resource consumption from thousands of metered endpoints. A faulty meter, a burst pipe, or a cyber-intrusion event can manifest as a consumption spike or drop. Manual threshold configuration is impractical at scale. An unsupervised anomaly detection engine must run in-browser on the streaming WebSocket data, compute sliding-window statistics (mean, stddev) per meter, and flag deviations exceeding a dynamic z-score threshold. Alerts must be surfaced as both UI badges and dispatched Redux actions for downstream logging.

Technical Invariants & Bounds

• Sliding window size: 1,200 observations per meter (10 minutes at 2 Hz update rate).
• Z-score threshold: adaptive at for spike detection, for sustained drift over 5 consecutive windows.
• Maximum tracked meters: 5,000 concurrently. Each window stores 1,200 float64 values ≈ 48 KB per meter → 240 MB worst-case. Must use a ring buffer compressed to running sums (M1, M2, M3, M4) for incremental mean/variance/skewness.
• Computation budget: anomaly scoring for all 5,000 meters must complete within 200 ms per tick (2 Hz → 500 ms interval).
• False positive suppression: ignore single-frame anomalies; require 3 of 5 consecutive windows to breach threshold before emitting alert.

Codebase Navigation Guide

• src/hooks/useWebSocket.ts — Raw WebSocket connection; emits parsed to Redux.
• src/store/slices/telemetrySlice.ts — Stores latest telemetry per meter ID; also holds anomaly flags.
• src/workers/anomalyDetect.worker.ts — Dedicated Web Worker holding per-meter ring buffers and running z-score computations.
• src/utils/stats.ts — Welford's online algorithm for running mean, variance, skewness.
• src/components/panels/AnomalyBadge.tsx — UI badge that blinks red/orange based on anomaly severity.
• src/pages/Dashboard.tsx — Top-level dashboard that instantiates the anomaly worker.

Implementation Blueprint

1. Write class in implementing Welford's online algorithm: , , , . Support reset.
2. In , maintain a and a ring buffer (implemented as a fixed Float64Array per meter with head pointer). On each message from the main thread:
   a. Push each reading into the meter's RunningStats and ring buffer.
   b. Compute z-score for latest value. Track consecutive-window breach counter.
   c. If 3 of 5 consecutive windows breach the z-score threshold, emit .
3. The main thread receives anomaly events from the worker and dispatches .
4. In the reducer, update with and optionally increment an alert counter.
5. reads for its assigned meter and renders a colored indicator; auto-dismiss after 300 seconds of no further alerts.
6. Add a memory-safety check: if the main thread detects the worker's exceeds 300 MB estimated usage, downgrade window to 600 observations and log a warning.

[Tobi-8]

please can you assign me to this issue