diff --git a/docs/index.md b/docs/index.md index be0107d98..e05158ec6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -141,7 +141,7 @@ Organized by domain (model line / subsystem / playbook / lesson) instead of by l | `subsystems/frontend/simulated-inference-engine.md` | CPU-only simulated model crate for vLLM/OpenAI frontend and `vllm bench serve` validation without CUDA, real model weights, or real-model performance claims. | | `subsystems/frontend/cpu-profiling-baseline.md` | Frontend CPU profiling baseline using `openinfer-sim` with fixed TTFT=5ms/TPOT=12ms: 200 req / concurrency=16 shows ~150ms TTFT overhead (no dominant hotspot), heap allocation ~10%, stream polling ~7.5%, IPC ~1%; reproducible benchmark command and perf evidence documented. | | `subsystems/frontend/startup-time.md` | Qwen3-4B warm startup-to-ready 3.25s → ~1.45s: frontend tokenizer load runs concurrently with the engine load (HTTP still binds only after the engine registers), and the source safetensors mmap is kept alive to dodge ~0.4s of munmap stalling the next cudaMalloc. | -| `subsystems/frontend/prometheus-metrics.md` | `/metrics` request histograms work for every model; Qwen3, Qwen3.5, and GLM5.2 schedulers also publish running/waiting/KV engine gauges through `LoadSnapshot` watches. | +| `subsystems/frontend/prometheus-metrics.md` | `/metrics` request histograms come from bridge events/PrefillStats; scheduler stats ride `LoadSnapshot`. Qwen3 now reports real prefix-cache query/hit counters via monotonic totals differenced by the bridge, plus running/waiting/KV gauges. | | `subsystems/frontend/dashboards/README.md` | Grafana 10.4-validated dashboard for OpenInfer's live `/metrics` surface: HTTP traffic, request outcomes, scheduler/KV state, token throughput, and request latency. | ## subsystems / correctness diff --git a/docs/subsystems/frontend/prometheus-metrics.md b/docs/subsystems/frontend/prometheus-metrics.md index e17df01c8..db422a56d 100644 --- a/docs/subsystems/frontend/prometheus-metrics.md +++ b/docs/subsystems/frontend/prometheus-metrics.md @@ -1,31 +1,23 @@ -# Prometheus /metrics via the vLLM frontend +# Prometheus `/metrics` via the vLLM frontend -**TL;DR:** `/metrics` exposes request histograms for every model and engine gauges for schedulers that publish `LoadSnapshot`: Qwen3 and Qwen3.5 use one logical engine, while GLM5.2 EP8/DP8 uses eight rank-local engines and GLM5.2 TP8 uses one logical engine. The bridge forwards each partition's stats under the same identity the vLLM frontend uses for least-load routing. +**TL;DR:** `/metrics` exposes request metrics for every model. Schedulers that publish `LoadSnapshot` also expose load and KV gauges; Qwen3 additionally reports real prefix-cache query and hit token counters. Last touched: 2026-07 -## How the numbers flow +## Metric sources -Two independent paths feed the upstream Prometheus registry (`vllm-metrics`, served by `vllm-server` at `/metrics` with its HTTP middleware counters): +- Request metrics come from frontend request events and include latency histograms, prompt/generated token totals, and request outcomes. +- Scheduler metrics come from `LoadSnapshot`. They are present only for model schedulers that publish a load watch. +- Each logical scheduler partition has its own `engine` label. -1. **Per-request path (works for every model crate).** The bridge stamps each request's first output with `Queued`/`Scheduled` timestamps and `PrefillStats` (prompt/computed/cached token split). The upstream `RequestMetricsTracker` turns those into `time_to_first_token_seconds`, `inter_token_latency_seconds`, `request_queue_time_seconds`, `prompt_tokens_total`, `generation_tokens_total`, `request_success_total`, `prompt_tokens_by_source_total`, … unconditionally — `disable_log_stats` only gates the periodic *text* logger, not Prometheus. -2. **Engine-gauge path (needs one `LoadSnapshot` watch per scheduler partition).** The scheduler publishes `LoadSnapshot { kv_used_blocks, kv_total_blocks, num_running_reqs, num_waiting_reqs }` at scheduler boundaries; one bridge identity per partition forwards its snapshot as a stats-only `RequestBatchOutputs`. The enclosing `engine_index` is both the routing identity and the Prometheus `engine` label. Watches coalesce to ≤1 message per scheduler step, and the scheduler's final idle publish settles the gauges back to 0. +Qwen3 publishes monotonic prefix-cache totals. The frontend converts them to interval deltas before passing them to vLLM's Prometheus collector, so coalesced load-watch updates do not lose or double-count increments. -For a single-partition model, `EngineHandle::with_load_watch` keeps the original one-engine contract. Qwen3.5 uses that contract for both its single-GPU backend and its TP backend because both execute one logical request stream through one scheduler. A partitioned scheduler uses `with_load_watches`, and the frontend launch declares the same engine count; a mismatch fails startup. GLM5.2 EP8 therefore registers engines 0–7, each bound to its own pending queue and KV pool. TP8 registers only engine 0 because its eight workers mirror one logical request stream. +`vllm:prefix_cache_queries_total` counts prompt tokens submitted to an actual first-prefix lookup. `vllm:prefix_cache_hits_total` counts tokens restored from matching full cache blocks. Echo requests, cache-disabled requests, and later chunks of the same prefill do not add queries. Qwen3 cache blocks contain 16 tokens. -Measured cost is noise in both covered configurations: +## Check prefix-cache counters -- Qwen3 TPOT: 10.6387 ms (main) vs 10.6395 ms (metrics branch) over 828 tokens. -- GLM5.2 EP8, three-run median at concurrency 64: 1268.58 vs 1264.82 output tok/s (-0.30%); TPOT p50 41.76 vs 41.35 ms. +Send the same prompt of at least 16 tokens twice, then scrape `/metrics` and inspect `vllm:prefix_cache_queries_total` and `vllm:prefix_cache_hits_total`. Both counters are cumulative. Queries should increase for each request; hits should increase when the repeated prompt reuses cached blocks. -## What deliberately reads zero (state at capture time) +## Coverage limits -- `prefix_cache_queries/hits` and the by-reason waiting split (`reason="deferred"` is driven by a skipped-request counter we don't report; all waiting shows as `reason="capacity"`). -- Spec-decode counters, per-GPU FLOPs/bytes estimates, KV-block residency histograms, cudagraph stats — the bridge sends `SchedulerStats::default()` for these fields. -- Every model crate whose scheduler doesn't publish a `LoadSnapshot` watch (currently deepseek and kimi) gets path 1 only; its engine gauges are absent, not lying-zero — the bridge skips the stats task for that partition when no watch exists. - -## Validated coverage and next step - -Qwen3.5 single-GPU live RTX 5090 validation confirmed that running and KV gauges rise during generation, waiting rises under batch-slot pressure, and all three return to zero after drain and recovery. The commands and metric samples are recorded in [Qwen3.5 Scheduler LoadSnapshot](../../models/qwen35/load-snapshot.md#validation-boundary). TP uses the same scheduler publication path but was not part of that live run. - -Next, wire the DeepSeek-V2-Lite and Kimi-K2 schedulers using the same recipe, and report real prefix-cache query/hit counters instead of zeros. A future partitioned model must expose its logical scheduler partitions instead of averaging them behind engine 0. +Unsupported scheduler fields remain at their upstream defaults, including speculative-decoding counters, GPU FLOP/byte estimates, KV-residency histograms, and CUDA Graph statistics. Models without a `LoadSnapshot` watch expose request metrics but no scheduler gauges. diff --git a/openinfer-engine/src/engine.rs b/openinfer-engine/src/engine.rs index a21323f42..ae1d1446b 100644 --- a/openinfer-engine/src/engine.rs +++ b/openinfer-engine/src/engine.rs @@ -352,6 +352,16 @@ pub struct LoadSnapshot { pub num_running_reqs: u64, /// Requests admitted but not yet running (KV pressure, prefetch wait). pub num_waiting_reqs: u64, + /// Prompt tokens queried against the local prefix cache since engine start. + /// + /// This is a monotonic total because the snapshot rides a coalescing watch + /// channel. Consumers that feed delta counters must difference consecutive + /// snapshots rather than forwarding this value directly. + pub prefix_cache_queries_total: u64, + /// Prompt tokens served from the local prefix cache since engine start. + /// Monotonic for the same coalescing-safe reason as + /// [`Self::prefix_cache_queries_total`]. + pub prefix_cache_hits_total: u64, } /// One full KV block that just became reusable from this engine's prefix cache. diff --git a/openinfer-glm52/src/scheduler/load.rs b/openinfer-glm52/src/scheduler/load.rs index 8578e0e33..2a0c214f1 100644 --- a/openinfer-glm52/src/scheduler/load.rs +++ b/openinfer-glm52/src/scheduler/load.rs @@ -36,6 +36,7 @@ pub(super) fn publish_load( kv_total_blocks: kv_total_blocks as u64, num_running_reqs: slots[rank].iter().flatten().count() as u64, num_waiting_reqs: pending[rank].len() as u64, + ..LoadSnapshot::default() }); } } diff --git a/openinfer-qwen3/src/executor.rs b/openinfer-qwen3/src/executor.rs index 3ffbfb620..d5b63561e 100644 --- a/openinfer-qwen3/src/executor.rs +++ b/openinfer-qwen3/src/executor.rs @@ -95,6 +95,9 @@ pub struct PrefillStepItem { /// Set by the executor after matching; the forward pass only computes /// the remaining suffix. pub(crate) cached_tokens: usize, + /// Whether this step performed the request's one local prefix-cache lookup. + /// False for later chunks, echo requests, and cache-disabled execution. + pub(crate) prefix_cache_queried: bool, /// Scheduler-set cap on prompt tokens forwarded this step (chunked /// prefill). The executor clamps it to the tokens actually remaining. pub(crate) chunk_budget: usize, @@ -125,6 +128,7 @@ impl PrefillStepItem { echo, lora_adapter: None, cached_tokens: 0, + prefix_cache_queried: false, chunk_budget: usize::MAX, chunk_start: 0, chunk_tokens, @@ -307,6 +311,7 @@ fn build_prefill_request_results( first_token_logprob: first_token_logprobs[i].take(), prompt_logprobs, cached_tokens: req.cached_tokens, + prefix_cache_queried: req.prefix_cache_queried, completed, prefill_pos: req.chunk_start + req.chunk_tokens, }); @@ -805,6 +810,10 @@ pub struct PrefillRequestResult { pub prompt_logprobs: Option>>, /// Prompt tokens served from the prefix cache (KV reused, not recomputed). pub cached_tokens: usize, + /// Whether the executor actually queried the local prefix cache for this + /// request. The scheduler uses this to exclude echo and cache-disabled + /// prefills from the aggregate query denominator. + pub prefix_cache_queried: bool, /// Whether the prompt is fully prefilled. When false this step ran a /// non-final chunk and `first_token` is meaningless. pub completed: bool, @@ -1841,6 +1850,7 @@ impl Qwen3Executor { // Echo needs logits for every prompt position; cached positions // are never forwarded, so echo requests prefill from scratch. if self.prefix_cache_enabled && !req.echo { + req.prefix_cache_queried = true; req.cached_tokens = rkv.match_and_add_prefix(self.kv_mgr.pool())?; } self.request_kvs.insert(req.request_id, rkv); diff --git a/openinfer-qwen3/src/scheduler.rs b/openinfer-qwen3/src/scheduler.rs index 56df3fac6..2401f0875 100644 --- a/openinfer-qwen3/src/scheduler.rs +++ b/openinfer-qwen3/src/scheduler.rs @@ -93,6 +93,23 @@ pub(super) struct PendingRequest { pub(super) cached_tokens: usize, } +/// Lifetime prefix-cache token totals published through `LoadSnapshot`. +/// Keeping totals here makes the coalescing watch lossless; the frontend bridge +/// differences observed snapshots into the interval deltas vLLM expects. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub(super) struct PrefixCacheTotals { + queries: u64, + hits: u64, +} + +impl PrefixCacheTotals { + fn record(&mut self, prompt_tokens: usize, cached_tokens: usize) { + debug_assert!(cached_tokens <= prompt_tokens); + self.queries = self.queries.saturating_add(prompt_tokens as u64); + self.hits = self.hits.saturating_add(cached_tokens as u64); + } +} + impl PendingRequest { fn from_scheduler_request(request_id: RequestId, req: GenerateRequest) -> Self { Self { @@ -498,12 +515,15 @@ fn publish_load( executor: &E, num_running_reqs: u64, num_waiting_reqs: u64, + prefix_cache_totals: PrefixCacheTotals, ) { load_tx.send_replace(LoadSnapshot { kv_used_blocks: kv_total.saturating_sub(executor.available_blocks() as u64), kv_total_blocks: kv_total, num_running_reqs, num_waiting_reqs, + prefix_cache_queries_total: prefix_cache_totals.queries, + prefix_cache_hits_total: prefix_cache_totals.hits, }); } @@ -532,6 +552,7 @@ fn scheduler_loop( // Decode-overlap async prefill: pending requests whose prefill is in-flight // on the prefill overlap stream. `None` when no async prefill is running. let mut inflight_prefill_pending: Option> = None; + let mut prefix_cache_totals = PrefixCacheTotals::default(); info!("Scheduler ready"); @@ -544,6 +565,7 @@ fn scheduler_loop( + prefilling.len() + inflight_prefill_pending.as_ref().map_or(0, Vec::len)) as u64, (deferred.len() + loading.len()) as u64, + prefix_cache_totals, ); // Flush the prior step's cache changes to a router (no-op unless the // event feed is on). Top-of-loop, like `publish_load`: one pass per @@ -570,7 +592,13 @@ fn scheduler_loop( scheduled_at_unix_s, }; let effects = resolve_step(&executor, &active, artifacts); - apply_effects(&mut executor, &mut active, &mut prefilling, effects); + apply_effects( + &mut executor, + &mut active, + &mut prefilling, + &mut prefix_cache_totals, + effects, + ); } } @@ -703,7 +731,13 @@ fn scheduler_loop( // Only apply decode effects from the unified result. let effects = resolve_step(&executor, &active, artifacts); - apply_effects(&mut executor, &mut active, &mut prefilling, effects); + apply_effects( + &mut executor, + &mut active, + &mut prefilling, + &mut prefix_cache_totals, + effects, + ); // Track the pending prefill for next-iteration polling. inflight_prefill_pending = Some(pending_for_poll); @@ -721,7 +755,13 @@ fn scheduler_loop( } }; let effects = resolve_step(&executor, &active, artifacts); - apply_effects(&mut executor, &mut active, &mut prefilling, effects); + apply_effects( + &mut executor, + &mut active, + &mut prefilling, + &mut prefix_cache_totals, + effects, + ); } } @@ -743,6 +783,7 @@ fn scheduler_loop_with_lora_control( let mut prefilling: Vec = Vec::new(); let mut pending_control: VecDeque = VecDeque::new(); let mut post_control_deferred: Vec = Vec::new(); + let mut prefix_cache_totals = PrefixCacheTotals::default(); info!("Scheduler ready with LoRA control"); @@ -753,6 +794,7 @@ fn scheduler_loop_with_lora_control( &executor, (active.len() + prefilling.len()) as u64, (deferred.len() + loading.len() + post_control_deferred.len()) as u64, + prefix_cache_totals, ); // 1. Drain incoming commands. Generation submitted after a pending @@ -873,7 +915,13 @@ fn scheduler_loop_with_lora_control( } }; let effects = resolve_step(&executor, &active, artifacts); - apply_effects(&mut executor, &mut active, &mut prefilling, effects); + apply_effects( + &mut executor, + &mut active, + &mut prefilling, + &mut prefix_cache_totals, + effects, + ); } } diff --git a/openinfer-qwen3/src/scheduler/effects.rs b/openinfer-qwen3/src/scheduler/effects.rs index 6a7eab47a..3b7eea712 100644 --- a/openinfer-qwen3/src/scheduler/effects.rs +++ b/openinfer-qwen3/src/scheduler/effects.rs @@ -5,6 +5,7 @@ use openinfer_core::engine::TokenSink; use super::ActiveRequestState; use super::PendingRequest; +use super::PrefixCacheTotals; use super::TokenEvent; use crate::executor::RequestId; @@ -24,6 +25,7 @@ pub(super) struct ScheduledEffect { pub(super) scheduled_at_unix_s: f64, pub(super) prompt_tokens: usize, pub(super) cached_tokens: usize, + pub(super) prefix_cache_queried: bool, } pub(super) enum PendingEffect { @@ -110,6 +112,7 @@ pub(super) fn apply_effects( executor: &mut impl crate::executor::ModelExecutor, active: &mut Vec, prefilling: &mut Vec, + prefix_cache_totals: &mut PrefixCacheTotals, effects: StepEffects, ) { // `Finished` events are not sent inline: they are collected here and @@ -121,6 +124,9 @@ pub(super) fn apply_effects( let mut finishes: Vec<(TokenSink, TokenEvent)> = Vec::new(); for scheduled in effects.scheduled { + if scheduled.prefix_cache_queried { + prefix_cache_totals.record(scheduled.prompt_tokens, scheduled.cached_tokens); + } let _ = scheduled.token_tx.send(TokenEvent::Scheduled { queued_at_unix_s: scheduled .queued_at_unix_s diff --git a/openinfer-qwen3/src/scheduler/plan.rs b/openinfer-qwen3/src/scheduler/plan.rs index fa5d479e7..70eea1250 100644 --- a/openinfer-qwen3/src/scheduler/plan.rs +++ b/openinfer-qwen3/src/scheduler/plan.rs @@ -225,6 +225,7 @@ fn build_prefill_items(pending: &[PendingRequest], indices: &[usize]) -> Vec
>>,
     prefetch_offers: Arc>>,
     stop_token: Option,
+    prefix_cache_hits: VecDeque,
 }
 
 impl FakeExecutor {
@@ -52,6 +53,7 @@ impl FakeExecutor {
             dropped,
             prefetch_offers: Arc::new(Mutex::new(Vec::new())),
             stop_token: None,
+            prefix_cache_hits: VecDeque::new(),
         }
     }
 
@@ -75,6 +77,11 @@ impl FakeExecutor {
         self
     }
 
+    fn with_prefix_cache_hits(mut self, hits: &[usize]) -> Self {
+        self.prefix_cache_hits = hits.iter().copied().collect();
+        self
+    }
+
     /// Advance a request's prompt by one chunk, mirroring the real
     /// executor: clamp the scheduler's budget to the tokens remaining
     /// and report the new authoritative position.
@@ -92,12 +99,16 @@ impl FakeExecutor {
         } else {
             self.prefill_positions.insert(req.request_id, prefill_pos);
         }
+        let cached_tokens = (start == 0)
+            .then(|| self.prefix_cache_hits.pop_front())
+            .flatten();
         PrefillRequestResult {
             request_id: req.request_id,
             first_token: 100 + req.request_id.get() as u32,
             first_token_logprob: None,
             prompt_logprobs: None,
-            cached_tokens: 0,
+            cached_tokens: cached_tokens.unwrap_or(0),
+            prefix_cache_queried: cached_tokens.is_some(),
             completed,
             prefill_pos,
         }
@@ -838,6 +849,36 @@ fn wait_until(timeout: Duration, mut predicate: impl FnMut() -> bool) -> bool {
     false
 }
 
+#[test]
+fn load_snapshot_accumulates_prefix_cache_query_and_hit_tokens() {
+    let dropped = Arc::new(Mutex::new(Vec::new()));
+    let executor = FakeExecutor::new(8, Arc::clone(&dropped)).with_prefix_cache_hits(&[0, 16]);
+    let handle = start_with_executor(executor, 42, DEFAULT_MAX_PREFILL_TOKENS);
+    let load_rx = handle.load_watch().expect("qwen3 publishes load snapshots");
+
+    for _ in 0..2 {
+        let (req, mut token_rx) = request(32, 1);
+        handle.submit(req).expect("submit cache-metrics request");
+        assert!(matches!(
+            recv_skipping_scheduled(&mut token_rx),
+            Some(TokenEvent::Token { .. })
+        ));
+        assert!(matches!(
+            recv_skipping_scheduled(&mut token_rx),
+            Some(TokenEvent::Finished { .. })
+        ));
+    }
+
+    assert!(
+        wait_until(Duration::from_secs(1), || {
+            let snapshot = *load_rx.borrow();
+            snapshot.prefix_cache_queries_total == 64 && snapshot.prefix_cache_hits_total == 16
+        }),
+        "two 32-token cache lookups should publish 64 queried and 16 hit tokens; got {:?}",
+        *load_rx.borrow()
+    );
+}
+
 #[test]
 fn unknown_lora_request_is_rejected_without_blocking_base_request() {
     let dropped = Arc::new(Mutex::new(Vec::new()));
@@ -962,6 +1003,7 @@ fn retiring_multiple_active_requests_tolerates_unsorted_indices() {
         &mut executor,
         &mut active,
         &mut Vec::new(),
+        &mut PrefixCacheTotals::default(),
         effects::StepEffects {
             scheduled: Vec::new(),
             prompt_echoes: Vec::new(),
diff --git a/openinfer-sim/tests/frontend_e2e.rs b/openinfer-sim/tests/frontend_e2e.rs
index f1ebcd246..ddf279a39 100644
--- a/openinfer-sim/tests/frontend_e2e.rs
+++ b/openinfer-sim/tests/frontend_e2e.rs
@@ -273,6 +273,7 @@ async fn one_http_endpoint_exports_per_engine_scheduler_metrics() -> Result<()>
             kv_total_blocks: 100,
             num_running_reqs: 1,
             num_waiting_reqs: 0,
+            ..LoadSnapshot::default()
         },
     )?;
     server.publish_load(
@@ -282,6 +283,7 @@ async fn one_http_endpoint_exports_per_engine_scheduler_metrics() -> Result<()>
             kv_total_blocks: 100,
             num_running_reqs: 0,
             num_waiting_reqs: 2,
+            ..LoadSnapshot::default()
         },
     )?;
     wait_for_metrics(
@@ -304,6 +306,7 @@ async fn one_http_endpoint_exports_per_engine_scheduler_metrics() -> Result<()>
             kv_total_blocks: 100,
             num_running_reqs: 3,
             num_waiting_reqs: 4,
+            ..LoadSnapshot::default()
         },
     )?;
     server.publish_load(
@@ -313,6 +316,7 @@ async fn one_http_endpoint_exports_per_engine_scheduler_metrics() -> Result<()>
             kv_total_blocks: 100,
             num_running_reqs: 5,
             num_waiting_reqs: 6,
+            ..LoadSnapshot::default()
         },
     )?;
     wait_for_metrics(
diff --git a/openinfer-vllm-frontend/src/bridge.rs b/openinfer-vllm-frontend/src/bridge.rs
index e1530a87b..92cdf30be 100644
--- a/openinfer-vllm-frontend/src/bridge.rs
+++ b/openinfer-vllm-frontend/src/bridge.rs
@@ -599,9 +599,20 @@ async fn publish_scheduler_stats(
     output_tx: mpsc::UnboundedSender,
     shutdown: CancellationToken,
 ) -> Result<()> {
+    let mut previous_prefix_cache_queries = 0;
+    let mut previous_prefix_cache_hits = 0;
     loop {
         let snapshot = *load_rx.borrow_and_update();
-        let stats = SchedulerStats {
+        let prefix_cache_queries = snapshot
+            .prefix_cache_queries_total
+            .saturating_sub(previous_prefix_cache_queries);
+        let prefix_cache_hits = snapshot
+            .prefix_cache_hits_total
+            .saturating_sub(previous_prefix_cache_hits);
+        previous_prefix_cache_queries = snapshot.prefix_cache_queries_total;
+        previous_prefix_cache_hits = snapshot.prefix_cache_hits_total;
+
+        let mut stats = SchedulerStats {
             num_running_reqs: snapshot.num_running_reqs,
             num_waiting_reqs: snapshot.num_waiting_reqs,
             kv_cache_usage: if snapshot.kv_total_blocks == 0 {
@@ -611,6 +622,12 @@ async fn publish_scheduler_stats(
             },
             ..SchedulerStats::default()
         };
+        // Upstream records these with Prometheus `inc_by`, so only the delta
+        // since this bridge's previous observation belongs in SchedulerStats.
+        // The scheduler-side values stay cumulative because watch updates may
+        // coalesce; differencing a jump preserves every skipped increment.
+        stats.prefix_cache_stats.base.queries = prefix_cache_queries;
+        stats.prefix_cache_stats.base.hits = prefix_cache_hits;
         let outputs = RequestBatchOutputs {
             engine_index,
             scheduler_stats: Some(Box::new(stats)),
diff --git a/openinfer-vllm-frontend/src/bridge/tests.rs b/openinfer-vllm-frontend/src/bridge/tests.rs
index ab9bddac8..e0a51c058 100644
--- a/openinfer-vllm-frontend/src/bridge/tests.rs
+++ b/openinfer-vllm-frontend/src/bridge/tests.rs
@@ -456,8 +456,9 @@ fn rejected_request_is_reported_as_error() {
 
 /// The scheduler-stats task turns each load-watch snapshot into a stats-only
 /// batch (no request outputs, no finished set) with the queue gauges and the
-/// fractional KV usage the frontend records into Prometheus, sends the current
-/// snapshot up front, and follows every watch update with exactly one message.
+/// fractional KV usage and interval prefix-cache deltas the frontend records
+/// into Prometheus, sends the current snapshot up front, and follows every
+/// watch update with exactly one message.
 #[tokio::test]
 async fn load_snapshots_become_stats_only_batches() {
     let (load_tx, load_rx) = tokio::sync::watch::channel(LoadSnapshot {
@@ -465,6 +466,8 @@ async fn load_snapshots_become_stats_only_batches() {
         kv_total_blocks: 100,
         num_running_reqs: 2,
         num_waiting_reqs: 1,
+        prefix_cache_queries_total: 32,
+        prefix_cache_hits_total: 16,
     });
     let (output_tx, mut output_rx) = mpsc::unbounded_channel();
     let shutdown = CancellationToken::new();
@@ -486,8 +489,17 @@ async fn load_snapshots_become_stats_only_batches() {
     assert_eq!(stats.num_running_reqs, 2);
     assert_eq!(stats.num_waiting_reqs, 1);
     assert!((stats.kv_cache_usage - 0.25).abs() < 1e-9);
-
-    load_tx.send_replace(LoadSnapshot::default());
+    assert_eq!(stats.prefix_cache_stats.base.queries, 32);
+    assert_eq!(stats.prefix_cache_stats.base.hits, 16);
+
+    // The cumulative source jumps over multiple possible scheduler steps; the
+    // bridge must forward only the unobserved interval, without losing it to
+    // watch-channel coalescing or replaying the earlier totals.
+    load_tx.send_replace(LoadSnapshot {
+        prefix_cache_queries_total: 96,
+        prefix_cache_hits_total: 48,
+        ..LoadSnapshot::default()
+    });
     let batch = match output_rx.recv().await.expect("drained stats batch") {
         EngineCoreOutputs::RequestBatch(batch) => batch,
         other => panic!("expected a stats batch, got {other:?}"),
@@ -495,6 +507,8 @@ async fn load_snapshots_become_stats_only_batches() {
     let stats = batch.scheduler_stats.expect("scheduler stats");
     assert_eq!(stats.num_running_reqs, 0);
     assert_eq!(stats.kv_cache_usage.to_bits(), 0.0_f64.to_bits());
+    assert_eq!(stats.prefix_cache_stats.base.queries, 64);
+    assert_eq!(stats.prefix_cache_stats.base.hits, 32);
 
     shutdown.cancel();
     task.await