feat(kv-events): publish the batch envelope, and honour kv_cache_report_mode (#352)

[localai-bot]

Row KV-EVENTS, W3. Issue #352. Spec .agents/specs/kv-events.md, committed before the implementation (3e379400). Claim CLAIM-KV-EVENTS-W3.

What landed

W1 built the whole producer side of the KV-cache event stream and nothing that could consume it. The Scheduler hard-coded /*enable_kv_cache_events=*/false into its KVCacheManager, owned no publisher and never called take_events(), so no KVEventBatch was ever constructed outside a unit test holding a BlockPool. And BlockPool::emit_cached_block_events — ported, tested, correct — had no caller at all, because upstream fires it only under request.kv_cache_report_mode == "full" and our Request had no such field.

This wires the CPU-reachable half, through the existing CollectingEventPublisher seam.

The envelope (scheduler.py:86,116-119,155-158, :1901-1915, :2456-2459). The Scheduler ctor takes an optional KVEventsConfig and a data-parallel rank, derives enable_kv_cache_events from it, and builds its publisher through EventPublisherFactory. update_from_output drains kv_cache_manager->take_events() and, only if the step raised any, publishes one KVEventBatch stamped with a wall-clock ts (time.time(), not a steady clock — a consumer correlates it against its own wall clock). The drain sits at the tail of update_from_output, where upstream puts it, because eviction events raised by allocate_slots during schedule() and store events raised by cache_blocks at the end of the step have to land in the SAME batch. shutdown() reaches the publisher.

The report mode (request.py:116-127, kv_cache_manager.py:262-280). SamplingParams gains extra_args; Request derives kv_cache_report_mode from it with upstream's exact .get(key, "incremental") / None -> "incremental" fallback; KVCacheManager retains enable_kv_cache_events and, under report_mode == "full", emits the reused prefix blocks per group after find_longest_cache_hit. A prefix-cache HIT is now reportable, which is the case a prefix-cache-aware router actually consumes.

The default path is untouched, and that is gated rather than asserted: a null KVEventsConfig — what every existing call site passes — resolves the flag to the same false that was hard-coded, the factory returns a NullEventPublisher whose publish is an empty body, and report mode is "incremental" on every request that does not ask otherwise. The inertness case is green in BOTH red stages and after.

Deliberately excluded

The live ZMQ transport is NOT ported, and that exclusion still holds on this head. EventPublisherFactory::create throws loudly on publisher == "zmq" rather than silently downgrading to a null publisher:

EventPublisherFactory: the 'zmq' publisher (live socket transport) is not yet
ported; event generation + payload are gated, the ZMQ transport is deferred
(KV-EVENTS spec).

tests/vllm/v1/test_kv_events.cpp:234-238 pins that throw. Why it is excluded rather than implemented: the PUB/ROUTER sockets, replay buffer, publisher thread and DP port offset need a third-party socket library that does not exist under third_party/. Vendoring one is a dependency decision, not an implementation one, so it is not a call this row gets to make silently. It is the row's named residual, which is why KV-EVENTS stays ACTIVE and does not move to DONE.

Also deferred, each recorded at its call site: the connector leg of the drain (scheduler.py:1903-1910 — our OffloadingEvent is a different type owned by KV-OFFLOAD), vllm_xargs (so report_mode is reachable from the C++ engine API but not over HTTP), and DP aggregation.

Found and filed rather than silently fixed: #353 — KVEventsConfig has no PostInit, so an enabled config with an unset publisher throws unknown event publisher '' where upstream (kv_events.py:50-52) resolves it to zmq. W3's tests set publisher = "null" explicitly.

RED-first

RED came in two stages, because a C++ port of a missing API cannot fail an assertion before it fails to compile, and only the second stage shows the tests discriminate.

• Stage 1, API absent: 14 compile errors naming extra_args, kv_cache_report_mode, set_kv_event_publisher and shutdown.
• Stage 2, API present, the four behaviours absent: 5 of 12 cases, 5 assertions, each pointing at one missing behaviour.
• GREEN: 12/12, 105 assertions.

Upstream has no test for either half — a grep over the pin returns three source sites and no test, and the event coverage in tests/v1/core/test_prefix_caching.py drives the pool directly rather than stepping a scheduler — so the new cases are written from the source anchors and recorded as written-from-scratch.

One of the author's own expectations was wrong on the first green run, and the correction is the most useful thing the gate produced: a "full" hit publishes two BlockStored in one batch — the reuse report, then the ordinary store of the single block the request still had to compute, parented on the last reused block. The test now asserts that whole shape, including the parent linkage that lets a consumer chain the prefix. The "incremental" control publishes only the second.

Gate numbers

Re-run on the merged head cbabbf88, CPU-only Release build-cpu (-DVLLM_CPP_CUDA=OFF -DVLLM_CPP_VULKAN=OFF -DVLLM_CPP_METAL=OFF), foreground, unbounded:

Gate	Result
build	0 warnings (-Werror)
./build-cpu/tests/test_kv_events	12/12 cases · 105 assertions
./build-cpu/tests/test_scheduler	36/36 cases · 423 assertions
./build-cpu/tests/test_llm_engine	24/24 cases · 493 assertions
ctest --test-dir build-cpu -j 6	385/385, 0 failed (35.8 s)
scripts/agent-preflight.sh	all gates green

test_openai_conformance was starvation, not a defect. It was the single failure in the pre-merge sweep. It passes in-sweep at 0.35-0.96 s on this head, and 0.59 s standalone on a quiet box at load 2.12, against failures recorded at load 121+. It is the documented parallel-ctest load-starver, and this sweep settles it — the qualifier is removed from the row's evidence rather than carried forward.

No GPU, SACRED or CUDA gate is implicated: no kernel, no vt op, no ABI, no CMake, no model file is touched.

Merging main, and how the keyed records were resolved

The branch was interrupted before it was ever pushed, so it absorbed 160 commits of main across four forward merges (157080c8, 211ff73a, 902b0e39, 50374199). Merged, never rebased — main is never force-pushed, so the branch absorbs main rather than rewriting over it.

No keyed record was left at git's three-way result, even where it merged cleanly (AGENTS.md §Records). Every one was reset to origin/main wholesale and this branch's single scoped edit reapplied against an anchor asserted in a script to match exactly once:

• .agents/roadmap_v1.md (conflicted, then auto-merged twice) — main re-sorted the intake table by row ID, so the KV-EVENTS: the batch envelope is never published and kv_cache_report_mode does not exist — events can only be observed by reaching into the BlockPool #352/KVEventsConfig has no __post_init__: an enabled config with an unset publisher reports "unknown event publisher ''" instead of resolving to zmq #353 rows are reinserted in the sorted position ahead of KV-EXTERNAL-CACHE, not where the old unsorted table had them.
• .agents/engine-matrix.md (auto-merged, redone) — main's KV-EVENTS row was byte-identical to the merge base's, so the row replacement reapplies cleanly. Lifecycle is ACTIVE on both sides, so no matrix count moves and none of the branch's totals are carried.
• .agents/coordination.md (auto-merged, redone) — reverted to origin/main byte-for-byte. CLAIM-KV-EVENTS-W3 moved to .agents/claims/CLAIM-KV-EVENTS-W3.md, the per-claim surface Shared record surfaces are a lock: 16/29 open PRs conflict, 13 of them in bookkeeping only #364 added while this branch was parked; its README says new claims go there, and the insert-at-one-anchor claims table is exactly the conflict source that made this row collide. check-agent-record.py reads both.
• scripts/check-gate-commands.py (conflicted, then auto-merged twice) — both sides grew RUNNABLE_BASELINE at the same anchor. Took main's set entire, each time, and reinserted only the KV-EVENTS entry with its re-pin rationale.
• docs/USAGE.md (conflicted) — both sides appended after the logprobs prose. Kept BOTH: main's logprob_token_ids paragraph, then this branch's "KV-cache events, and kv_cache_report_mode" section.
• .agents/benchmark-record.md needed no union: this branch never wrote it.

scripts/check-public-doc-tables.py is taken from main unchanged — the STATUS_RATCHET["chars"] pin retired by #364/#368 is not reintroduced. docs/BENCHMARKS.md is untouched: this row carries no measurement, so nothing is charged against its 45,000-char cap.

Found on main while merging

#420, filed not fixed. 211ff73a (#416) moves ENG-TRAILER-MERGE-ARTIFACTS ACTIVE -> DONE in the engine matrix and touches no public document; python3 scripts/check-doc-checkpoint.py --commit 211ff73a errors. CI misses it because ci.yml:240 runs that checker over a --base/--head range rather than per commit. It is another row's DONE record and docs/BENCHMARKS.md is cap-constrained, so repairing it inside a KV-EVENTS PR would surprise a reviewer. This PR's own range is clean: check-doc-checkpoint.py --base origin/main --head HEAD reports OK.

Records

Issue #352 appears in the three places AGENTS.md requires them to agree: the intake table in .agents/roadmap_v1.md, the spec's header and ## Evidence, and this body.

---

Landing note: a GitHub squash merge duplicates the trailers and adds a Co-authored-by (#406). A local git merge --no-ff preserves them.