Fix v0.25 W2 starvation and structured-output MTP failures

[OmarB97]

Summary

Publishes the starvation-corrected vLLM v0.25.0 W2 overlay and contains the structured-output failure found during the real DeepSeek-V4-Flash W2 speculative-decoding canary. Normal requests retain MTP acceleration; structured/tool-call requests use the target-token path until vLLM can commit speculative grammar state sequentially.

Canonical production source: OmarB97/vllm branch moet-v0.25.0 at 6023898a814230ea839107ad82ca0141b71062b6

Distribution head: b52b5c737e9844b375f7d8c801eb7b681252874f

Official base: vLLM v0.25.0 at 702f4814fe54fabff350d43cb753ae3e47c0c276

Overlay SHA-256: ea1e8462008e8d3530e8938483a4f8974258196acc6a0bbcc4124bc4a719ed5d

Overlay scope: 61 files, +13,342/-134

The production fork branch and patch/SOURCE-v025.txt are the rollout authority. A contribution PR to another fork or official vLLM upstream is optional follow-up work and never gates this overlay.

Root cause

The earlier overlay omitted the W2 starvation correction because rebasing the distribution branch did not regenerate its patch payload.

The hardware canary then exposed a separate vLLM 0.25 defect under native MTP K=1. Consecutive forced tool calls could commit duplicate or otherwise grammar-invalid target/bonus blocks. xgrammar rejected the block and the OpenAI endpoint returned HTTP 500. A grammar-mask copy, a CUDA-stream fence, and a post-sampling validate/rollback defense did not solve it; the rollback defense also corrupted valid arguments.

The fail-closed boundary is the scheduler: discard speculative drafts for structured-output requests before num_new_tokens and the model-runner path are selected. Ordinary requests remain speculative.

Related upstream PR vllm-project/vllm#48116 fixes a reasoning-aware structural-tag path and reports 60/60 Qwen MTP tool calls. This containment is materially different: the DeepSeek launch had no reasoning parser configured and reproduced a target/bonus block failure directly.

What changed

• Retains the W2 starvation recovery correction, routing-window close, conditional replay pinning, DSpark live re-derivation, and method-agnostic spec guard.
• Clears structured-output drafts at the scheduler boundary and in both draft-update paths.
• Adds a scheduler regression for synchronous scheduling and asynchronous placeholder padding.
• Binds the v0.25 distribution artifact to the exact pushed production source commit and official release-tag SHA.
• Makes CI clone that production branch and fail closed on a missing source, unpublished commit, moved tag, or normalized source/patch mismatch.
• Adds six focused patch-guard regressions and restores tools/test_nvme_store.py, which the candidate branch had accidentally deleted relative to upstream main.
• Regenerates the lost-line manifest and Docker identity.
• Records the real taro throughput, context, acceptance, memory, tool-call, and production-restore receipts.

Hardware evidence

Real DeepSeek-V4-Flash W2 checkpoint on taro, RTX 5090, FP8 KV, 131,072-token model length, max sequences 1, 100 GiB cgroup with 96 GiB memory.high.

Mode	Warm median	Change vs no-spec
No speculation	23.56 tok/s	baseline
n-gram K=3	39.13 tok/s	+66.1%
n-gram K=4	41.05 tok/s	+74.3%
native MTP K=1	32.46 tok/s	+37.8%
contained MTP K=1	32.86 tok/s	+39.5%

• n-gram K=3 exact 120K retrieval: 250.3 s vs 260.8 s no-spec.
• Warm acceptance: 98.2% at n-gram K=3; 97.8% at K=4.
• No cgroup high, max, OOM, or OOM-kill events.
• Scheduler-delta disposable canary: 20/20 exact report_result(ok=true, label="SPEC_OK"), 20 HTTP 200s, zero grammar rejections, zero HTTP 500s.
• Named tool choice continues to report finish_reason="stop", matching existing vLLM serving behavior; the tool call and exact arguments are present.
• Disposable canary removed. Production llama-swap, ai-gate, and cloudflared were restored; ports 8080, 8081, and 9090 returned 200; the Qwen GPU process was restored.

The 20/20 tool-call result proves the scheduler containment delta on the real checkpoint. It is not a claim that the complete ea1e8462... overlay image has been served.

Source and distribution verification

• Production source branch is pushed and resolves to 6023898a814230ea839107ad82ca0141b71062b6.
• patch/SOURCE-v025.txt records that exact commit; it descends directly from the bound official v0.25.0 tag commit 702f4814....
• A fresh public clone of OmarB97/vllm, remote rename, official tag fetch, and strict source guard passed exactly as CI runs it.
• Normalized git diff v0.25.0 6023898a... is identical to the committed 61-file overlay; both hash to ea1e8462....
• Fresh exact-v0.25 apply and reverse-apply checks passed; the applied tree and distribution diff pass git diff --check.
• Six of six patch-guard regressions passed, including missing-source, unpublished-source, and missing-bound-tag fail-closed cases.
• Patch manifests pass: v0.24 has 66 files; v0.25 has 61 files.
• Bench lint: 0 errors, 0 warnings. README render check and workflow YAML parse pass.
• All 14 recipe launcher --print smokes pass.
• Exact full-overlay image exists on taro as vllm-moet-sm120:v025-w2candidate-ea1e8462, image ID sha256:3acf5a707966; its patch label matches this overlay.
• The focused vLLM source pytest could not collect in the existing source environment because that environment lacks tblib; the exact scheduler behavior is covered by the 20-request hardware canary, while the source-binding machinery is covered by the six green distribution regressions.

Risk and rollout

This is ready for exact-head review and does not promote v0.25 into production. The complete ea1e8462... overlay image is built side-by-side but has not been served. The scheduler delta was tested as a derived disposable image on the real checkpoint. Keep the current production image and packs as the immediate rollback boundary.

The next hardware gate must retain fail-closed cgroup memory and swap accounting, deliberately drain and restore the current model, use the already validated v0.25 eval store, and run on an isolated port. Source publication or green CI alone is not hardware acceptance.

Relationship to PR #10

This is the fresh replacement head. PR #10 remains untouched so its exact artifact and review history stay auditable.