docs: M1.5 InputBatch + BlockTable (MRV2) plan

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: mudler

docs/superpowers/plans/2026-07-03-m1.5-input-batch.md

@@ -0,0 +1,36 @@
+# M1.5 InputBatch + BlockTable (MRV2)
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development. Steps use checkbox (`- [ ]`) syntax.
+
+**Goal:** The persistent batch + step-input build — `BlockTable` (per-request block-id tensors + slot mapping), `InputBatch` (the long-lived per-slot state updated incrementally from `SchedulerOutput`: add new / apply cached diffs / swap-remove finished), and the step-input construction (`query_start_loc`, `seq_lens`, `positions`, `slot_mapping`, logits indices) — ported 1:1 from vLLM V1's **Model Runner V2** (`vllm/v1/worker/gpu/{input_batch,block_table}.py`) so the model runner (M0.9 forward, later the batched runner) can consume a scheduled step. Behavioral (no CUDA/model in the tests — the tensors are plain host arrays here; device placement is the runner's concern), CI-testable.
+
+**Architecture:** Mirrored `src/vllm/v1/worker/gpu/{input_batch,block_table}.{h,cpp}` (← `vllm/v1/worker/gpu/`). This is MRV2 (the project's committed direction; the scheduler already emits the MRV2 shape with `prefill_token_ids` + resumed-as-new). The InputBatch is a **persistent** structure: it holds per-slot arrays (token ids, positions, block tables, sampling params, num_computed_tokens) mutated in place across steps, NOT rebuilt each step — this is the vLLM performance design. Scope T0: the single-sequence + small-batch path the gate models need; use plain host `std::vector`/arrays for the "tensors" (the vt-device version is a later perf concern). Defer LoRA/spec-decode/multimodal/structured-output slot state behind marked stubs.
+
+## Global Constraints
+
+- 1:1 with pinned e24d1b24 MRV2 — VERIFY the pinned `vllm/v1/worker/gpu/input_batch.py` + `block_table.py` CURRENT API before porting (proven-necessary habit). Same class/method/field names. Ported files carry `// Ported from: vllm/v1/worker/gpu/<file> @ e24d1b24`.
+- The persistent-batch update contract: `add_request` (new + resumed, from `NewRequestData` incl. `prefill_token_ids`), `remove_request`/`condense` (swap-remove finished, keep the batch dense), the cached-diff apply (`num_computed_tokens` + `new_block_ids` append/replace per the M1.4 `CachedRequestData` semantics). The req_id↔slot-index mapping must match upstream (the runner indexes num_reqs-major arrays by it).
+- Step-input build: `query_start_loc` (batch+1 cumulative offsets), `seq_lens`, `positions`, `slot_mapping` (block_id*block_size + offset per token), logits indices (last token per sequence). Match upstream's exact construction — the attention op (M1.6) + sampler (M1.7) consume these.
+- **M1.4 watch-item — `new_block_ids_to_zero` / GDN state zeroing:** when a fresh block is allocated for a GDN/mamba-state group, upstream zeros it to prevent stale NaN in SSM computation. Confirm how MRV2's InputBatch/block-table handles this for the hybrid gate models; if the T0 path reuses freed blocks without zeroing, record it as a correctness dependency for the GDN forward.
+- Port upstream's OWN tests from `/home/mudler/_git/vllm/tests/v1/worker/` (test_input_batch / test_gpu_input_batch if present). Cite. Behavioral oracle.
+- Warnings-as-errors; TDD; push authorized (CPU, all CI).
+
+---
+
+### Task 1: BlockTable
+Read `/home/mudler/_git/vllm/vllm/v1/worker/gpu/block_table.py` (+ `MultiGroupBlockTable` if the hybrid path uses one). Port `BlockTable` (per-request block-id storage as a 2D host array [max_reqs, max_blocks_per_req] + num_blocks_per_row, `add_row`/`append_row`/`move_row`/`swap_row`/`clear`, `compute_slot_mapping`, `commit_block_table`), and `MultiGroupBlockTable` (one BlockTable per KV cache group — the gate models have 2 groups). Match the exact array bookkeeping. Unit tests ported (add/append/move/swap rows, slot mapping = block_id*block_size + offset, multi-group).
+
+### Task 2: InputBatch (persistent) — add / remove / condense
+Read `/home/mudler/_git/vllm/vllm/v1/worker/gpu/input_batch.py` — the `InputBatch`/`CachedRequestState` (or the MRV2 states in `gpu/states.py`). Port the persistent structure: per-slot arrays (req_ids indexed by slot, token_ids_cpu, num_computed_tokens_cpu, num_prompt_tokens, block_table via MultiGroupBlockTable, sampling metadata), `add_request(new_req_data, slot)`, `remove_request(req_id) → slot`, `condense()` (swap-remove to keep dense), `num_reqs`, the req_id↔index maps. Match upstream's incremental-mutation contract. Defer LoRA/spec/mm/structured slot state (marked). Unit tests ported (add fills a slot, remove + condense keeps dense + fixes indices, num_reqs).
+
+### Task 3: Step-input build (from SchedulerOutput)
+Port the step-input construction the MRV2 model_runner does before the forward: given a `SchedulerOutput` (new reqs + cached diffs) and the persistent InputBatch, apply the diffs (new_computed_tokens, new_block_ids), then build the flattened step inputs: `input_token_ids` (the scheduled tokens per request, flattened), `positions`, `query_start_loc` (cumulative per-request token offsets), `seq_lens` (computed tokens per req), `slot_mapping` (per token: block_id*block_size + within-block offset), and `logits_indices` (index of each sequence's last scheduled token). Read the pinned MRV2 `model_runner.py` `_prepare_inputs`/`prepare_inputs` (the part that builds these from the persistent batch) — port the T0 subset. Match the exact offsets/ordering the attention op (M1.6) needs. Unit tests: a fake SchedulerOutput (2 reqs, one prefill chunk + one decode) produces the right query_start_loc/seq_lens/slot_mapping/positions/logits_indices; the GDN-state-group slot handling.
+
+### Task 4: Records
+roadmap M1.5 ✅; ledger row; state (NEXT M1.6 paged attention backend — the KV-cache-aware attention that replaces M0.9's dense attention for the batched/cached path, + CommonAttentionMetadata builder consuming these step inputs); inventory §3 mark InputBatch/BlockTable rows ✅ (note deferred LoRA/spec/mm); CI green. Record whether GDN-state zeroing is handled or a carried dependency.
+
+## Self-review notes
+- The persistent-batch incremental update + the step-input offsets are the correctness core the attention/sampler consume — port exactly, validate with upstream's worker tests.
+- VERIFY current pinned MRV2 API per file (proven habit; the scheduler already emits the MRV2 shape this consumes).
+- Host arrays for "tensors" at T0 (device placement is the runner's job); defer LoRA/spec/mm/structured slot state behind 1:1 stubs.
+- Resolve/record the GDN-state-zeroing question (M1.4 watch-item) — real correctness dependency for the hybrid gate models.