feat(ENG-MM-INPUT-PIPELINE): the multimodal limits reach a live request, so the flags stop being decoration (#607, #686)

.agents/roadmap_v1.md

@@ -192,7 +192,8 @@ issue is not yet placed. Keyed record: update in place, never append.
 | [#558](https://github.com/mudler/vllm.cpp/issues/558) | — | `tests/parity/hf_snapshot.h` has no guard against declaration-order breaks: the C++ build catches them, but the records-only lane that broke it never builds C++, and all 14 TUs that include the header are checkpoint-gated so `ctest` reports the break as `***Not Run`. `fafa16f0f` (#546, #551) fixed the ordering and carried no guard | bug |
 | [#603](https://github.com/mudler/vllm.cpp/issues/603) | — | `windows-msvc-cpu` / `windows-msvc-vulkan` are RED on `main`: `test_backend_cross_device.cpp` calls POSIX `setenv`/`unsetenv`, which MSVC does not provide | bug |
 | [#606](https://github.com/mudler/vllm.cpp/issues/606) | — | `vllm-serve` aborts on `--enable-auto-tool-choice` (89/157 recipes) and `--trust-remote-code` (82/157), both of which are no-ops for us, so a copy-pasted official recipe command never reaches model load. Needs an accepted-and-inert seam with a per-flag reason; no row owns serve CLI recipe compatibility | feature |
-| [#607](https://github.com/mudler/vllm.cpp/issues/607) | `ENG-MM-INPUT-PIPELINE` | **Premise corrected 2026-08-13, see `specs/multimodal-track.md` §1.5.** Not "skip the vision encoder": `--language-model-only` sets every modality limit to **0** (`multimodal.py:78,321-327`) and is sugar over `--limit-mm-per-prompt`. Two consequences follow, and we have NEITHER — upstream then **refuses every multimodal request** (`processing/context.py:409-428` raises "At most 0 image(s) may be provided in one prompt", from `:461` and `chat_utils.py:662`), and builds the tower uninitialised (`interfaces.py:293`). This is a PORT of the limits mechanism (L1-L4), not the exposure of a boolean. **L1 LANDED 2026-08-13:** `vllm::MultiModalConfig` + `GetLimitPerPrompt` (`include/vllm/config/multimodal.h`) and the refusal it carries (`include/vllm/multimodal/processing/context.h`) are in, unit-gated, with NO serve surface — nothing constructs the config on a live request yet, so the two call sites still validate nothing. L2 (the flags + C-ABI field + wiring those call sites), L3 (tower skip) and L4 (kernel gate) are still owed. 43 of 157 recipes pass the flag and we abort on it. The flag appears in this repo only in `tools/bench/run_serve_low.py`, which passes it to the ORACLE — a grep reads as coverage and is not | feature |
+| [#607](https://github.com/mudler/vllm.cpp/issues/607) | `ENG-MM-INPUT-PIPELINE` | **Premise corrected 2026-08-13, see `specs/multimodal-track.md` §1.5.** Not "skip the vision encoder": `--language-model-only` sets every modality limit to **0** (`multimodal.py:78,321-327`) and is sugar over `--limit-mm-per-prompt`. Two consequences follow, and we have NEITHER — upstream then **refuses every multimodal request** (`processing/context.py:409-428` raises "At most 0 image(s) may be provided in one prompt", from `:461` and `chat_utils.py:662`), and builds the tower uninitialised (`interfaces.py:293`). This is a PORT of the limits mechanism (L1-L4), not the exposure of a boolean. **L1 LANDED 2026-08-13:** `vllm::MultiModalConfig` + `GetLimitPerPrompt` (`include/vllm/config/multimodal.h`) and the refusal it carries (`include/vllm/multimodal/processing/context.h`) are in, unit-gated, with NO serve surface. **L2 LANDED 2026-08-14 (also closing [#686](https://github.com/mudler/vllm.cpp/issues/686)):** both serve flags (`--[no-]language-model-only`, `--limit-mm-per-prompt '<json>'` with upstream's legacy/configurable/mixed spellings and its refusals, `src/vllm/config/multimodal.cpp`), the C-ABI fields (`vllm_model_params.language_model_only` / `.limit_mm_per_prompt`, **ABI v18 → v19**), and the CALL SITE — `MakeQwen3VLImageChatFn` validates through `ValidateChatMmLimits` (port of `chat_utils.py:648-662`) over the seam's declared ceiling `{"image": 1}`, so a three-image request is `400 "At most 1 image(s) may be provided in one prompt."` instead of a truncated answer. The 43 recipes now reach model load. Gated on aarch64: `test_serve_mm_limits` 11/11, `test_chat_mm` 11/11, `test_capi` 58/58, `test_openai_api_server` 56/56 (the HTTP 400 arm proven e2e against both 500 and a truncated 200). Repaired in review (#758 filed): the v19 ABI header claimed an enforcement no C-ABI caller can reach — `set_multimodal_chat_fn` has one caller, `server_main.cpp` — and `ParseLimitMmPerPromptJson` refused unknown option keys for EVERY modality, where upstream's `BaseDummyOptions` fallback has no `extra="forbid"` and drops them. Still owed: L3 (tower skip — **no memory claim is made by L2**), L4 (kernel gate), and the second call site `process_inputs_mm`, which is blocked on the per-model `get_supported_mm_limits()` hook the M2 towers own. The flag appears in this repo only in `tools/bench/run_serve_low.py`, which passes it to the ORACLE — a grep reads as coverage and is not | feature |
+| [#758](https://github.com/mudler/vllm.cpp/issues/758) | `ENG-MM-INPUT-PIPELINE` | A multimodal refusal cannot distinguish a configured limit from an UNIMPLEMENTED arm. `Qwen3VLChatSupportedMmLimits()` declares the seam's honest ceiling `{"image": 1}` with video/audio absent, but the message a client gets is upstream's generic `At most 0 video(s) may be provided in one prompt.` — identical to what `--limit-mm-per-prompt '{"video": 0}'` produces. AGENTS.md requires an unimplemented arm be refused "with a message naming the missing piece"; #749 claimed the ceiling satisfies that and its review found it does not. The only present signal is by OMISSION (`ValidateNumItems` withholds the `--limit-mm-per-prompt` hint when raising the limit would not help). Not fixed in flow: naming the arm diverges from a verbatim-ported message three suites assert byte-for-byte, so it needs its own spec and fresh review. Found in the #749 review round (#607 wave L2, #686) | bug |
 | [#651](https://github.com/mudler/vllm.cpp/issues/651) | — | `test_agent_record`'s MODEL-ratchet docstring is two contradictory paragraphs spliced together, and the surviving half records a pin transition that never happened | bug |
 | [#652](https://github.com/mudler/vllm.cpp/issues/652) | — | `model-matrix.md` prose counters drifted: LTX-2.5 reached the rows and the CI-enforced rollup but none of the five sentences that count them | bug |
 | [#659](https://github.com/mudler/vllm.cpp/issues/659) | — | LTX-2.5 device select adopts M3a's platform seam but not its companion capability guard: `ltx2_video.cpp` asks `CurrentPlatform().device_type()` and `TryGetBackend(...)` but never `supports_model_architecture`, so a PARTIAL backend (Metal 15/75 ops, Tenstorrent) is handed a queue and dies in a kernel bind where it used to be refused BY NAME (found while reviewing #553 for landing) | bug |
@@ -619,7 +620,7 @@ degraded run — it is no run at all.
 | `--tool-call-parser` | 42 names, **84/90 recipe uses (93%)** — the healthy axis; `inkling` (2 uses) landed 2026-08-13 under #608 W1 | [#608](https://github.com/mudler/vllm.cpp/issues/608) for the last 6 |
 | `--reasoning-parser` | 10 of 28 names, **15/76 uses (20%)**; `qwen3` (18) rejected on our own gate models | [#605](https://github.com/mudler/vllm.cpp/issues/605) |
 | `--enable-auto-tool-choice`, `--trust-remote-code` | no-ops for us, yet **abort startup** on 89 and 82 recipes | [#606](https://github.com/mudler/vllm.cpp/issues/606) |
-| `--language-model-only` | absent; 43 recipes use it. It zeroes every modality limit — the freed encoder VRAM **and** the refusal of every mm request both follow from that | [#607](https://github.com/mudler/vllm.cpp/issues/607) |
+| `--language-model-only`, `--limit-mm-per-prompt` | **ACCEPTED + ENFORCED 2026-08-14** (#607 L2): the 43 recipes that pass the flag reach model load, and it zeroes every modality limit so mm requests are REFUSED with upstream's message and HTTP 400. The freed encoder VRAM is the half still owed (wave L3), and the flag must not be described as freeing memory until that lands and is measured | [#607](https://github.com/mudler/vllm.cpp/issues/607) |
 | `--kv-cache-dtype` | not a serve flag; residual on the `KV-FP8` row | — |
 | `--speculative-config` | MTP + DFlash land; `eagle`/`eagle3` (7 uses) do not | — |
 | TP / EP / multi-node (`--tensor-parallel-size`, `--enable-expert-parallel`, `--mm-encoder-tp-mode`) | absent by scope, not by defect — single-box engine | see the TP W-plan above |

.agents/specs/mm-serving.md

@@ -140,6 +140,24 @@ chat request content-part array          [MM-SERVE-PARSE — CPU, THIS BRICK]
   #607's L1 has now ported that refusal (`BaseProcessingInfo::ValidateNumItems`);
   wiring this call site to it is L2, and this is the reason L2 exists.
 
+  **CLOSED 2026-08-14 by #607 wave L2** (`row/mm-limits-l2`), with one correction
+  to the description above that the RED run produced and that is recorded rather
+  than quietly dropped. Through the PRODUCTION seam the pre-L2 behaviour was not
+  a truncated 200: `MakeQwen3VLImageChatFn` injects one placeholder marker per
+  image part while routing only the first image, so `ExpandImagePlaceholders`
+  raised *"more image placeholders than grids"* and the client got an **HTTP 500**
+  carrying an internal message. The silent truncation is real for the
+  validate-then-build seam shape (pinned as its own leg in
+  `test_api_server.cpp`), so the issue's diagnosis — first image, `break`, no
+  refusal — stands; only its stated consequence was understated. Both cases are
+  now the same answer: `ValidateChatMmLimits` (`chat_mm.cpp:311`, ahead of the
+  unchanged first-image loop at `:313-326`) runs first, and a request over the
+  seam's declared ceiling (`Qwen3VLChatSupportedMmLimits() == {"image": 1}`,
+  the `min()` fold operand the issue asked for) is refused with
+  **HTTP 400 "At most 1 image(s) may be provided in one prompt."** Video and
+  audio parts are refused the same way rather than dropped, because a modality
+  absent from the seam's supported limits reads as limit 0 (`context.py:414-415`).
+
 ## Brick 2 (`MM-SERVE-ENGINE`) — landed
 
 - `include/vllm/v1/engine/input_processor.{h,cpp}` — `process_inputs_mm`: the