record(TOOLS-PARSER-BREADTH): backfill the 41 parsers already shipped, and correct three counts (#608)

THIS IS A RECORD REPAIR, NOT AN ACHIEVEMENT. No parser was ported, no `src/`
or `tests/` file changed, and no behaviour moved. Every parser named below was
already on `main` before this commit; it landed incrementally under other rows
and this row simply never recorded it. Read the diff as "the row caught up with
the tree", never as "this change delivered forty-one parsers".

The row read:

    Qwen-Coder XML, Mistral, pythonic, and remaining parsers
    Our code: -  ·  Our tests/evidence: -  ·  INVENTORIED

All three families it names by title are implemented, alongside 34 more. This
is the same defect `SAMPLE-REASONING` carried before its own W0 ("the seam
shipped under da93382 but this row was never advanced").

W0 of .agents/specs/tool-parser-breadth.md. W1 (openai, inkling, minimax_m3),
W2 (cohere_command3/4) and W3 (port upstream's ToolParserTestConfig harness)
are separate changes and remain owed.

What is now recorded on the row: 41 accepted parser names over 37 families;
the factory and `tool_parser_names()` anchors in `tool_parsers/abstract.cpp`;
the 27-row ordered autodetect table and its `ORDER MATTERS` collision analysis
in `tool_parsers/detect.cpp`; and the 38 test files, including the registry
test that PINS the count so a factory branch added without listing its name
fails the suite instead of shipping an unreachable dialect.

State `INVENTORIED` -> `PARTIAL`, and not further. The matrix header defines
`ANCHOR-BACKFILL` as "code and tests but still lacks its required leaf spike";
that spec is committed, so that half is no longer true. What remains true is
the other half — "the implementation is also known to omit upstream behavior"
— which is exactly `PARTIAL`. Summary counts moved with it (Structured output
and tools PARTIAL 3->4 / INVENTORIED 2->1; Total PARTIAL 16->17 /
INVENTORIED 39->38), both re-derived from the rows rather than incremented.

Three counts in the spec were re-derived from the two registries at the pin
5559679229bc961848b121ccdeaa8fa5d79bec98 and corrected:

  * We register 41 names, not 40. `muse_glimmer` took it 40 -> 41 on
    2026-08-10 under MODEL-MUSE-GLIMMER-W7, and `test_detect.cpp:221` already
    pinned 41 — so the spec was stale against a committed test.
  * Upstream registers 44, not 43.
  * The five upstream-only names are confirmed exactly as the spec listed
    them. But two of OUR names are absent from upstream's registry at the pin:
    `qwen3` (our local alias for the Hermes-JSON Qwen dialect) and
    `muse_glimmer` (decorator-registered at the older e24d1b24 port anchor,
    gone from vllm/tool_parsers/ at 555967922). 44 - 5 = 39 shared, 39 + 2 =
    41. The old "40 = 43 - 3" arithmetic only worked because two errors
    cancelled.

`docs/STATUS.md` claimed "36 parser families / 40 accepted names" and "every
vLLM tool parser at the pin except the three Rust/Harmony-backed ones". Both
halves were wrong: five are missing, not three, and of those five only
`minimax_m3` is Rust-backed (`inkling` is a ParserEngine adapter; `openai`,
`cohere_command3` and `cohere_command4` are plain Python). Repaired, and the
unported `ToolParserTestConfig` harness is now stated rather than implied by
"all held to the upstream test suites".

The `roadmap_v1.md` issue row still carried the earlier usage-derived framing
of #608 — "six names missing", which listed `nemotron_json` as portable (it is
registered nowhere at the pin) and missed both Cohere entries. AGENTS.md
requires the issue table, the spec and the PR body to agree, so it is restated
from the registries. `nemotron_json`, `kimi_k3` and `ling3` are in NEITHER
registry and arrive with the pin advance, not here.

Every anchor recorded here was re-derived and read at this head, upstream ones
against the pinned checkout. One was wrong on the way in and fixed before
commit: `abstract_tool_parser.py:235` is a bare `@classmethod` decorator, so
the manager anchor is now `:223,236,318`.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Author: mudler

.agents/engine-matrix.md

@@ -42,13 +42,13 @@ forensics: roadmap_v1.md and the parity ledger.
 | KV cache and memory | 22 | 7 | 3 | 2 | 2 | 1 | 2 | 0 | 5 |
 | Parallelism | 6 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 5 |
 | Sampling and generation | 15 | 4 | 2 | 0 | 0 | 4 | 0 | 1 | 4 |
-| Structured output and tools | 7 | 0 | 3 | 0 | 0 | 2 | 0 | 0 | 2 |
+| Structured output and tools | 7 | 0 | 4 | 0 | 0 | 2 | 0 | 0 | 1 |
 | Speculative decoding | 21 | 0 | 0 | 1 | 0 | 5 | 0 | 4 | 10 |
 | Serving, API, CLI, library | 33 | 10 | 2 | 0 | 2 | 10 | 2 | 3 | 4 |
 | LoRA and adapters | 2 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 |
 | Long context and attention | 10 | 5 | 0 | 0 | 1 | 0 | 1 | 0 | 3 |
 | Loading, tokenizer, config | 10 | 3 | 3 | 0 | 0 | 1 | 1 | 1 | 1 |
-| **Total** | **153** | **35** | **16** | **4** | **9** | **30** | **8** | **11** | **39** |
+| **Total** | **153** | **35** | **17** | **4** | **9** | **30** | **8** | **11** | **38** |
 
 ## Engine core and scheduling
 
@@ -155,7 +155,7 @@ lifecycle are unchanged.
 | `TOOLS-GUIDANCE-OUTLINES` | Guidance, outlines, and LM-format-enforcer backends | T2 | `vllm/v1/structured_output/__init__.py:140-159` | - | - | `planned: specs/guidance-outlines-backends.md` | `INVENTORIED` | - |
 | `TOOLS-CALLING-CORE` | Auto, required, named tool choice; streaming deltas; Hermes and Qwen3 parsers | T0 | `vllm/entrypoints/openai/chat_completion/serving.py:428,688,872`; `vllm/tool_parsers/hermes_tool_parser.py:34`; `vllm/tool_parsers/qwen3_engine_tool_parser.py:7` | `src/vllm/entrypoints/openai/serving_chat.cpp:98,224,239`; `src/vllm/entrypoints/openai/tool_parsers/hermes.cpp:124,180`; `src/vllm/entrypoints/openai/tool_parsers/qwen3.cpp:13` | `tests/vllm/entrypoints/openai/tool_parsers/test_tool_parsers.cpp:27,121,159`; `tests/vllm/entrypoints/openai/tool_parsers/test_tool_choice_grammar.cpp:248,260,280`; `tests/vllm/entrypoints/openai/test_serving.cpp:625,741,903`; **SELECTION SURFACE 2026-07-24 (`CLAIM-DOCS-T2-FIXES`)** — the bundled OpenAI server no longer hardcodes `"hermes"`/`""`: `--tool-call-parser` and `--reasoning-parser` (vLLM's own flag names) select any registered dialect, `auto` runs the same chat-template detection the C ABI uses, `none` disables, and an unknown name aborts startup listing the registry. The name lists are ENUMERATED from the factories (`tool_parser_names()` 40 names / `reasoning_parser_names()` 7), not hand-written at the flag; the whole flag behaviour lives in `ResolveToolParserName` / `ResolveReasoningParserName` so it is unit-tested without a server. DEFAULTS REPRODUCE THE OLD HARDCODE EXACTLY (`hermes` / disabled), so an invocation naming neither flag is unchanged. Anchors: `examples/server/main.cpp` (flags), `src/vllm/entrypoints/openai/tool_parsers/{abstract,detect}.cpp`, `src/vllm/entrypoints/openai/reasoning_parsers/{abstract,detect}.cpp`; tests `tests/vllm/entrypoints/openai/tool_parsers/test_detect.cpp`, `tests/vllm/entrypoints/openai/reasoning_parsers/test_detect.cpp` | `planned: specs/tool-calling.md` | `PARTIAL` | - |
 | `TOOLS-STREAMING-PARSER` | Unified streaming parser engine for reasoning and tool calls, including token-ID scanning, coalesced deltas, replay adapters and parser-specific configurations | T1 | `vllm/parser/engine/streaming_parser_engine.py:89`; `vllm/parser/engine/token_id_scanner.py:29`; `vllm/parser/engine/incremental_lexer.py:80`; `vllm/parser/engine/events.py:11,22`; `vllm/parser/qwen3.py:88`; `vllm/parser/kimi_k2.py:52`; `vllm/parser/engine/parser_engine.py:79` (assembly, residual); `tests/parser/engine/test_engine.py`, `test_token_id_scanner.py`, `test_qwen3.py` @ `555967922` | CORE engine landed 2026-07-27 (`CLAIM-ROADMAP-C8-PARSER`): `include/vllm/parser/engine/{events,parser_engine_config,incremental_lexer,token_id_scanner,streaming_parser_engine,configs,registry}.h` + `src/vllm/parser/engine/{incremental_lexer,token_id_scanner,streaming_parser_engine,configs,registry}.cpp` (scanner + prefix-buffering lexer + transition state machine + JSON-arg brace hold-back + drop-info + qwen3/seed_oss/kimi_k2 configs + unified name->config registry). ASSEMBLY landed 2026-07-27 (`CLAIM-ROADMAP-C8-ASSEMBLY`): `include/vllm/parser/engine/{parser_engine,py_json}.h` + `src/vllm/parser/engine/parser_engine.cpp` (`ParserEngine`: SemanticEvent -> streaming `DeltaMessage` + one-shot `ExtractedToolCallInformation`, held-back streaming-arg prefix, tool_index++, finish() flush, qwen3 `<parameter=>` arg-converter) + `include/vllm/parser/{kimi_k2,parser_manager}.h` + `src/vllm/parser/{kimi_k2,parser_manager}.cpp` (kimi native-header id/name overrides + name->parser dispatch) + assembly fields on `parser_engine_config.h`/`configs.cpp`. SERVING-SSE dispatch swap landed 2026-07-27 (`CLAIM-ROADMAP-C8-SERVING`): `src/vllm/entrypoints/openai/serving_chat.cpp` (`ShapeChatDeltaEngine`/`ShapeChatMessageEngine`/`MakeParserEngine` + 3 name-selected drive-site branches + `ChatSseStream` engine member) + `include/vllm/entrypoints/openai/serving_chat.h`; faithful `include_reasoning` request field on `protocol.{h,cpp}`. CONFIG FAMILIES landed 2026-07-27 (`CLAIM-ROADMAP-C8-CONFIGS`): 5 more engine-backed families ported as additive `ParserEngineConfig` builders in `src/vllm/parser/engine/configs.cpp` (`minimax_m2_config`, `glm47_moe_config`, `deepseek_v4_config`, `deepseek_v32_config`, `nemotron_v3_config` + `_minimax_m2`/`_glm47`/`_dsml` std::regex arg-converters) + `registry.cpp`/`parser_manager.cpp` dispatch + `include/vllm/parser/glm47_moe.{h,cpp}` (name-`.strip()` over the existing hooks). CONFIG FAMILIES C8-2 landed 2026-07-27 (`CLAIM-ROADMAP-C8-CONFIGS-2`): the last 2 deferred families gemma4 + inkling PORTED — added 4 additive assembly-core virtual seams (default-inert for the other 8 families) `preprocess_feed` (parser_engine.py:210), virtual `events_to_delta` (:706), virtual `single_pass_parse` (:645), `args_wrapper_keys` from `_extract_args_value` (:1064) + virtual `reset`/`extract_reasoning`; `src/vllm/parser/engine/configs.cpp` (`gemma4_config` + `_gemma4_arg_converter` key:value scanner; `inkling_config` + `_inkling_arg_converter` JSON-span carver) + `include/vllm/parser/{gemma4,inkling}.{h,cpp}` (gemma4 `_preprocess_feed` channel-injection + `_events_to_delta` `thought\n`-strip + `extract_reasoning`; inkling `args_wrapper_keys` unwrap + `_single_pass_parse` trailing flush) + `registry.cpp`/`parser_manager.cpp` dispatch. JSON-SCHEMA ARG-TYPE COERCION landed 2026-07-28 (`CLAIM-C8-ARG-COERCION`): `_fix_arg_types` / `_streamable_string_keys` / `find_tool_properties` (parser_engine.py:227,269,365,348) ported over the ALREADY-ported `extract_types_from_schema` / `coerce_to_schema_type` helpers (`tool_parsers/utils.cpp`) — `ParserTool` now carries the function `parameters` JSON-schema (threaded from `serving_chat.cpp` `ToParserRequest`), so a request whose tools declare typed params (int/number/bool/string/array/null) has its assembled `tool_calls[].function.arguments` coerced to the declared types in BOTH streaming (`parse_delta`) and one-shot (`extract_tool_calls`/`parse`); no-schema/absent-tools = identity (byte-identical). Additive to `parser_engine.{h,cpp}` (recursive `_coerce_dict`/`_coerce_value` + `find_tool_properties`) + `serving_chat.cpp` schema threading; no other TU changed — anchor `src/vllm/parser/engine/streaming_parser_engine.cpp:143` | `tests/vllm/parser/engine/test_streaming_parser_engine.cpp` (586/586, 8 scenarios) + `test_parser_engine_assembly.cpp` (`..._goldens.inc`): 5038/5038 field-for-field over 30 scenarios (streaming DeltaMessage + one-shot extract_tool_calls + non-streaming parse()) vs vLLM 0.26 assembly — scenarios 27-30 (`CLAIM-C8-ARG-COERCION`) add qwen3 typed-schema coercion (whole+char, days`5`/active`true`/temp`3.14`/tags`[1,2,3]` coerced, unit stays string), qwen3 schema-mismatch (uncoercible `"abc"` left as-is + nullable `"null"`->null), kimi_k2 JSON-native `"5"`->int in extract (converter-less: streaming stays raw, extract coerces — divergence gated); RED-first proven (38 asserts, first boundary `qwen3_typed_schema_wholedelta extract tc[0] arguments`: identity `{"days": "5", …}` vs coerced `{"days": 5, "unit": "celsius", "active": true, "temp": 3.14, "tags": [1, 2, 3]}`); scenarios 10-19 (`CLAIM-ROADMAP-C8-CONFIGS`) add minimax_m2 / glm47_moe / deepseek_v4 / deepseek_v32 / nemotron_v3, scenarios 20-26 (`CLAIM-ROADMAP-C8-CONFIGS-2` 2026-07-27) add gemma4 (explicit + elided channel, whole+char) + inkling (think/tool/trailing-text + non-object-args fallback, whole+char), each whole-delta AND char-by-char; RED-first proven for all seams (32 asserts `_safe_arg_prefix`; 2 asserts glm47 name-`.strip()`; 13 asserts gemma4 `_events_to_delta` at `gemma4_channel_tool_wholedelta delta[0] reasoning`; 5 asserts gemma4 `_preprocess_feed` at `gemma4_elided_channel_wholedelta delta[0] content`; 4 asserts inkling `args_wrapper_keys` at `inkling_nonobject_args_wholedelta extract tc[0] arguments`; 2 asserts inkling `_single_pass_parse` at `inkling_think_tool_text_wholedelta parse content`); serving `tests/vllm/entrypoints/openai/test_serving_chat_stream.cpp` (`..._goldens.inc`): 210/210 chunk-for-chunk SSE parity over the 9 scenarios vs vLLM 0.26 `chat_completion_stream_generator` (role frame + per-delta reasoning/content/tool-call deltas + terminal `tool_calls` flip + name-selected dispatch), RED-first proven (6 CHECKs, first boundary `chunk[1]` reasoning-vs-raw-content); goldens byte-reproduced by `tools/parity/dump_{streaming_parser_engine,parser_engine_assembly,serving_chat_stream}.py` — anchor `tests/vllm/parser/engine/test_streaming_parser_engine.cpp:99` | [specs/streaming-parser-engine.md](specs/streaming-parser-engine.md), [specs/parser-assembly-c8.md](specs/parser-assembly-c8.md) | `ACTIVE` | `CLAIM-ROADMAP-C8-PARSER`, `CLAIM-ROADMAP-C8-ASSEMBLY`, `CLAIM-ROADMAP-C8-SERVING`, `CLAIM-ROADMAP-C8-CONFIGS`, `CLAIM-ROADMAP-C8-CONFIGS-2`, `CLAIM-C8-ARG-COERCION` |
-| `TOOLS-PARSER-BREADTH` | Qwen-Coder XML, Mistral, pythonic, and remaining parsers | T1 | `vllm/tool_parsers/__init__.py:16-206` | - | - | [tool-parser-breadth.md](specs/tool-parser-breadth.md) | `INVENTORIED` | - |
+| `TOOLS-PARSER-BREADTH` | Qwen-Coder XML, Mistral, pythonic, and the remaining `--tool-call-parser` dialects. **RECORD BACKFILL 2026-08-13 (#608 W0) — NO code shipped in this move; the surface below landed incrementally under other rows and was never recorded here, the same defect `SAMPLE-REASONING` carried before its own W0.** Landed: **41 accepted parser names over 37 parser families**, the count PINNED by the registry test rather than asserted in prose. Aliases account for the difference: `llama3_json`/`llama4_json` → one class, `qwen3_coder`/`qwen3_xml`/`mimo` → one, `glm45`/`glm47` → one. All three families this row names by title are among them (`qwen3_coder`, `mistral`, `pythonic`). Selection is three-way — explicit `--tool-call-parser NAME`, `auto` template sniffing over a **27-row ordered marker table**, or `none`; an unknown name throws enumerating the whole registry. **Known omission vs the pin, which is why this is `PARTIAL` and not further:** upstream's lazy registry carries **44** names to our 41, and **exactly 5 are upstream-only** — `openai` (`GptOssToolParser`), `inkling` (`InklingEngineToolParser`, ParserEngine-backed), `minimax_m3` (`MinimaxM3ToolParser`, the ONLY Rust-backed one of the five), `cohere_command3` + `cohere_command4` (plain Python, referenced by ZERO recipes, so a usage-driven audit cannot see them). Conversely **2 of our names are absent from upstream's registry at the pin**: `qwen3` (our local alias for the Hermes-JSON Qwen dialect) and `muse_glimmer` (decorator-registered at the older `e24d1b24` port anchor, gone from `vllm/tool_parsers/` at `555967922`) — so 44 − 5 = 39 shared, 39 + 2 = 41. `nemotron_json`, `kimi_k3` and `ling3` are in NEITHER registry (post-pin; they arrive with the pin advance). W1 (`openai`, `inkling`, `minimax_m3`), W2 (both Cohere) and W3 (port `ToolParserTestConfig` as a shared harness) remain owed | T1 | registry `vllm/tool_parsers/__init__.py:24-201` (the 44-name `_TOOL_PARSERS_TO_REGISTER` dict) + `:204,210` (`register_lazy_tool_parsers`); manager `vllm/tool_parsers/abstract_tool_parser.py:223,236,318` (`ToolParserManager`, `get_tool_parser`, `register_module`); the 5 upstream-only classes `gptoss_tool_parser.py:17`, `inkling_tool_parser.py:7`, `minimax_m3_tool_parser.py:7`, `cohere_command_tool_parser.py:125,138` | factory `src/vllm/entrypoints/openai/tool_parsers/abstract.cpp:72` (`get_tool_parser`, 41 name branches → 37 classes) + `:269` (`tool_parser_names`, the enumeration the flag's error message is built from); autodetect `src/vllm/entrypoints/openai/tool_parsers/detect.cpp:65` (`kToolParserMarkers`, 27 ordered rows) + `:17` (the `ORDER MATTERS` collision analysis recording which families are deliberately EXPLICIT-ONLY) + `:100` (`DetectToolParser`, first-match with hermes fallback) + `:124` (`ResolveToolParserName`, `auto`/`none`/explicit plus the enumerating throw); per-family parsers under `src/vllm/entrypoints/openai/tool_parsers/` | registry pin `tests/vllm/entrypoints/openai/tool_parsers/test_detect.cpp:206` — every enumerated name resolves and every marker-table name is registered — with `:221` pinning `names.size() == 41`, so a factory branch added without listing its name fails the suite instead of silently shipping an unreachable dialect; **38 test files** under `tests/vllm/entrypoints/openai/tool_parsers/` = 34 per-family + 4 cross-cutting (`test_detect.cpp`, `test_tool_parsers.cpp:27` covering hermes+qwen3, `test_tool_choice_grammar.cpp`, `test_structural_tags.cpp`), and `deepseek_v31` shares `test_deepseek.cpp`, so all 37 families are covered. NOT a ported common suite: every file is hand-written per parser, so the floor differs per parser — upstream's `ToolParserTestConfig` is unported (W3) | [tool-parser-breadth.md](specs/tool-parser-breadth.md) | `PARTIAL` | - |
 
 ## Speculative decoding
 

.agents/roadmap_v1.md

@@ -181,7 +181,7 @@ issue is not yet placed. Keyed record: update in place, never append.
 | [#558](https://github.com/mudler/vllm.cpp/issues/558) | — | `tests/parity/hf_snapshot.h` has no guard against declaration-order breaks: the C++ build catches them, but the records-only lane that broke it never builds C++, and all 14 TUs that include the header are checkpoint-gated so `ctest` reports the break as `***Not Run`. `fafa16f0f` (#546, #551) fixed the ordering and carried no guard | bug |
 | [#606](https://github.com/mudler/vllm.cpp/issues/606) | — | `vllm-serve` aborts on `--enable-auto-tool-choice` (89/157 recipes) and `--trust-remote-code` (82/157), both of which are no-ops for us, so a copy-pasted official recipe command never reaches model load. Needs an accepted-and-inert seam with a per-flag reason; no row owns serve CLI recipe compatibility | feature |
 | [#607](https://github.com/mudler/vllm.cpp/issues/607) | — | No `--language-model-only`: 43 recipes skip the vision encoder to hand its VRAM to the KV cache and we load the tower unconditionally. The flag appears in this repo only in `tools/bench/run_serve_low.py`, which passes it to the ORACLE — a grep reads as coverage and is not | feature |
-| [#608](https://github.com/mudler/vllm.cpp/issues/608) | `TOOLS-PARSER-BREADTH` | Six `--tool-call-parser` names missing (`openai`, `inkling`, `minimax_m3`, `nemotron_json`, `kimi_k3`, `ling3`), closing the last 8 of 90 official-recipe uses; four are portable at the pin, two are post-pin | feature |
+| [#608](https://github.com/mudler/vllm.cpp/issues/608) | `TOOLS-PARSER-BREADTH` | **W0 (record backfill) landed 2026-08-13; row `INVENTORIED` → `PARTIAL`, 41 shipped names / 37 families recorded.** Re-derived from the two REGISTRIES rather than from recipe usage: **five** `--tool-call-parser` names are portable at the pin — `openai`, `inkling`, `minimax_m3` (W1, recipe demand) and `cohere_command3`, `cohere_command4` (W2, ZERO recipe uses, so usage-driven audits miss them). `nemotron_json`, `kimi_k3` and `ling3` are in NEITHER registry and arrive with the pin advance, not here. W3 ports upstream's shared `ToolParserTestConfig` harness. The earlier "six missing" framing was usage-derived: it listed `nemotron_json` as portable (it is not registered at the pin) and missed both Cohere entries | feature |
 
 ## Top-level portfolio