Skip to content

Commit 983b4aa

Browse files
authored
feat(SAMPLE-REASONING): --reasoning-parser qwen3 (and its mimo alias) resolve (#605) (#630)
FOLLOWING_AGENTS_PROTOCOL qwen3 is the most common --reasoning-parser value in the official recipe corpus (18 of 76 uses) and is what the published Qwen3.5/3.6 recipes pass to models we already ship token-exact. Until now the engine served the model and rejected its own recipe's flag. Coverage 10 -> 12 names; 20 of the 76 uses. PORTED FRESH, NOT ALIASED -- the load-bearing finding. qwen3 and our existing think_auto are OPPOSITE on the most common case: qwen3_config(thinking=True) sets initial_state=REASONING (qwen3.py:100), so a marker-less stream is ALL REASONING, which upstream's own WITHOUT_THINK fixture pins (reasoning set, content None). think_auto.cpp:30 does the exact opposite by design, because the generic <think> template row covers hybrid-thinking models that may answer with no think block. Aliasing would have misclassified every marker-less Qwen3.5 response. Two further behaviours no <think>-splitting text parser can see: an unpaired <tool_call> ends reasoning with no </think> at all (qwen3.py:137), and a duplicate </think> in the content span is absorbed (qwen3.py:132). Landed as an engine face per the spec's W3 intent, so the base is reusable rather than qwen3-shaped: ParserEngineReasoningAdapter (adapters.py:35) plus the Qwen3Parser engine subclass (qwen3.py:201), which now also backs seed_oss exactly as upstream does (class SeedOssParser(Qwen3Parser)). No detect.cpp marker row: qwen3's only template literal is the generic <think>, and that row must keep resolving to think_auto -- a row before it would hijack every hybrid-thinking template, a row after it would be dead code. RED 5 cases / 0 passed, 6 assertions / 0 passed; GREEN 7/7, 157/157. A fresh review reproduced GREEN exactly and ran EIGHT mutations, verifying rather than crediting the implementer's self-declared blind spot: the three ported THINKING_DISABLED_CASES do NOT distinguish the thinking-off override -- upstream shares that gap -- and only the added engine-level case catches it. Verdict PASS. Its full serial gate read 404/404, better than the implementer's 403/404, which was the known test_engine_core_proc -j starvation. Two LOW findings are follow-up, not blockers: qwen3.cpp:48's reasoning-reopen guard is correct but unpinned by any test, and the seed_oss refactor's thinking-off arm changes behaviour toward upstream fidelity with no test pinning it. The production path is byte-identical -- MakeParserEngine passes thinking=true. Windows lanes red from pre-existing #584 only. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
1 parent eba6ab7 commit 983b4aa

19 files changed

Lines changed: 665 additions & 23 deletions

File tree

.agents/engine-matrix.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -137,7 +137,7 @@ lifecycle are unchanged.
137137
| `SAMPLE-LOGIT-FILTERS` | Logit bias, allowed-token IDs, bad words | T1 | `vllm/sampling_params.py:318,321,337,341,388-413,659-698`; `vllm/v1/sample/sampler.py:396`; `vllm/v1/worker/gpu_input_batch.py:446-471`; `vllm/entrypoints/openai/completion/protocol.py:369-371`; `tests/v1/sample/test_sampler.py:367,413`; `tests/v1/sample/test_sampling_params_e2e.py:106,147` | `include/vllm/sampling_params.h`+`src/vllm/sampling_params.cpp` (fields+validation); `src/vllm/entrypoints/openai/protocol.cpp` (`ParseLogitFilters`/`ApplyLogitFilters` clamp); `src/vllm/v1/worker/gpu/input_batch.cpp:255,344` (per-slot wiring+condense/swap); `src/vllm/v1/engine/input_processor.cpp` (bad_words tokenization); `src/vllm/v1/sample/sampler.cpp:239`; `src/vllm/v1/sample/logits_processor/builtin.cpp:41`; `src/vllm/v1/sample/ops/bad_words.cpp:13,55` | `tests/vllm/v1/sample/test_logits_processors.cpp:121,163,200`; `tests/vllm/test_sampling_params.cpp` (bad_words/allowed_token_ids validation); `tests/vllm/entrypoints/openai/test_protocol.cpp` (logit_bias clamp+parse); `tests/vllm/v1/worker/test_input_batch.cpp` (wiring, RED-first); `tests/vllm/v1/test_input_processor.cpp` (bad_words tokenization) | [sampling-controls-c7.md](specs/sampling-controls-c7.md) (`SAMPLE-LOGIT-FILTERS`) | `ANCHOR-BACKFILL` | `CLAIM-ROADMAP-C7` |
138138
| `SERVE-COMPLETION-LONGTAIL` | Best-of, echo, suffix, user request fields | T1 | `vllm/entrypoints/openai/completion/protocol.py:56,67,70`; `tests/entrypoints/openai/completion/test_token_in_token_out.py:56` | echo parse only `include/vllm/entrypoints/openai/protocol.h:196`; `src/vllm/entrypoints/openai/protocol.cpp:204,295` | acceptance-only `tests/vllm/entrypoints/openai/test_conformance.cpp:589` | `planned: specs/completions-longtail-fields.md` | `PARTIAL` | - |
139139
| `SAMPLE-BEAM` | Beam search: an OUTER loop over the engine (NOT a core-sampler param). Each step runs ONE decode per active beam (`logprobs=2*beam_width`, `max_tokens=1`, the beam temperature), expands each beam to those next tokens (`cum_logprob += logprob`), keeps the top-`beam_width` by the length-penalty score `get_beam_search_score = cum_logprob / seq_len**length_penalty` (`seq_len` INCLUDES the prompt, −1 when the last token is EOS), retires EOS-terminated beams into `completed`, and after `max_tokens` (or once all beams complete) returns the top-`beam_width` completed beams as multiple outputs (reuses the `SAMPLE-N` multi-output aggregation seam). The scoring + top-k-beam selection + EOS + length-penalty are DETERMINISTIC ⇒ token-EXACT vs vLLM, gated model-free on a hand-computed toy tree. `std::stable_sort` DESCENDING reproduces vLLM's `sorted(reverse=True)` tie behaviour. OpenAI-endpoint `use_beam_search` is WIRED on both `/v1/completions` and `/v1/chat/completions` over BOTH engine seams, REAL vLLM-0.26 surface: the SYNC `LLMEngine` (`BeamSearch`, offline.py) AND the PRODUCTION AsyncLLM HTTP server (`BeamSearchAsync`, online.py) — the server (`examples/server/main.cpp`) holds an AsyncLLM, so a beam request there now RUNS instead of raising "requires the synchronous engine". `BeamSearchAsync` drives the AsyncLLM per-beam single-token `generate` (pre-tokenized overload added to `AsyncLLM`) and calls the SAME model-free `BeamSearchStep`/`get_beam_search_score` — the algorithm is shared verbatim via a template driver body, only the engine object differs (mirrors online.py mirroring offline.py). GATE: `BeamSearchAsync` returns beams token-IDENTICAL to sync `BeamSearch` over the same synthetic CPU model (tokens/order/scores/text), for beam_width 1/2/3. CONCURRENCY FINDING: per-step beam decodes are issued SEQUENTIALLY (one isolated request each), byte-identical to the sync driver; online.py's `asyncio.gather` per-beam CONCURRENT stepping is a NAMED RESIDUAL (AsyncLLM supports concurrent requests — a future throughput optimization, correctness-first here). OTHER RESIDUALS: streaming beam (rejected like upstream), C-ABI beam params, grammar-constrained beam search (structured-output bitmask branch), encoder-decoder/LoRA beams | T1 | `vllm/entrypoints/generate/beam_search/utils.py:18,102,112,137,156`; `vllm/entrypoints/generate/beam_search/offline.py:58,118,160,193,291-327`; `vllm/entrypoints/generate/beam_search/online.py:28-220` (the OpenAI-serving beam generator); `vllm/entrypoints/openai/completion/protocol.py:260`/`chat_completion/protocol.py:589` (`to_beam_search_params`); `vllm/entrypoints/openai/completion/serving.py:173-205`/`chat_completion/serving.py:319-343` (`use_beam_search` routing); `vllm/sampling_params.py:1114` (`BeamSearchParams`) | `include/vllm/entrypoints/beam_search.h` + `src/vllm/entrypoints/beam_search.cpp` (model-free core + shared template `BeamSearchDrive` + `BeamSearch(LLMEngine&, …)` sync driver + `BeamSearchAsync(AsyncLLM&, …)` production driver); `include/vllm/v1/engine/async_llm.h`+`src/vllm/v1/engine/async_llm.cpp` (pre-tokenized `add_request`/`generate` overloads the async beam driver steps on); `include/vllm/entrypoints/openai/protocol.h`+`src/vllm/entrypoints/openai/protocol.cpp` (`use_beam_search`/`length_penalty` fields + `to_beam_search_params`, both requests); `src/vllm/entrypoints/openai/serving_completion.cpp` + `serving_chat.cpp` (`use_beam_search` routes to `BeamSearchAsync` when async-backed, else `BeamSearch` + `set_beam_search_tokenizer`); `examples/server/main.cpp` (wires `set_beam_search_tokenizer` on the production handlers so beam runs on the HTTP server); `CMakeLists.txt` — anchor `src/vllm/entrypoints/beam_search.cpp:59` | `tests/vllm/entrypoints/test_beam_search.cpp` (model-free token-EXACT tree) + `tests/vllm/v1/test_llm_engine.cpp` (e2e beam over the CPU engine; `BeamSearchAsync` == sync `BeamSearch` token-identical for bw 1/2/3) + `tests/vllm/entrypoints/openai/test_serving.cpp` (endpoint `use_beam_search` choices IDENTICAL to the direct driver, completion + chat, over BOTH the sync AND the production AsyncLLM engine; `to_beam_search_params` round-trip; streaming-beam + tokenizer-less async beam rejected) — anchor `tests/vllm/entrypoints/test_beam_search.cpp:82` | [sampling-controls-c7.md](specs/sampling-controls-c7.md) (`SAMPLE-BEAM`) | `ACTIVE` | `CLAIM-C7-BEAM-ASYNC` |
140-
| `SAMPLE-REASONING` | Reasoning parsers (`<think>` reasoning/content split, streamed as `reasoning` deltas + non-stream `reasoning_content`) and reasoning-gated grammar integration | T1 | `vllm/reasoning/abs_reasoning_parsers.py:26,213`; `vllm/reasoning/__init__.py:22` (registry, 28 names); `vllm/reasoning/basic_parsers.py:18` (BaseThinking); `deepseek_r1_reasoning_parser.py:10`; `deepseek_v3_reasoning_parser.py:20,83`; `identity_reasoning_parser.py:17` | **SEAM LANDED (record backfill 2026-07-28 — the seam shipped under `da933828`/`eb9d1291`/`5fffe7e6` but this row was never advanced):** base+registry `src/vllm/entrypoints/openai/reasoning_parsers/abstract.cpp:19` + `include/vllm/entrypoints/openai/reasoning_parsers/abstract.h:53`; `BaseThinkingReasoningParser` `src/vllm/entrypoints/openai/reasoning_parsers/basic.cpp:27`; parsers `deepseek_r1.cpp`, `mistral.cpp`, `minimax_m2.cpp`, `step3.cpp`, `olmo3.cpp`, `think_auto.cpp` (auto-detect default); template detection + `--reasoning-parser` resolve `src/vllm/entrypoints/openai/reasoning_parsers/detect.cpp:59`; C ABI v5 `src/capi/vllm_c.cpp` (`reasoning_parser`); serving `src/vllm/entrypoints/openai/serving_chat.cpp` (reasoning-before-tools routing, `reasoning` SSE delta). **W1 2026-07-28 (`CLAIM-SAMPLE-REASONING`):** + `src/vllm/entrypoints/openai/reasoning_parsers/identity.cpp:8` (passthrough delegate) + `src/vllm/entrypoints/openai/reasoning_parsers/deepseek_v3.cpp:9` (thinking-gated: `deepseek_v3`→Identity / `holo2`→R1) → 9 registered names | `tests/vllm/entrypoints/openai/reasoning_parsers/test_deepseek_v3.cpp:40` (ports `tests/reasoning/test_deepseekv3_reasoning_parser.py`: thinking-gated selection + identity passthrough + no-think edge, RED-first) + `tests/vllm/entrypoints/openai/reasoning_parsers/test_detect.cpp:102` (name-count 7→9) + existing `test_{base_thinking,deepseek_r1,mistral,minimax_m2,step3,olmo3,detect,think_auto}.cpp` + `reasoning_test_utils.h` (ports `tests/reasoning/utils.py`) | [specs/reasoning-parsers.md](specs/reasoning-parsers.md) | `ANCHOR-BACKFILL` | `CLAIM-SAMPLE-REASONING` |
140+
| `SAMPLE-REASONING` | Reasoning parsers (`<think>` reasoning/content split, streamed as `reasoning` deltas + non-stream `reasoning_content`) and reasoning-gated grammar integration | T1 | `vllm/reasoning/abs_reasoning_parsers.py:26,213`; `vllm/reasoning/__init__.py:22` (registry, 28 names); `vllm/reasoning/basic_parsers.py:18` (BaseThinking); `deepseek_r1_reasoning_parser.py:10`; `deepseek_v3_reasoning_parser.py:20,83`; `identity_reasoning_parser.py:17` | **SEAM LANDED (record backfill 2026-07-28 — the seam shipped under `da933828`/`eb9d1291`/`5fffe7e6` but this row was never advanced):** base+registry `src/vllm/entrypoints/openai/reasoning_parsers/abstract.cpp:19` + `include/vllm/entrypoints/openai/reasoning_parsers/abstract.h:53`; `BaseThinkingReasoningParser` `src/vllm/entrypoints/openai/reasoning_parsers/basic.cpp:27`; parsers `deepseek_r1.cpp`, `mistral.cpp`, `minimax_m2.cpp`, `step3.cpp`, `olmo3.cpp`, `think_auto.cpp` (auto-detect default); template detection + `--reasoning-parser` resolve `src/vllm/entrypoints/openai/reasoning_parsers/detect.cpp:59`; C ABI v5 `src/capi/vllm_c.cpp` (`reasoning_parser`); serving `src/vllm/entrypoints/openai/serving_chat.cpp` (reasoning-before-tools routing, `reasoning` SSE delta). **W1 2026-07-28 (`CLAIM-SAMPLE-REASONING`):** + `src/vllm/entrypoints/openai/reasoning_parsers/identity.cpp:8` (passthrough delegate) + `src/vllm/entrypoints/openai/reasoning_parsers/deepseek_v3.cpp:9` (thinking-gated: `deepseek_v3`→Identity / `holo2`→R1) → 9 registered names. **W3 first brick 2026-08-13 (#605, `CLAIM-SAMPLE-REASONING`):** the first ENGINE-BACKED reasoning adapter — `include/vllm/entrypoints/openai/reasoning_parsers/parser_engine_adapter.h:54` (`ParserEngineReasoningAdapter`, ports `vllm/parser/engine/adapters.py:35`) + `Qwen3ParserReasoningAdapter` (`registered_adapters.py:48`, registered under BOTH `qwen3` `__init__.py:115` and `mimo` `:87`) over the already-landed `src/vllm/parser/engine/` parser; engine-side overrides `src/vllm/parser/qwen3.cpp:30,40` (`Qwen3Parser`, ports `vllm/parser/qwen3.py:201,247,256` — thinking-off passthrough + unpaired-`<tool_call>` reasoning end; `seed_oss` now shares this class as upstream does) + `ParserEngine::extract_reasoning_streaming` / `is_reasoning_end(text)` `src/vllm/parser/engine/parser_engine.cpp:349,361` (`parser_engine.py:519,595`) → 12 registered names | `tests/vllm/entrypoints/openai/reasoning_parsers/test_qwen3.cpp:148` (ports `tests/reasoning/test_qwen3_reasoning_parser.py`: all 10 TEST_CASES fixtures × non-streaming AND streaming, the 5 MULTI_TOKEN_DELTA_CASES, THINKING_DISABLED_CASES, and is_reasoning_end incl. the unpaired-tool-call rule; RED-first) + `tests/vllm/entrypoints/openai/reasoning_parsers/test_deepseek_v3.cpp:40` (ports `tests/reasoning/test_deepseekv3_reasoning_parser.py`: thinking-gated selection + identity passthrough + no-think edge, RED-first) + `tests/vllm/entrypoints/openai/reasoning_parsers/test_detect.cpp:105` (pinned name-count, 10→12) + existing `test_{base_thinking,deepseek_r1,mistral,minimax_m2,step3,olmo3,detect,think_auto}.cpp` + `reasoning_test_utils.h` (ports `tests/reasoning/utils.py`) | [specs/reasoning-parsers.md](specs/reasoning-parsers.md) | `ANCHOR-BACKFILL` | `CLAIM-SAMPLE-REASONING` |
141141
| `SAMPLE-THINKING-BUDGET` | Thinking budget state and logit combination | T1 | `vllm/v1/sample/sampler.py:381-386` | - | - | `planned: specs/thinking-budget.md` | `INVENTORIED` | - |
142142
| `SAMPLE-REPETITION` | Repetition detection and penalty state | T1 | `vllm/v1/sample/sampler.py:437` | - | - | `planned: specs/repetition-detection.md` | `INVENTORIED` | - |
143143
| `SAMPLE-CUSTOM-PROCESSORS` | Custom logits-processor plugin point — a host-registered per-request callback the sampler invokes each decode step (generated token-ids + a mutable logits view) BEFORE sampling, at vLLM's non-argmax-invariant stage (after allowed_token_ids/bad_words/min_tokens/logit_bias, before penalties). Exposed through the C-ABI (`vllm_logits_processor`, ABI v8); default (no processor) byte-identical. Mirrors vLLM's `SamplingParams.logits_processors` structure/ordering; also satisfies SGLang's `custom_logit_processor`. Residual: single per-request C callback (not a batched plugin graph); no Python-side registration; on the async scheduler the generated-token view is fed by the scheduler (may lag) — the strict token-ids contract is gated at the sampler level | T2 | `vllm/v1/sample/logits_processor/__init__.py:49-97`; `vllm/v1/sample/logits_processor/interface.py:60`; `vllm/v1/sample/sampler.py:399`; sglang `python/sglang/srt/sampling/custom_logit_processor.py:24` | ABI `include/vllm.h:186` (typedef) + `:239` (field, v8); `include/vllm/logits_processor_callback.h:40`; `src/capi/vllm_c.cpp:207`; `include/vllm/sampling_params.h:214`; `include/vllm/v1/sample/metadata.h:101`; `src/vllm/v1/sample/logits_processor/builtin.cpp:75` (`apply_logits_processors`) + `include/vllm/v1/sample/logits_processor/builtin.h:59`; wired `src/vllm/v1/sample/sampler.cpp:308`; per-slot `src/vllm/v1/worker/gpu/input_batch.cpp:284` + emit `:468` | `tests/vllm/v1/sample/test_sampler.cpp:310,353,378` (forces-token EXACT + per-request + inert; RED-first); `tests/vllm/v1/sample/test_logits_processors.cpp:236,253,264` (mutate/no-op/null-skip); `tests/capi/test_capi.cpp:409` (ABI v8 e2e forces token, fires per step) | [sampling-controls-c7.md](specs/sampling-controls-c7.md) (`SAMPLE-CUSTOM-PROCESSORS`) | `ANCHOR-BACKFILL` | `CLAIM-C7-CUSTOM-LOGITS` |

0 commit comments

Comments
 (0)