You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(SAMPLE-REASONING): --reasoning-parser qwen3 (and its mimo alias) resolve (#605) (#630)
FOLLOWING_AGENTS_PROTOCOL
qwen3 is the most common --reasoning-parser value in the official recipe corpus
(18 of 76 uses) and is what the published Qwen3.5/3.6 recipes pass to models we
already ship token-exact. Until now the engine served the model and rejected its
own recipe's flag. Coverage 10 -> 12 names; 20 of the 76 uses.
PORTED FRESH, NOT ALIASED -- the load-bearing finding. qwen3 and our existing
think_auto are OPPOSITE on the most common case: qwen3_config(thinking=True) sets
initial_state=REASONING (qwen3.py:100), so a marker-less stream is ALL REASONING,
which upstream's own WITHOUT_THINK fixture pins (reasoning set, content None).
think_auto.cpp:30 does the exact opposite by design, because the generic <think>
template row covers hybrid-thinking models that may answer with no think block.
Aliasing would have misclassified every marker-less Qwen3.5 response. Two further
behaviours no <think>-splitting text parser can see: an unpaired <tool_call> ends
reasoning with no </think> at all (qwen3.py:137), and a duplicate </think> in the
content span is absorbed (qwen3.py:132).
Landed as an engine face per the spec's W3 intent, so the base is reusable rather
than qwen3-shaped: ParserEngineReasoningAdapter (adapters.py:35) plus the
Qwen3Parser engine subclass (qwen3.py:201), which now also backs seed_oss exactly
as upstream does (class SeedOssParser(Qwen3Parser)). No detect.cpp marker row:
qwen3's only template literal is the generic <think>, and that row must keep
resolving to think_auto -- a row before it would hijack every hybrid-thinking
template, a row after it would be dead code.
RED 5 cases / 0 passed, 6 assertions / 0 passed; GREEN 7/7, 157/157. A fresh
review reproduced GREEN exactly and ran EIGHT mutations, verifying rather than
crediting the implementer's self-declared blind spot: the three ported
THINKING_DISABLED_CASES do NOT distinguish the thinking-off override -- upstream
shares that gap -- and only the added engine-level case catches it. Verdict PASS.
Its full serial gate read 404/404, better than the implementer's 403/404, which
was the known test_engine_core_proc -j starvation.
Two LOW findings are follow-up, not blockers: qwen3.cpp:48's reasoning-reopen
guard is correct but unpinned by any test, and the seed_oss refactor's
thinking-off arm changes behaviour toward upstream fidelity with no test pinning
it. The production path is byte-identical -- MakeParserEngine passes thinking=true.
Windows lanes red from pre-existing #584 only.
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
| `SAMPLE-BEAM` | Beam search: an OUTER loop over the engine (NOT a core-sampler param). Each step runs ONE decode per active beam (`logprobs=2*beam_width`, `max_tokens=1`, the beam temperature), expands each beam to those next tokens (`cum_logprob += logprob`), keeps the top-`beam_width` by the length-penalty score `get_beam_search_score = cum_logprob / seq_len**length_penalty` (`seq_len` INCLUDES the prompt, −1 when the last token is EOS), retires EOS-terminated beams into `completed`, and after `max_tokens` (or once all beams complete) returns the top-`beam_width` completed beams as multiple outputs (reuses the `SAMPLE-N` multi-output aggregation seam). The scoring + top-k-beam selection + EOS + length-penalty are DETERMINISTIC ⇒ token-EXACT vs vLLM, gated model-free on a hand-computed toy tree. `std::stable_sort` DESCENDING reproduces vLLM's `sorted(reverse=True)` tie behaviour. OpenAI-endpoint `use_beam_search` is WIRED on both `/v1/completions` and `/v1/chat/completions` over BOTH engine seams, REAL vLLM-0.26 surface: the SYNC `LLMEngine` (`BeamSearch`, offline.py) AND the PRODUCTION AsyncLLM HTTP server (`BeamSearchAsync`, online.py) — the server (`examples/server/main.cpp`) holds an AsyncLLM, so a beam request there now RUNS instead of raising "requires the synchronous engine". `BeamSearchAsync` drives the AsyncLLM per-beam single-token `generate` (pre-tokenized overload added to `AsyncLLM`) and calls the SAME model-free `BeamSearchStep`/`get_beam_search_score` — the algorithm is shared verbatim via a template driver body, only the engine object differs (mirrors online.py mirroring offline.py). GATE: `BeamSearchAsync` returns beams token-IDENTICAL to sync `BeamSearch` over the same synthetic CPU model (tokens/order/scores/text), for beam_width 1/2/3. CONCURRENCY FINDING: per-step beam decodes are issued SEQUENTIALLY (one isolated request each), byte-identical to the sync driver; online.py's `asyncio.gather` per-beam CONCURRENT stepping is a NAMED RESIDUAL (AsyncLLM supports concurrent requests — a future throughput optimization, correctness-first here). OTHER RESIDUALS: streaming beam (rejected like upstream), C-ABI beam params, grammar-constrained beam search (structured-output bitmask branch), encoder-decoder/LoRA beams | T1 | `vllm/entrypoints/generate/beam_search/utils.py:18,102,112,137,156`; `vllm/entrypoints/generate/beam_search/offline.py:58,118,160,193,291-327`; `vllm/entrypoints/generate/beam_search/online.py:28-220` (the OpenAI-serving beam generator); `vllm/entrypoints/openai/completion/protocol.py:260`/`chat_completion/protocol.py:589` (`to_beam_search_params`); `vllm/entrypoints/openai/completion/serving.py:173-205`/`chat_completion/serving.py:319-343` (`use_beam_search` routing); `vllm/sampling_params.py:1114` (`BeamSearchParams`) | `include/vllm/entrypoints/beam_search.h` + `src/vllm/entrypoints/beam_search.cpp` (model-free core + shared template `BeamSearchDrive` + `BeamSearch(LLMEngine&, …)` sync driver + `BeamSearchAsync(AsyncLLM&, …)` production driver); `include/vllm/v1/engine/async_llm.h`+`src/vllm/v1/engine/async_llm.cpp` (pre-tokenized `add_request`/`generate` overloads the async beam driver steps on); `include/vllm/entrypoints/openai/protocol.h`+`src/vllm/entrypoints/openai/protocol.cpp` (`use_beam_search`/`length_penalty` fields + `to_beam_search_params`, both requests); `src/vllm/entrypoints/openai/serving_completion.cpp` + `serving_chat.cpp` (`use_beam_search` routes to `BeamSearchAsync` when async-backed, else `BeamSearch` + `set_beam_search_tokenizer`); `examples/server/main.cpp` (wires `set_beam_search_tokenizer` on the production handlers so beam runs on the HTTP server); `CMakeLists.txt` — anchor `src/vllm/entrypoints/beam_search.cpp:59` | `tests/vllm/entrypoints/test_beam_search.cpp` (model-free token-EXACT tree) + `tests/vllm/v1/test_llm_engine.cpp` (e2e beam over the CPU engine; `BeamSearchAsync` == sync `BeamSearch` token-identical for bw 1/2/3) + `tests/vllm/entrypoints/openai/test_serving.cpp` (endpoint `use_beam_search` choices IDENTICAL to the direct driver, completion + chat, over BOTH the sync AND the production AsyncLLM engine; `to_beam_search_params` round-trip; streaming-beam + tokenizer-less async beam rejected) — anchor `tests/vllm/entrypoints/test_beam_search.cpp:82` | [sampling-controls-c7.md](specs/sampling-controls-c7.md) (`SAMPLE-BEAM`) | `ACTIVE` | `CLAIM-C7-BEAM-ASYNC` |
0 commit comments