Add LagunaXS2Renderer for poolside/Laguna-XS.2#21
Merged
hallerite merged 16 commits intoMay 13, 2026
Conversation
Hard-coded renderer mirroring the laguna_glm_thinking_v5_1 chat template. The format uses block-style role markers (<system>/</system>, <user>/</user>, <assistant>/</assistant>, <tool_response>/</tool_response>) — of these only <assistant>/</assistant> are single (added) tokens. Tool calls wrap with single-token <tool_call>/</tool_call>, but inner <arg_key>/<arg_value> tags are plain text and parsed via regex on the decoded block. Other notable properties: - Prefix is <|EOS|> (BOS=EOS in this tokenizer) emitted unconditionally. - Default system prompt baked into the template; consumed from messages[0] if present, attributed to msg_idx=0 so build_training_sample sees it. - Reasoning is rendered for every assistant message (no last-user-index gating), so the renderer is listed in NO_OP_MODELS for the preserve_* thinking tests. - _visible_text accepts list-form content with TextPart entries; the new _thinking_text helper routes ThinkingPart entries to reasoning_content so a parse → reserialize → re-render round-trip preserves reasoning. Wires the renderer through __init__, MODEL_RENDERER_MAP, _populate_registry, and adds the model to the standard test conftest + roundtrip matrices. Adds tests/test_laguna_xs2.py with five focused regressions covering the ThinkingPart round-trip path and degenerate content shapes.
The list-form content behaviors these tests exercised (TextPart extraction, ThinkingPart routing, reasoning_content precedence, degenerate-shape robustness) are generic Renderer-protocol invariants rather than Laguna-specific quirks. Better suited to the shared conftest matrix with opt-in subsets, see PR description for upstreaming suggestions.
Hard-coded renderer mirroring the laguna_glm_thinking_v5_1 chat template. The format uses block-style role markers (<system>/</system>, <user>/</user>, <assistant>/</assistant>, <tool_response>/</tool_response>) — of these only <assistant>/</assistant> are single (added) tokens. Tool calls wrap with single-token <tool_call>/</tool_call>, but inner <arg_key>/<arg_value> tags are plain text and parsed via regex on the decoded block. Other notable properties: - Prefix is <|EOS|> (BOS=EOS in this tokenizer) emitted unconditionally. - Default system prompt baked into the template; consumed from messages[0] if present, attributed to msg_idx=0 so build_training_sample sees it. - Reasoning is rendered for every assistant message (no last-user-index gating), so the renderer is listed in NO_OP_MODELS for the preserve_* thinking tests. - _visible_text accepts list-form content with TextPart entries; the new _thinking_text helper routes ThinkingPart entries to reasoning_content so a parse → reserialize → re-render round-trip preserves reasoning. Wires the renderer through __init__, MODEL_RENDERER_MAP, _populate_registry, and adds the model to the standard test conftest + roundtrip matrices. Adds tests/test_laguna_xs2.py with five focused regressions covering the ThinkingPart round-trip path and degenerate content shapes.
The list-form content behaviors these tests exercised (TextPart extraction, ThinkingPart routing, reasoning_content precedence, degenerate-shape robustness) are generic Renderer-protocol invariants rather than Laguna-specific quirks. Better suited to the shared conftest matrix with opt-in subsets, see PR description for upstreaming suggestions.
…t_never After rebasing onto current main, the Laguna parser broke against ParsedResponse.tool_calls' new list[ParsedToolCall] shape (introduced in PrimeIntellect-ai#22). Mirror parse_glm's structure: emit ParsedToolCall with status (UNCLOSED_BLOCK / MISSING_NAME / INVALID_JSON / OK) and token_span relative to the stop-stripped stream. Also drop ``assert_never(unexpected_role)``: msg["role"] is plain ``str`` (TypedDict), so ty flags a type-assertion-failure and any unknown role would crash at runtime — every other renderer silently skips unknown roles. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brings in ParsedToolCall API (PrimeIntellect-ai#22), hatch-vcs versioning (PrimeIntellect-ai#20), Apache 2.0 license (PrimeIntellect-ai#27), and other commits landed after the PR branched. Resolves conflicts in renderers/__init__.py, renderers/base.py, renderers/parsing.py, and tests/test_roundtrip.py by taking main's ParsedToolCall shape and re-applying the Laguna additions on top.
…t_never After merging main, the Laguna parser was still on the old ``list[dict]`` shape and 4 tests failed against ``ParsedResponse.tool_calls``'s new ``list[ParsedToolCall]`` type (introduced in PrimeIntellect-ai#22). Mirror ``parse_glm``'s structure: emit ``ParsedToolCall`` with a ``status`` enum (UNCLOSED_BLOCK / MISSING_NAME / INVALID_JSON / OK) and a ``token_span`` relative to the stop-stripped stream. Also drop ``assert_never(unexpected_role)``: ``msg["role"]`` is plain ``str`` (TypedDict), so ``ty`` flags a type-assertion-failure and any unknown role would crash at runtime — every other renderer silently skips unknown roles. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
Merge main + migrate parser to ParsedToolCall API
…aguna-xs2-renderer
hallerite
approved these changes
May 13, 2026
hallerite
added a commit
that referenced
this pull request
May 13, 2026
hallerite
added a commit
that referenced
this pull request
May 13, 2026
LagunaXS2Renderer landed in #21; its parser has the same string-type corruption as the other XML parsers (5/5 cases fail). Count is now 20 failed, 10 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
hallerite
added a commit
that referenced
this pull request
May 13, 2026
XML-style chat templates (Qwen3.5, GLM-5/4.5, MiniMax-M2, Laguna) render
tool-call argument values verbatim inside ``<arg_value>X</arg_value>``
(or ``<parameter>X</parameter>``) tags with no quoting. A value of
``true`` and the string ``"true"`` produce identical wire bytes; without
the tool schema, the parser has no signal to choose between them and
defaults to ``json.loads`` — silently corrupting string args that look
like JSON.
This adds ``tools: list[ToolSpec] | None = None`` to ``parse_response``
on the ``Renderer`` Protocol and every concrete renderer. When supplied,
the four XML-style parsers (``parse_qwen35``, ``parse_glm``,
``parse_minimax``, ``parse_laguna_xs2``) consult each parameter's
declared JSON-schema ``type`` and preserve declared-string params
verbatim. Without ``tools``, behavior is unchanged.
Two new helpers in ``parsing.py``:
- ``_build_param_type_index`` — accepts either the flat ``ToolSpec``
shape or the OpenAI ``{"type":"function","function":{...}}`` envelope
and returns ``{tool_name: {param_name: schema_fragment}}``.
- ``_coerce_arg_value`` — returns ``(value, used_json_fallback)``; the
bool is True only when ``json.loads`` was tried and raised, so the
``INVALID_JSON`` status fires only for genuine parse failures, not
for schema-driven string preservation.
The JSON-shaped parsers (Qwen3 hermes, Qwen3-VL, DeepSeek-V3, Kimi K2,
Kimi K2.5, Nemotron3, gpt-oss harmony, Default) sidestep this bug
because their wire format quotes strings; they accept the ``tools``
kwarg for API uniformity but ignore it.
Matches the reference behavior of vLLM / SGLang's
``glm45_tool_parser.py`` and ``hermes_tool_parser.py``.
Raised by Robin (Poolside) on PR #21.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a renderer for
poolside/Laguna-XS.2. This renderer broadly follows the shape of the GLM renderers as both model families use largely identical reasoning and tool call syntax. The main delta is that the Laguna XS.2 model lacks special tokens for markers inside the<tool_call>tags, e.g.<arg_key>so we can't re-use the GLM tool parsing as-is.