Add MiniCPM5 XML tool-call parser#1317
Open
scrappylabsai wants to merge 1 commit into
Open
Conversation
openbmb/MiniCPM5-1B emits tool calls in XML, not JSON:
<function name="get_weather">
<param name="city">Tokyo</param>
<param name="date">2024-06-27</param>
</function>
The mlx_lm.server warning "Received tools but model does not
support tool calling" fires for this model because no parser
recognizes the format. This adds one.
Validates schema: rejects unknown functions, unknown params,
duplicate params, missing required params, and <param> tags
without a name= attribute. Supports CDATA-wrapped multi-line
values and single/double-quoted attributes.
Auto-detected by `<function name=` + `<param name=` markers in
the chat template.
Ported from SGLang's MiniCPM5Detector (sgl-project/sglang#25600).
Smoke-tested end-to-end against openbmb/MiniCPM5-1B-MLX: single
and multi-call requests return finish_reason=tool_calls with
correctly typed arguments.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a tool-call parser for openbmb/MiniCPM5-1B (and its
-MLX,-GGUF,-SFTvariants).The model emits tool calls in XML — not JSON — so generic parsers don't pick it up, and
mlx_lm.servercurrently warns:The MiniCPM5 wire format:
Multi-line / special-character values may be wrapped in
<![CDATA[...]]>. Attributes can be single- or double-quoted.Implementation
mlx_lm/tool_parsers/minicpm5.py— statelessparse_tool_call(text, tools)returning a single{"name", "arguments"}dict; raisesValueErrorfor theToolCallFormatterto log + drop.<param>tags without aname=attribute.ast.literal_eval, then fall back to raw.<function name=+<param name=in the chat template.Logic ported from SGLang's
MiniCPM5Detector.Tests
tests/test_tool_parsing.py:test_minicpm5covering: CDATA multi-line params, non-string typed params (array, boolean), body-only segment form, missing-required validation, unknown-param validation, duplicate-param validation, missing-name-attr validation, single-quoted attributes.Smoke test
Live tested against
openbmb/MiniCPM5-1B-MLXrunning undermlx_lm.server:Single call:
{ "finish_reason": "tool_calls", "tool_calls": [{ "function": { "name": "get_weather", "arguments": "{\"city\": \"Tokyo\", \"date\": \"2024-06-27\"}" }, "type": "function", "id": "..." }] }Two consecutive
<function>...</function>blocks both extracted cleanly via the state machine (one segment per pair).Test plan
test_tool_parsing.pycontinues to pass; newtest_minicpm5passes.mlx_lm.server --model openbmb/MiniCPM5-1B-MLXacceptstools=[...]requests without warning and returnstool_callsin the response.