fix(llama-tool-parser): recognize Llama 3.1+ / 3.3 tool-call formats

[CBribiescas]

First — apologies for another PR; I know I've been busy in the issue tracker lately. After this one, the only thing I'm still investigating locally is the gpt-oss harmony rendering side from #568 (waiting on maintainer direction there per @Thump604's reply). Putting this up because it's a narrow, well-scoped fix in a different parser.

The bug

LlamaToolParser only matches the legacy <function=name>{"json"}</function> XML format. But Llama 3.1+ and Llama 4 emit calls with the <|python_tag|>{"name":..., "parameters":...} JSON envelope per Meta's model card, and Llama 3.3 emits the same JSON shape bare, without any marker:

{"type": "function", "name": "read_file", "parameters": {"path": "foo.py"}}

Both currently leak through to message.content untouched and downstream callers see tool_calls: [], treating the assistant turn as a clean stop.

Fix shape

Two extraction helpers added ahead of the existing XML matcher:

• _extract_python_tag_calls — scans for <|python_tag|> markers and uses json.JSONDecoder().raw_decode() to extract each following JSON object (so multiple tool calls in one response work).
• _extract_bare_json_call — only fires when the whole stripped content is a JSON object with name AND (parameters OR arguments). Intentionally strict to avoid mis-parsing user-visible JSON the model might emit for unrelated reasons.

Existing <function=...>{...}</function> XML behavior is unchanged.

Empirical signal

On a multi-turn Claude-Code-style agentic benchmark (20 tasks, T=0):

variant	tasks_passed	avg_turns_taken	score
Llama-3.3-70B-Instruct-4bit, pre-patch	0/20	1.00	0.000
Llama-3.3-70B-Instruct-4bit, post-patch	4/20	3.40	0.200

Same model weights, same harness, same prompt. The avg_turns lift from 1.00 -> 3.40 is the signal: the model was emitting valid bare-JSON tool calls on every turn — they were just being silently dropped.

Test coverage

8 new tests + 3 existing — all passing:

• python_tag JSON, with parameters and the arguments alias
• multiple python_tag calls in one response
• bare-JSON envelope, with and without the type:function field
• precedence: python_tag short-circuits the bare-JSON path (no double-count)
• negative cases: plain text and non-tool JSON must NOT be flagged

$ pytest tests/test_tool_parsers.py::TestLlamaToolParser -v
============================== 11 passed in 0.48s ==============================

Relation to #568

Same class of bug as the gpt-oss rendering issue I reported in #568 — vllm-mlx code paths haven't kept up with formats the models actually emit at inference time. This one is on the output side (parsing) rather than the input side (prompt rendering), which is why it's a much cleaner, smaller patch: no template-format design choices, no openai-harmony vs in-tree decision, no maintainer-direction blocker. Just two regex/decoder helpers and tests.

Happy to address review feedback.

[Thump604]

Thanks for narrowing this. The complete-response extraction path looks like the right direction, but I think one served path needs to be covered before merge.

Blocking items:

1. The streaming parser still only recognizes the legacy XML marker. extract_tool_calls_streaming() returns {"content": delta_text} whenever "<function=" not in current_text, so streamed Llama 3.1+/3.3/4 outputs using <|python_tag|>{...} or the bare JSON envelope will still be emitted as assistant content and never become tool_calls. Since the server calls this streaming parser for streamed chat completions, this leaves stream=True tool-use clients in the old failure mode. Please add streaming coverage for the new formats, or otherwise make the streaming path use equivalent detection once the tagged/bare JSON object is complete.

2. CI lint is currently failing because Black would reformat vllm_mlx/tool_parsers/llama_tool_parser.py.

Not claiming the non-stream extraction fix is wrong; this is about making the existing streaming parser path match the new non-stream behavior before merge.

[CBribiescas]

Pushed the fix in 9222c4e:

• Streaming parser now detects <|python_tag|> and bare-JSON envelopes alongside legacy <function=. 4 new tests in TestLlamaToolParserStreaming cover both new shapes plus regression guards for plain-text and legacy-XML paths.
• Black-formatted vllm_mlx/tool_parsers/llama_tool_parser.py and tests/test_tool_parsers.py.

16 Llama parser tests + 124 tool-parser suite green; no regression in test_native_tool_format. Ready for another look.