Skip to content

[Rust Frontend] Fix macro-based content format detection#49042

Open
reidliu41 wants to merge 1 commit into
vllm-project:mainfrom
reidliu41:fix/qwen35-content-format-detection
Open

[Rust Frontend] Fix macro-based content format detection#49042
reidliu41 wants to merge 1 commit into
vllm-project:mainfrom
reidliu41:fix/qwen35-content-format-detection

Conversation

@reidliu41

@reidliu41 reidliu41 commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Purpose

The Rust chat template detector only recognized loops directly over
message.content. It missed templates that pass the content into a macro and
iterate with {% for item in content %}.

Qwen3.5 uses the macro form, causing auto mode to incorrectly select the
string content format. Multimodal messages consequently lost the
<|vision_start|> and <|vision_end|> wrappers around <|image_pad|>, which
could produce incorrect prompt tokens and multimodal positions.

Recognize loops over a content parameter while preserving the existing
direct message.content detection. Add regression coverage using both a
minimal macro template and the repository's Qwen3.5 template.

The corrected auto-detection matches the explicit OpenAI content format and
the corresponding Python detector behavior.

Test Plan

  cargo nextest run \
    -p vllm-chat \
    -E 'test(detects_openai_template_with_content_parameter_loop) | test(qwen35_template_auto_detects_openai_multimodal_content)'

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Signed-off-by: reidliu41 <reid201711@gmail.com>
@mergify mergify Bot added the rust label Jul 18, 2026
@reidliu41
reidliu41 marked this pull request as ready for review July 18, 2026 15:39

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant