Skip to content

[Rust Frontend] Add Harmony Renderer for GPT-OSS#46800

Merged
BugenZhao merged 3 commits into
mainfrom
bz/oss-harmony-renderer
Jun 29, 2026
Merged

[Rust Frontend] Add Harmony Renderer for GPT-OSS#46800
BugenZhao merged 3 commits into
mainfrom
bz/oss-harmony-renderer

Conversation

@BugenZhao

@BugenZhao BugenZhao commented Jun 26, 2026

Copy link
Copy Markdown
Member

Purpose

Add a native Harmony chat renderer for GPT-OSS in the Rust frontend.

With this change, RendererSelection::Auto chooses the new Harmony renderer when model_type == "gpt_oss". The renderer uses oss-harmony to render the Chat Completions input directly into Prompt::TokenIds, matching the existing Harmony output processor path and avoiding the generic Jinja/HF rendering path for GPT-OSS.

The renderer covers the current Rust Chat Completions surface:

  • leading system/developer instructions, including the VLLM_SYSTEM_START_DATE and VLLM_GPT_OSS_HARMONY_SYSTEM_INSTRUCTIONS behavior
  • reasoning_effort validation for Harmony-supported values
  • request-level and developer-local function tools
  • assistant reasoning/final/commentary/tool-call history
  • tool responses mapped back through assistant tool-call IDs
  • Python-parity stale analysis dropping before calling into oss-harmony

The existing HF renderer remains available through explicit --chat-renderer hf selection for debugging and comparison.

Test Plan

  • cargo fmt --manifest-path rust/Cargo.toml --all -- --check
  • cargo nextest run --manifest-path rust/Cargo.toml -p vllm-chat harmony
  • cargo nextest run --manifest-path rust/Cargo.toml -p vllm-chat renderer
  • cargo nextest run --manifest-path rust/Cargo.toml -p vllm-chat output::harmony
  • cargo check --manifest-path rust/Cargo.toml -p vllm-chat --locked
  • cargo doc --manifest-path rust/Cargo.toml -p vllm-chat --no-deps --locked

Test Result

All focused Rust commands passed locally.

cargo doc completed successfully. It reports three pre-existing rustdoc private-link warnings outside the Harmony renderer, in rust/src/chat/src/lib.rs and rust/src/chat/src/output/default/mod.rs.

Ad hoc fixture comparison against the current Python Harmony path matched the simple, leading-instruction, request-tool, assistant-history, and stale-analysis cases. The remaining differences are intentional/current gaps: Rust preserves developer-local tools expressed in the Rust request schema, and Rust follows oss-harmony v0.0.11 / <|constrain|>json tool-call rendering while the local Python environment used openai-harmony 0.0.8.

GB200 E2E smoke validation passed.


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

@BugenZhao

Copy link
Copy Markdown
Member Author

@mergify mergify Bot added rust gpt-oss Related to GPT-OSS models labels Jun 26, 2026
@BugenZhao
BugenZhao marked this pull request as ready for review June 26, 2026 06:45
@BugenZhao
BugenZhao requested a review from njhill as a code owner June 26, 2026 06:45

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

Base automatically changed from bz/oss-harmony to main June 26, 2026 15:24
@github-project-automation github-project-automation Bot moved this from To Triage to Ready in gpt-oss Issues & Enhancements Jun 26, 2026
@njhill njhill added the ready ONLY add when PR is ready to merge/full CI is needed label Jun 28, 2026
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@BugenZhao
BugenZhao force-pushed the bz/oss-harmony-renderer branch from 4feb4e2 to 1fdc599 Compare June 29, 2026 03:08
@BugenZhao
BugenZhao enabled auto-merge (squash) June 29, 2026 03:08
@BugenZhao
BugenZhao merged commit 5274c11 into main Jun 29, 2026
28 checks passed
@BugenZhao
BugenZhao deleted the bz/oss-harmony-renderer branch June 29, 2026 03:39
wincent8 pushed a commit to wincent8/vllm that referenced this pull request Jun 29, 2026
SURUJ404 added a commit to SURUJ404/vllm that referenced this pull request Jun 30, 2026
Long system instructions injected into the Harmony model identity could produce unexpectedly large prompts with no upper bound. Truncate at 4096 characters with a clear '... [truncated]' suffix.

This is a follow-up improvement to vllm-project#46800 (Harmony Renderer for GPT-OSS).
rjrock pushed a commit to rjrock/vllm that referenced this pull request Jul 1, 2026
noooop pushed a commit to noooop/vllm that referenced this pull request Jul 9, 2026
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gpt-oss Related to GPT-OSS models ready ONLY add when PR is ready to merge/full CI is needed rust

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants