[Rust Frontend] Add Harmony Renderer for GPT-OSS#46800
Merged
Merged
Conversation
Member
Author
|
This change is part of the following stack: Change managed by git-spice. |
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.
Tip: disable this comment in your organization's Code Review settings.
njhill
approved these changes
Jun 26, 2026
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
BugenZhao
force-pushed
the
bz/oss-harmony-renderer
branch
from
June 29, 2026 03:08
4feb4e2 to
1fdc599
Compare
BugenZhao
enabled auto-merge (squash)
June 29, 2026 03:08
wincent8
pushed a commit
to wincent8/vllm
that referenced
this pull request
Jun 29, 2026
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
SURUJ404
added a commit
to SURUJ404/vllm
that referenced
this pull request
Jun 30, 2026
Long system instructions injected into the Harmony model identity could produce unexpectedly large prompts with no upper bound. Truncate at 4096 characters with a clear '... [truncated]' suffix. This is a follow-up improvement to vllm-project#46800 (Harmony Renderer for GPT-OSS).
4 tasks
rjrock
pushed a commit
to rjrock/vllm
that referenced
this pull request
Jul 1, 2026
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
noooop
pushed a commit
to noooop/vllm
that referenced
this pull request
Jul 9, 2026
Signed-off-by: Bugen Zhao <i@bugenzhao.com> Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Add a native Harmony chat renderer for GPT-OSS in the Rust frontend.
With this change,
RendererSelection::Autochooses the new Harmony renderer whenmodel_type == "gpt_oss". The renderer usesoss-harmonyto render the Chat Completions input directly intoPrompt::TokenIds, matching the existing Harmony output processor path and avoiding the generic Jinja/HF rendering path for GPT-OSS.The renderer covers the current Rust Chat Completions surface:
VLLM_SYSTEM_START_DATEandVLLM_GPT_OSS_HARMONY_SYSTEM_INSTRUCTIONSbehaviorreasoning_effortvalidation for Harmony-supported valuesoss-harmonyThe existing HF renderer remains available through explicit
--chat-renderer hfselection for debugging and comparison.Test Plan
cargo fmt --manifest-path rust/Cargo.toml --all -- --checkcargo nextest run --manifest-path rust/Cargo.toml -p vllm-chat harmonycargo nextest run --manifest-path rust/Cargo.toml -p vllm-chat renderercargo nextest run --manifest-path rust/Cargo.toml -p vllm-chat output::harmonycargo check --manifest-path rust/Cargo.toml -p vllm-chat --lockedcargo doc --manifest-path rust/Cargo.toml -p vllm-chat --no-deps --lockedTest Result
All focused Rust commands passed locally.
cargo doccompleted successfully. It reports three pre-existing rustdoc private-link warnings outside the Harmony renderer, inrust/src/chat/src/lib.rsandrust/src/chat/src/output/default/mod.rs.Ad hoc fixture comparison against the current Python Harmony path matched the simple, leading-instruction, request-tool, assistant-history, and stale-analysis cases. The remaining differences are intentional/current gaps: Rust preserves developer-local tools expressed in the Rust request schema, and Rust follows
oss-harmonyv0.0.11 /<|constrain|>jsontool-call rendering while the local Python environment usedopenai-harmony0.0.8.GB200 E2E smoke validation passed.
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.