Skip to content

[Rust Frontend] Use oss-harmony for Harmony output processing#46799

Merged
WoosukKwon merged 1 commit into
mainfrom
bz/oss-harmony
Jun 26, 2026
Merged

[Rust Frontend] Use oss-harmony for Harmony output processing#46799
WoosukKwon merged 1 commit into
mainfrom
bz/oss-harmony

Conversation

@BugenZhao

@BugenZhao BugenZhao commented Jun 26, 2026

Copy link
Copy Markdown
Member

Purpose

Switch the library used for Harmony output processing from Inferact/openai-harmony (introduced in #46696) to oss-harmony, a more actively maintained fork of the original library.

This gives the Rust frontend two practical benefits:

  • picks up upstream Harmony bug fixes and improvements
  • embeds the tokenizer vocabularies in the library itself, removing the runtime download path and the related reqwest / TLS backend dependency from Harmony output processing

This PR is the dependency-only base for the follow-up GPT-OSS Harmony renderer work.

Test Plan

  • cargo nextest run --manifest-path rust/Cargo.toml -p vllm-chat output::harmony
  • cargo check --manifest-path rust/Cargo.toml -p vllm-chat --locked

Test Result

Both commands passed locally.


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@BugenZhao BugenZhao requested a review from njhill as a code owner June 26, 2026 06:25

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@BugenZhao

Copy link
Copy Markdown
Member Author

@mergify mergify Bot added rust gpt-oss Related to GPT-OSS models labels Jun 26, 2026
@BugenZhao BugenZhao added the ready ONLY add when PR is ready to merge/full CI is needed label Jun 26, 2026
@github-project-automation github-project-automation Bot moved this from To Triage to Ready in gpt-oss Issues & Enhancements Jun 26, 2026
@WoosukKwon WoosukKwon merged commit e71bc6d into main Jun 26, 2026
29 of 35 checks passed
@WoosukKwon WoosukKwon deleted the bz/oss-harmony branch June 26, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gpt-oss Related to GPT-OSS models ready ONLY add when PR is ready to merge/full CI is needed rust

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants