Skip to content

feat(eagle3): add per-layer FC normalization (--fc-norm) - #1

Closed
orestis-z wants to merge 21 commits into
eagle3.1from
feat/fc-norm
Closed

feat(eagle3): add per-layer FC normalization (--fc-norm)#1
orestis-z wants to merge 21 commits into
eagle3.1from
feat/fc-norm

Conversation

@orestis-z

@orestis-z orestis-z commented Jun 23, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add --fc-norm flag for per-layer RMSNorm on each auxiliary hidden state before concatenation and FC projection: concat(Norm(h_a), Norm(h_b), Norm(h_c))
  • This matches the Eagle 3.1 paper specification and vLLM's fc_norm config field
  • Differs from --norm-before-fc which applies a single norm to the full concatenation: Norm(concat(h_a, h_b, h_c))
  • Defaults to True for --draft-arch llama (along with --norm-output); --norm-before-fc no longer defaults to True for llama since --fc-norm is the more principled alternative

Stacks on top of vllm-project#610.

Test plan

pytest tests/unit/test_config.py -k "fc_norm" -v
pytest tests/integration/models/test_model_forward.py -k "fc_norm" -v
  • test_fc_norm — Eagle3 with fc_norm + norm_output, verifies fc_norms module created and forward + backward pass
  • test_peagle_fc_norm — P-Eagle with fc_norm
  • test_eagle3_config_fc_norm_roundtrip — config serialization roundtrip

🤖 Generated with Claude Code

dependabot Bot and others added 3 commits June 23, 2026 14:18
@dependabot
Bump actions/checkout from 6 to 7 (vllm-project#641)
c1db4de 
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to
7.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/checkout/releases">actions/checkout's
releases</a>.</em></p>
<blockquote>
<h2>v7.0.0</h2>
<h2>What's Changed</h2>
<ul>
<li>block checking out fork pr for pull_request_target and workflow_run
by <a href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2454">actions/checkout#2454</a></li>
<li>Bump actions/publish-immutable-action from 0.0.3 to 0.0.4 in the
minor-actions-dependencies group across 1 directory by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2458">actions/checkout#2458</a></li>
<li>Bump flatted from 3.3.1 to 3.4.2 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2460">actions/checkout#2460</a></li>
<li>Bump js-yaml from 4.1.0 to 4.2.0 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2461">actions/checkout#2461</a></li>
<li>Bump <code>@​actions/core</code> and
<code>@​actions/tool-cache</code> and Remove uuid by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2459">actions/checkout#2459</a></li>
<li>upgrade module to esm and update dependencies by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2463">actions/checkout#2463</a></li>
<li>Bump the minor-npm-dependencies group across 1 directory with 3
updates by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2462">actions/checkout#2462</a></li>
<li>getting ready for checkout v7 release by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2464">actions/checkout#2464</a></li>
<li>update error wording by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2467">actions/checkout#2467</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/checkout/pull/2454">actions/checkout#2454</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v6.0.3...v7.0.0">https://github.com/actions/checkout/compare/v6.0.3...v7.0.0</a></p>
<h2>v6.0.3</h2>
<h2>What's Changed</h2>
<ul>
<li>Update changelog by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2357">actions/checkout#2357</a></li>
<li>fix: expand merge commit SHA regex and add SHA-256 test cases by <a
href="https://github.com/yaananth"><code>@​yaananth</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2414">actions/checkout#2414</a></li>
<li>Fix checkout init for SHA-256 repositories by <a
href="https://github.com/yaananth"><code>@​yaananth</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2439">actions/checkout#2439</a></li>
<li>Update changelog for v6.0.3 by <a
href="https://github.com/yaananth"><code>@​yaananth</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2446">actions/checkout#2446</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/yaananth"><code>@​yaananth</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/checkout/pull/2414">actions/checkout#2414</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v6...v6.0.3">https://github.com/actions/checkout/compare/v6...v6.0.3</a></p>
<h2>v6.0.2</h2>
<h2>What's Changed</h2>
<ul>
<li>Add orchestration_id to git user-agent when ACTIONS_ORCHESTRATION_ID
is set by <a
href="https://github.com/TingluoHuang"><code>@​TingluoHuang</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/2355">actions/checkout#2355</a></li>
<li>Fix tag handling: preserve annotations and explicit fetch-tags by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2356">actions/checkout#2356</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v6.0.1...v6.0.2">https://github.com/actions/checkout/compare/v6.0.1...v6.0.2</a></p>
<h2>v6.0.1</h2>
<h2>What's Changed</h2>
<ul>
<li>Update all references from v5 and v4 to v6 by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2314">actions/checkout#2314</a></li>
<li>Add worktree support for persist-credentials includeIf by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2327">actions/checkout#2327</a></li>
<li>Clarify v6 README by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2328">actions/checkout#2328</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v6...v6.0.1">https://github.com/actions/checkout/compare/v6...v6.0.1</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/actions/checkout/blob/main/CHANGELOG.md">actions/checkout's
changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<h2>v7.0.0</h2>
<ul>
<li>Block checking out fork PR for pull_request_target and workflow_run
by <a href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2454">actions/checkout#2454</a></li>
<li>Bump actions/publish-immutable-action from 0.0.3 to 0.0.4 in the
minor-actions-dependencies group across 1 directory by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2458">actions/checkout#2458</a></li>
<li>Bump flatted from 3.3.1 to 3.4.2 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2460">actions/checkout#2460</a></li>
<li>Bump js-yaml from 4.1.0 to 4.2.0 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2461">actions/checkout#2461</a></li>
<li>Bump <code>@​actions/core</code> and
<code>@​actions/tool-cache</code> and Remove uuid by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2459">actions/checkout#2459</a></li>
<li>upgrade module to esm and update dependencies by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2463">actions/checkout#2463</a></li>
<li>Bump the minor-npm-dependencies group across 1 directory with 3
updates by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2462">actions/checkout#2462</a></li>
</ul>
<h2>v6.0.3</h2>
<ul>
<li>Fix checkout init for SHA-256 repositories by <a
href="https://github.com/yaananth"><code>@​yaananth</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2439">actions/checkout#2439</a></li>
<li>fix: expand merge commit SHA regex and add SHA-256 test cases by <a
href="https://github.com/yaananth"><code>@​yaananth</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2414">actions/checkout#2414</a></li>
</ul>
<h2>v6.0.2</h2>
<ul>
<li>Fix tag handling: preserve annotations and explicit fetch-tags by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2356">actions/checkout#2356</a></li>
</ul>
<h2>v6.0.1</h2>
<ul>
<li>Add worktree support for persist-credentials includeIf by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2327">actions/checkout#2327</a></li>
</ul>
<h2>v6.0.0</h2>
<ul>
<li>Persist creds to a separate file by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2286">actions/checkout#2286</a></li>
<li>Update README to include Node.js 24 support details and requirements
by <a href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/2248">actions/checkout#2248</a></li>
</ul>
<h2>v5.0.1</h2>
<ul>
<li>Port v6 cleanup to v5 by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2301">actions/checkout#2301</a></li>
</ul>
<h2>v5.0.0</h2>
<ul>
<li>Update actions checkout to use node 24 by <a
href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2226">actions/checkout#2226</a></li>
</ul>
<h2>v4.3.1</h2>
<ul>
<li>Port v6 cleanup to v4 by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2305">actions/checkout#2305</a></li>
</ul>
<h2>v4.3.0</h2>
<ul>
<li>docs: update README.md by <a
href="https://github.com/motss"><code>@​motss</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1971">actions/checkout#1971</a></li>
<li>Add internal repos for checking out multiple repositories by <a
href="https://github.com/mouismail"><code>@​mouismail</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1977">actions/checkout#1977</a></li>
<li>Documentation update - add recommended permissions to Readme by <a
href="https://github.com/benwells"><code>@​benwells</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2043">actions/checkout#2043</a></li>
<li>Adjust positioning of user email note and permissions heading by <a
href="https://github.com/joshmgross"><code>@​joshmgross</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2044">actions/checkout#2044</a></li>
<li>Update README.md by <a
href="https://github.com/nebuk89"><code>@​nebuk89</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2194">actions/checkout#2194</a></li>
<li>Update CODEOWNERS for actions by <a
href="https://github.com/TingluoHuang"><code>@​TingluoHuang</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/2224">actions/checkout#2224</a></li>
<li>Update package dependencies by <a
href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2236">actions/checkout#2236</a></li>
</ul>
<h2>v4.2.2</h2>
<ul>
<li><code>url-helper.ts</code> now leverages well-known environment
variables by <a href="https://github.com/jww3"><code>@​jww3</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/1941">actions/checkout#1941</a></li>
<li>Expand unit test coverage for <code>isGhes</code> by <a
href="https://github.com/jww3"><code>@​jww3</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1946">actions/checkout#1946</a></li>
</ul>
<h2>v4.2.1</h2>
<ul>
<li>Check out other refs/* by commit if provided, fall back to ref by <a
href="https://github.com/orhantoy"><code>@​orhantoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1924">actions/checkout#1924</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/actions/checkout/commit/9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0"><code>9c091bb</code></a>
update error wording (<a
href="https://redirect.github.com/actions/checkout/issues/2467">#2467</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/1044a6dea927916f2c38ba5aeffbc0a847b1221a"><code>1044a6d</code></a>
getting ready for checkout v7 release (<a
href="https://redirect.github.com/actions/checkout/issues/2464">#2464</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/f0282184c7ce73ab54c7e4ab5a617122602e575f"><code>f028218</code></a>
Bump the minor-npm-dependencies group across 1 directory with 3 updates
(<a
href="https://redirect.github.com/actions/checkout/issues/2462">#2462</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/d914b262ffc244530a203ab40decab34c3abf34d"><code>d914b26</code></a>
upgrade module to esm and update dependencies (<a
href="https://redirect.github.com/actions/checkout/issues/2463">#2463</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/537c7ef99cef6e5ddb5e7ff5d16d14510503801d"><code>537c7ef</code></a>
Bump <code>@​actions/core</code> and <code>@​actions/tool-cache</code>
and Remove uuid (<a
href="https://redirect.github.com/actions/checkout/issues/2459">#2459</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/130a169078a413d3a5246a393625e8e742f387f6"><code>130a169</code></a>
Bump js-yaml from 4.1.0 to 4.2.0 (<a
href="https://redirect.github.com/actions/checkout/issues/2461">#2461</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/7d09575332117a40b46e5e020664df234cd416f3"><code>7d09575</code></a>
Bump flatted from 3.3.1 to 3.4.2 (<a
href="https://redirect.github.com/actions/checkout/issues/2460">#2460</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/0f9f3aa320cb53abeb534aeb54048075d9697a0e"><code>0f9f3aa</code></a>
Bump actions/publish-immutable-action (<a
href="https://redirect.github.com/actions/checkout/issues/2458">#2458</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/f9e715a95fcd1f9253f77dd28f11e88d2d6460c7"><code>f9e715a</code></a>
block checking out fork pr for pull_request_target and workflow_run (<a
href="https://redirect.github.com/actions/checkout/issues/2454">#2454</a>)</li>
<li>See full diff in <a
href="https://github.com/actions/checkout/compare/v6...v7">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/checkout&package-manager=github_actions&previous-version=6&new-version=7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
Update torch requirement from <=2.12.0,>=2.9.0 to >=2.9.0,<=2.12.1 (v…
456b9ba 
…llm-project#642)

Updates the requirements on [torch](https://github.com/pytorch/pytorch)
to permit the latest version.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pytorch/pytorch/releases">torch's
releases</a>.</em></p>
<blockquote>
<h2>PyTorch 2.12.1 Release, bug fix release</h2>
<p>This release is meant to fix the following regressions and silent
correctness issues:</p>
<h2>Regression fixes</h2>
<ul>
<li>Fix nondeterministic outputs in test_batch_invariance with
FLASH_ATTN on NVIDIA B200 GPUs (<a
href="https://redirect.github.com/pytorch/pytorch/issues/181248">#181248</a>),
fixed by updating Triton to 3.7.1 (<a
href="https://redirect.github.com/pytorch/pytorch/pull/186814">#186814</a>)</li>
<li>Fix illegal memory access in the Triton convolution2d_bwd_weight
kernel on B100/B200 (sm100) GPUs (<a
href="https://redirect.github.com/pytorch/pytorch/issues/187081">#187081</a>),
fixed by updating Triton to 3.7.1 (<a
href="https://redirect.github.com/pytorch/pytorch/pull/186814">#186814</a>)</li>
<li>Fix fill_ on byte-dtype views with misaligned storage offset (<a
href="https://redirect.github.com/pytorch/pytorch/pull/186821">#186821</a>)</li>
</ul>
<h2>Releng / Build</h2>
<ul>
<li>Drop CPython 3.13t from the binary build matrix (<a
href="https://redirect.github.com/pytorch/pytorch/pull/182951">#182951</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pytorch/pytorch/blob/main/RELEASE.md">torch's
changelog</a>.</em></p>
<blockquote>
<h1>Releasing PyTorch</h1>
<!-- raw HTML omitted -->
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#release-compatibility-matrix">Release
Compatibility Matrix</a>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#pytorch-cuda-support-matrix">PyTorch
CUDA Support Matrix</a></li>
</ul>
</li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#release-cadence">Release
Cadence</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#general-overview">General
Overview</a>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#frequently-asked-questions">Frequently
Asked Questions</a></li>
</ul>
</li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#cutting-a-release-branch-preparations">Cutting
a release branch preparations</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#cutting-release-branches">Cutting
release branches</a>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#pytorchpytorch"><code>pytorch/pytorch</code></a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#pytorch-ecosystem-libraries">PyTorch
ecosystem libraries</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#making-release-branch-specific-changes-for-pytorch">Making
release branch specific changes for PyTorch</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#making-release-branch-specific-changes-for-ecosystem-libraries">Making
release branch specific changes for ecosystem libraries</a></li>
</ul>
</li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#running-launch-execution-team-core-xfn-sync">Running
Launch Execution team Core XFN sync</a></li>
<li><a
href="#drafting-rcs-release-candidates-for-pytorch-and-domain-libraries">Drafting
RCs (https://github.com/pytorch/pytorch/blob/main/Release Candidates)
for PyTorch and domain libraries</a>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#release-candidate-storage">Release
Candidate Storage</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#release-candidate-health-validation">Release
Candidate health validation</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#cherry-picking-fixes">Cherry
Picking Fixes</a>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#how-to-do-cherry-picking">How
to do Cherry Picking</a></li>
</ul>
</li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#cherry-picking-reverts">Cherry
Picking Reverts</a></li>
</ul>
</li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#preparing-and-creating-final-release-candidate">Preparing
and Creating Final Release Candidate</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#promoting-rcs-to-stable">Promoting
RCs to Stable</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#additional-steps-to-prepare-for-release-day">Additional
Steps to prepare for release day</a>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#modify-release-matrix">Modify
release matrix</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#open-google-colab-issue">Open
Google Colab issue</a></li>
</ul>
</li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#patch-releases">Patch
Releases</a>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#patch-release-criteria">Patch
Release Criteria</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#patch-release-process">Patch
Release Process</a>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#patch-release-process-description">Patch
Release Process Description</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#triage">Triage</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#issue-tracker-for-patch-releases">Issue
Tracker for Patch releases</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#building-a-release-schedule--cherry-picking">Building
a release schedule / cherry picking</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#building-binaries--promotion-to-stable">Building
Binaries / Promotion to Stable</a></li>
</ul>
</li>
</ul>
</li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#hardware--software-support-in-binary-build-matrix">Hardware
/ Software Support in Binary Build Matrix</a>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#python">Python</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#accelerator-software">Accelerator
Software</a>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#special-support-cases">Special
support cases</a></li>
</ul>
</li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#operating-systems">Operating
Systems</a></li>
</ul>
</li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#submitting-tutorials">Submitting
Tutorials</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#special-topics">Special
Topics</a>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#updating-submodules-for-a-release">Updating
submodules for a release</a></li>
<li><a
href="https://github.com/pytorch/pytorch/blob/main/#triton-dependency-for-the-release">Triton
dependency for the release</a></li>
</ul>
</li>
</ul>
<!-- raw HTML omitted -->
<h2>Release Compatibility Matrix</h2>
<p>Following is the Release Compatibility Matrix for PyTorch
releases:</p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pytorch/pytorch/commit/7269437d655783a26cba32aa88195b741ff496aa"><code>7269437</code></a>
Update triton to 3.7.1 release (<a
href="https://redirect.github.com/pytorch/pytorch/issues/186814">#186814</a>)</li>
<li><a
href="https://github.com/pytorch/pytorch/commit/88f16c2e68c0bcd24fcc974014c044a4995b26f4"><code>88f16c2</code></a>
[MPS] Fix fill_ on byte-dtype views with misaligned storage offset (<a
href="https://redirect.github.com/pytorch/pytorch/issues/186821">#186821</a>)</li>
<li><a
href="https://github.com/pytorch/pytorch/commit/ccf6e670f128f7f64492468581f89a14fc127eb1"><code>ccf6e67</code></a>
[release-only] Update version to 2.12.1 (<a
href="https://redirect.github.com/pytorch/pytorch/issues/186813">#186813</a>)</li>
<li><a
href="https://github.com/pytorch/pytorch/commit/88a6dc788f17f2bf1cc53672b8c40f13cda0673e"><code>88a6dc7</code></a>
Revive CUDA 12.9 nightly binary builds (<a
href="https://redirect.github.com/pytorch/pytorch/issues/186015">#186015</a>)</li>
<li><a
href="https://github.com/pytorch/pytorch/commit/ded55054597e87d73d21266c3099d71943d0515f"><code>ded5505</code></a>
[CD] Drop CPython 3.13t from binary build matrix (<a
href="https://redirect.github.com/pytorch/pytorch/issues/182951">#182951</a>)
(<a
href="https://redirect.github.com/pytorch/pytorch/issues/186654">#186654</a>)</li>
<li><a
href="https://github.com/pytorch/pytorch/commit/0d62256a2b23365f8e1604297eb23a6545102aa8"><code>0d62256</code></a>
[release] Dockerfile: skip torchaudio install when CUDA_PATH=cu132 (<a
href="https://redirect.github.com/pytorch/pytorch/issues/183346">#183346</a>)</li>
<li><a
href="https://github.com/pytorch/pytorch/commit/7661cd9c6b841b62b7f411aa52ec51f05457263b"><code>7661cd9</code></a>
[MPS] Fix SDPA wrong output for permuted q/k/v with B &gt; 1 (<a
href="https://redirect.github.com/pytorch/pytorch/issues/181886">#181886</a>)</li>
<li><a
href="https://github.com/pytorch/pytorch/commit/9da6087ab64ab6a2118686420ca5353c90dd7e1f"><code>9da6087</code></a>
Fix stale PYTORCH_RELEASES_CODE_CC dict (fixes <a
href="https://redirect.github.com/pytorch/pytorch/issues/182250">#182250</a>)
(<a
href="https://redirect.github.com/pytorch/pytorch/issues/182369">#182369</a>)</li>
<li><a
href="https://github.com/pytorch/pytorch/commit/e4c37cc011d47246ce8ea4b99c9b28fb7f400224"><code>e4c37cc</code></a>
Avoid raw stream name collisions in Inductor (<a
href="https://redirect.github.com/pytorch/pytorch/issues/182178">#182178</a>)</li>
<li><a
href="https://github.com/pytorch/pytorch/commit/822d047dc8dd8d919f83c3ad5c786e405075d1f0"><code>822d047</code></a>
[MPS] Fix bool mask handling in 1-pass SDPA decode kernel (<a
href="https://redirect.github.com/pytorch/pytorch/issues/182285">#182285</a>)
(<a
href="https://redirect.github.com/pytorch/pytorch/issues/182311">#182311</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/pytorch/pytorch/compare/v2.9.0...v2.12.1">compare
view</a></li>
</ul>
</details>
<br />

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@fynnsu @thechaos16
Add Mlflow metric handler (vllm-project#623)
f261a74 
<!-- markdownlint-disable -->

<!-- PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS
(AT THE BOTTOM) HAVE BEEN CONSIDERED. -->

## Purpose
This pr adds a commit on top of vllm-project#565, which simplifies some of the
logger handling logic.

## Tests

Manually ran a test training job and confirmed logging worked, including
resuming existing runs.

## Checklist

I have filled in:

- [x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x] The test plan/results, such as providing test command and pasting
the results.
- [ ] (Optional) The necessary documentation update.
- [x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

---------

Signed-off-by: thechaos16 <thechaos16@gmail.com>
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: thechaos16 <thechaos16@gmail.com>
eldarkurtic and others added 8 commits June 24, 2026 13:54
@eldarkurtic
Ignore ruff "too many branches" (vllm-project#650)
f307053
@fynnsu
Suppress uv and apt progress bars in logs (vllm-project#653)
162a6c5 
<!-- markdownlint-disable -->

<!-- PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS
(AT THE BOTTOM) HAVE BEEN CONSIDERED. -->

## Purpose

Currently all of the buildkite logs are getting truncated because they
exceed 2MB. The reason for this is the uv (and to a lesser degree apt)
installs print and then delete progress updates as they go. This looks
nice locally but results in massive log files in non-terminal
environments.

Edit: It looks like we're also currently downloading uv packages to a
default cache dir, which then get copied for use in the environment.
This seems to be happening because the default cache dir isn't on the
same volume as the workplace where the env is created. I added a commit
to set a new local cache dir.

## Tests

Will run buildkite ci and confirm changes improve logs.

## Checklist

I have filled in:

- [x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x] The test plan/results, such as providing test command and pasting
the results.
- [ ] (Optional) The necessary documentation update.
- [x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

---------

Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
@fynnsu
Create buildkite sections in logs (vllm-project#654)
d55207b 
<!-- markdownlint-disable -->

<!-- PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS
(AT THE BOTTOM) HAVE BEEN CONSIDERED. -->

## Purpose

Currently all of the buildkite steps (apt package install, nvidia-smi
printing, python venv setup, and pytest tests) end up under the same
collapsible header "Running commands". This pr adds subsections for each
part so that it is easier to view the different steps and jump to the
relevant parts.

## Tests

Will run CI and review results. 

## Checklist

I have filled in:

- [x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x] The test plan/results, such as providing test command and pasting
the results.
- [ ] (Optional) The necessary documentation update.
- [x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
@GIREESH7963 @claude @fynnsu
Add total variation (TV) loss for speculative decoding (vllm-project#648
fadff3e 
)

## Summary
Adds a total variation (TV) loss function to the speculators loss
interface.

## Motivation
The speculative-decoding acceptance rate equals the target/draft
distribution overlap,
alpha = sum_v min(p_v, q_v) = 1 - d_TV(p, q). TV loss therefore
optimizes acceptance rate
directly, whereas the existing cross-entropy and KL losses do so only
indirectly (KL is a
loose upper bound on TV via Pinsker). Source: arXiv 2606.12370, Eq. 10
(per-step TV loss).

## Changes
- `tv_loss(logits, targets) -> [1, seq_len]` in
`src/speculators/models/metrics.py`,
  matching the existing `kl_div_loss` / `ce_loss` interface.
- Registered as "tv" in `resolve_loss_fn`.
- Unit tests covering correctness, shape/range, equivalence to the L1
form, and resolution.

## Scope / notes
- Implements the regular per-step TV loss only. 
- Target distribution is treated as the (detached) reference, consistent
with the existing
  kl_div/ce losses which assume frozen-target logits.

## AI assistance
Implemented with assistance from Claude code

Signed-off-by: GIREESH7963 <abburigireesh@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
@guan404ming @shanjiaz
Add DFlash model converter (vllm-project#617)
31374a5 
<!-- markdownlint-disable -->

<!-- PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS
(AT THE BOTTOM) HAVE BEEN CONSIDERED. -->

## Purpose

Close vllm-project#616

External DFlash checkpoints (e.g. z-lab/*-DFlash) had no path into the
Speculators format, so they could not be loaded or finetuned via
from_pretrained.

## Tests

- add mocked _build_config unit tests
- test on a real checkpoint: converted `z-lab/Qwen3-8B-DFlash-b16` with
verifier `Qwen/Qwen3-8B`, reloaded via `from_pretrained` with weights
bit-identical to source

## Checklist

I have filled in:

- [x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x] The test plan/results, such as providing test command and pasting
the results.
- [ ] (Optional) The necessary documentation update.
- [x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

---------

Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: shanjiaz <zsjwpianpian@gmail.com>
@eldarkurtic
Handle nested rope in transformers v5 (vllm-project#649)
26e0074 
Models like Laguna have nested rope like:
```json
"rope_parameters": {
  "full_attention": {
    "rope_theta": 500000.0,
    "rope_type": "yarn",
    "factor": 64.0,
    "original_max_position_embeddings": 4096,
    "beta_slow": 1.0,
    "beta_fast": 64.0,
    "attention_factor": 1.0,
    "partial_rotary_factor": 0.5
  },
  "sliding_attention": {
    "rope_type": "default",
    "rope_theta": 10000.0,
    "partial_rotary_factor": 1.0
},
```
This PR enables inferring the correct `rope_theta` for such
configurations. When both attns are present (full and sliding), we
prefer theta from the sliding one.

Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
@shubhra @cursoragent Shubhra Pandit
feat(train): mid-epoch checkpoint save and resume (vllm-project#603)
9e322bd 
<!-- markdownlint-disable -->

<!-- PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS
(AT THE BOTTOM) HAVE BEEN CONSIDERED. -->

## Purpose
Previously, fractional checkpoints (checkpoint_freq < 1) were named by
epoch index only (e.g. 0/, 1/) and resume always jumped to the *next*
epoch, discarding any training progress within the interrupted epoch.

This change adds proper mid-epoch checkpoint state tracking so training
can resume from the exact batch where it was interrupted:

- Save training_state.json alongside each checkpoint recording the
current epoch, local_step within that epoch, and global_step. This
allows the trainer to distinguish a mid-epoch save (local_step > 0) from
an end-of-epoch save (local_step == 0).

- On resume, if training_state.json indicates a mid-epoch checkpoint,
stay in the same epoch and fast-skip the already-processed batches by
slicing the batch sampler's pre-generated batch list. This avoids
calling __getitem__ (and thus vLLM) for skipped batches, making resume
essentially instant regardless of how many batches are skipped.

- Create a human-readable epoch{N}_step{S} symlink pointing to the
checkpoint folder whenever a mid-epoch checkpoint is saved, and an
epoch{N}_end symlink for end-of-epoch checkpoints. The symlink is
updated on each new checkpoint for that epoch so there is always exactly
one readable pointer per epoch.

- Skip symlinks in BaseCheckpointer._get_previous_epoch so the new
descriptive symlinks do not interfere with epoch detection.

The training_state.json is not written for interrupted checkpoints
(epoch="interrupted") since those are emergency saves that require
manual intervention to use.

## Tests
Ran the targeted resume regression test:
- Command:
- `/home/shubhra/fp8_experiments/venv/bin/python
test_mid_epoch_resume.py`
  
What this validates:
- `training_state.json` is written at fractional checkpoints with
correct `epoch`, `local_step`, and `global_step`.
- Mid-epoch resume restores `current_epoch`, `_resume_local_step`, and
`global_step` correctly.
- Fast-skip behavior works: already-processed batches are skipped
without replay.
- Resume continues from the next unprocessed batch and keeps
`global_step` continuity.
- Descriptive symlink creation/update works (e.g. `epoch0_step3`).

## Checklist

I have filled in:

- [ x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x ] The test plan/results, such as providing test command and pasting
the results.
- [ ] (Optional) The necessary documentation update.
- [ x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

---------

Signed-off-by: Shubhra Pandit <shpandit@redhat.com>
Co-authored-by: Shubhra Pandit <shpandit@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shubhra Pandit <shubhra@a100-01.nemg-001.lab.rdu2.dc.redhat.com>
@GIREESH7963
Add negative log-acceptance (LK) loss for speculative decoding (vllm-…
669b878 
…project#657)

## Summary
Adds a negative log-acceptance loss to the speculators loss interface —
the
hyperparameter-free LK loss variant.

## Motivation
Acceptance rate equals the target/draft overlap, `alpha = sum_v min(p_v,
q_v) = 1 - TV`.
This loss is `-log(alpha)`, which optimizes acceptance directly. Its
gradient is
`(1/alpha)*grad(TV)`: the `1/alpha` factor amplifies TV's
otherwise-vanishing gradient when
overlap is low, so it trains well from a cold start where pure TV
stalls. At a point-mass
target it reduces to cross-entropy. Source: Samarin et al., "LK Losses:
Direct Acceptance
Rate Optimization for Speculative Decoding" (arXiv 2602.23881).

## Changes
- `neg_log_acceptance_loss(logits, targets) -> [1, seq_len]` in
`metrics.py`, reusing the
  same overlap term as `tv_loss`, with an `_EPS` floor before the log.
- Registered as `"nla"` in `resolve_loss_fn` (short name open to
maintainer preference).
- Unit tests: identical-dist = 0, equals `-log(overlap)`, reduces to
cross-entropy at a
  point-mass target, finite at zero overlap, and resolution.

## Scope / notes
- Negative-log-acceptance variant only. The hybrid LK loss (`lambda*KL +
(1-lambda)*TV` with an
adaptive, eta-controlled weight) is deferred to a follow-up, since it
needs a
loss-hyperparameter interface decision. The e2e variant and kernel
optimizations are out of
  scope.
- Target distribution is treated as the (detached) reference, consistent
with the existing
  `kl_div`/`ce`/`tv` losses which assume frozen-target logits.

Signed-off-by: GIREESH7963 <abburigireesh@gmail.com>
@orestis-z orestis-z mentioned this pull request Jun 26, 2026
Merged
4 tasks
@orestis-z @claude
feat(eagle3): add Eagle 3.1 training support (vllm-project#610)
aba50b0 
## Purpose

Adds Eagle 3.1 training support with two config flags (matching vLLM's
config fields):

- **`--norm-before-fc`**: Apply a single RMSNorm to the concatenated
target hidden states before the FC projection layer — i.e.
`Norm(concat(h₂, h₁₈, h₃₃))`. This was already implemented on main for
gpt-oss checkpoint compatibility; this PR adds the training CLI flag and
wires it through P-EAGLE. **Note:** the original Eagle 3.1 paper
specifies per-layer normalization (`concat(Norm(h₂), Norm(h₁₈),
Norm(h₃₃))`), which corresponds to vLLM's `fc_norm` field. Per-layer
norm support (`--fc-norm`) will be added in a follow-up PR.
- **`--norm-output`**: Feed post-norm hidden states back across TTT
steps to stabilize magnitude drift across speculation depths.

Both flags default to `False` (opt-in).

Also fixes P-EAGLE missing `input_norm` before FC and wires
`norm_before_fc` / `norm_output` through the converter and P-EAGLE's
`from_training_args`.

### vLLM compatibility

- `norm_before_fc` works out of the box with vLLM
- `norm_output` requires a one-line patch in vLLM's `algos.py` to
propagate the config field (model code already supports it via
`getattr`): vllm-project/vllm#46488

## Tests

```bash
pytest tests/unit/test_config.py -k "eagle3_config" -v    # config roundtrip
pytest tests/integration/models/test_model_forward.py -k "NormOutput" -v  # forward pass
```

- `test_norm_output` — both flags enabled, verifies input_norm + forward
+ backward
- `test_norm_output_without_norm_before_fc` — norm_output alone (no
input_norm)
- `test_peagle_norm_before_fc` — P-EAGLE with norm_before_fc
- 4 config roundtrip tests (model_dump, to_dict, save/load_pretrained,
defaults)

## Checklist

- [x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x] The test plan/results, such as providing test command and pasting
the results.
- [x] The necessary documentation update.
- [x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

---------

Signed-off-by: Orestis Zambounis <orestis.zambounis@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@orestis-z
orestis-z marked this pull request as draft June 26, 2026 14:07
@orestis-z
orestis-z force-pushed the feat/fc-norm branch 2 times, most recently from 8725922 to 2764cd1 Compare June 26, 2026 14:33
orestis-z and others added 3 commits June 26, 2026 15:19
@orestis-z @claude
Add per-layer FC normalization (--fc-norm)
fd2230b 
Add fc_norm: per-layer RMSNorm on each auxiliary hidden state before
concatenation and FC projection — concat(Norm(h_a), Norm(h_b), Norm(h_c))
— matching the Eagle 3.1 paper specification. This differs from
norm_before_fc which applies a single norm to the full concatenation.

Defaults to True for llama draft arch (along with norm_output).
norm_before_fc no longer defaults to True for llama arch since
fc_norm is the more principled alternative.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Orestis Zambounis <orestis.zambounis@gmail.com>
@orestis-z @claude
refactor: rename fc_norms to fc_norm for vLLM weight compatibility
30422f4 
The vLLM model uses `self.fc_norm` (singular) for the ModuleList,
producing weight names like `fc_norm.0.weight`. Align the speculators
attribute name so checkpoints are directly loadable without remapping.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Orestis Zambounis <orestis.zambounis@gmail.com>
@orestis-z @claude
refactor: simplify norm flags to store_true, remove defaulting logic
e5652d6 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Orestis Zambounis <orestis.zambounis@gmail.com>
shanjiaz and others added 6 commits June 26, 2026 15:59
@shanjiaz shanjiaz
fix dflash converter issues (vllm-project#662)
7c50e60 
<!-- markdownlint-disable -->

<!-- PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS
(AT THE BOTTOM) HAVE BEEN CONSIDERED. -->

## Purpose
Fixing two separate issues:
1. For z-lab style dflash models, older models have `block_size` on top
level but newer models have them nested under `dflash_config`.

## Tests
Tested both locally.

## Checklist

I have filled in:

- [x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x] The test plan/results, such as providing test command and pasting
the results.
- [ ] (Optional) The necessary documentation update.
- [x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

<!-- codesmith:footer -->
---
<a
href="https://app.blacksmith.sh/vllm-project/codesmith/speculators/pr/662"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-light-v2.svg"><img
alt="View with Codesmith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/view-with-codesmith-dark-v2.svg"></picture></a>
<a
href="https://backend.blacksmith.sh/track/enable-autofix?expires=1785011677&installation_id=142609222&pr_number=662&repository=vllm-project%2Fspeculators&return_to=https%3A%2F%2Fgithub.com%2Fvllm-project%2Fspeculators%2Fpull%2F662&signature=fba84640716e0e5676c7dd7ff179a2da1bd1724d2cb2e279d7472e09fbb84f6b"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-light.svg"><img
alt="Autofix with Codesmith"
src="https://pr-comments-assets.blacksmith.sh/codesmith/autofix-with-codesmith-dark.svg"></picture></a>
<sup>Need help on this PR? Tag <code>/codesmith</code> with what you
need. Autofix is disabled.</sup>

<!-- codesmith:autofix:disabled -->
<!-- /codesmith:footer -->

---------

Signed-off-by: shanjiaz <hezhao@redhat.com>
Co-authored-by: shanjiaz <hezhao@redhat.com>
@shanjiaz shanjiaz
remove eagle converter and reference (vllm-project#672)
bcdc38a 
<!-- markdownlint-disable -->

<!-- PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS
(AT THE BOTTOM) HAVE BEEN CONSIDERED. -->

## Purpose
Deprecate old eagle converter pathway. It's incompatible with rest of
our converters and we haven't seen enough community engagement.

## Tests

## Checklist

I have filled in:

- [x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x] The test plan/results, such as providing test command and pasting
the results.
- [ ] (Optional) The necessary documentation update.
- [x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

Signed-off-by: shanjiaz <hezhao@redhat.com>
Co-authored-by: shanjiaz <hezhao@redhat.com>
@rahul-tuli shanjiaz
Fix verifier kwarg leaking into draft model constructors (vllm-projec…
0451356 
…t#669)

<!-- markdownlint-disable -->

> ⚠️ **DO NOT MERGE before vllm-project#672.** This PR is stacked on vllm-project#672 (remove
eagle converter) and its base is `remove-eagle-converter`. Merging this
first would pull in vllm-project#672's changes prematurely and/or land against the
wrong base. Merge order: **vllm-project#672 first, then this.** Once vllm-project#672 merges, I
will rebase this onto `main`. The diff here is the 3-line loader change
only.

## Purpose

Fix `TypeError: Eagle3DraftModel.__init__() got an unexpected keyword
argument 'verifier'` raised during draft model construction in
`scripts/train.py --from-pretrained`.

This breaks the EAGLE3 finetuning smoke test:

```
tests/e2e/smoke/test_finetuning_sanity.py::test_finetuning_weight_sanity
TypeError: Eagle3DraftModel.__init__() got an unexpected keyword argument 'verifier'
```

`build_draft_model` passes `verifier=args.verifier_name_or_path` to
`from_pretrained` (for the external-checkpoint auto-convert path added
in vllm-project#617). `SpeculatorModel.from_pretrained` had no `verifier` parameter,
so it stayed in `**kwargs`, was forwarded to HF's `from_pretrained`, and
reached the model constructor `cls(config, **kwargs)`. Draft models take
only `config` and reject it.

## Fix

Make `verifier` an explicit named parameter of
`SpeculatorModel.from_pretrained` so it is consumed at the loader
boundary (never forwarded into the constructor), used directly for
auto-conversion, and forwarded through the base-class re-dispatch
alongside `t2d`/`d2t`.

With the attachment-style `EagleSpeculator` removed in vllm-project#672, no model
constructor takes `verifier` anymore, so consuming it is sufficient — no
per-model special-casing needed.

## Tests

- Failing test fixed:
`tests/e2e/smoke/test_finetuning_sanity.py::test_finetuning_weight_sanity`
(covered on CI).
- `make style` / `make quality`: pass.
- `pytest tests/unit/test_model.py
tests/unit/convert/test_entrypoints.py`: 20 passed.

## Checklist

I have filled in:

- [x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x] The test plan/results, such as providing test command and pasting
the results.
- [ ] (Optional) The necessary documentation update.
- [x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

---------

Signed-off-by: shanjiaz <hezhao@redhat.com>
Signed-off-by: Rahul-Tuli <rtuli@redhat.com>
Co-authored-by: shanjiaz <hezhao@redhat.com>
@WindChimeRan @rahul-tuli @shanjiaz
test(mtp): pin packed-batch attention to document-local (vllm-project…
5aa4da9 
…#621) (vllm-project#627)

## Purpose

Regression test for
vllm-project#621. Full
investigation in the issue thread.

MTP builds its mask on the dense `create_causal_mask` path, where
transformers auto-derives the block-diagonal packed mask from the
per-document `position_ids` the collate emits (positions reset per
document, attention_mask=None). So MTP training attention is already
document-local — implicitly via position_ids, not via an explicit
document_ids mask.

This test pins that invariant: a packed batch with per-document
position_ids must yield a causal and same-document mask in the MTP
forward. It fails if positions stop carrying segment boundaries or
packed-mask detection changes — the silent failure mode where loss still
decreases.

## Tests

This PR is test only. 

## Checklist

I have filled in:

- [x] The purpose of the PR, such as "Fix some issue (link existing
issues this PR will resolve)".
- [x] The test plan/results, such as providing test command and pasting
the results.
- [ ] (Optional) The necessary documentation update.
- [x] I (a human) have written or reviewed the code in this pr to the
best of my ability.

---------

Signed-off-by: Ranran Haoran Zhang <ranzhang@redhat.com>
Co-authored-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: shanjiaz <zsjwpianpian@gmail.com>
@orestis-z
Merge branch 'main' into feat/fc-norm
3e6c699
@orestis-z @claude
style: fix ruff formatting in peagle/core.py
fbedd79 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Orestis Zambounis <orestis.zambounis@gmail.com>
@orestis-z orestis-z closed this Jun 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@orestis-z @fynnsu @eldarkurtic @GIREESH7963 @guan404ming @shubhra @shanjiaz @rahul-tuli @WindChimeRan