Skip to content

Add paged MLA/KDA serving for Ling-3.0 - #607

Draft
FENP wants to merge 3 commits into
vllm-project:mainfrom
FENP:bailing_v3_support
Draft

Add paged MLA/KDA serving for Ling-3.0#607
FENP wants to merge 3 commits into
vllm-project:mainfrom
FENP:bailing_v3_support

Conversation

@FENP

@FENP FENP commented Aug 14, 2026

Copy link
Copy Markdown

Note

This is a work-in-progress PR for early design and implementation review.
It is blocked on:

  • a vLLM release containing Bailing V3 and FP8 support;
  • merging MLX-LM PR #1711 and updating the pinned revision.

Please do not merge until these dependencies are available.

Summary

  • Add paged serving support for Ling-3.0 Bailing V3 hybrid models.
  • Support hybrid MLA and KDA layers with paged latent attention and recurrent-state caching.
  • Integrate KDA with the existing hybrid recurrent paged-attention runtime for continuous batching.
  • Restrict model detection to BailingMoeV3ForCausalLM and the expected KDA configuration fields.
  • Document Ling-3.0 in the supported-model matrix.

Dependencies

This change has two upstream prerequisites:

Validation

  • 54 passed, 1 skipped in the existing focused vLLM-Metal test suites.
  • git diff --check passed.
  • The full Ling-3.0 Tiny-FP8 checkpoint was previously served through the paged path using local vLLM and MLX-LM source revisions containing the required Bailing V3 support.
  • The engine detected native FP8 weights, 6 MLA layers, 18 KDA layers, and patched all 24 hybrid layers.
  • Chat API requests for factual answering, multiplication, and translation returned the expected results with thinking disabled.

Scope

  • Supports BF16 and native FP8 Ling-3.0 checkpoints through MLX-LM.
  • MTP weights may be ignored during loading; this change does not execute MTP decoding.

FENP added 3 commits August 14, 2026 10:58
Signed-off-by: FENP <yuanyongjie.yyj@antgroup.com>
Signed-off-by: FENP <yuanyongjie.yyj@antgroup.com>
Signed-off-by: FENP <yuanyongjie.yyj@antgroup.com>
@FENP FENP changed the title Add paged MLA/KDA serving for Ling-3.0 Bailing V3 Add paged MLA/KDA serving for Ling-3.0 Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant