Skip to content

Add encoder pooling backend foundation - #608

Open
LxYuan0420 wants to merge 12 commits into
mainfrom
refactor/encoder-pooling-backend
Open

Add encoder pooling backend foundation#608
LxYuan0420 wants to merge 12 commits into
mainfrom
refactor/encoder-pooling-backend

Conversation

@LxYuan0420

Copy link
Copy Markdown
Collaborator

This PR is:

  • To add the encoder pooling runtime path without using decoder KV cache or paged attention.
  • To add a minimal native XLM-R/RoBERTa backbone as the foundation for BGE-M3 pooling.
  • To keep decoder pooling on the existing generation loader while encoder pooling owns its model-family loader.
  • To preserve vLLM’s configured tokenizer and resolved model config contracts.

Note:

This is the PR2 foundation for BGE-M3 support. It does not claim broad XLM-R/RoBERTa model support as a product feature. The local backbone exists because mlx-lm / mlx-vlm do not currently provide a license-compatible encoder implementation for this path.

Follow-up plan:

  • PR3: Add BGE-M3 dense embedding behavior on top of this encoder backend.
  • PR4: Add BGE-M3 sparse token_classify / lexical-weight output.

Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
Signed-off-by: Yuan Lik Xun <lxyuan0420@gmail.com>
@LxYuan0420 LxYuan0420 self-assigned this Aug 14, 2026

@ricky-chaoju ricky-chaoju left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for working on this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants