[Frontend][Bugfix] support prefill decode disaggregation on deepseek

[billishyahao]

This patch aims to provides the following:

1. fix for issue [Bug]: 'DeepseekV2Model' object has no attribute 'config' when enabling P/D Disaggregation #14324
2. support prefill decode disaggregation on deepseek

After applying the PR, we got the correct output from 1P1D case on deepseek v2 :

curl -X POST -s http://localhost:8000/v1/completions -H "Content-Type: application/json" -d '{
"model": "deepseek-ai/DeepSeek-V2-Lite",
"prompt": "San Jose is a",
"max_tokens": 50,
"temperature": 0
}'

{"id":"cmpl-e85d4162e19942d5a5f0a31304b31188","object":"text_completion","created":1741965377,"model":"deepseek-ai/DeepSeek-V2-Lite","choices":[{"index":0,"text":" city in the Bay Area of California. It is the largest city in Northern California and the third largest in the state. It is the county seat of Santa Clara County.\nSan Jose is a major center for high-tech industry, with many major","logprobs":null,"finish_reason":"length","stop_reason":null,"prompt_logprobs":null}],"usage":{"prompt_tokens":5,"total_tokens":55,"completion_tokens":50,"prompt_tokens_details":null}}

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[ZhaiFeiyue]

@billishyahao code style check failed, please re-format your code follow the guide here

[LucasWilkinson]

Ideally we should base this on the use_mla flag instead of the presence of kv_lora_rank so that it respects the VLLM_MLA_DISABLE env var

[billishyahao]

Hi @LucasWilkinson Thanks for the comment. According to this, I introduced those flags to make sure we are safe on both VLLM_MLA_DISABLE=0 and VLLM_MLA_DISABLE=1.

self.is_deepseek_mla = config.model_config.is_deepseek_mla
self.use_mla_opt = not envs.VLLM_MLA_DISABLE

[billishyahao]

@billishyahao code style check failed, please re-format your code follow the guide here

Good point! Fixed this. @ZhaiFeiyue

[LucasWilkinson]

LGTM to me from an MLA perspective but im not spun up on prefill decode disaggregation (cc @KuntaiDu )

[billishyahao]

Hi @KuntaiDu , Could you please take a look at the code change? Thanks!

[KuntaiDu]

The logic LGTM, some small suggestions on improving the code readability.

[KuntaiDu]

Would be great if we can move this logic to a separate utility file, and move model-specific logics there, say creating a file called model_aware_kv_ops.py and have something like
put_kv_to_vllm(kv_cache, keys, values, **kwargs)

[KuntaiDu]

I talked to @billishyahao and he/she will follow-up a PR on this, so we're good to go!

[billishyahao]

Thank you! @KuntaiDu for the comments from PD disaggregation design perspective. As we talk offline, I am going to keep function of current patch, and there is another xPyD PR: [Feature][Disaggregated] Support XpYd disaggregated prefill with MooncakeStore by ShangmingCai · Pull Request #12957 · vllm-project/vllm . It introduced new mooncakestore connector, which also need this dispatcher for different models. I plan to make a following up code change to add a unify function put_kv_to_vllm .

[ShangmingCai]

Thank you! @KuntaiDu for the comments from PD disaggregation design perspective. As we talk offline, I am going to keep function of current patch, and there is another xPyD PR: [Feature][Disaggregated] Support XpYd disaggregated prefill with MooncakeStore by ShangmingCai · Pull Request #12957 · vllm-project/vllm . It introduced new mooncakestore connector, which also need this dispatcher for different models. I plan to make a following up code change to add a unify function put_kv_to_vllm .

Sure thing. If this pr gets merged, you can support it for MooncakeStoreConnector too, I will review it.

[billishyahao]

Thank you! @KuntaiDu for the comments from PD disaggregation design perspective. As we talk offline, I am going to keep function of current patch, and there is another xPyD PR: [Feature][Disaggregated] Support XpYd disaggregated prefill with MooncakeStore by ShangmingCai · Pull Request #12957 · vllm-project/vllm . It introduced new mooncakestore connector, which also need this dispatcher for different models. I plan to make a following up code change to add a unify function put_kv_to_vllm .

Sure thing. If this pr gets merged, you can support it for MooncakeStoreConnector too, I will review it.

Sounds good! Thank you @ShangmingCai