What are the minimum inference-engine / connector version requirements for PD disaggregation?

[0xjasoncao]

When using kthena router for Prefill-Decode disaggregated serving, the router connectors (nixl, mooncake, sglang, http in pkg/kthena-router/connectors/) inject engine/connector-specific parameters into the request body so the prefill and decode workers can coordinate KV-cache transfer.

These injected parameters are tied to a specific version of the inference engine's built-in KV connector. For example:

• NIXL connector injects kv_transfer_params, matching vLLM's built-in NixlConnector.
• SGLang connector injects bootstrap_room / bootstrap_host, matching SGLang's disaggregation bootstrap protocol.

The current PD docs only give concrete image tags in examples (e.g. vllm-openai:v0.10.0-cu128-nixl-v0.4.1-lmcache-0.3.2, sglang:latest) without stating the minimum required versions of the inference engines / built-in connectors.

Question: Does kthena define minimum supported versions for the inference engine and its built-in KV connector when using PD disaggregation? If the engine / connector version is older or newer than the version the router connector was written against, is there any risk of silent degradation (e.g. decode re-computes KV instead of receiving it from prefill)?

It would be great to know the recommended / minimum tested engine version for each PD scenario (vLLM+NIXL, vllm-ascend+Mooncake, SGLang disaggregation).

Environment

• Kthena version: v0.4.0
• Kubernetes version: N/A
• Others: PD disaggregation docs and router connector code

---

[avinxshKD]

Hey thanks for raising this. Short answer: we do not currently have a formally documented minimum engine / connector version matrix for PD disaggregation. That should be added.

What Kthena supports today is effectively tied to the request contract implemented by the router connector and the image versions shown in the PD examples.

Documented example images today:

• vLLM + NIXL: ghcr.io/volcano-sh/vllm-openai:v0.10.0-cu128-nixl-v0.4.1-lmcache-0.3.2
• vLLM Ascend + Mooncake: ghcr.io/volcano-sh/kthena-engine:vllm-ascend_v0.10.1rc1_mooncake_v0.3.5
• SGLang + Mooncake: the docs currently use docker.io/lmsysorg/sglang:latest, which is not ideal. We should replace that with a pinned tested tag before calling it a supported baseline.

The router is not doing generic PD handling here. It depends on engine-specific behavior:

• For nixl / mooncake, the router injects kv_transfer_params, expects the prefill response to return kv_transfer_params, then forwards those params to the decode worker.
• For SGLang, the router uses the bootstrap protocol: same bootstrap_room on prefill/decode, plus bootstrap_host on decode, and both requests need to be in flight at the same time.

So yes, using an older engine-side connector, or a newer one that changes this schema/behavior, can break PD routing.

I think the right follow-up is to add a compatibility table in the PD docs and stop using floating tags such as latest in these examples. Until that exists, the image tags above should be treated as documented examples rather than a broad compatibility guarantee.

cc @hzxuzhonghu @YaoZengzeng

[hzxuzhonghu]

Does kthena define minimum supported versions for the inference engine and its built-in KV connector when using PD disaggregation? If the engine / connector version is older or newer than the version the router connector was written against, is there any risk of silent degradation (e.g. decode re-computes KV instead of receiving it from prefill)?

Good question, from what i understand both vllm and sglang should not break backward compatablity. @avinxshKD 's view is reasonable, we should supplement a compatibility table.

[0xjasoncao]

Thanks @avinxshKD @hzxuzhonghu.

Two questions:

1. Even though the docs still use sglang:latest, which SGLang version was the kthena router written and tested against, so we can pin it on our
   side?

2. Beyond the three PD scenarios above, are there other validated inference-engine versions that the kthena router has been tested against?

[hzxuzhonghu]

Even though the docs still use sglang:latest, which SGLang version was the kthena router written and tested against, so we can pin it on our
side?

The commit showed it was proposed on April

Beyond the three PD scenarios above, are there other validated inference-engine versions that the kthena router has been tested against?

MoonCake and also lmcache connector both tested