Support routed experts replay for vLLM P/D

[S1ro1]

Summary

Adds Prime-RL support for the compact split routed-experts path used by the patched vLLM/router/verifiers stack:

• consumes split prompt/completion routed experts as compact (shape, bytes) payloads only
• removes support for the old nested-list/base85 routed-experts paths
• aligns and pads routed experts fail-fast, allowing only the expected missing final-token entry
• replays experts through trainer MoE layers using the sparse MoE-layer index emitted by vLLM
• pads/truncates routed experts correctly during sample and micro-batch preparation
• exposes inference.routed_experts_replay_max_blocks for vLLM routed-experts replay cache sizing
• retries the W&B shared-mode non-primary init race surfaced by GPU integration CI
• records the required vLLM fork state and wheel build in src/prime_rl/inference/vllm_state.md

Cross-repo PRs:

• vLLM fork: Support routed-experts replay with prefix caching S1ro1/vllm#3
• router: Merge routed experts for vLLM P/D responses router#28
• verifiers: Compose split routed experts from vLLM responses verifiers#1349

Pinned stack:

• verifiers: 162cffb
• vllm: 0.20.2rc1.dev214+g24c0208fc.precompiled
• vLLM wheel: https://github.com/PrimeIntellect-ai/prime-rl/releases/download/v0.5.0/vllm-0.20.2rc1.dev214%2Bg24c0208fc.precompiled-cp312-cp312-linux_x86_64.whl
• vLLM wheel sha256: 437a618dd32400d2636e17de266061aa6685001653e6b9a78751e3ae53036e51
• vllm-router: kept at upstream 0.1.22 with a TODO: update router wheel when ready comment; P/D routed experts require router PR Add qwq #28 once the wheel is published

Local validation:

• uv sync --all-extras
• uv run ruff check .
• uv run ruff format --check .
• PYTEST_OUTPUT_DIR=/tmp/outputs uv run pytest tests/unit -m "not gpu" (342 passed, 65 deselected)
• uv run ruff check src/prime_rl/utils/monitor/wandb.py
• uv run ruff format --check src/prime_rl/utils/monitor/wandb.py

---

Note

Medium Risk
Medium risk because it changes the routed-experts HTTP/transport payload format and replay indexing across multiple trainer models, and pins a custom vllm wheel/verifiers revision that can affect inference behavior.

Overview
Adds end-to-end support for split prompt vs completion routed-experts in the vLLM P/D path by emitting prompt_routed_experts plus per-choice routed_experts from /inference/v1/generate using a compact base64 int16 payload, and removing the chat endpoint’s custom routed-experts capture override.

Switches trainer/orchestrator transport from nested lists to a new RoutedExperts bytes+shape struct (transport/routed_experts.py), with strict alignment/concat/slice/padding helpers applied during trajectory interleaving, batch packing, and tensor materialization.

Updates MoE model router-replay to index routed experts by sparse MoE-layer order (not decoder layer index) and asserts layer-count consistency, and exposes inference.routed_experts_replay_max_blocks for the vLLM replay cache.

Pins the patched stack by updating verifiers and the x86_64 vllm wheel source, and documents the required vLLM fork/build contract in inference/vllm_state.md; unit tests are updated to match the new payload/validation behavior.

^{Reviewed by Cursor Bugbot for commit 7c8adcf. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit cd4ec22. Configure here.}

[cursor]

Missing null check crashes extend_routed_experts on None

High Severity

The condition guarding the extend_routed_experts call was narrowed from checking both tokens.get("routed_experts") is not None and sample.routed_experts is not None to only sample.routed_experts is not None. If the first step has routed experts (making sample.routed_experts non-None) but a subsequent step's tokens["routed_experts"] is None, then step_routed will be None and extend_routed_experts(sample.routed_experts, None, prefix_len) will crash inside validate_routed_experts when accessing None.dtype.

 

^{Reviewed by Cursor Bugbot for commit cd4ec22. Configure here.}