feat: renderer-only multimodal path — rip MITO branch, pack pixel_values from renderer

[hallerite]

Summary

VLM RL now goes through the renderer path exclusively. The renderer owns the processor, ships byte-identical pixel_values to vLLM (via /inference/v1/generate multi_modal_data) and to the trainer (via mm_kwargs). Net: -1112 LOC, no more legacy chat-completions / MITO image cache.

Companion PRs:

• renderers: PrimeIntellect-ai/renderers#17 — image support for Qwen3-VL, Qwen3.5/3.6, Kimi-K2.5
• verifiers: PrimeIntellect-ai/verifiers#1346 — RendererClient threads multi_modal_data through the rollout + transport layer

Commits

1. a479a028 feat: drop use_renderer=True VLM skip; pack pixel_values from renderer

• interleave_rollout consumes the renderer-emitted multi_modal_data on each trajectory step (when present), packs per-image pixel_values + image_grid_thw onto the TrainingSample, and computes mm_token_type_ids — no VLMImageCache lookup required when the rollout came from a multimodal renderer
• VLMImageCache remains as the fallback for MITO / chat-completions rollouts
• Orchestrator config: drop the validate_renderer_vs_vlm validator that previously rejected use_renderer=True for VLMs (it's now supported)
• e2e test: real Qwen3VLRenderer + RendererClient → mocked /inference/v1/generate POST, plus a roundtrip through vLLM's GenerateRequest pydantic model + decode_mm_kwargs_item. Strongest end-to-end check we can run without a GPU
• A/B configs for color-codeword (feat-renderer vs main-mito), logging to W&B project multimodal-renderer

2. b385fbf2 refactor(orchestrator): rip MITO multimodal path, renderer-only for VLMs

• Delete the MITO chat-completions multimodal branch from the orchestrator and the ~370 lines of image-cache/preprocess machinery in trajectories.py that supported it
• Renderer-shipped mm_token_type_ids: orchestrator reads renderer.mm_token_type_id_map (1 = image_pad, 2 = video_pad) and stamps a per-token list onto each TrainingSample. Trainer's _get_qwen3_vl_mm_token_type_ids auto-path remains as a fallback but the renderer is now the source of truth
• forward() now takes a generic mm_kwargs: dict (e.g. {pixel_values, image_grid_thw}) instead of the Qwen3-VL-specific (pixel_values, image_grid_thw) keyword pair, so adding new VLM families (Gemma3, LLaVA, etc.) doesn't require touching forward
• Config validator: orchestrator.use_renderer must be true when model.vlm is set — fail at config-load instead of producing cryptic runtime errors
• Test cleanup: drop 25 tests for removed helpers (VLMImageCache, _extract_images, etc.); update the one remaining renderer-mm trajectory test to pass mm_token_type_ids_mapping directly

Net diff

12 files changed, 666 insertions(+), 1778 deletions(-)

Deps

The companion verifiers + renderers PRs need to land first. Once merged:

• bump verifiers rev in pyproject.toml to the merged commit
• bump renderers to the next release that contains PrimeIntellect-ai/renderers#17

Test plan

• Existing orchestrator unit tests pass (126 + 2 new)
• e2e test exercises real Qwen3VLRenderer → /inference/v1/generate payload roundtrip
• A/B run against MITO baseline on color-codeword for ≥20 steps with reward / KL parity
• No regressions on text-only RL (the renderer/orchestrator paths are dispatched on model.vlm)
• Trainer accepts the generic mm_kwargs dict for both image and video families

🤖 Generated with Claude Code

---

Note

High Risk
High risk because it rewires the multimodal RL data path end-to-end (orchestrator tokenization/packing, transport serialization, and trainer forward inputs) and removes the previous MITO/VLM image-cache fallback, so any mismatch can break VLM training or silently corrupt model inputs.

Overview
VLM RL now requires the renderer path: configs set use_renderer=true/use_token_client=false, and OrchestratorConfig enforces renderer usage when model.vlm is present while removing the old “renderer unsupported for VLMs” restriction.

Removes the legacy MITO/VLM preprocessing stack (orchestrator-side AutoProcessor, image cache building, and related helpers/tests) and instead consumes renderer-emitted multi_modal_data on trajectory steps, packing it into model-agnostic mm_kwargs plus explicit mm_token_type_ids derived from renderer token IDs.

Trainer/transport become model-agnostic for multimodal inputs: replaces Qwen3-VL-specific pixel_values/image_grid_thw fields with a serialized mm_kwargs dict (EncodedTensor values), updates batch loading to decode and move these tensors to CUDA, and updates forward() to **-unpack mm_kwargs (with a small Qwen-VL MRoPE special-case).

Adds a CPU-only integration test that round-trips a real Qwen3VLRenderer + RendererClient features payload through vLLM request parsing/decoding, and introduces a dedicated renderer A/B config for color-codeword. Dependency sources are updated to renderer/verifiers feature branches and renderers is unpinned.

^{Reviewed by Cursor Bugbot for commit bf26a06. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit b385fbf. Configure here.}

[cursor]

Renderer-shipped mm_token_type_ids silently ignored for Qwen3-VL

High Severity

The precedence of mm_token_type_ids in forward() is inverted relative to the stated design. The auto-computed _get_qwen3_vl_mm_token_type_ids is preferred over the renderer-shipped mm_token_type_ids parameter. For Qwen3-VL, the auto function always returns non-None (a tensor of zeros filled with image/video markers), so the explicitly shipped mm_token_type_ids from the orchestrator is never used. The PR description says "the renderer is now the source of truth" and the comment in interleave_rollout notes the orchestrator-shipped explicit list "produces ~30x lower mismatch KL," yet forward() discards it.

Additional Locations (1)

• src/prime_rl/orchestrator/trajectories.py#L363-L374

 

^{Reviewed by Cursor Bugbot for commit b385fbf. Configure here.}

[cursor]

Breaking VLM config change missing from CHANGELOG

Medium Severity

The new vlm_requires_renderer validator makes orchestrator.use_renderer = true mandatory when model.vlm is set. Previously, VLM configs used use_renderer = false (the default). Any existing VLM config will now fail at load time with a ValueError. This is a breaking configuration change (effectively a removed valid-config combination) that requires a CHANGELOG.md entry per project rules.

 

^{Triggered by project rule: BugBot Instructions}

^{Reviewed by Cursor Bugbot for commit b385fbf. Configure here.}