feat: renderer emits numpy, not torch#18
Merged
Conversation
ebc1f1b to
a2aa322
Compare
HF processors return torch by default; multimodal renderers were passing those tensors through to ``mm_items``, which leaked torch into the renderer's data model and forced downstream transport layers (verifiers) to handle torch tensors despite not declaring torch as a dependency. Switch all three multimodal processors (Qwen3-VL, Qwen3.5/3.6, Kimi-K2.5) to ``return_tensors="np"``. ``mm_items[i]["pixel_values"]`` and friends now ship numpy arrays. Renderer is torch-free; downstream consumers (vLLM-glue helper here in client.py, trainer in prime-rl) convert via ``torch.from_numpy`` / ``torch.as_tensor`` at their boundary where torch is already a hard dependency. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
a2aa322 to
4c78db5
Compare
hallerite
added a commit
that referenced
this pull request
May 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
HF processors return torch by default; multimodal renderers were passing those tensors through to
mm_items, which leaked torch into the renderer's data model and forced downstream transport layers (verifiers — which doesn't declare torch as a dep) to handle torch tensors.Switch all three multimodal processors (Qwen3-VL, Qwen3.5/3.6, Kimi-K2.5) to
return_tensors=\"np\".mm_items[i][\"pixel_values\"]and friends now ship numpy arrays. Renderer is torch-free; downstream consumers (vLLM-glue helper inclient.py, trainer in prime-rl) convert viatorch.from_numpy/torch.as_tensorat their boundary, where torch is already a hard dependency.Why
Architectural cleanup: the renderer shouldn't know about tensorization. numpy is the lowest-common-denominator format that's already a dep of every realistic consumer (HF, torch, vLLM, jax) and is natively serializable by verifiers' msgpack encoder. This lets verifiers drop its torch handling branch (companion PR opens shortly).
What changes
Qwen3VLRenderer._process_image—return_tensors=\"pt\"→\"np\"Qwen35Renderer._process_image— sameKimiK25Renderer._process_image— sameclient._build_qwen_vl_features— wrap numpy items intorch.as_tensorbeforetorch.cat/BatchFeature. Torch is already imported here (the vLLM payload encoder is a torch-required code path).Test plan
input_ids, not pixel arrays)msgpack_encoderand bumps torenderers>=0.1.8renderers>=0.1.8; trainer-sidedecode_tensor_payloadalready handles the encoded wire format identically (numpy/torch source produces the same__torch_tensor__-tagged payload)