Request: Support for nvidia/MiniMax-M3-NVFP4

[DikiyLifter]

Hi,

I've been running DeepSeek-V4-Flash on my RTX Pro 6000 (96 GB VRAM) + 96 GB RAM with great results. I'd like to request support for nvidia/MiniMax-M3-NVFP4 (https://huggingface.co/nvidia/MiniMax-M3-NVFP4).

Model: 428B total / 23B active, MoE (128 experts, top-4 + 1 shared), 60 layers (3 dense + 57 MoE), NVFP4 checkpoint, ~250 GB on disk.

My hardware: 1× RTX Pro 6000 (96 GB VRAM) + 96 GB RAM.

What needs to be supported:

MSA (MiniMax Sparse Attention) — block-sparse attention with index branch, different from DS4's hybrid attention. Reference: https://github.com/MiniMax-AI/MSA
ViT vision encoder — M3 is natively multimodal.
NVFP4 checkpoint format — quantized via NVIDIA Model Optimizer, different from FP4/FP8 currently supported.
Expert dimensions — different from DS4/GLM, new SASS kernel sharding needed.
Sigmoid router with expert-bias correction (DeepSeek-V3-style routing).
SwiGLU-OAI activation with clamped gate/up + linear offset.
Gemma-style RMSNorm (x * (1 + w)).

I'm happy to test on my hardware once there's any preliminary support. Thanks.

[kmpoaquests]

this would be greatly appreciated , I've got a rtx 6000 pro blackwell + 5090 + 192gb ram ddr5

[lovedheart]

@kmpoaquests This project might be fit for you: https://github.com/guqiong96/Lvllm/releases#release-lvllm-v2.3.4

[HH1162]

Hi DikiyLifter, I have the exact same hardware (RTX Pro 6000 96GB VRAM + 94GB RAM) but I keep hitting OOM when trying to launch DeepSeek-V4-Flash with vLLM-Moet. The planes are 72.6GB and the loading peak seems to exceed available memory.

Could you share your launch configuration? Specifically:

1. Did you use VLLM_MOE_W2_FORCE_RESIDENT=1?
2. What was your free VRAM before launch (nvidia-smi output)?
3. Did you close all other CUDA processes before launch?
4. Any other environment variables or settings that helped?
5. Did you use VLLM_MOE_W2_STORE_DIR for planes cache?

Thanks!