Add MORI as a Token Dispatcher#135

Merged

sudhu2k merged 65 commits into

sudhu/mori_deepseek_qwen

Jul 9, 2026

Commits on Apr 28, 2026

Add Qwen2 and Qwen3 training scripts and documentation
sudhu2k
committed

Commits on May 4, 2026

Initial commit with MORI
sudhu2k
committed

Commits on May 6, 2026

Merge remote-tracking branch 'origin/rocm_dev' into sudhu/mori_deepseek_qwen
sudhu2k
committed

Commits on May 7, 2026

IFU changes to mori
sudhu2k
committed

Commits on May 8, 2026

Enhance MORI dispatch and combine functionality by ensuring receiver-side global indices are used for backward operations. Updated `MoriDispatch` and `MoriCombine` to save and pass the correct indi…
sudhu2k
committed
Refactor MORI integration by removing `MORI_SHMEM_MODE=vmm` because of bug and adding a helper function to ensure non-empty weights for MORI operations. This change prevents potential issues with …
sudhu2k
committed
Enhance MORI operator management by improving the `get_mori_op` and `reset_mori_op` functions. The `get_mori_op` now ensures a resolved kernel type is used and clarifies the operator's lifecycle in…
sudhu2k
committed

Commits on May 11, 2026

Make MORI EP dispatch/combine sync-free
sudhu2k
committed

Commits on May 12, 2026

Commits on May 19, 2026

Remove initialization of shmem buffers for MORI and make the initialization lazy to call it just before combine or dispatch ops
sudhu2k
committed

Commits on May 20, 2026

Refactor MORI combine and dispatch operations to clarify sender-receiver distinctions. Updated argument naming conventions in `MoriCombine` to include `sender_` and `recv_` prefixes, enhancing code…
sudhu2k
committed

Commits on May 24, 2026

Commits on May 26, 2026

Refactor MORI combine op to use source token indices instead of dispatch_indices.
sudhu2k
committed

Commits on May 28, 2026

Commits on May 29, 2026

Commits on May 31, 2026

fix(moe): isolate MORI Ep state between tests and add shared-op coverage
sudhu2k
committed

Commits on Jun 3, 2026

Commits on Jun 4, 2026

Commits on Jun 5, 2026

Commits on Jun 9, 2026

Commits on Jun 10, 2026

Update Dockerfile to disable UMBP during MORI installation for improved compatibility
sudhu2k
committed

Commits on Jun 12, 2026

Remove unnecessary None argument from MoriCombine function in fused_a2a.py for cleaner code.
sudhu2k
committed

Commits on Jun 14, 2026

Merge branch 'rocm_dev' into sudhu/mori_deepseek_qwen
sudhu2k
committed

Commits on Jun 15, 2026

Commits on Jun 16, 2026

Remove unnecessary destructor from MoEModelTestContainer in test_token_dispatcher.py
sudhu2k
committed

Commits on Jul 1, 2026

Commits on Jul 6, 2026

Address PR comments
sudhu2k
committed

Commits on Jul 7, 2026

Commits on Jul 8, 2026

Refactor MORI EP dispatch configuration in `fused_a2a.py`
sudhu2k
committed