v1.5.2

新特性

1. 新增 model_type 支持：GLM-5.2（glm_moe_dsa 新增 indexer_type = 'shared' 支持）、minicpmv4_6。
2. moe_router_load_balancing_type 支持同时设置多个类型。
3. 修复 DeepSeek-V4 FP8 相关问题。

New Features

1. Added support for new model_type: GLM-5.2 (with indexer_type = 'shared' support added for glm_moe_dsa) and minicpmv4_6.
2. moe_router_load_balancing_type now supports specifying multiple types simultaneously.
3. Fixed issues related to DeepSeek-V4 FP8.

What's Changed

• [compat] MTP compat megatron-core main branch / dev branch by @Jintao-Huang in #127
• [bugfix] regenerate columnwise data after FP8 weight loading by @HorizonChaser in #126
• [bugfix] fix get_logger by @Jintao-Huang in #131
• [compat] moe_router_load_balancing_type compat mcore>=0.16 by @Jintao-Huang in #130
• [bugfix] fix GLM 5.1 FP8 by @Jintao-Huang in #132
• [bugfix] Fix deepseek-v4 fp8 by @Jintao-Huang in #133
• [bugfix] fix glm5.2 indexer_type 'shared' by @Jintao-Huang in #134
• [bugfix] fix deepseek v4 fp8 by @Jintao-Huang in #135
• [bugfix] fix multi-LoRA load assertion failure on idle adapter slots by @Yunnglin in #136
• [model] Add MiniCPM-V 4.6 model support by @randydl in #137
• update minicpm4_6 by @Jintao-Huang in #138

New Contributors

• @HorizonChaser made their first contribution in #126
• @Yunnglin made their first contribution in #136

Full Changelog: v1.5.1...v1.5.2

Patch release v1.5.1

Full Changelog: v1.5.0...v1.5.1

v1.5.0

新特性

1. generative_reranker 任务训练 lm_head 部分显存占用优化，只提取 positive / negative token 位置的 logits 而不是完整的 logits。
2. 不再兼容 megatron-core 0.15。
3. 修复若干 Bugs。

New Features

1. Optimized GPU memory usage for the generative_reranker task during training of the lm_head component, by extracting logits only at positive/negative token positions instead of computing the full logits.
2. Dropped compatibility with megatron-core 0.15.
3. Fixed several bugs.

What's Changed

• optimize generative_reranker memory by @Jintao-Huang in #115
• update mm_gpt_model input_ids by @Jintao-Huang in #117
• Fix NPU LoRA for MindSpeed MoE grouped linear by @addsubmuldiv in #118
• [bugfix] fix embedding sharded_state_dict by @Jintao-Huang in #119
• update mcore requirements by @Jintao-Huang in #120
• update requirements by @Jintao-Huang in #122
• update requirements by @Jintao-Huang in #123
• [bugfix] fix linear_decoupled_in_proj CP by @Jintao-Huang in #124

Full Changelog: v1.4.3...v1.5.0

v1.4.3

新特性

1. 新增 model_type 支持：gemma4_unified；kimi_k25 新增多模态支持。
2. 新增 language_model_only 参数，启用后仅创建语言模型部分，并只加载与保存语言模型相关权重。
3. 修复若干 Bug。

New Features

1. Added model_type support for gemma4_unified; added multimodal support for kimi_k25.
2. Added language_model_only parameter, which when enabled, only creates the language model component and exclusively loads/saves language model weights.
3. Fixed several bugs.

What's Changed

• [bugfix] fix: clamp num_tokens=0 in MTP loss & add normalized scale for MTP per token loss by @YaoweiFan in #104
• [bugfix] fix tie_word_embeddings by @Jintao-Huang in #105
• [bugfix] fix deepseek-v4 dev branch by @Jintao-Huang in #107
• [model] support gemma4_unified by @Jintao-Huang in #108
• update batch_p2p_comm by @Jintao-Huang in #111
• support language_model_only by @Jintao-Huang in #112
• support kimi_k25 mm by @Jintao-Huang in #113
• update mla rope mcore>=0.18 (0.15-0.18 compat) by @Jintao-Huang in #114

New Contributors

• @YaoweiFan made their first contribution in #104

Full Changelog: v1.4.2...v1.4.3

v1.4.2

新特性

1. 新增 model_type 支持：bailing_hybrid。
2. 修复 olmoe/bailing_moe 在TP > 1时的损失异常。

New Features

1. Add model_type support: bailing_hybrid.
2. Fix abnormal loss for olmoe/bailing_moe when TP > 1.

What's Changed

• [bugfix] fix bug by @Jintao-Huang in #99
• [bugfix] fix qwen3_next norm sp by @Jintao-Huang in #100
• [model] Support bailing_hybrid by @Jintao-Huang in #85
• refactor olmoe by @Jintao-Huang in #101
• [bugfix] fix npu GDN by @Jintao-Huang in #103

Full Changelog: v1.4.1...v1.4.2

v1.4.1

中文版

新特性

1. 新增 model_type 支持：gemma4、deepseek_v4。
2. README 新增使用 Mcore-Bridge 创建模型并执行 forward、计算损失的最简示例。
3. 兼容 megatron-core main 与 dev 分支。

English Version

New Features

1. Added model_type support for: gemma4, deepseek_v4.
2. Added a minimal example in README demonstrating how to create a model using Mcore-Bridge to perform forward pass and compute loss.
3. Compatible with both megatron-core main and dev branches.

What's Changed

• [model] Support gemma4 by @Jintao-Huang in #56
• [docs] update readme by @Jintao-Huang in #84
• compat megatron dev branch by @Jintao-Huang in #87
• [model] support gemma4 padding_free by @Jintao-Huang in #88
• [docs] update docs by @Jintao-Huang in #89
• update gemma4 rope by @Jintao-Huang in #90
• refactor MLA by @Jintao-Huang in #91
• compat mtp megatron_core main branch by @Jintao-Huang in #92
• [model] Support deepseek-v4 by @Jintao-Huang in #86
• [bugfix] fix bugs by @Jintao-Huang in #95
• [model] support deepseek v4 mtp by @Jintao-Huang in #93
• Support fp4 blockwise load by @Jintao-Huang in #96
• [bugfix] fix gdn conv1d by @Jintao-Huang in #97
• update lora add by @Jintao-Huang in #98

Full Changelog: v1.4.0...v1.4.1

v1.4.0

中文版

新特性

1. 新增 model_type 支持：bailing_moe、qwen3_asr。
2. 支持 Qwen3-Next 以 Mcore-GDN 方式运行（默认），从而支持序列 packing、FP8 及 CP。
3. 对 transformer_block / transformer_layer 进行重构，通过可继承的方式便于新模型的接入。
4. 兼容 Python 3.13。
5. 支持 transformers 中以 grouped 方式组织专家的 MoE 模型的 LoRA 权重存储与读取。（注意：该 LoRA 权重不支持通过 transformers 直接加载，但可通过 Megatron 加载以用于后续继续训练。）
6. 新增 padding_mask 支持，修复了在 padding_free=False 时，moe_aux_loss 对 padding token 错误计算 routing loss 的问题。

English Version

New Features

1. Added model_type support for bailing_moe and qwen3_asr.
2. Support running Qwen3-Next with Mcore-GDN (default), enabling sequence packing, FP8, and CP.
3. Refactored transformer_block / transformer_layer with an inheritable design to simplify the integration of new models.
4. Added compatibility with Python 3.13.
5. Support LoRA weight saving and loading for MoE models whose experts are organized in grouped mode in transformers. (Note: these LoRA weights cannot be loaded directly via transformers, but can be loaded via Megatron for continued training.)
6. Added padding_mask support, fixing an issue where moe_aux_loss incorrectly computed routing loss on padding tokens when padding_free=False.

What's Changed

• [bugfix] fix MTP & mcore 0.15 (NPU) by @Jintao-Huang in #67
• compat python 3.13 by @Jintao-Huang in #68
• compat lint py313 by @Jintao-Huang in #69
• compat lint py3.13 by @Jintao-Huang in #70
• [model] support bailing by @Jintao-Huang in #55
• update gpt_model by @Jintao-Huang in #71
• refactor transformer_block by @Jintao-Huang in #72
• [bugfix] fix tie_word_embeddings by @Jintao-Huang in #74
• [bugfix] fix qwen3_vl by @Jintao-Huang in #73
• remove hf_grouped lora error by @Jintao-Huang in #75
• [model] support qwen3_next gdn by @Jintao-Huang in #76
• compat megatron.core 0.18 by @Jintao-Huang in #77
• [model] support qwen3_asr by @Jintao-Huang in #78
• Support padding mask by @Jintao-Huang in #79
• compat peft 0.19 by @Jintao-Huang in #80
• [readme] Update readme by @Jintao-Huang in #81
• [docs] update readme by @Jintao-Huang in #82
• [bugfix] fix minimax qk_norm sp by @Jintao-Huang in #83

Full Changelog: v1.3.0...v1.4.0

Patch release v1.3.2

Full Changelog: v1.3.1...v1.3.2

Patch release v1.3.1

Full Changelog: v1.3.0...v1.3.1

v1.3.0

中文版

新特性

1. 新增 model_type 支持：kimi_k25、hy_v3、llava_onevision。
2. mlp_padding_free 兼容 Sequence Parallelism。
3. 移除对 megatron-core 0.12 - 0.14 版本的依赖支持。

English Version

New Features

1. Added model_type support: kimi_k25, hy_v3, llava_onevision.
2. mlp_padding_free is now compatible with Sequence Parallelism.
3. Removed dependency support for megatron-core versions 0.12 - 0.14.

What's Changed

• [docs] update readme by @Jintao-Huang in #49
• update requirements by @Jintao-Huang in #51
• npu qwen3.5 megatron padding_free fix by @addsubmuldiv in #50
• [model] support kimi_k25 by @Jintao-Huang in #52
• [model] support hy_v3 by @Jintao-Huang in #53
• Add support for LLaVA-OneVision-1.5 model by @randydl in #54
• [bugfix] fix torch_dtype by @Jintao-Huang in #57
• fix qwen3_next by @Jintao-Huang in #58
• remove mcore0.12-mcore0.14 by @Jintao-Huang in #59
• fix kwargs by @Jintao-Huang in #61
• [megatron] support mlp_padding_free & sp; refactor TransformerLayer by @Jintao-Huang in #62
• [bugfix] fix gather_from_sp by @Jintao-Huang in #63
• update transformers by @Jintao-Huang in #65
• update requirements by @Jintao-Huang in #66

New Contributors

• @randydl made their first contribution in #54

Full Changelog: v1.2.0...v1.3.0