Skip to content

[core] first commit#2

Merged
Jintao-Huang merged 8 commits into
mainfrom
dev2
Mar 30, 2026
Merged

[core] first commit#2
Jintao-Huang merged 8 commits into
mainfrom
dev2

Conversation

@Jintao-Huang

Copy link
Copy Markdown
Collaborator

No description provided.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the project documentation and refactors the handling of MoE shared expert keys and rotary positional embeddings (RoPE). Specifically, it removes automatic configuration patching for rotary_interleaved and introduces manual tensor reshaping for Multi-Latent Attention (MLA) within the model logic. Feedback identifies a critical bug where the rotary_interleaved flag must be explicitly disabled after reshaping to prevent incorrect RoPE calculations. Additionally, it is recommended to use explicit model type checks instead of substring matching for better robustness.

Comment thread src/mcore_bridge/model/gpt_model.py
Comment thread src/mcore_bridge/bridge/gpt_bridge.py
@Jintao-Huang

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates MCore-Bridge to version 1.0.0.dev0, introducing architectural refinements and expanded documentation. Key changes include the removal of the processor attribute from ModelConfig to centralize its handling within specific bridge classes, and the addition of a set_random_seed utility for improved reproducibility in distributed settings. The documentation now includes a comprehensive model support list and a guide for using PEFT with LoRA. Technical updates also feature support for multi_latent_attention in rotary embeddings and refined logic for MoE shared experts. I have no feedback to provide.

@Jintao-Huang Jintao-Huang merged commit 06818b3 into main Mar 30, 2026
2 checks passed
@Jintao-Huang Jintao-Huang deleted the dev2 branch March 30, 2026 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant