[core] first commit#2
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the project documentation and refactors the handling of MoE shared expert keys and rotary positional embeddings (RoPE). Specifically, it removes automatic configuration patching for rotary_interleaved and introduces manual tensor reshaping for Multi-Latent Attention (MLA) within the model logic. Feedback identifies a critical bug where the rotary_interleaved flag must be explicitly disabled after reshaping to prevent incorrect RoPE calculations. Additionally, it is recommended to use explicit model type checks instead of substring matching for better robustness.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request updates MCore-Bridge to version 1.0.0.dev0, introducing architectural refinements and expanded documentation. Key changes include the removal of the processor attribute from ModelConfig to centralize its handling within specific bridge classes, and the addition of a set_random_seed utility for improved reproducibility in distributed settings. The documentation now includes a comprehensive model support list and a guide for using PEFT with LoRA. Technical updates also feature support for multi_latent_attention in rotary embeddings and refined logic for MoE shared experts. I have no feedback to provide.
No description provided.