Skip to content

[compat] MTP compat megatron-core main branch / dev branch #127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Jintao-Huang merged 5 commits into from
Jun 17, 2026
Merged

[compat] MTP compat megatron-core main branch / dev branch #127

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
822f0d0
update
Jintao-Huang Jun 17, 2026
d11d3a5
fix
Jintao-Huang Jun 17, 2026
c672d7f
fix
Jintao-Huang Jun 17, 2026
f61cab9
update
Jintao-Huang Jun 17, 2026
1c75cc3
fix
Jintao-Huang Jun 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 18 additions & 6 deletions src/mcore_bridge/model/gpt_model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
logger = get_logger()

mcore_016 = version.parse(megatron.core.__version__) >= version.parse('0.16.0rc0')
mcore_019 = version.parse(megatron.core.__version__) >= version.parse('0.19.0rc0')
Comment thread
Jintao-Huang marked this conversation as resolved.

@gemini-code-assist gemini-code-assist Bot Jun 17, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To avoid importing _compute_mtp_acceptance_counts repeatedly inside the training loop, we can conditionally import it at the module level using a try...except ImportError block. This adheres to PEP 8 guidelines which recommend placing imports at the top of the file.

mcore_019 = version.parse(megatron.core.__version__) >= version.parse('0.19.0rc0')

try:
    from megatron.core.transformer.multi_token_prediction import _compute_mtp_acceptance_counts
except ImportError:
    _compute_mtp_acceptance_counts = None
References
  1. PEP 8 recommends placing imports at the top of the file, just after any module comments and docstrings, and before module globals and constants. (link)



class OutputLayerLinear(TELinear):
Expand Down Expand Up @@ -507,12 +508,23 @@ def _postprocess(self,
if self.training:
mtp_loss_for_log = (
torch.sum(mtp_loss) / num_tokens if num_tokens > 0 else mtp_loss.new_tensor(0.0))
MTPLossLoggingHelper.save_loss_to_tracker(
mtp_loss_for_log,
mtp_layer_number,
self.config.mtp_unroll_steps,
avg_group=parallel_state.get_data_parallel_group(with_context_parallel=True),
)
avg_group = parallel_state.get_data_parallel_group(with_context_parallel=True)
if hasattr(MTPLossLoggingHelper, 'save_metrics_to_tracker'):
# mcore >= 0.19 main branch: save_metrics_to_tracker with correct/total
from megatron.core.transformer.multi_token_prediction import _compute_mtp_acceptance_counts
correct, total = _compute_mtp_acceptance_counts(
mtp_logits, mtp_labels, loss_mask_, output_layer=None, runtime_gather_output=True)
MTPLossLoggingHelper.save_metrics_to_tracker(
mtp_loss_for_log,
correct,
total,
mtp_layer_number,
self.config.mtp_unroll_steps,
avg_group=avg_group)
Comment on lines +512 to +523

@gemini-code-assist gemini-code-assist Bot Jun 17, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Importing _compute_mtp_acceptance_counts inside the training loop (which runs mtp_unroll_steps times per forward pass) introduces unnecessary overhead. By using the module-level imported function, we can avoid this overhead and improve performance.

                    if hasattr(MTPLossLoggingHelper, 'save_metrics_to_tracker') and _compute_mtp_acceptance_counts is not None:
                        # mcore >= 0.19 main branch: save_metrics_to_tracker with correct/total
                        correct, total = _compute_mtp_acceptance_counts(
                            mtp_logits, mtp_labels, loss_mask_, output_layer=None, runtime_gather_output=True)
                        MTPLossLoggingHelper.save_metrics_to_tracker(
                            mtp_loss_for_log,
                            correct,
                            total,
                            mtp_layer_number,
                            self.config.mtp_unroll_steps,
                            avg_group=avg_group)

else:
# mcore < 0.19: original signature
MTPLossLoggingHelper.save_loss_to_tracker(
mtp_loss_for_log, mtp_layer_number, self.config.mtp_unroll_steps, avg_group=avg_group)
mtp_loss_scale = self.config.mtp_loss_scaling_factor / self.config.mtp_unroll_steps
# Clamp to avoid 0/0=NaN when a CP rank's tokens are all rolled out of range.
safe_num_tokens = num_tokens.clamp(min=1)
Expand Down
Loading