Require effective GRPO updates

[bingran-you]

Summary

• align the production Qwen3.5 recipe with TRL's official eight-generation default
• record the exact GRPO software/code/config recipe and invalidate stale checkpoints on any drift
• require complete reward groups, nonzero within-group reward variance, finite training loss, and finite nonzero LoRA-B updates before publishing a GRPO adapter
• preserve detailed reward/update/trainer diagnostics and expose the effective-update result in score and leaderboard records
• allow monotonic GRPO sampling upgrades on resume while restarting only GRPO and downstream evaluation from the preserved SFT checkpoint
• make SFT/GRPO dry-runs non-mutating and cascade artifact invalidation when a stale SFT or GRPO checkpoint is retrained

Evidence

The previous 16-task live run produced 32 rollouts in two-generation groups. Every group had identical binary rewards, train_loss=0, and all 248 GRPO LoRA-B tensors remained exactly zero. This change makes that state fail closed and increases production groups to eight candidates.

Verification

• pytest pipelines/benchflow-task-posttrain/tests -q (237 passed)
• ruff check pipelines/benchflow-task-posttrain/src pipelines/benchflow-task-posttrain/tests
• ruff format --check on all changed Python files
• python -m compileall -q pipelines/benchflow-task-posttrain/src
• git diff --check
• independent and adversarial review passes; all actionable findings fixed

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6d3e45fb10

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Avoid loading train-only deps during resume checks

When resuming or dry-running a run that already has a GRPO train_metrics.json, this new checkpoint-current predicate calls grpo_training_recipe(), whose version fields raise if optional training packages such as trl, peft, transformers, or torch are not installed. Because the base package keeps those dependencies optional, a resume that could otherwise reuse the saved GRPO checkpoint, or at least dry-run the restart, now aborts during the freshness check before reaching either path; consider treating an unavailable current recipe as non-current or only resolving package versions inside actual GRPO training.

Useful? React with 👍 / 👎.