fix(train): add retrieval replay to SFT workflow; repair broken replay paths - #138
Merged
Conversation
… paths Two problems with the SFT workflow, one measured and one that would have failed the run at launch. Measured: retrieval capability decays during causal-only training. Held-out top-1 on the advanced split fell 42.5% (step 249k peak) -> 34.4% -> 28.8% -> 21.3% (step 392k) across ~143k causal-only steps, with university 31.9% -> 17.2% and school 22.4% -> 12.5%; recall@10 is now falling too, so the embedding is losing documents outright rather than merely ranking them worse. Training loss cannot show this because it only reflects the objective currently running. SFT is five consecutive supervised-generation phases, so it would erode the retriever -- the actual product -- the same way. Four retrieval_representation replay phases are now interleaved after each SFT phase at layer 24 / temperature 0.05, using each pretraining stage's own retrieval shard and geometry: 8.1% of tokens, alongside the existing 15.7% causal foundation replay. The replay dose is a starting point, not a derived optimum. Verify it by running `hermes-train eval --objective contrastive_retrieval` on all three held-out splits before and after SFT and comparing against the 249k peak; if retrieval still regresses, raise the dose rather than accepting it. Broken paths: all four causal replay phases pointed at /opt/hermes-run/moe-300m-v5-optimized-20260731/data/, which does not exist on the training host. The shards live under /opt/hermes-run/moe-300m-v4/data/. Every replay phase would have failed to open its data. Retrieval phase geometry is sized against the validator's worst case of 33 sequences per example (MAX_RETRIEVAL_REPRESENTATION_DOCUMENTS), not the 4 our data actually carries, so the batch and shuffle bounds hold for any negatives count the schema permits. Workflow validates against retriever_300m_moe.mal: 13 phases, 18,100 steps, 915.9M compute tokens.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two problems with
workflow.sft.example.json— one measured, one that would have failed the run at launch.Retrieval decays during causal-only training (measured)
Held-out retrieval top-1 across the current pretraining run:
Advanced MRR 0.541 → 0.325 over ~143k causal-only steps. Recall@10 is now falling too (university 0.656 → 0.553), so the embedding is losing documents outright, not just ranking them worse. Training loss cannot show any of this — it only reflects the objective currently running.
SFT is five consecutive supervised-generation phases, so it would erode the retriever the same way. Four
retrieval_representationreplay phases are now interleaved after each SFT phase at layer 24 / temperature 0.05, each using its own pretraining stage's retrieval shard and geometry: 8.1% of tokens, alongside the existing 15.7% causal foundation replay.The dose is a starting point, not a derived optimum. Verify with
hermes-train eval --objective contrastive_retrievalon all three held-out splits before and after SFT against the 249k peak, and raise it if retrieval still regresses.All four causal replay paths were broken
They pointed at
/opt/hermes-run/moe-300m-v5-optimized-20260731/data/, which does not exist on the training host — the shards live under/opt/hermes-run/moe-300m-v4/data/. Every replay phase would have failed to open its data.Note on geometry
Retrieval phases are sized against the validator's worst case of 33 sequences per example (
MAX_RETRIEVAL_REPRESENTATION_DOCUMENTS), not the 4 our data carries, so batch and shuffle bounds hold for any negatives count the schema permits.Validates against
retriever_300m_moe.mal: 13 phases, 18,100 steps, 915.9M compute tokens.