SP8192 + LongCtx NoQV QK5.25 Prefix2750 — 1.05827 BPB (seed 42)

[someone114514]

Summary

Single-seed result: val_bpb=1.05826976 on seed 42.

This PR is a deliberately narrow follow-up to #1953. It keeps the #1953 training, quantization, tokenizer, CaseOps data, long-context eval, no_qv TTT mask, TTT local LR multiplier, and QK gain stack intact, and changes only the phased-TTT prefix schedule:

• PHASED_TTT_PREFIX_DOCS: 2500 -> 2750
• PHASED_TTT_NUM_PHASES: unchanged at 3
• no tokenizer change
• no PPM, n-gram, SLOT, or logit bias
• score-first TTT behavior inherited from Record: PR #1945 base + 2560 long-context + no_qv TTT mask + TTT LR 0.75 + QK_GAIN 5.25 — val_bpb 1.05855 (3-seed mean) #1953

Result

Run	Seed	Prefix docs	Final BPB	Eval time	Total bytes
#1953 seed42 reference	42	2500	1.05824720	430.0s	15,988,861
This PR	42	2750	1.05826976	495.0s	15,978,173

The larger prefix lands within 0.00003 BPB of the #1953 seed42 reference while staying under the 16 MB cap.

Files

• train_gpt.py: Record: PR #1945 base + 2560 long-context + no_qv TTT mask + TTT LR 0.75 + QK_GAIN 5.25 — val_bpb 1.05855 (3-seed mean) #1953-derived script with default PHASED_TTT_PREFIX_DOCS=2750
• download_caseops_data.py: reproducible CaseOps dataset downloader/validator
• train_seed42.log: full seed42 run log
• README.md: setup and run command
• submission.json: seed42 result metadata

Reproduce

The README contains the full 8x GPU command. CaseOps data should be prepared with:

python3 records/track_10min_16mb/2026-04-30_LongCtx_NoQV_QK525_Prefix2750/download_caseops_data.py --local-dir /workspace/caseops_data

[aquariouseworkman]

must beat current SOTA by >or - 0.005 nats

[cocohearts]

Leaderboard audit note (pre-cutoff state): I don't think this is record-ready as submitted. The PR reports only a single seed and no std/p-value evidence, and the seed-42 number is essentially a narrow follow-up around PR #1953 rather than a statistically supported new frontier row. Please rerun a matching 3-seed package if this is intended as a leaderboard claim.