11L + XSA4 + EMA(0.997) + seq2048 + Int5-MLP + MuonWD=0.04 + LateK-FP16 | val_bpb=1.1361#372

Closed

HyperPotatoNeo wants to merge 3 commits intoopenai:mainfrom

HyperPotatoNeo:submission/2026-03-21_11L_XSA4_EMA097_seq2048_Int5MLP_MuonWD04

Commits on Mar 21, 2026

Add 11L+XSA4+EMA(0.997)+seq2048+Int5MLP+MuonWD=0.04: 1.1357 BPB
HyperPotatoNeo
committed
Add 3-seed results: avg_bpb=1.1361 (seeds 1337, 42, 123)
HyperPotatoNeo
committed
Fix stale single-seed references: update to 3-seed avg (1.1361)
HyperPotatoNeo
committed