Record: SP4096 + 3-Layer Recurrence + GPTQ Embeddings + SDClip + ETLB — val_bpb 1.0913 (3-seed mean)

[bigbag]

Summary

• val_bpb = 1.0913 (3-seed mean, std 0.0012) | ~14.75 MB | 8×H100 SXM
• SP4096 with GPTQ on embeddings, SDClip, 3-layer depth recurrence, eval-time logit bias
• No SLOT, no TTT — fully compliant

3-Seed Results

Seed	Sliding BPP	ETLB BPP	Artifact
42	1.0901	1.0900	14,748,056
314	1.0920	1.0919	14,744,841
999	1.0922	1.0921	14,746,245
Mean	1.0914	1.0913	14,746,381

Key Techniques

1. GPTQ on Embeddings (int8) — PR Record: SP8192 + GPTQ Embeddings + Depth Recurrence + MuonEq-R + SDClip — val_bpb 1.08563 (5 seed mean) #1394 @clarkkev
2. SDClip — std-dev based quantization (PR Record: SP8192 + GPTQ Embeddings + Depth Recurrence + MuonEq-R + SDClip — val_bpb 1.08563 (5 seed mean) #1394)
3. 3-Layer Depth Recurrence (layers 3,4,5) — PR Record: MuonEq-R + 3-Layer Recurrence + WD=0.095 + MLR=0.022 + All-Int6 — val_bpb 1.0900 (3-seed mean) #1331 @dexhunter
4. ETLB: Eval-Time Logit Bias — PR Record: Pre-Quant TTT + ETLB: Eval-Time Logit Bias for Neural Language Model Compression 1.0898 BPB on PR #1285 base #1399 @resouer
5. QK-Gain 5.0 — PR Non Record: MuonEq-R + Context-Only SLOT + QK_GAIN=5.0 — val_bpb 1.1027 (3-seed mean) #1217 @bigbag
6. WD=0.095 + MLR=0.022 — PR Record: MuonEq-R + 3-Layer Recurrence + WD=0.095 + MLR=0.022 + All-Int6 — val_bpb 1.0900 (3-seed mean) #1331 @dexhunter
7. MuonEq-R — PR Record: MuonEq-R + Depth Recurrence + Mixed Int5/Int6 GPTQ — val_bpb 1.0929 (3-seed mean) #1260 @dexhunter
8. LZMA code wrapper — 18KB code saves ~40KB artifact

Credits

PR #1394 @clarkkev, PR #1399 @resouer, PR #1331 @dexhunter, PR #1217 @bigbag, PR #1260 @dexhunter, PR #1204 @msisovic

Test plan

• 3-seed validation (42, 314, 999)
• All artifacts under 16,000,000 bytes
• No SLOT, no TTT
• Seed 42 reproduced twice (1.0900 both times)

🤖 Generated with Claude Code

[bigbag]

Thanks to OpenAI's Advanced Competitor grant ($500 compute credit via RunPod) — this has been extremely helpful for running experiments at scale on 8×H100. We're continuing to iterate with the grant support.

[MatoTeziTanka]

Community Review — Record: SP4096 + 3-Layer Recurrence + GPTQ Embeddings + SDClip + ETLB — val_bpb 1.0913 (3-seed mean)

BPB: 1.0913 | Compliance: LOOKS CLEAN — pure-neural submission, no TTT/SLOT/n-gram-cache

What I found in the code (head SHA 4949b8172b12, file records/track_10min_16mb/2026-04-06_SP4096_3LayerRecur_GPTQ-Embed_SDClip_ETLB/train_gpt.py):

Static code review found no TTT adaptation function, no SLOT optimization loop, no n-gram-cache class, and no pre-quant val-token fine-tune. The eval path uses the standard sliding-window stride-64 pattern. The submission is a pure-neural architecture iteration on the standard SP1024/SP4096/SP8192 baseline.

CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 5.52s, dim=512, layers=11, vocab=8192, code=62133 B, SMOKE_TEST_PASS

Verdict: LOOKS CLEAN.

Recommendation to @cocohearts @valerio-oai @0hq @yuzhougu-oai @notapplica: MERGE pending the usual record-track checks (3-seed validation, under-16MB artifact cap, ≤600s train + ≤600s eval on 8×H100 SXM). No compliance flags from the classification pass — this looks like a clean pure-neural iteration on the standard baseline.

Auto-classification caveat: this review was drafted by the AST-based classifier. If there's a non-standard eval mechanism (logit postprocessing, hedge mixing, etc.) that I missed because it's factored into a helper file or a non-standard function name, please flag it and I'll re-run the audit manually.

---

Reviewed by @MatoTeziTanka — The Agora. CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 5.52s, dim=512, layers=11, vocab=8192, code=62133 B, SMOKE_TEST_PASS. Classification via deterministic AST-based classify_prs.py (pattern bank derived from ~65 manually-reviewed PRs earlier in the 2026-04-11 sweep). This review was auto-drafted from a template and spot-checked before posting — if the template misread your code, please call it out so I can iterate the classifier.