diff --git a/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/README.md b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/README.md
new file mode 100644
index 0000000000..f5e680ebd9
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/README.md
@@ -0,0 +1,65 @@
+# Record: QK-Gain 5.5 + SP8192 + 3-Layer Recurrence + Parallel Residuals + Legal TTT
+
+**val_bpb = 1.0810** (3-seed mean, std 0.0005) | **< 16 MB** | 8xH100 SXM
+
+## 3-Seed Results
+
+| Seed | **TTT BPB** | Artifact |
+|------|-------------|----------|
+| 42   | **1.0804**  | 15,994,470 |
+| 314  | **1.0812**  | 15,993,777 |
+| 999  | **1.0814**  | 15,991,277 |
+| **Mean** | **1.0810** | |
+| **Std** | **0.0005** | |
+
+## Key Change
+
+**QK_GAIN_INIT=5.5** (up from 5.25). The monotonic improvement trend in query-key gain scaling continues past 5.25. This extends the finding from PR #1394 (@clarkkev) which documented improvement from 4.0 to 5.25.
+
+## Base Architecture
+
+Built on the SOTA foundation from:
+- **@clarkkev** — SP8192 + GPTQ SDClip + MuonEq-R + depth recurrence (PR #1394)
+- **@dexhunter** — 3-layer depth recurrence (PR #1331, #1437), legal TTT on SP8192 (PR #1413)
+- **@abaybektursun** — Score-first TTT framework (PR #549)
+- **@Robby955** — Parallel residuals on SP8192 (PR #1412)
+- **@msisovic** — Parallel residuals concept (PR #1204)
+- **@X-Abhishek-X** — Hyperparameter tuning (PR #1445, #1471)
+
+## Architecture
+
+11L x 512d x 8H / 4KV, MLP 4x, LeakyReLU(0.5)^2, Partial RoPE (16/64 dims), layerwise LN scale, tied embeddings, logit softcap=30.0. Depth recurrence: layers 3-5 loop (num_loops=2, activated at frac=0.35). Parallel residuals from layer 7. Skip gates (sigmoid-gated U-Net connections). XSA on all layers.
+
+## Training
+
+~4600 steps in ~588s on 8xH100 SXM. EMA decay 0.9965. Warmdown frac 0.72. WD=0.095. MuonEq-R (row-normalized, Newton-Schulz 5 steps).
+
+## Quantization
+
+Full-Hessian GPTQ with SDClip: int6 for attention/MLP matrices, int8 for token embeddings. Brotli-11 compression.
+
+## TTT (Test-Time Training)
+
+Legal score-first TTT: SGD (lr=0.005, momentum=0.9), 3 epochs per 32K-token chunk, cosine LR decay. Each chunk scored under `torch.no_grad()` before any SGD update. Each token scored exactly once.
+
+## Compliance
+
+Per Issue #1017 (Track B — legal eval-time adaptation):
+- Condition 1 (Causality): Sliding-window eval is strictly causal
+- Condition 2 (Normalized distribution): Standard softmax over full vocab
+- Condition 3 (Score before update): Each chunk scored under torch.no_grad() before SGD update
+- Condition 4 (Single pass): Each token scored exactly once
+- All artifacts under 16,000,000 bytes on all 3 seeds
+- Training under 600s on all 3 seeds (~588s actual)
+- Eval (sliding + TTT) under 600s on all 3 seeds
+
+## Reproduction
+
+```bash
+pip install brotli sentencepiece
+pip install flash_attn_3 --no-deps --find-links https://windreamer.github.io/flash-attention3-wheels/cu128_torch291/
+MATCHED_FINEWEB_REPO_ID=kevclark/parameter-golf python3 data/cached_challenge_fineweb.py --variant sp8192
+
+SEED=42 QK_GAIN_INIT=5.5 TTT_ENABLED=1 TTT_LR=0.005 TTT_EPOCHS=3 \
+  torchrun --standalone --nproc_per_node=8 train_gpt.py
+```
diff --git a/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/submission.json b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/submission.json
new file mode 100644
index 0000000000..ea29f72584
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/submission.json
@@ -0,0 +1,15 @@
+{
+  "val_bpb_mean": 1.08102,
+  "val_bpb_std": 0.00051,
+  "seeds": {
+    "42": {"val_bpb": 1.08044, "artifact_bytes": 15994470},
+    "314": {"val_bpb": 1.08120, "artifact_bytes": 15993777},
+    "999": {"val_bpb": 1.08141, "artifact_bytes": 15991277}
+  },
+  "hardware": "8xH100 80GB SXM",
+  "training_time_seconds": 588,
+  "eval_method": "sliding_window + legal_ttt",
+  "key_change": "QK_GAIN_INIT=5.5 (up from 5.25)",
+  "base": "SP8192 + 3-Layer Recurrence + Parallel Residuals + Legal TTT (PR #1394, #1331, #1412, #549)",
+  "author": "G3sparky (Gavin Saunders)"
+}
diff --git a/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_gpt.py b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_gpt.py
new file mode 100644
index 0000000000..1201aa4ffa
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_gpt.py
@@ -0,0 +1,2 @@
+import lzma as L,base64 as B
+exec(L.decompress(B.b85decode(";Jw{5fL#DIn@VT6Qap3bt~@<3h>ok~)Km^%<bI~`7~^@P9dNt*OmJtouSV|m@^}~LJOY#qcoM@9BaGiY8ypPvdJq=NbK}E`t%**OWHq5Yg*w`LcO2`&Ki8P4h{e@}LyJM^e5fwE@n-bff1Ph*mUliW#yCOV*=I+v<n|QZ$y02ziZN=i)3}Qx?Dtm=+{LAgGTC@~>c^ys%R{D_%yAk9-_tV7^coUOo3$w>`(`ci)t`2F7>r>Ltx>>S2CRw|7ov>Wn1e~_!RLQ=%V9g?)G3yPsu%SBy!lj1PaC-x%dDmCDOZ^r^!)+WWz}ejKXTJ#^U6Ra!};QocHHXQC+4UM!QQ!-N5Xd|%~a(9)bTYIO+>B~8~@lqmri%^qEkQUy074Rh6w7V_#^s9J-3BNA`G;qyR$LYcI?e+loZVWi~B$n=TKFp{%SeHYp{oNWh;U@Ahk8M2$OU%K8B$lb*dRQXd-GR_@*KAZdRdwSd#v=LSq1v@Puul=a7WXDmh1^kBj}Y2XlER!D2E{&{%lV(hz$#n5%+%sk&Q}>{y0xpRgiQQBJeVV0hy8UD3ntyo@(Pv+K7^zVRDt4bah<BW;bd+Cav&<Xg-~v4E$T<1_zFr^A}Ais)|vX*pO`3FxNShvEi59luzir1VmzF1ek|5CGkl?5qc}ko)L1|6lu6I`m+*bf%)mk)RQ$_JmfE%eS#TV<mEIvI}8;<$6J^=w~cJs5Q7=n#vHa4v1R&1_9=;Q2@QfQORDBW6CkyC4%tXHv%r3D5rF;6FZ>(r8kfsZThb+H1)~K-lIr4`|V#-2R>G7pP*N!fwWd&Dq8C)y=NrG_U_Oz6Q?+@ok1?(VJ5?ZT~&}C4Ks38WRB>3i=I!}H-8qq=&yKJ;tbpwwn~lAseD^q1C*u5T;l<jmq^*|gW0-BBpM5(&T>KQtF;?zv@u0f36%6SXU~txi3v5iSPK*`fNE9531KaQDL`zTPF$MX4U(-3sY-&?>QJe)giBQzpor7H)AZ#4=Hn#`AoAL7tT){&bw(fgz|eQRt`#6-<>;m*+&$!nf|od6&lVKYYHuOoNgZU_L>E@!O%__mlt=);Hwdc43+CM?sh5y+my3XSVYMO8F1pXuq$fvTU<$mpDjr>Lm){DeV)>4AKAhA?jxjH<-3yYQ<QU|mI~;=9eDwNwPw;)v(Mu^%YrcdcwEH`*s$RtDvffGRv||Tctzy{1@`75RzWmpHgSnSab&b&Fyos-0wdom9#6ey3CnhkwfuHfVJc+7~(p~gN6Q?zV1K>#5qz+4c`Utifny+Ydmr4?c_z60#9@FU+U1&O$Lfg$WrX7gCj50O1t`1A`k04LVr;^*~{|@(TS5>#TAjL(B`umc8bVA$bS|F?^2A7E}z7IIgZlY(8Ex#K+nLh0vzlKK=74U!g+sX4T?e3_^_7XB1A(HB{pYd{vHYcak_P3DZ2LAB20wAP+C_9p7R|0}wA=p~JFi&xD8H}n(LxCc5rcmw<AY&of601+p3__P4;S)9wD!!2(*apEWmyI^3Nlf>F`!s(tSf_<O3Y4@Lb^pR}8!JWi5F-x7_`)oM#pVlXHEPzi0_w{=OcHSWtLTLJZ%v~xfE-VLIhk+V3}!ek?mRs^iiZfQr<Cz8wcNzPD3j5B{~Al3+P&zpBEN%rOTMb?hCP$lA?OeffC?6d+83}cL8pBK3e!YNJ~9kmX-Ex)_GbEil+L!j=<aZeLrs`Qx2fCt`V(`tk{k`tP!^_a&cJLbp-5%L!|FGej~o0jPXI6XZYF*KMbV3}lTx@Ln<Lua-O#Rd6MFf|hg{hiGOq{aNC;vw8&q5AhpiiicR}mCe^oxpwpei+d|T&qL>l_TXk(cPZJ_z`)iV4#r^gzawYQ%HE1iaUF=(KAcKXE%%6Hx0i;?;p1w#dN7!-y!(2GUw()t4|BXt%+05bu$y<wAnZSUY=-zn2daV7XQ!FRQ&E39a_e-evNQeH$%?k$j{<f2ReW`))eJS=r+^AO%o{S-|mcA#-J)~|Na8W7N&x7Y%qc}W|PQU3N_33v}9rHTp1gPXKFiZ=ghykdcGL*?F!Cgp^gGOHsjlG6z&`N$_rfTirGgHnbxUmK`ZYAYjsz^O;`V+8FnR!3RjxgtUMj}@iffiyK*40=kBdOKbvmxC&OtL?AD%tI-f#Ue!e@rkDfVbz!EEW?KyfWIX?60i{``J&VX2%Z3u0DFujf-mL7MLYzyLu&oz@SWnnB8as3Q2~X{m^08~{^CZm!I=WZQ(|Ri`w4QD6oqUIp&ghyb1r3)XQIiwz*99LjnA%97e+<(Dy<%Le~b6b^n_z}O<Z)T6;z(P@UO8JGf2q~fNKR$4R3eV1RzaSj7^{@Y+^a&@GhCY(7b`{jL2ry%;_^Q!l>ea+{f!deuk%(g-o}&XEEWm!lO+1^On#i#4rhP{bDYb9ZnbGd5n{*P->hZxI<{=3c-92I#g*mTey8O>cuw%hdwjB!#=GH_0?hY|Lf@L$0Qp+k03PROh)o-cMOQ!b>qfvPNJLTVvCBX24BGgI=_|35Bd%&Vq&!LWECF4!1&J?@uRfe=N2mi{l-S0aW101I*cY_A&2R~zfij0IZST;@;xJYti>{)weL@A2ZOGrq(U-ibnWz0BL;s!=S<i~82|F>;`!K6@M501z-dU(OqY0?!!!Tk6Z{9!iH*tDZsjYG`J8UEqJz~7cNEPh1A#iz_$2*Nxo;S{UehRA^{Mf3GV^9hBEKSL(iD!=aKpjYCTqmhYA|h4zASL-v?9UWx8tzm#N5eHo2h3w*`(kHM2e}vDviz3$~a(Y#jlJL?*}m9&(Oqs1+CNUy7g~Z#lRN#>g9{7u~tot;|0qTu4G4xwdXk3eT+l1l$)Vq%}j^^1b(jIvF|OcNb1Jz&)>b)qGiC5P7yS}AvC}VDK<woZNv4t;T(e%=S+yy9Wl!Mk?)7e{+73*u$1ks?HX%Es<Jb3p=`hTp!8!8l%0)~O(9avW&Tp9s8lf<<!52RHDrrWVny4z`ftyF2ZJ6}qhUWU$@+y3>ORD$#^Ydjg!zDuM#$J+k<}O|o#9dvGrp)*yShv3->joMiF%~orV4^0cl9F!@VqwD`5f<QPLctdikXXI0rQ$mAAaeO)6}Zwp1jH%fdXVoxoZ`z5lV>jekV@3E`STlX!=JDxbOQiv?Jp)$Xy|Z>g)@q_QYKopPeu&ghhPNw0y&{j?$GHwDQoztHvVU)a0ca6}7{#3^KK1uTcBkMSF$IDQp#Nhy>JTHPK2w+%N#FZ(D)=sw?BkfduBK{Owa(SkBq^_S*|NP@DI(VEWNqETjYsZ@bch5-dlWjP>|)xv+AknhsqQ!j3!=TT%CYvl>o#XU4AApoVBJ;db<uv@;Zle>=W0m0#FH7by8-Z*V~$!QptJuPqLkH-bA#`L?*g#-60qO9x7)rWh{~YY77{NX_v_!Mc-#`(n{>OHy|HxotTLyFAdqCe^bQsveKyNdxf%^ECJD<Wa?}FX&QnOVoQL$TZ#&f##R{pUuK(_{#}CVvVm7@+&m{N8R2nI$9!ALHZPv*Rk%b#hP48Q>w1jQaV{3doP1nC-IuYoJBS)BjwI+@*WRZpBQq--|WAHx2WWVue@lE`*T9AY1=3wKyIT}9Ss;d##=nZ>!%19!lx_0W91se3dXzq5oIE}=)Lf4xkby*McKg=z&Qh>gK5l~kV<Y3G|XD^O|uI3K$}GY)tl)92}xD#u_|>7t^lQK_s8TXQqiwiAz+UT7qOmRWvI~~2y<G+o%s+*8rsYGCml^rm)<LgsXmv>t~5_%swZRW#&SB~-uWRFRPsi5WZAzJp1&o@T%9?d1EUEpyP1v5zSN9`Nzg4<9J>%D?ZP~-T(dwGEKqZMPuhgN<|K<!knseUP=32Qh=k%@&WeMpwkJyP5WK?vq@D7&kY*+seMo3rb+KxP@Zkyra8qm)C>igW>x{H0%t_&ya;8v^0F=-)sK*R85LA|5>|ZYqA#XmVbFc92&H9WjeO5C!FX%LsiXg6}0#l(Pg(=6hjd7H_7$6NLIA+rCa;GE_3R75D#&J(7=>z|LN$87?M}UpavJo5jeYlJy>3UxeQ{duojamIjZmWv|*!Tjr5rT-K7C#w~_vZ!oIz>O;(D%nYcBK)`IjO`=SDZo*4vJ4V2bFcFg(2@0lt|4uUCPbO&N6^dv4y}sPBwT(0$|M|*?y;Jv@#8^JCr!hxD0=c#R8ALJkOUZ5;?_TS5GI^kyB>q;{eo<-=N|;JU?~G80$0+y}Bn>nRaoX5bq_lK8&2G2D0K(N6U_xX}HirikYywzHoCpo)+j^d}t`9sXluV$o6?ewHe5Ui+m5Y9oyhGHXI2OTu~#~ow24E&_|NZmvkjEEo{?lrj>I+3}kwNN$<|WFHD7&hT*J`96C?gGpoC>Df4aU8P&s$90m&Ugy{5AY@?hVDTc{$QAjsoHqS{ck6snhl_)o^474tl{Idqaq1M4gTm}}RWDGq`oxuuW;}<=ge*lXX+3Fk7GOExz3(~A%nd`>nKrLxi-U((%)yeb9`|*{}XN<!aZ;K4$uq0Oaz(HwoaYiOebn91@RynR!R8j7s;7Hra4}H!6j$8JPBT!r&nU1}{`9xQ4xJ`pVq(u7*0xR)<BH-?0@E)`61eEbr2+TPRO!dQhz6=IyRVaDDLEI15s(e*LMiSoTYZ)%g@+%W997F?^5x*mgOyrJBo>04z>c7Ok#4FmP|M+baT*Fuu4_Vg~%D*h%0S&xmIOVhF)nXWYKj;F_kSpCA=E?|IY3TP731i4AmJ9{uIj}nvkZA~PCqrx=X%FiI(pX*UhQoq9-g$`cDVZp7ZcaGt$}M)fe&uhR9-a*yxklx#6Au8ICI}z@KWrk3Obx%(^tG4C1D@?bgq2jBnZ(O?j&jyR!8J6j%~%zfEtIAPDKu$5hd4V~`Wco06Vlpvp}HuyKK}fz6t3mD@K4unNqV2DlC&6KFhx<%Ai%h1TodO4gAJ51=G}q7(kO9N(i4X$<eW-!*R9YA;7)@J*L9oP=fK!D6|+fcPENd)?Uh;;D^?V=^lDx@B&4gKK``fb35<plFL>8MWnd^UE!-v`^1{5*9tbIpjgUqjwfygmDW~97EWp8~n;>1xM9pw3FxHUd_tgGkMlT)SNNWx(w9cg>9zUZzX{Q^w*d#U194JF-wwlAz0yZ)tH+4r_JXO{=3ej6LQd~!naPgtZ;`)wmc&a4dppCSVTiQcc8UfugJge8Gerop`ck3K6;T*^kZp<9Hf-jlYfu+Ncv+`5i&JOwCg|NE;{ox`VY;Ub%E~4C%G@CLJi@0X_co$N5u2iD|I6taml4pwulJ)p!O69folAr>W$wc_@9v9VvU6N+O#a=gX;BimpVU*u)q2XscX=lx^Zn-BN-;_}UkjSaA)uW&+!Y^2T_W8ydgx(`~YE%eEmgcn-wY0id8}v=bSq#xRdSY89OVSts;`59NKaqnBJzq!px7xcxQkwC!%nzq;ACIq57xa9CNC~WPR=_N|sDZ*mo*d={3!J1&)iMAr6Ji^XPO(S$Grm|<fw5+?L~eAHeAbwmOWw>U4{YYnUtbS1j`5tgw6*^XZGHcn{g#{1)Vn2E4|r#ek2MWQ9#>rzQ3zDgm9txa-#PSX0d6dM`yebz%Twh(oT!pmA+0Mo^hZ(f*S=n@m;h!I4LZqoenOsP%;j#RNdY2pvZFx4)uHo@w&tRGaMg#!x08$i%Dckl;@(Bn`nq>l-xa+7FYu&~4Y<w4YkQu%DC`x>q@Vdt+c}5e4r$M9h_8xqG>aHp&?q8)HWa0Sz{jdpt`RF2TQ#ak`F0Ku`0Z2b&|K}-j9!Ag!ByLopEz?X#Iz-hH-XI{L8;k+U1StEfZI9`UEgh8v|1fNGF!KlPDs)l0lp4YCF?m_QdcJRn{uyNT<tv(XvN3Q$usCUQMQPBAr(r!9GoxzDxD$8giB^ipm=QSOMrmo2|~`WMQD6#wk8V{@d4E^0=<9y1;)!H0-b%jS!&}s$c`c%#yqM0P6bK(eh4|=KIQ1#uuTTdW8X=~5cnI1Xg{T-P}=H_!t(4`KrHAMMa}8a^T=5L+o^Q+-&a;8iKn|MVWYVH!K>lYhoL&(q5rP2Z9SU>2tYy;UDp^wa%Fg~Hhk(2S`|te$Gx+Ii0m3i){{7ug{x+aM|ib=hj3X>-y&H^d1>Yk&CE~y^DBjF0EgaKtEaKI7Wky8h&Y@UJ7P7{NwNv$D(&at=BZzc4W-VudsUAV!}qh{Bx~xqASW>QiwS&Pp?nj~MhW!&p&ZK!3XYPt|0r=y$NiXqT5r|a3t~awh`ELbbKti#lGw4_Z}(|4CO9mS3kjLy9es(hZKys_D+DbXrMuaf<R7ECuCc``XbNXj>)dQf+fd=>LNl`QmMe|5HAZu3&M|Qc$e&*iiiM1kpmZL3f%xon_T*7vFZe9~Mf+S+JioV)e9`9pu;^&wln&a4*Ffme0YR3eyDzaPl4eJOEcxo}+_<$hvv>BL)>?w7w?zq|h-nRo`f!5Z^Z0EYf3QZs73GD`>p#=LV@%sLW3~V7UBob6G;zP|6$TYwVbdcqN#-p-y330KXa?YLk+7|%B|s655R+FA)bcA;K2bp!effB5mS5M{$`4RUF)D-}S2zDLqE^vwi-W%}HG7E`-9#iZY98C2*wag^L8FGhTg>4~1t<927~iA%U4&gdRuPFnqSz#Pp^sW|GtL`42h!3vJLPs{Yk+FgK!+g~a3qqe(;LMP&qfS#hT%mKhNhcUPsWZ`K`DcL?sHamF=Hhu*!?d1u{s#h*t!VqsV3y_mGfSdb^JScFk8jUdO?g(o(2gC%_^uh1$<-K*o~`R)Y+;0K`JnLDY6s#;ZWB1vt<!42|CAdo@qaF&WNqL(1q`9x@WZs4=&f^2FP5u!K#I<Vv3;TVszC!BIc(+apK;;U41_qd_w7i6^n9**N*@iEb)_x{QQ1@lka-;!7IY}I8x=Oa$WGlOg81lKk4@Y8af5?hU+DCwpk*Fc<FOpLP(K$4d~Ns^W&1>nry0W&135=PkgRY7sTp#l@E7$0*w4Kr<2ke+_dq6&O6JW$*9x7%6WHy4`^yxw#Al;YVTb<VV-Mu@9r37=E`e#uVZlaC%s#Gcr?rD(q(p)OSLJK`vJ*d=};9~C#_8YOgxIv&j3f&Y*~uTDDmRa!#J}XYUc^bJb&jTe1A_s&r&85yBMlRi^|$*qDyP)y|m!f7~L@N9AsB@$Ut#ICLclm6{ggs&QQ@@81!$X&@+zKkQq(imrNNx0_}e8gB75FgqrV-rTF+UBUlaLsPfE@@F}Ip(PR^lXW!89kvOOeGKK=#<Q@S4Y5Pl&7~CWdN;xfb7t18SRixp=XcD6=A)f?7Z0?0mAa@g-Tt@73YTtuRrf}Pq2}-<O8@CD=_C1N&%?dT?wH1|mKOOQ89r`v1YYh295gB11aT)w8Wyu;8ZG0pvm?(7>3CPy~Z;@}r_c~jG3g)%+iPOR2*zg^<$yYfn!U2MNM&RjuDjCBg+-)-Z5SNK$@YUY3LX2>lQFN54`UZRmO7GTUTn(e0S)DjFC0qZn#`W~1Tlo=z>)mCXdAK+UBFc44FEMl2e8`+5f&30d-K)8VmW}JZU`gXNq<|eO?7R^m97LpKwJUX;MC3lNB6M=aQZ4k|KAi4}mlh$J>ZI31_<gO6&4?-oT3)ZsqeVA_FZt-v($R1eLWvHLAlcV|(V7$TWAD54CEB=92b!p$kTd6DOv7^I7ob)k2S*q*CJ7g{>j^-<fuNrH`)ya0qk;hNK;k+#kpkNho}%l(g@6auo9#GKQFl0QhbGHsJ5)p0f?1@b1VrF>*scF&{J^lIKauwG>6f7cviw1wPmS*>ozfDBu}n&hyDs>b^#XsvZ6btR8gNpO*h-+X7gLNr4n;Gb4JJKwCEXrC|9KHY`Ml?jmzn}65<LpQ589VSmo#O(E^qbg6VYtS{MjvoDeOiW>x`!Z9+@iql=}{4Sy>*h6-b&m)4X&g069dds355-{$xntPJ3I?LHZW}y2gLcFlfsp-|cUJQ&8%}s)J*SWRvBXz8D#>Iq4gVhCIUE!HeV@S_J2Dur1!X`9pd4?`0|?>oic&j~O{wG^t#stUrKuH6`vf2HSEUw6s0YDS2H)1kTD&9r^(5Qv(H~=!>`wBE$nA1xS@CN9|rYBQ*A$Qq0u0CJW$8-(-8+kascp$ekk)<v|S@%MifMRrK@Lu#p#YHU_YITltkd0)jZ295@PYWmxMTEdRRPY&)DHr@VdkKNjdQ-U{>Q&nC3GRn=<2>E=P{Pyu$WKo(P}w#Ev_QQhD`4O6wVp{Mlj>5E?0R|`+6E(*x4_XsAG7WwH{+OyB?=ghZ@+HKNED%{1R->XEtpR39%RfO}y&~Gd4Za-_fyku%XTl!YIn_7xmt;p>Sa2&&w%UZM^kj0SYG(|W);}iWTOn3K+J$i-2zRX8sXx*y6dS@(}ufe>$Uz?NfWN(#!ifPmpZgXdw_7O>%j_22is^1IeqCfLH9t{@P8g87^ivbb@wKFj2^e}tN(KeUrP8j9I;T&Htn`L1$uT#j*m8QV%*=6l<WN4Y9$a1klym77ZbwJaeIm0aj>2#CLn_{d&x*L(SHHrfk(@zl+sA`8%&J6h%+LKk4<NO~F43e>09)E83-HAcGBHdA=gP4(Ej<e+^&3O9n4SP^WTxr%Iy^?-za?*M=o0Uzb3@aTJTFVZjJkz8_mA;U0h4Bd^UCs$udW3;%)C3F`*Xl2Wwon2M0Nwz{VC+Wa#o3ZM965i7KX<W}Pq!g4&RX*om|o|cnqg>t2;;%*Bsrr~L4C69Vu~e<Dks(g2HQg=Z7H6abGUYuk7GTT!3Q8JWG{Knp|2hhH~e#0s>vF{x7H+zM)QqA?I)_vIV*?PhK-EXVsG%&TJrwZL-kU5)Hb_A5!MFXh=aX|#Ch|gVNS*LWR|DtrBT=n03RP}e@3qiS>LeIKWv`n4>Lp5A3A|o<p(R`Igdq)7Df~|WB_E}iO@8>ZDQN8f4YLl0|Xo+)2kyWF_3tQB>g?YL69?$9rVe0*e|h_mYIQtQ@Nr`+n#Jpupgn(<Zkd$^&kE$)(TisXne}VN(jA@f0gIKz1p#~l~epX+FUH*WZ_UXGV|p)Q47;%#>3qhzsS+S=bniZdRV%q>WR6We6Z3y_O>GitqnkrZ%n&Zv`5}ANvM`Z%Wqsdb4CiWJ80w_@d$<hINYP6gYu=r?zN#)MRPE={Nnye#hNYey)R`#-?mRxaqv?L+3ed5ipNanXaO^{gq6jOkP%;Dx}c9wc}r`)DGtpIlj}o<LZax2x1#J^qPK&-(!JZDA@{yje^pNTUZ+<=f;itDFX*|5fP>0o*uEutYe_>#QH;E=Jt#$%`22Dm6la?)Q!AYPAJI6^-jd3}0vG`y4~t!iWU@C@WY_)6t{}K*Tw^!mrdMz0C-Qs+6KnYkgaQIpQinp4ybr13`soBJxjD*zfjwFjN(90&1Tb)%ssq%NDXFd&wyWQ!ff_Jx#i9g6mpp2UoHbF=#F!I9%(cTajqvE=9^5TEdMjoe-p4S?;>3urimS1<{3)vL2o2sD`WVjTtZk;B%<Ln|SW7i`I}rroHr(rvF5=+W7B&8#NCTQ2s{Mp+Wg?PijdeLWU3^fG#&fAouE{X`LVNY^^%`of^mH}8d+i+y@lP&y_>azGTCVbpq(K=M=zpQ2pfR2VOOF8O`B=fvWO}LPJI2)YXhduxYeS58MA|e`z1l_C4af?0DM=hV&77&eK4hLNa!C-+*9el4sHKSW8mx6QI&P4^TCNr;na8jouqxDI_gg9+<Ax9sftaZ{oi||#=_7|DtJlzabtB~#dFNM|3UC0n3v_<j8R|4Q!}Uo|85c1ok$4<O13bH-*u9u$jH3;lH@X-|&-w_yDJ&*c^vY}wD}NLzx2SdrQYZ>&PaR6uuy0kX+y>Z7ol}UA_$}a1hajh`6(Lkz!sHo%rRy9&bQqR!F`AUNI<&@w4=g#)AdbNWa=y;Shlf&Ub}?bii#;w*e=GGtsJ$(anmWG46}d)%8UN#04~LZ1@1AC7e#wB5oK5bSm=;v@l46vYr5<-t(d`rE2<Si%dLlES&y!E+WCvJQA>T5ild@WFcEH{-e)BkFo8cV$!`Dv!rxp)qCch+I8Y~4e#Ud#I|M}LLJgSM0NeUv}8FvrwtL4$X_O$wJybTIUyp7I3?tEiMkR|_U9UqDPU!<k3HYcBU@SPgJUfy+v8Vrm2E;^&7Gck!=ivwB>YQF}Wsg^5%>ArjK{@-<aF9~>hYQ$VTD!_k~Q%3%EVJv;~S0Z1;=)mif+b?l>WC)YA$1vN+r3@>41_3`qR+DPKr=inwgPDlKcF?xsRl9Ty11f7|qm_%Q?BvSXuP(`vARP;^0jRy#I8_4-31Z8L?R7OqkHB9O5?y~}>$CFqEY*Sm+0``e?j6>#Ig-Gcg(uCnAPxl|RBOi<((`X1ZmMd(Hj^>|+AbcGqBzm&?Lu8ZX-L`JCdYK1o5{X|71cL!hX`7hf6X+G-P@iAE_x!+)y;mr=Ud_lc+MZ(T_%(c*IFfTP!+%RR6<2UAzCXCg^}^HN5qjG=qSib-u<DhKPGPJY%U26xjLkr$y|@9svyp<!fd&fKFEnl<D{R9{y%g>hE}oenz$u$>B^#;*a|$f8f*uqy365bw<|;d6a5AbU4gj&yu>)j4gF0C*(RgpD;Yne#M>6$5U8rCxw}F0N=tAi9UxWwR24?P#}7Ip4LvVvLd!Qv=_^Ks#~Nhq`x{JZcc+wFD>Z!@?n9Bk7&&-iGfM^9-%T^IZiVhbpwfanW9XZBlgurGVRJrO1NWRUo`);F8@~u)6oi&reB%IfG^ANZ>eT3aLbyfrSr3McF%s)6*rJE3h`i8xVHf62X~KDuuwHU9s|Y=<tAi-oer?RAaJll8B(;YLV=>5(pG;AVOZk-iJ`_dSUjaebDh3<O>j9I3K40LwOzRh~+^0&(YF!8Pb<M0>?q#~>Vj|H45G)Arn*Na_asO2@X|{8r+f{f$1sj~!LQ-Gwi3H1y@n%A<fIR*&zt+yoC7OMfa^aomd(Ml{<h5LOm3n?-Si&=JpI!sNqF`YWK25<gH|~yLPz_Yt&2Kz?C(2?eJ6y^yCM1z%S<o4qvP^C_q)EKvNVD-;3!@o&q*x!iQOTisY-KGzviOdp+RTzjT#cEIatpT3tg7U_i&0iST>Dq5$q?C%uudKU4}m3WufA%~+bd(?pIJ(oTJ_1h!-cjIZGnc+Af?*-l>aNJ54H?`1>$zqex@s2P^`JC_s3M^f;oHW(tD^yD2sJ6+YL7wZxM30@HXM!>Cy}ZO&GYPv3!BM0(?46N$eqaY~q(OpnVI$7o&QWb`$1AM)byJ?1*+FC~#u&UYfdda)h-*DL*~Y>F>~rNDs+inr&%+Ok>n<L5h$i9zc@2(5W$4KM2i#1(E+Jjupv--M(d<BdI3zwsDzp=d7)Am4=6|P$CCL_nH3uJkSh>&S_JZ>t)&!uiQs@%VgldfwmHIhZ?wFq*=n6oWJ~xNE>aqtuRcz*?o~bnPk=sFv2vKh685~Bx)yeOJJ50g?*3pf6^qybBzH-=8p5Z*opY512*lcID-WqC)RYk-ZcqYEAqFfn+ngrhxYo1&+M#D50d6_&p>_hZc2b9j_d7zkmleAn^DX%=$?m`Wf}}Uj!1q-eEC;Dbg`Nq5PdFFfprqW;X^_0__{-Zi<alsY26WV=E}YK!K0Nvm4O{!qOpmrn)W6Wthu_aBIj#;lamDw+-ZLTc_%u4FS1eK=<LgKck(<a4L!5HM&kTqh~fGEZXscOmwqhJa>(WM6r0{q!<_><nc!9BxkEx03P6S~Hf>hf=monpZb$s~iluogT>_TH++QiV-ts;nL(3D~F;GwJ;FA0|cc-3Jz<W1F3l1z~6GXTe>d045!qzb#a~K2lKK691(Mga^I?tYe^WnFm?h0!fXwFjwj3>jk#bBAI#HIzS<N0Q!LRt0l<D0z3&>Nk{XZca9M1A6otwIjWu+@=7n&T>EOoY`=r7NzO)juywu|R`-Lyo&ZR7cQ9i2wYzmYp}7WA68KnAjvTX%g1JWcyBQ`sw0HBRO0<kSVLA6D!eB@v^sUz<>~$NmLvGX!f;w$ZH#Lip1A9IKR3HSeB+TlOd|f#e@V$#gXT6sY%e1+6?H|S_0d|6bmZz6t$+$+!Uytx#uCw6CghnjG>4P&N0jn3+7-w&i6x-0UGe7B2T^@6@KeAZN5zy4Z_dO8lhB0jmRq%-cEbsxce%FxADG|xEKQMDK$~H;AhE^#$fmec}L>7JpcACqnKVNs(YsMbaT+FD(BsJC8elam>;9hdJ$^_W0?yqI2Q0kR2@NBP6Y~0NUa8`*AeqaSts`A>uR7cXejd0RyQGF^JT_OyHam<<}2eRQ#<b~e6r7@fTG26<H;OQcLvR<Z2to!pQ{F!i#`5;K`Frip<u_Uykmf^Skm?)Z&ZinI&oCzC~!*xOK;Tu&U@XU`2lE=nM6z$QI{{Q)%a<Cz>3e=m)WvVt!H09hyU;b6<6lFcxb=V(gEaR_nM#p{XNRjPB5Hy21YzZ+CC56Ru3kjIj^u!<dD_bmWX~Xb6>E^gCg06`p6}#Gi*yz=6{rXNghv3c-#r8Z8o!jLb((2&nCk|x9dQW0L7BD2%mbu=5;fwpfRVrHT+&FuQ(^iORSOo5FcL}2QWr5B3eVcBh};48vDhWqjNq6%P4T28-^jEjv0X-sELV-ovQ6QC}Xu;1u=#riZe>Rw3wt*9IBsN-?;dP_>iGF3*BTef<0{8y{ftBfGYdh+yg(zqcZlCeG_dt#P#Cgu)-8QE|y9Ty<XOO`{Y-Pn~u2s)20&mvx^83zLvcZA|N(<8#Nnot{O;iB032g)`TN>gV%QCLM*>%T;3cGQa;<daRh-dbz>i72H2H*)K)EvMqj)N(zsp7O6kW<Z*(SrFG9>JqMzClxalG;_ygKk5jj^qrU-~>9wYEzrWV*%LxGtZD13Rj7Tu=3LW7(Bfz#O?uRyQ3i3X{A=|0%$;R7Lk7YB-4_hJaP@G}_GH;5v-qZOMTUCkRI@vx*zPUfwx%b0PV-lCsh;^O1UM&0uR_Jt&h(C1Px6Pijn6_OPNU~rGxhqq`4Q<F1}eOhyqxIGb&r>Io@XQ(Ju$1)|Oz^lXAQU0;s$9Q9(3;<h$$EE)?UsA}Q$%ApcnV6*7cE|iRJbNhm-X_Z!8o-F$yQ`fWGDu4Yz__fu$BYxEvwC}m41{nO<<*o>VW?Yf2HMnf2wl5u6>-m)xio_>`<*|S?AP9ve;6=UjysPF2ox7>64wxKFP85m0?o?ib7x3Q%Yzr7IqnKe6zn7>3F;Ug{njL8u$322c^XzU$DNk_PkYIP+5j$nM8=q{qp7<?uPvMvgj?NeT$f?v*tz>h@=HB?(m8358k`^N&YkcRBqO(ehovSLnB0r}F-8Lg*auiO<FriQy3(`Ggz)ylmGf;nnb%5RITeegg$-NtZD2<{Ag-sC_$-3h-w&p>`KmD|-kVAB;+uL|*LX0HsRhPGM>i+hb;L9(7H;NdtI6C+;&B{_=ZBuvl*ETW7gN_4J*Ah~*FzK!e4;(z?6FSr^hAUSx~pb}#j;xy$>YKZY_R<Ysk<c4d$qNYK!_KvxJ^XD%nr67Y2M)hb~<l&guH$VX0xGb$H&qLW5Cf<8fC>*WR2}^CW(7%s)67`t-}ePyC7ckk^_DsoHa)7F|#G;C;+dZK-G5A>_WxIEXU}9CtMU^Xz%fhcYh*F;(-;Nbgi)s+<wAr+Dn2W`n*S+)A)%CCG9xqqa*fJ9FP9m^CCKbE|?v)^O5>r$muj)wst+afHQ}bFU9^98<Nx}@T601Il5FfP!NIQ--}2o*D<vvGMDr|Rj~^6QN71%ijm(tDr|gbh{krh#i4W@GJ3PZyT(Z{*xp;dpxE$U3CIVQH{oY3_dcS#E37Ak5#hv7i)<2qwS}>@(TjUf;!k@Dc&SnXqr>X9Xn3qRoST&yotQCq;j!^UR-7~1l1ELI1RM)L)*Xkt0r~uL_8d>!3VTgl2bh0|R!;RYCZyd;Dqz)Q(qWVQYXc+UbiAZ#nH<bro3SU4{8>Dc1s=eP-_jRBh!1z)MUAxeZ`~Rrttwm-84-@&_~7l2Dq#Bl6;stcT=$VBWi$*%s`VY&yqS#0W@b&cug^(2*p<w_8ESFn+G&v(NNkjI<YJeAPnp^IwbUh}JioE_^9OfKFS4Bih7f3S?dzZ~n2(P!Io0!B=$VC)I5b;Gq)a}eg&cL;5-=+{HuqtxOU9*TJTWNpXp-6%y4jp+zifM7lTv51D)vgB=hL{fC~h1m>n1B5X2USbivm{h?fTs~f5wCF`IrA;18VOC5P;KBqDUDQFlcY02azpequ{MjdOm7ee$u*~h%;DRFn<g)9p>)i^y8uCsEiFd!5C`Z>$ON@dCpHc&c<JanB2d(65jPN`W&l?MD?G|)<q0zfjGE*e?G^~^4``yXbD}(^tQ^_G9bDIWU9ySM4-ZJW4kbXAFE(WE0dM`{r4J#MEd^UD2Z>(VmeCdP8{&Bk`+VuWi<A`R-^RU*RPZ3bB_wV89a%howQWsfy9#+jo!}AZTB6AV{?4aEFDs^g)Z&J;?8T-cbW5Itse~oVvM4gbf+J{cJk=Ku<mmyI$B9GU!gE7(Uy6_=w7}BJfcz?bDS76VItNQneukvOrSXnv!1X;X2PAH>Y0X|i>H=`td<SXne#t=&=Fk)x6RC_aC&f_A>*wD{pP3(aH>_KZa_Nw;RL^+Jk_Y5#^3LBUZO_h)`ZcJ4~G?<#zXNd{&Ea)+qSWO8$2fpK-;V~vPE13hpeumQ1N|k+ML=4Fchb7w8I^CL=2A#d}h+{A3v1JFKcBI#ekvq@E>4fQb9kS40Q`ANCBLcJwxRv^WJUh|NT_;f|~4>ybtUgpu!W$Zst8-`s}o0kPI9M{PIK)dDZG3`!r_Ya->&71-3D=RQ|WO1SK$9HNopj%7eKocI+D1#-8$siRS<gieWGmx~TN7PFvgTA?<wWap1g9md1TBX|?lEj6<Y!kM_aB6kK_ILmU~MTw_j3G!9Ohu*gJNgj?};xZpbSmhDFpK0g?`1vtM*s|1~UT|Q{|S|b#7&;w?R2`vwi+t72xF)72JNw(fUuL#gD)eJ`alvA&%39LVV1=I~MiH?#f$%Z!}dl*6TA&^O_h@B1QoK)<#O5<`$1fT;MP*iZU9YUPf!7~n?b`^kC6_Qw{EyOS%?5TF|Soj)bI=!PruOstVvi_lP{LflCI{Jcead0+gg)u15=&N|04t~HhGNrHkkRNDrkh3=*7zDITLyv$TS>6abh^#>q%+*6rh_8WE$Nc%mH_-vB-IsKcCx8q!dISq&SJDwU+&&zEMg#4C)k!Pb|MsscYNtd@u}lvQb*R;=@hk%j$I>Rs<VZ9wb?nzkKSh%S9hGd5<%EcinP}}OA#XTK@k@!qU9yJV^qN@f9@pi4kpW13v+@A^+nnSlhVC@sB(*!(UA;rQ;4?j}Ckjs<gYmv&PUkCGxm3T&mWRuo)=SAg_awfmO5te&1P+bF^W_e@{77|i=p(Yr7Dgvbdsv^h^n|wbvta8US4H&8TQbWKm8VlM0P-INqu#WIb=`mDz2K@U_Qk5o1;A$p#kbF5PMsOhmL=F1FcUAuC)p!*!E3eG0OF5A2O8x?^&^I9H}6}B2_&Q_bHt#|1*!FC2>bH>^_R=ed7i{xNTk?@jj1)BGn((UQLD^T(XS6tLQpBzbwvdL<#?7%TVJ+)W4D7r^K7p(o(m^t1jDP99ihO;CzD}AQXUparM@hMrHOa;Be6zoDS6h!nJwnt-c(=2vnSm8z8#E^e0WXEXK{zYHkf9-$5liX$w_pcHE3XK1zr|80|>iRXeR#t)L12d*IsAuCDPd1iA}9}m*`XjsE|0bHTO<HPH}UFb74Yi&_kasE?iVn*D-Q>+YGZzC}m^&wML@%8x3Egy@%oeQ1XycK011sIDcX^jv*wsYj0=*i6Yqo6#&Ep)sn>+(Vmk+Hg$gKF%ydh>{VQMetdD{ZbN+IVPVU3bJZCpT27<K7tpj0D9$b!;PE&>uxY=(4VwnJ+7MPN<cNa?=aV!6x}8X9&@62x^&feUQo+rQ>))lXp#5BiHm0B6*Zxr*riAAQ-#5HtzwQt2RK$mVJvpZsNYT_DsT<!E4sW+<4CZ(^^za_)B$cFnXdnM2lvxql<Kn+!%?FW>$YYWfuv=u6expY-X^`VP&yB^Nq98GU8pj7$jdt?-uPm(BFYwVQ&SyM+f84Hrf^p4~LRUHJX$BNeQN4h0+ltIIyuO@|)J!|-ehlYn2k;mu3kr;WEocAFCpV^I#b^6shvee*cMJX2>wTkBXwXC2<vs+8dCpjGBWcS*Mth5~yy;#q&4$+h(E&>?CGptv353qQ61OUMNkSAj!W0OoKk5)t@pidDoOae=SXEaA4q}0I5C2$=TvaZN)bE&PS!vb(h6VtM<B#O2Q&!OW%ipMp`pBD$*9t|32)%YsCLOBbvhozsmctBk>9BM|ODxqn{QAuzmgEwRZRf#;+@V|=)?0Cr{w4rt+4>T+0Q-q-jdC!2Imz(a%b?Tg;uMxv#26}jF6Xk8PqwuLurP_8RTU|9V7ktCZlABdG~Cx-Phn@pq)nM}B17=8Lbfi(cK!hLt@-48o~tGA=hYt_mut=DDL2eyM};-@H^p4LNidIRGEVj+;)BEf_r)_m+#w|)he5k%L`L7Ps>(8Z*a<w|tl^mC`5veCGsub^aqE_ORLTG&IAsMul7#M%wszq@03TDdJ89#!1teoDxi-*ipZvq&a2!a4W$a*Be}~CSGK45lO)`z9*25!NGCRr%4#*R&uKZRv$S^z^cI-ArACxQr"),format=L.FORMAT_RAW,filters=[{"id":L.FILTER_LZMA2}]))
diff --git a/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_seed314.log b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_seed314.log
new file mode 100644
index 0000000000..9130592890
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_seed314.log
@@ -0,0 +1,198 @@
+====================================================================================================
+Hyperparameters:
+  adam_eps: 1e-08
+  adam_wd: 0.02
+  beta1: 0.9
+  beta2: 0.95
+  compressor: brotli
+  data_dir: ./data/
+  datasets_dir: ./data/datasets/fineweb10B_sp8192
+  distributed: True
+  ema_decay: 0.9965
+  embed_bits: 8
+  embed_clip_sigmas: 20.0
+  embed_lr: 0.6
+  embed_wd: 0.085
+  embedding_dim: 512
+  enable_looping_at: 0.35
+  etlb_clip: 3.0
+  etlb_enabled: False
+  etlb_lr: 0.05
+  etlb_steps: 5
+  eval_seq_len: 2048
+  eval_stride: 64
+  gptq_calibration_batches: 64
+  gptq_reserve_seconds: 12.0
+  grad_accum_steps: 1
+  grad_clip_norm: 0.3
+  head_lr: 0.008
+  is_main_process: True
+  iterations: 20000
+  ln_scale: True
+  local_rank: 0
+  logfile: logs/47f77d33-b999-421c-a300-0071da6ab5d8.txt
+  logit_softcap: 30.0
+  loop_end: 5
+  loop_start: 3
+  matrix_bits: 6
+  matrix_clip_sigmas: 12.85
+  matrix_lr: 0.022
+  max_wallclock_seconds: 600.0
+  min_lr: 0.0
+  mlp_mult: 4.0
+  model_dim: 512
+  model_path: final_model.pt
+  muon_backend_steps: 5
+  muon_beta2: 0.95
+  muon_momentum: 0.99
+  muon_momentum_warmup_start: 0.92
+  muon_momentum_warmup_steps: 1500
+  muon_row_normalize: True
+  muon_wd: 0.095
+  num_heads: 8
+  num_kv_heads: 4
+  num_layers: 11
+  num_loops: 2
+  parallel_residual_start: 7
+  qk_gain_init: 5.5
+  quantized_model_path: final_model.int6.ptz
+  rank: 0
+  rope_base: 10000.0
+  rope_dims: 16
+  rope_train_seq_len: 2048
+  run_id: 47f77d33-b999-421c-a300-0071da6ab5d8
+  scalar_lr: 0.02
+  seed: 314
+  skip_gates_enabled: True
+  sliding_window_enabled: True
+  tie_embeddings: True
+  tied_embed_init_std: 0.005
+  tied_embed_lr: 0.03
+  tokenizer_path: ./data/tokenizers/fineweb_8192_bpe.model
+  train_batch_tokens: 786432
+  train_files: ./data/datasets/fineweb10B_sp8192/fineweb_train_*.bin
+  train_log_every: 500
+  train_seq_len: 2048
+  ttt_chunk_tokens: 32768
+  ttt_enabled: True
+  ttt_epochs: 3
+  ttt_lr: 0.005
+  ttt_momentum: 0.9
+  val_batch_tokens: 524288
+  val_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_*.bin
+  val_loss_every: 4000
+  vocab_size: 8192
+  warmdown_frac: 0.72
+  warmup_steps: 20
+  world_size: 8
+  xsa_last_n: 11
+====================================================================================================
+Running Python 3.11.10 | packaged by conda-forge | (main, Oct 16 2024, 01:27:36) [GCC 13.3.0]
+Running PyTorch 2.9.1+cu128
+Sat Apr 18 10:16:31 2026       
++-----------------------------------------------------------------------------------------+
+| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 13.0     |
++-----------------------------------------+------------------------+----------------------+
+| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
+| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
+|                                         |                        |               MIG M. |
+|=========================================+========================+======================|
+|   0  NVIDIA H100 80GB HBM3          On  |   00000000:04:00.0 Off |                    0 |
+| N/A   48C    P0            122W /  700W |    1521MiB /  81559MiB |      3%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   1  NVIDIA H100 80GB HBM3          On  |   00000000:05:00.0 Off |                    0 |
+| N/A   43C    P0            131W /  700W |    1521MiB /  81559MiB |      4%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   2  NVIDIA H100 80GB HBM3          On  |   00000000:0B:00.0 Off |                    0 |
+| N/A   47C    P0            124W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   3  NVIDIA H100 80GB HBM3          On  |   00000000:0C:00.0 Off |                    0 |
+| N/A   41C    P0            121W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   4  NVIDIA H100 80GB HBM3          On  |   00000000:84:00.0 Off |                    0 |
+| N/A   46C    P0            124W /  700W |    1521MiB /  81559MiB |      2%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   5  NVIDIA H100 80GB HBM3          On  |   00000000:85:00.0 Off |                    0 |
+| N/A   40C    P0            125W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   6  NVIDIA H100 80GB HBM3          On  |   00000000:8B:00.0 Off |                    0 |
+| N/A   45C    P0            129W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   7  NVIDIA H100 80GB HBM3          On  |   00000000:8C:00.0 Off |                    0 |
+| N/A   41C    P0            120W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+
++-----------------------------------------------------------------------------------------+
+| Processes:                                                                              |
+|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
+|        ID   ID                                                               Usage      |
+|=========================================================================================|
+|  No running processes found                                                             |
++-----------------------------------------------------------------------------------------+
+
+====================================================================================================
+train_shards: 80
+val_tokens: 40540160
+model_params:35944536
+gptq:reserving 12s, effective=588000ms
+warmup_step: 1/20
+warmup_step: 2/20
+warmup_step: 3/20
+warmup_step: 4/20
+warmup_step: 5/20
+warmup_step: 6/20
+warmup_step: 10/20
+warmup_step: 20/20
+loop_warmup:enabled encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+loop_warmup_step: 1/20
+loop_warmup_step: 2/20
+loop_warmup_step: 3/20
+loop_warmup_step: 4/20
+loop_warmup_step: 5/20
+loop_warmup_step: 6/20
+loop_warmup_step: 10/20
+loop_warmup_step: 20/20
+0/20000 val_loss: 9.0096 val_bpb: 3.4879
+1/20000 train_loss: 9.0109 train_time: 0.0m tok/s: 8383441
+2/20000 train_loss: 12.3539 train_time: 0.0m tok/s: 8252407
+3/20000 train_loss: 11.0268 train_time: 0.0m tok/s: 8117879
+4/20000 train_loss: 9.4777 train_time: 0.0m tok/s: 8067384
+5/20000 train_loss: 8.3397 train_time: 0.0m tok/s: 8030280
+500/20000 train_loss: 3.3840 train_time: 0.8m tok/s: 7802964
+1000/20000 train_loss: 3.2934 train_time: 1.7m tok/s: 7825725
+1500/20000 train_loss: 3.1883 train_time: 2.5m tok/s: 7806606
+2000/20000 train_loss: 3.0719 train_time: 3.4m tok/s: 7811848
+layer_loop:enabled step:2045 frac:0.350 encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+2500/20000 train_loss: 3.1268 train_time: 4.6m tok/s: 7194032
+3000/20000 train_loss: 2.9016 train_time: 5.8m tok/s: 6792026
+3500/20000 train_loss: 2.9469 train_time: 7.0m tok/s: 6531408
+4000/20000 train_loss: 2.8257 train_time: 8.3m tok/s: 6349219
+4000/20000 val_loss: 2.8825 val_bpb: 1.1159
+4500/20000 train_loss: 2.8464 train_time: 9.5m tok/s: 6214871
+4626/20000 val_loss: 2.8133 val_bpb: 1.0891
+stopping_early: wallclock_cap train_time: 588077ms step: 4626/20000
+peak memory allocated: 39046 MiB reserved: 39070 MiB
+ema:applying EMA weights
+pre-quantization post-ema val_loss:2.81014268 val_bpb:1.08789380 eval_time:5689ms
+Serialized model: 135431033 bytes
+Code size: 16607 bytes
+GPTQ:collecting Hessians from calibration data...
+GPTQ:collected 67 Hessians in 12.6s
+Quantized weights:
+  gptq (int6): blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight
+  gptq (int8): tok_emb.weight
+  passthrough (float16): blocks.attn.q_gain, blocks.attn_scale, blocks.mlp_scale, blocks.resid_mix, skip_gates, skip_weights
+Serialized model quantized+brotli: 15977170 bytes
+Total submission size quantized+brotli: 15993777 bytes
+quantized val_loss:2.83942473 val_bpb:1.09922980 eval_time:8007ms
+quantized_sliding_window val_loss:2.79635283 val_bpb:1.08255532 eval_time:91515ms
+ttt:start chunks=1238 ttt_lr=0.005 ttt_epochs=3
+quantized_ttt val_loss:2.79284037 val_bpb:1.08119554 eval_time:321805ms
diff --git a/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_seed42.log b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_seed42.log
new file mode 100644
index 0000000000..8d7b37a6da
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_seed42.log
@@ -0,0 +1,198 @@
+====================================================================================================
+Hyperparameters:
+  adam_eps: 1e-08
+  adam_wd: 0.02
+  beta1: 0.9
+  beta2: 0.95
+  compressor: brotli
+  data_dir: ./data/
+  datasets_dir: ./data/datasets/fineweb10B_sp8192
+  distributed: True
+  ema_decay: 0.9965
+  embed_bits: 8
+  embed_clip_sigmas: 20.0
+  embed_lr: 0.6
+  embed_wd: 0.085
+  embedding_dim: 512
+  enable_looping_at: 0.35
+  etlb_clip: 3.0
+  etlb_enabled: False
+  etlb_lr: 0.05
+  etlb_steps: 5
+  eval_seq_len: 2048
+  eval_stride: 64
+  gptq_calibration_batches: 64
+  gptq_reserve_seconds: 12.0
+  grad_accum_steps: 1
+  grad_clip_norm: 0.3
+  head_lr: 0.008
+  is_main_process: True
+  iterations: 20000
+  ln_scale: True
+  local_rank: 0
+  logfile: logs/31c8ee29-fd15-472e-979a-b06f278ee8c0.txt
+  logit_softcap: 30.0
+  loop_end: 5
+  loop_start: 3
+  matrix_bits: 6
+  matrix_clip_sigmas: 12.85
+  matrix_lr: 0.022
+  max_wallclock_seconds: 600.0
+  min_lr: 0.0
+  mlp_mult: 4.0
+  model_dim: 512
+  model_path: final_model.pt
+  muon_backend_steps: 5
+  muon_beta2: 0.95
+  muon_momentum: 0.99
+  muon_momentum_warmup_start: 0.92
+  muon_momentum_warmup_steps: 1500
+  muon_row_normalize: True
+  muon_wd: 0.095
+  num_heads: 8
+  num_kv_heads: 4
+  num_layers: 11
+  num_loops: 2
+  parallel_residual_start: 7
+  qk_gain_init: 5.5
+  quantized_model_path: final_model.int6.ptz
+  rank: 0
+  rope_base: 10000.0
+  rope_dims: 16
+  rope_train_seq_len: 2048
+  run_id: 31c8ee29-fd15-472e-979a-b06f278ee8c0
+  scalar_lr: 0.02
+  seed: 42
+  skip_gates_enabled: True
+  sliding_window_enabled: True
+  tie_embeddings: True
+  tied_embed_init_std: 0.005
+  tied_embed_lr: 0.03
+  tokenizer_path: ./data/tokenizers/fineweb_8192_bpe.model
+  train_batch_tokens: 786432
+  train_files: ./data/datasets/fineweb10B_sp8192/fineweb_train_*.bin
+  train_log_every: 500
+  train_seq_len: 2048
+  ttt_chunk_tokens: 32768
+  ttt_enabled: True
+  ttt_epochs: 3
+  ttt_lr: 0.005
+  ttt_momentum: 0.9
+  val_batch_tokens: 524288
+  val_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_*.bin
+  val_loss_every: 4000
+  vocab_size: 8192
+  warmdown_frac: 0.72
+  warmup_steps: 20
+  world_size: 8
+  xsa_last_n: 11
+====================================================================================================
+Running Python 3.11.10 | packaged by conda-forge | (main, Oct 16 2024, 01:27:36) [GCC 13.3.0]
+Running PyTorch 2.9.1+cu128
+Sat Apr 18 09:54:00 2026       
++-----------------------------------------------------------------------------------------+
+| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 13.0     |
++-----------------------------------------+------------------------+----------------------+
+| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
+| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
+|                                         |                        |               MIG M. |
+|=========================================+========================+======================|
+|   0  NVIDIA H100 80GB HBM3          On  |   00000000:04:00.0 Off |                    0 |
+| N/A   35C    P0            113W /  700W |    1521MiB /  81559MiB |      2%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   1  NVIDIA H100 80GB HBM3          On  |   00000000:05:00.0 Off |                    0 |
+| N/A   34C    P0            123W /  700W |    1521MiB /  81559MiB |      2%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   2  NVIDIA H100 80GB HBM3          On  |   00000000:0B:00.0 Off |                    0 |
+| N/A   35C    P0            115W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   3  NVIDIA H100 80GB HBM3          On  |   00000000:0C:00.0 Off |                    0 |
+| N/A   33C    P0            116W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   4  NVIDIA H100 80GB HBM3          On  |   00000000:84:00.0 Off |                    0 |
+| N/A   34C    P0            117W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   5  NVIDIA H100 80GB HBM3          On  |   00000000:85:00.0 Off |                    0 |
+| N/A   33C    P0            119W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   6  NVIDIA H100 80GB HBM3          On  |   00000000:8B:00.0 Off |                    0 |
+| N/A   35C    P0            121W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   7  NVIDIA H100 80GB HBM3          On  |   00000000:8C:00.0 Off |                    0 |
+| N/A   33C    P0            116W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+
++-----------------------------------------------------------------------------------------+
+| Processes:                                                                              |
+|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
+|        ID   ID                                                               Usage      |
+|=========================================================================================|
+|  No running processes found                                                             |
++-----------------------------------------------------------------------------------------+
+
+====================================================================================================
+train_shards: 80
+val_tokens: 40540160
+model_params:35944536
+gptq:reserving 12s, effective=588000ms
+warmup_step: 1/20
+warmup_step: 2/20
+warmup_step: 3/20
+warmup_step: 4/20
+warmup_step: 5/20
+warmup_step: 6/20
+warmup_step: 10/20
+warmup_step: 20/20
+loop_warmup:enabled encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+loop_warmup_step: 1/20
+loop_warmup_step: 2/20
+loop_warmup_step: 3/20
+loop_warmup_step: 4/20
+loop_warmup_step: 5/20
+loop_warmup_step: 6/20
+loop_warmup_step: 10/20
+loop_warmup_step: 20/20
+0/20000 val_loss: 9.0090 val_bpb: 3.4877
+1/20000 train_loss: 9.0104 train_time: 0.0m tok/s: 8323419
+2/20000 train_loss: 12.3654 train_time: 0.0m tok/s: 8223529
+3/20000 train_loss: 11.0100 train_time: 0.0m tok/s: 8119130
+4/20000 train_loss: 9.4547 train_time: 0.0m tok/s: 8083206
+5/20000 train_loss: 8.3267 train_time: 0.0m tok/s: 8054328
+500/20000 train_loss: 3.3764 train_time: 0.8m tok/s: 7814000
+1000/20000 train_loss: 3.2872 train_time: 1.7m tok/s: 7829403
+1500/20000 train_loss: 3.1850 train_time: 2.5m tok/s: 7840973
+2000/20000 train_loss: 3.0729 train_time: 3.3m tok/s: 7846825
+layer_loop:enabled step:2054 frac:0.350 encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+2500/20000 train_loss: 3.1287 train_time: 4.5m tok/s: 7214375
+3000/20000 train_loss: 2.8977 train_time: 5.8m tok/s: 6726913
+3500/20000 train_loss: 2.9404 train_time: 7.1m tok/s: 6480876
+4000/20000 train_loss: 2.8203 train_time: 8.3m tok/s: 6308009
+4000/20000 val_loss: 2.8776 val_bpb: 1.1140
+4500/20000 train_loss: 2.8408 train_time: 9.5m tok/s: 6180179
+4604/20000 val_loss: 2.8112 val_bpb: 1.0883
+stopping_early: wallclock_cap train_time: 588018ms step: 4604/20000
+peak memory allocated: 39045 MiB reserved: 39124 MiB
+ema:applying EMA weights
+pre-quantization post-ema val_loss:2.80812820 val_bpb:1.08711394 eval_time:6975ms
+Serialized model: 135431033 bytes
+Code size: 16607 bytes
+GPTQ:collecting Hessians from calibration data...
+GPTQ:collected 67 Hessians in 12.7s
+Quantized weights:
+  gptq (int6): blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight
+  gptq (int8): tok_emb.weight
+  passthrough (float16): blocks.attn.q_gain, blocks.attn_scale, blocks.mlp_scale, blocks.resid_mix, skip_gates, skip_weights
+Serialized model quantized+brotli: 15977863 bytes
+Total submission size quantized+brotli: 15994470 bytes
+quantized val_loss:2.83705302 val_bpb:1.09831164 eval_time:21932ms
+quantized_sliding_window val_loss:2.79417179 val_bpb:1.08171097 eval_time:120357ms
+ttt:start chunks=1238 ttt_lr=0.005 ttt_epochs=3
+quantized_ttt val_loss:2.79087872 val_bpb:1.08043613 eval_time:364112ms
diff --git a/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_seed999.log b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_seed999.log
new file mode 100644
index 0000000000..d4c0c1f394
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-18_QK525to55_DepthRecur_ParResid_SP8192_LegalTTT/train_seed999.log
@@ -0,0 +1,198 @@
+====================================================================================================
+Hyperparameters:
+  adam_eps: 1e-08
+  adam_wd: 0.02
+  beta1: 0.9
+  beta2: 0.95
+  compressor: brotli
+  data_dir: ./data/
+  datasets_dir: ./data/datasets/fineweb10B_sp8192
+  distributed: True
+  ema_decay: 0.9965
+  embed_bits: 8
+  embed_clip_sigmas: 20.0
+  embed_lr: 0.6
+  embed_wd: 0.085
+  embedding_dim: 512
+  enable_looping_at: 0.35
+  etlb_clip: 3.0
+  etlb_enabled: False
+  etlb_lr: 0.05
+  etlb_steps: 5
+  eval_seq_len: 2048
+  eval_stride: 64
+  gptq_calibration_batches: 64
+  gptq_reserve_seconds: 12.0
+  grad_accum_steps: 1
+  grad_clip_norm: 0.3
+  head_lr: 0.008
+  is_main_process: True
+  iterations: 20000
+  ln_scale: True
+  local_rank: 0
+  logfile: logs/10d124a7-7aba-4766-b190-93767717f0c5.txt
+  logit_softcap: 30.0
+  loop_end: 5
+  loop_start: 3
+  matrix_bits: 6
+  matrix_clip_sigmas: 12.85
+  matrix_lr: 0.022
+  max_wallclock_seconds: 600.0
+  min_lr: 0.0
+  mlp_mult: 4.0
+  model_dim: 512
+  model_path: final_model.pt
+  muon_backend_steps: 5
+  muon_beta2: 0.95
+  muon_momentum: 0.99
+  muon_momentum_warmup_start: 0.92
+  muon_momentum_warmup_steps: 1500
+  muon_row_normalize: True
+  muon_wd: 0.095
+  num_heads: 8
+  num_kv_heads: 4
+  num_layers: 11
+  num_loops: 2
+  parallel_residual_start: 7
+  qk_gain_init: 5.5
+  quantized_model_path: final_model.int6.ptz
+  rank: 0
+  rope_base: 10000.0
+  rope_dims: 16
+  rope_train_seq_len: 2048
+  run_id: 10d124a7-7aba-4766-b190-93767717f0c5
+  scalar_lr: 0.02
+  seed: 999
+  skip_gates_enabled: True
+  sliding_window_enabled: True
+  tie_embeddings: True
+  tied_embed_init_std: 0.005
+  tied_embed_lr: 0.03
+  tokenizer_path: ./data/tokenizers/fineweb_8192_bpe.model
+  train_batch_tokens: 786432
+  train_files: ./data/datasets/fineweb10B_sp8192/fineweb_train_*.bin
+  train_log_every: 500
+  train_seq_len: 2048
+  ttt_chunk_tokens: 32768
+  ttt_enabled: True
+  ttt_epochs: 3
+  ttt_lr: 0.005
+  ttt_momentum: 0.9
+  val_batch_tokens: 524288
+  val_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_*.bin
+  val_loss_every: 4000
+  vocab_size: 8192
+  warmdown_frac: 0.72
+  warmup_steps: 20
+  world_size: 8
+  xsa_last_n: 11
+====================================================================================================
+Running Python 3.11.10 | packaged by conda-forge | (main, Oct 16 2024, 01:27:36) [GCC 13.3.0]
+Running PyTorch 2.9.1+cu128
+Sat Apr 18 10:35:55 2026       
++-----------------------------------------------------------------------------------------+
+| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 13.0     |
++-----------------------------------------+------------------------+----------------------+
+| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
+| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
+|                                         |                        |               MIG M. |
+|=========================================+========================+======================|
+|   0  NVIDIA H100 80GB HBM3          On  |   00000000:04:00.0 Off |                    0 |
+| N/A   49C    P0            124W /  700W |    1521MiB /  81559MiB |      1%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   1  NVIDIA H100 80GB HBM3          On  |   00000000:05:00.0 Off |                    0 |
+| N/A   44C    P0            132W /  700W |    1521MiB /  81559MiB |      1%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   2  NVIDIA H100 80GB HBM3          On  |   00000000:0B:00.0 Off |                    0 |
+| N/A   49C    P0            126W /  700W |    1521MiB /  81559MiB |      2%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   3  NVIDIA H100 80GB HBM3          On  |   00000000:0C:00.0 Off |                    0 |
+| N/A   42C    P0            121W /  700W |    1521MiB /  81559MiB |      2%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   4  NVIDIA H100 80GB HBM3          On  |   00000000:84:00.0 Off |                    0 |
+| N/A   48C    P0            124W /  700W |    1521MiB /  81559MiB |      3%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   5  NVIDIA H100 80GB HBM3          On  |   00000000:85:00.0 Off |                    0 |
+| N/A   42C    P0            126W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   6  NVIDIA H100 80GB HBM3          On  |   00000000:8B:00.0 Off |                    0 |
+| N/A   46C    P0            131W /  700W |    1521MiB /  81559MiB |      4%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+|   7  NVIDIA H100 80GB HBM3          On  |   00000000:8C:00.0 Off |                    0 |
+| N/A   41C    P0            123W /  700W |    1521MiB /  81559MiB |      0%      Default |
+|                                         |                        |             Disabled |
++-----------------------------------------+------------------------+----------------------+
+
++-----------------------------------------------------------------------------------------+
+| Processes:                                                                              |
+|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
+|        ID   ID                                                               Usage      |
+|=========================================================================================|
+|  No running processes found                                                             |
++-----------------------------------------------------------------------------------------+
+
+====================================================================================================
+train_shards: 80
+val_tokens: 40540160
+model_params:35944536
+gptq:reserving 12s, effective=588000ms
+warmup_step: 1/20
+warmup_step: 2/20
+warmup_step: 3/20
+warmup_step: 4/20
+warmup_step: 5/20
+warmup_step: 6/20
+warmup_step: 10/20
+warmup_step: 20/20
+loop_warmup:enabled encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+loop_warmup_step: 1/20
+loop_warmup_step: 2/20
+loop_warmup_step: 3/20
+loop_warmup_step: 4/20
+loop_warmup_step: 5/20
+loop_warmup_step: 6/20
+loop_warmup_step: 10/20
+loop_warmup_step: 20/20
+0/20000 val_loss: 9.0076 val_bpb: 3.4871
+1/20000 train_loss: 9.0093 train_time: 0.0m tok/s: 8315858
+2/20000 train_loss: 12.2938 train_time: 0.0m tok/s: 8245725
+3/20000 train_loss: 11.0090 train_time: 0.0m tok/s: 8139921
+4/20000 train_loss: 9.5038 train_time: 0.0m tok/s: 8089411
+5/20000 train_loss: 8.3677 train_time: 0.0m tok/s: 8055328
+500/20000 train_loss: 3.3802 train_time: 0.8m tok/s: 7811481
+1000/20000 train_loss: 3.2835 train_time: 1.7m tok/s: 7824525
+1500/20000 train_loss: 3.1869 train_time: 2.5m tok/s: 7831888
+2000/20000 train_loss: 3.0733 train_time: 3.3m tok/s: 7834040
+layer_loop:enabled step:2051 frac:0.350 encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+2500/20000 train_loss: 3.1239 train_time: 4.5m tok/s: 7220102
+3000/20000 train_loss: 2.9057 train_time: 5.8m tok/s: 6814014
+3500/20000 train_loss: 2.9491 train_time: 7.0m tok/s: 6543897
+4000/20000 train_loss: 2.8301 train_time: 8.2m tok/s: 6361116
+4000/20000 val_loss: 2.8835 val_bpb: 1.1163
+4500/20000 train_loss: 2.8477 train_time: 9.5m tok/s: 6225947
+4633/20000 val_loss: 2.8140 val_bpb: 1.0894
+stopping_early: wallclock_cap train_time: 588073ms step: 4633/20000
+peak memory allocated: 39046 MiB reserved: 39070 MiB
+ema:applying EMA weights
+pre-quantization post-ema val_loss:2.81087049 val_bpb:1.08817556 eval_time:5690ms
+Serialized model: 135431033 bytes
+Code size: 16607 bytes
+GPTQ:collecting Hessians from calibration data...
+GPTQ:collected 67 Hessians in 12.7s
+Quantized weights:
+  gptq (int6): blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight
+  gptq (int8): tok_emb.weight
+  passthrough (float16): blocks.attn.q_gain, blocks.attn_scale, blocks.mlp_scale, blocks.resid_mix, skip_gates, skip_weights
+Serialized model quantized+brotli: 15974670 bytes
+Total submission size quantized+brotli: 15991277 bytes
+quantized val_loss:2.83992111 val_bpb:1.09942196 eval_time:8036ms
+quantized_sliding_window val_loss:2.79695691 val_bpb:1.08278918 eval_time:91137ms
+ttt:start chunks=1238 ttt_lr=0.005 ttt_epochs=3
+quantized_ttt val_loss:2.79338992 val_bpb:1.08140829 eval_time:319853ms