diff --git a/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/README.md b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/README.md
new file mode 100644
index 0000000000..b2833547b2
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/README.md
@@ -0,0 +1,56 @@
+# Record: SP8192 + 3-Layer Recurrence + Parallel Residuals + QK-Gain 5.25 + Legal TTT + V-Gated
+
+**val_bpb = 1.0796** (3-seed mean, std 0.00025) | **~15.99 MB** | 8xH100 SXM
+
+## 3-Seed Results
+
+| Seed | Sliding BPP | **TTT BPP** | Artifact |
+|------|-------------|-------------|----------|
+| 42   | 1.08112468     | **1.07985553**  | 15985814 |
+| 314  | 1.08057225      | **1.07927887**  | 15983675 |
+| 999  | 1.08110726    | **1.07973035**  | 15986648 |
+| **Mean** | **1.0809** | **1.0796** | **15985379** |
+| **Std** | **0.00025** | **0.00025** | |
+
+## Key Techniques
+
+1. Based on **SP8192 + 3-Layer Recurrence + Parallel Residuals + QK-Gain 5.25 + Legal TTT** (PR #1493 by @bigbag).
+2. Added a learnable **final norm scale** and a **Smear gate** to make representations smoother and slightly more compression-friendly, reducing the compressed artifact size by about **40 KB** and freeing space for additional parameters.
+3. Added a **per-head V-Gate**, where the V projection jointly determines both **what information is fed into attention** and **how much each head contributes to the output**. This significantly improved model performance.
+4. Improved the quantized compression pipeline with **per-matrix automatic layout selection**, giving a small further reduction in final artifact size of about **10 KB**.
+5. Performed extensive hyperparameter search, including settings such as `MUON_BACKEND_STEPS=4` and `TTT_LR=0.01`. This made training more stable and improved reproducibility.
+
+## Architecture
+
+11L x 512d x 8H / 4KV, MLP 4x, LeakyReLU(0.5)^2, Partial RoPE (16/64 dims), layerwise LN scale, tied embeddings, logit softcap=30.0. Depth recurrence: encoder [0,1,2,3,4,5,3,4] decoder [5,3,4,5,6,7,8,9,10] (loops layers 3-5, activated at step ~2016). Parallel residuals from layer 7: attention and MLP operate on same pre-residual input. Skip gates (sigmoid-gated U-Net connections). Final norm scale, Smear gate and V-Gate.
+
+## Reproduction
+
+```bash
+pip install brotli sentencepiece
+pip install flash_attn_3 --no-deps --find-links https://windreamer.github.io/flash-attention3-wheels/cu128_torch291/
+MATCHED_FINEWEB_REPO_ID=kevclark/parameter-golf python3 data/cached_challenge_fineweb.py --variant sp8192
+
+SEED=314 QK_GAIN_INIT=5.25 TTT_ENABLED=1 TTT_LR=0.01 TTT_EPOCHS=3 \
+MLP_MULT=4 MUON_BACKEND_STEPS=4 torchrun --standalone --nproc_per_node=8 train_gpt.py
+```
+
+## Credits
+
+- **@bigbag** — SP8192 + 3-Layer Recurrence + Parallel Residuals + QK_GAIN_INIT=5.25 (PR #1493)
+- **@clarkkev** — SP8192 + GPTQ Embeddings + SDClip + MuonEq-R + depth recurrence (PR #1394)
+- **@dexhunter** — 3-layer depth recurrence (PR #1331, #1437), legal TTT on SP8192 (PR #1413)
+- **@abaybektursun** — Score-first TTT framework (PR #549, merged precedent)
+- **@Robby955** — Parallel residuals on SP8192 (PR #1412)
+- **@msisovic** — Parallel residuals concept (PR #1204)
+- **@X-Abhishek-X** — Hyperparameter tuning: WD=0.095, MLR=0.022, EMA=0.9965 (PR #1445, #1471)
+- **@kellerjordan** -- SmearGate concept (originally from modded-nanogpt)
+
+## Included Files
+
+- `README.md` (this file)
+- `submission.json`
+- `train_gpt.py`
+- `train_seed42.log`
+- `train_seed314.log`
+- `train_seed999.log`
\ No newline at end of file
diff --git a/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/submission.json b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/submission.json
new file mode 100644
index 0000000000..65b342d70e
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/submission.json
@@ -0,0 +1,30 @@
+{
+  "author": "liujshi",
+  "github_id": "liujshi",
+  "name": "SP8192 + 3-Layer Recurrence + Parallel Residuals + QK-Gain 5.25 + Legal Score-First TTT + V Gate",
+  "date": "2026-04-22",
+  "track": "10min_16mb",
+  "val_bpb": 1.0796,
+  "val_bpb_std": 0.00025,
+  "seeds": [42, 314, 999],
+  "seed_results": {
+    "42": {"val_bpb": 1.07985553, "artifact_bytes": 15985814},
+    "314": {"val_bpb": 1.07927887, "artifact_bytes": 15983675},
+    "999": {"val_bpb": 1.07973035, "artifact_bytes": 15986648}
+  },
+  "hardware": "8xH100 80GB SXM",
+  "pytorch_version": "2.9.1+cu128",
+  "technique_summary": "SP8192 + 3-Layer Depth Recurrence (L3-5) + Parallel Residuals (L7+) + QK-Gain 5.25 + EMA 0.9965 + WD 0.095 + Score-First TTT (SGD 3ep) + GPTQ SDClip + Brotli + V Gate + Smear",
+  "compliance": {
+    "train_under_600s": true,
+    "artifact_under_16mb": true,
+    "eval_under_600s": true,
+    "no_slot": true,
+    "no_pre_quant_ttt": true,
+    "no_etlb": true,
+    "no_ngram_cache": true,
+    "score_first_ttt": true,
+    "three_seeds": true
+  },
+  "based_on": "PR #1493 by @bigbag"
+}
diff --git a/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_gpt.py b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_gpt.py
new file mode 100644
index 0000000000..2585e41f49
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_gpt.py
@@ -0,0 +1 @@
+import lzma,base64;exec(lzma.decompress(base64.b85decode('{Wp48S^xk9=GL@E0stWa761SMbT8$j;RmrhAYA}7n@VT6Qap3bt~@<3h>ok~)KUn-s84rm43}oGUMAQ*q|8c_SvdluBKzp|7vc;L3kQ3*yTeDgV-Kbn2(}yE((tCe3*AJ7KGrvI+$jd9UE6jR<G~0Xcsa!kh(D<wO*J|0Qyv({^<{5Z)`J#<4_@7$YZ>~NN&A#=mLUwf%=f6b(fp)>cqs6u7Fj@y6R<8e-1zW{{S1=C2y0*wS>nEYNNhohM=s@L2X0>;M*V&@oCBd0WEzMVI4X>-*<iF$z##97>a|g?bcT5T?`(hp$y{EUy$D}(2V>W{LIRth!XuG<h9QJi#N;`2VynpHosQnjQO7$K`AT!u2wY6DDNt<!OdZbIF8{)T9!g_lVBN=e>v_-<$gL~GaI2o(zU;3BSzp~w=wYz(Ctex-#{d)P8`6oRDpGFWC(gbV`suRD(Q1p<iG&nu?M#6Vr6K|<P64nNAV4>oPUM>?Jk?d*T=xWM@1pk@!N(knWuZ{c=u_9hZiXe?=3$rBj~~GXr=gK49)biW6}c?6S4Ac*y*N=ISWCN%_05jQ{!<;+T;kIN0FoBrKSOLm_`cP@+2W>Qlzq$usBaQ^$9`Hbos_yD(kThAF>HY{llh*{pn*{CxoUCUWZ54@G^%;Ed6zPAS`CMuRGT)py@zO5^rs31NMl3?{X~~jcn8eKDJxyYnjY}d-=KlI%9hkO7G}C-4fLk*kq(hg#*T#7Vs;&NU=yYCq5v?urr2gbuI-dvKtL|5x;9j$s#0wINI%}@R%;&*OR5nL*g&srNzX5gFC+n9h^SlpHi0hKXdmBm$Cn?rCsjoS%zQnZA*J^;h~267eZCDuZBTO-Z(<mKuF$2xtkTKIgi9l3h}h%*F!OGrsWLc6#%W9V8Xn=Pd7+r#j-B1tXPE<HD?jv@#GA~M)R1(Cy%-fRmwcASMHWWNp5~izM$-3udYk4vbg3E?A})H_`?pu7>-i?-7_c)xjOs`2&2#63tpw%6wT@vap+C?JUr6vPmimc_Rz&kvdRH!j4M((<!eqVefEoRoXkGU#q%sB3{LzJoP}>q&Mo^ZXhnQv+?FAq_lOzvi3zo9XK9fq{(5F5uiIDBB%R<%h2z&P@jX}rbQ)HKoP|mPn_U$IqfA{LYTp2vqiApF9ARb(~NN^H05>^9XQs{}F_z-!8d;ygf%TpBjs9MQOuf9Jm6r~Vzpw0&;O_-GIZ>3??*wV~7WhawuTDz`L@b2Tk>WdK&Wq3_LrRds_0A4V4Dy|UMPpi6LKNn^FiCnbe!{u5xJd%LzbINujkkw<}arFJHGn7-e=RGa9on|@mi56kzAFR^~_J9%MbGDHBje~a*z5iT|RN<*t#9}o9C78v+32w-Jlu7zMwX;HXb_(w3g|Dk_ylBwzS3|Fw#G*Dr4sd|V>aix=vrimd&cWk<cB0&~Wh;rY(uF+vZ)1eRTRgwT0?sH=*~uwxY}(SM=f`PJ5+~w$hc?C`WTMl3>KToTocpJ(EAQjbWz$+qXN{Nn9wvU;*pmIqPDX&|;6%wYZITks^lUV$6OLuLhPTojm>DrA{Y=2$13;Fr?plW{GKrc$7T#UEBK2FS24|q9H!g<nD;X;dPrd!5U_M%Vi9W*WEy?%OJ&RnIZ6`_;d4_&{ImxWWO@%Ez<M<3ZwKt4Y?)P}eis(Wtp$=@Y6<z?|kbts3T>XmG4yRZ(*K({T{iff+<7^=4*5sOYcF5aW<;30Rwl9)ZwV9Qdy4DYm{h@-$+4yBlb7T(>%5!H;_F!`-$8dJgJf%%bsX2oZ0SqRTHmKf%b>r{lL479fdGK{PABj3Qlo=mJ$$J31M@1R7<>;6M$M^ZGGWQmm{N_V31g-unme-qvdWBp#cR+cF7_Isy3+@3P*;9qmsaAU>@xk>bd+1+J^jibahC%z(;1r1kAtnem3AIT279YGX9=IqMX|w2ud*qMYoi0rrWzsStLkg;9*KE?%si+D}K(nX7KQyd2z*nq|+mqFh7=<bdm31!X$_Kbaq=|&#hTLdOCgnSPXCrq(X(CvCb?!J1`k$F9%LdE%<scYdC41LwA6G_i6ktF*1!cAQ>>zh7+FNQdf2f7TzdM)4qO_a}B}Of&>`N4R=~(_%`?cs~VK+rT(Tl%)O*AWxVD=_qk+3s5EW3lpva3A30qt`-EJmTd<5xMV+}Fn&wo_@^!p3KQ>staMQnIDTjPxE`%gpOaqD7m>CPFP#v`w_ZPDte4Wfebn3G5HsP0~gXzK&-lkXi<il_aeLl0yoiqPt_Yi<f*Y)dI`kj=AbaHH26Rp=VOY`^IPI?<MNdMA@2b^#hjeRR;uRCZ>^kckBsM1(Z?m%&O>Quc7Az#(xRXTq(&#_*z!xGBbdD+mr$78M8=L3OdnEkg7rzJ)A|Sabym#&E-($rviIA2?cfRk?kn|?N&bIiteb9Gh}~R!+=a*DbXo!P4n{@#>pH}+YoW$@%?lRy(VB54OS+=tJrG0dF)|S9}#YQNNy5d{+h<Z$yR-UGuBrXMl&#O#e6#gT5KPsokDtYDpOOQfEvEvM_*K-PW8YRWaMDn8y7zYq3%E+wZ>7$Mc{HkodD*>*LyF@^M6Cq&o$Ufj7BDs{88gp+BAR`jf>AxFRss#y>PHW(79`N^rhdx_eug|ak$-A{c}}Lmi`adUdOk0Qt*pTVD(_t$p=#0-S1}%es>AO!u176es9fZrg4PNIok*JYE_9#&j0wF!*0ISA7q$03B?CiBg}pO)bx0X6)&YLA%&qGepfr2S0zSsLj}F$b+8ZG-AS(Av<7u{-uiT4FLcq{LYVQC%Izs|BR+tv?9jn@8{U1wSC^Ld$o$qY<xZeGJaB`#-<+yruv_Oo-o+mwf-@N0K??2Y)wbbNvd+ZKP%@FHa2Fz6tR0nuyt^GG{+ab@D;q+-4GC){v((GehZY?qqIpUJ{y3udo!jnskMaH?a68(Y{V%?z;fKVohjjVRhH*bXXPzl>8<AUxJp($Y&6Z03hrPcm)6NR!9lZ%8q)p$j2#e5SD&eiK#<iIF=`5`$rg!n!z$#TyPgp9kkj2<%|83Gesxa3O=A7G$mh_J?L+S#0rw=)M<9yk0`eSJTI|_6Ps5|E<Sb4c|zBoX{o!fUZ<QFH*5b_+#ta>5~I*fY6@J;AX4l2}V_A5`YmWXEc$KXTJ)f_hWd^WeJO)rm}@_bH1p4xL17<3Or<LY-ZOSk?+pbyiCu<E)!W?M2Zj8a<svQhLhHQC~nt9=34fbN-*s%&!ydG)OPzC*fFa2C)9x!#4fG&Q(&cWoW~24(mDuLs8ovzveXsa9g@K^2&FS9mHpGN3l;fwL@285S?YvNdPgMGtV{bz<@UMIfTlD}JGr0K9QvO7$EDXH8m^w%2v+zHs&D`(Ce0vLK739;O9=eFt4{W=HT!>~Ul#TC#7hRr*mXM>5=`r9z|a5yIDPH&fmC-2-pYoQ8i|Ojkm|bgzNC8eY(&w(*rEQ4gjhxFX6EVGMuA+I2m9kr=2Z<cYzcu)>Hf#mIsb7#aT|Q=|e(WDC$SMwh~04xCVwJmeA&cdxvyrCQ2U+wl?^1%>}JTL3JU6ZzUNyp?k*s07UywZy)3J5@NbGB*ZX`Kck-QiKx`cA3OOB;h66D!<~1s@Jh8OjHV^A#|WWuOHv9BJh;bE-NVG;Q(B@gW}(Q!yb6{L>oo%@pW7a5YaSuJe8>EaZMC8d0JkH@&%mvpvkBE^|e3~Ttd`vwH40cdRys_)YPrt@e(ppvGZ-f<Q)rtqvE=Gzvti5XRtI_jOBvRE73Ldoz_@J>LeF41K=|#*;MpdIYC=$j&v=G4A%E=<JuDBB&<)Qw5As$qj<f6yeW`A3{qk*H%GjrEbdv5YsIC`y(gH;*|IspYBs<;Ts#A>jLeatt$Sk+Us<n+<39ibwjp7QWkM$pfOLa4paT772-9{>%N}XHjcZew&&TpK%Zr@e(f8jXLR~YCC`zw)%s<>#>#d$+b0yZ$Tu+pP2@TG7_tv6rmG!}NFwhO4Xfd}Axg1iXy6!xVA2g>6LBVQiyyBJ3<Hlo5C}iuht}5q?+=&RM&p?xzF;mX?Se6v}eh6rXm=5ndAG`%5LSlG`V+#%ais%lOWI4LhZG~?KN)_x<%95}lFWkV(v;GxpJMlE0^c|u^s_DK?%AXuH5qSI8u3*&@Ygct^TA`7$m>U=0BDAyP122|W!_{z!-7LDyUiumFmSWZKh{VTz#G|FWf&-2PAx?RV|AM3(@=xGTysyM!*4Mc5r)HBqa>{}*__RCHxKbaxOM1GdIjEI#_xf8*QHiKSG)e)<?O=zAt?BRhos+1)d4oN<8IaC_q(hNVvHHJD;hJ;0@CzKSRndqM8cE}VLVvN?c}x^wv+@6jK1P>{GnW*7d_Fm2<7tTj_&Pa^DrsH>+mPL`<Debr?JwW^EEbwPlb-+#&<h6QBwO~0EASwAX4@wnoT-}0<EsHH)2-!#k4|D1gg}GH0~mJA!y#+67n~PG)V9JUYkIta8V01dBj+}55-oMe+X)>(QDoKS1!CQ4@X{abo_UhL>tL$eyt}1zwR?q)&t>>({c#p!*$oD&I0X8TH#T{QNJ@S)sy$n(j=udoGZFoHy5MShH=I0Gv2r_r>6%M_*j6Z?sie?D)UhhwaT-p)lN?EXXDbrboqPth#+lwfb0#;$oJDvgPW}w))fsU3o-1bio@LV;x0eP&;c6n$sOi4uKn}9{_iq2fY<A7@;F_Ym4*^WL>?3EThqIG;r`#Ak4m!RC#gFv=N9*ImJa$_6?rre`1><)iw$_LEANlDa4g6^#WiDsZj;7-4+Fe|mX%}R<vHhI;_W#aA-u=n##Hs^@=bXnlzHnIkf?JodmCGgEv3LVeeYLrv;iRd;ocg{iDg*s@eOGu~=N~O!Fb_&*9X<5=A1$D^7*vFwDNvs0;a=t_&^0{AnUbe>_bT*Vyd}w0qf)b=EwKkrDcw8-_gQEG)m49{2-#Q&L{x9<E;}#Hr)t4%{~7`S8=FO(QBqa-mR|j*O#l=bX)HXG8MdQ!U|nq+mAO2INvua*3&_p~>>IoY6IA7NS2<Pip|~8+i419d8W5!y9{pf+TLkpsq^<j7#Lk9Gd!nwgIXt7^^Xtf2SMm1y)xcLy3?u7w<$1G|9%5RE&{LDvng1=B{)5qRDhde;TDCBBfCJsNzC=Ek7t#sOklOdgUXpuEMG7LYTE3T$19JZR7pc<63c;GNV#v3-)=6$erYY}d=LB+}@2|7%(<Kyt3=9ZMer!<tnFwH`v-K9P6x|e^#BkGvG95Pq&y$^9D@jW?7<9ecp$n#Hr$N@(fU@N=?A*1#!D-T^ztg^UjIAGy;i%sUwp+p6YB?a^+n8)MJ%$}#oiTztW6UScPWZS}*sZuRe_mSaYgMI<^7=N18pSd1QX1NxOjUizK=LY`$e6c59ns7uHyYDK+0*q%Hj~6Zd8N|}XT7OBik2dTb@GHbCro%Fz9oN^>e7TyPj*96m}eH5NS64zBuW`0BF<UY!%VO#0lPpN%njIYcu6q<c#Yo>z;{?<B}o-y%v?tPM8%16P_r%7yqDP{<9wE`kcbAdq@6{+eWO#~^zPZ9eR}F!j#N7rNu0J7iB|bz=u`OZ^>ci06%q6vvCnqUo*|!qISP^7&2{v0X@8-8G(94hvRmJU0bhlXO7)~blee{NPPFhA1FzD~ow=a#oKT5nP}~^p*h5+b%4?Xa@uoLSGoX>vB^qans&O4zhb+@xUWUbmFcXzdp@^*lZtOa0{H9NvZbr~k=>u%}7&?T?F-^lxat=tZ6YU*75FEd?G-Rl{J{{#RyLc8zsCMpXW#JX&Yc%3qODuT77r>w&Uwpt!pA3EXW;K{GSn510bk`O>O3*v*4|83ql#bpCJtO5BL~S!H1$Fh-LV2xEP!#ktENBLPanJ*|&$lQ}nQBlkIUuX3mhl%{ZaFE&W~5m+J=45=X3}IYiI3HwY52akoZ2uC0O=+;%(I5V@jmn3Ou)YdQ=Q=uS=LP?!lx78hZkW~n%m!f<IsfdM1sC>Q2MlzZ_j{U)gVv_(33N?X!+ZX9Uc;m+e07<Q(uxb{F|b0ANP^fO+!#=@k|Pz5}{fbz{OJp>Wm$6Hi5zrZ3d4U@+kf89&i>Ndp4jHa2Fz$PNxk&j+XluKBXDVM%3DFp)20!Qp6w#`RcyNW4zs$jF4aO+$@z$1vhSf^+*R8VXpOG?Owfv^Fsi5iVFwvsGu@88nM>yhJ8S6C2YGo$(zvU@0T;aBEZZ+FrTtS81BT+4FylC4Pz4gB9<t(#oIHty2&&f!F6*f!yCU}r}2h>zJ_m6k)bSj$Ci#*wwEg1(dAtZnDwox^fRWAr-D6A??T)?$63=B<O4k*?h@3U>#Xd^TgyXl126aZLexZl%fUA($n)|m6~nXJJM7Wx;~IGJ2U4xAtEBC}LI<jIG7d(aW10eXeUU$U67^gJ#S=MkICnn=V(kvu_8-X&8Vn$X?g1#vDC0bxll^2TR8n~O3hP7RcXkNZBd6Q83w_ayn&tI4@`2m>W6(cj_Ko6NqPlQ7c!Zj+G86?GZ7|K|RQ_H$_bd(V$nMwxg;m<ppx@&+Sc?W?u2tSf>!(+%fE6JQ7PHUjxWA7xnWJ>LyEf~v4rZ6P$elqmp7m!~;GP*Vr17`RBIevn`3sP9407_ttsH4kjhA&`=l_D#zK<|Ta7B$E$WY;Z%5HYq%x&&q3gmM;8A2NdquI4@U}a;utq4X2rE>r?Id@889J-U~BFi`|>ez~n(taXUK3Iheex@A)<^otiMT);eU$M(hyoHXD#(?Y?6CqTm26ilsy3VM}QG72`PKK9oPmoQry;SbPBODMCW58r@d&8M=4pgCm%}zywk@D1!pz9)ota{L4Q|R{=CW>gHp5tEkaeRh8DYfYI#U%4L=E)$hA8JwG`M5nguYw%h-$&8NTtt+(B1@Pn)gQKCAWnUd`VR^xS>$RHgjmyYHR%Ch?@Zyj0!c?UvcD4%UQ(lXwcAz*PxPV)@HRVDaL&h#-i3wyo70O#+&}_XHd>23)-l@xCyn4%<<5k&p$lpW76-}l-x}{1XvNnw_ILzt=xi-S<HjL(Ah1fn{^)sqt&9TQ{ElEq;V>%ZkwxfDummTN04PKCnU~iTs@Xo*e*QsaP)|vmucW`R@0-4AeSk>YbU9cqOGj$Lp~Non+8QK<nrMC;L1g}>ePn4%t_xLlL+-g^WGWgc1Ui7R@}1?s0+E8)pyYP?qn-wHHmiSBa9AfjWdKrm4CD)X;&-S@;9DTn7Td<mBwZtkzVlF*QrZdxMQvEryOtipLbcpzaFWB1)M2s@g0p(P%*CPSxTeV_=F`~zs<2Xd5s$M+)YT>`^=NBCoZn4#sM3OYhC~(eK*^Ht_HdVtHFx+QD5o%Y)czB^{8*md0JOs8%#B@%87#H!*$3(=zBqji&W-2#pmRhtUoyEHKleK#c<iL6{)9-3Ib$61Oy-Y|!$U=W&g-_<z}s+g0H_(&Q)dw0i#V~3ggtESiiiE|9zq76`BVJThD&5o@Q)Sd4jmDpriFA>6dN+*c|un(7)6b$dLf$h_F8s>0t<0I&CK`AGA<nl%?f;3J+qxUajEmW2~c{0gZA9b&qj~O7|~&(OL^tJe;k<nZ-|Z^;Aa>zBrLWS11@W0bqS4hRa(}zmeCz<$HpvMG->%wDsq$(%E$o;O*Eb2MP`BkbAb~@byb;yJQj%q6e~Ib>Q4|J44Qf~ZCwqY-SW}npE>ioe_WZ#MP@u_%#~55htcen9FB}f*|d0${@D1~r{2f{aRcZE7T6%PAb}+}WzB`K10Nv8i0qwP68>3~XpN(Pl<PYN0X~0Iisc=XT3|7FQzTjW#>}T=JaZ3dwO99HhC!Ac6d}{d$|i{&+Ek~ez?z+~$qA`2r6XhuNC%-29dZTi&5W`DBKh{i1$ykGq3-Y0=jk!y^&X(l{A8u7;MsvxV7XKQyLw&S&<Jcqr{kc1TE0!rp@n@6C8Fb*jiVYAK|K!a-_b@!ohw*iLcgGClv)PeAa-55H_Vk=3hjdxVisbG-$hOc+-^)gA50<EoYpjr!*A9=9a7uf6A@b~qjEfl7G$7??_i4u`&QgDLB^L-P;0+9y2OR9Z-5^;xjWa77P6OvoaK9_%NO?#HE4x4N0ey%CONVmE<|N~yI6=^7V;Wf@k7q1stL!F&@|73hn~$KV?OT2px-x@h3>X64^yh#ZbA6CzOuO;>7hdX!zMZ*j_40PXHu37+n4p@r>_j%72-*|k*eFZs0+uwwLb$RinD10qJk#f8czEBJ2(;6@?>c>oROWl$bU@ewv25XY@=zV|2v@#5eaj&=4^V1h=!c<C+SXGoq0e64zmhQW#IIkKDo1I-XW_bZJC~6QAt*Tm~(QI{zz{%!iu&>OKkk+R;y(^xVCGz*`vgzoHlM+)mA`eR;IzFJaaf8oXR>t`u(ll$J2&Pss?TJO1Tk6h8tzKoDEN!UKdHQ67~X0O@w$xi7jzQN<IJkp{O+i_YN~EnYVUeMHf#*6QMk8JU!eoK#DnGiJqE`g-4*n1Jh`HxHV3wcG8B-vQKI$65lGnzd9bm*!)WyK-D2AsAY^d^=uhP{ueXL^Yh3=09A3Gq2`B=rKhOYQ!@+EqQx(-lA3Hv(Asu=Y>!6j%L9zJ5tbxWb<@V+Gq!y6qpkKWDDZmD(-bdq809O2R*25@;0+e?7*!-Q%j7G+vn@-vfFpK!z;4k2ACq`HniS#MW*d-I+>z9@cBWvK%72Meym<KT?aygCQS)T^OAS=?O+djhSggmlq?k#JLgICo!+wogUacj;k{LQ_-09-t6^G#2j5r8fQ;R369#lS@nQy%-S$-p}Rp?efMyL8|!ZFG^c14mKH!Cs?j2I@;YJtYkDamx6mA730_?Wl$F-dmtBTDB|QumABj9Sg28Aq@9n!j!I-%uCfZBa9WE8Sc^21VgZ{kdJ3=oZU{dUr_Wd<02Mu*)3=(@f@Mh`0nk8XEFwf647;6~C*b>P_tDBDuJ9sI2SuUQfl&&qU_a-YcV4G4?<v<be322G@wQp&HrnkJ41;icFTD3efoSQZwj?)&Dur*Y&sGscX<^9*dw1qBn7xujevQN<VAC!g#A~VT?UJ7AI&ENq!!4P12xa3+{d&tsZLdR6sp6qOB;#c>u?DaE>;GEX{!4G~<jRhS!O)VcQ3wb3$I8OGUFl>#u&VXQ3k<w|Q`)jOv_fU^}oaDFgdJh^X(htWz2TC6&8HYdrl9bt*11k#DzbzX-^Sccp6fPI6@g{C1n*=QEoFNI>48gtND{+Dr)#_9A<TMLu;<^@J+uUHq0^-I(7I9Qt49sq=^KOr}pp*9ex+1l8|`R132&?CIww(3`F+jj)2~R8<IRnymQaNJr=9w!6S|RA9^hV1Knpw+QBnNqY43e(8)jQR>QI!a`A)JdRxw<@}&U@dJbvxZGV#C-BeUa}xC77yd|KfYx;+g=w+-7d_|%=UY0Sp~2@o>q%1gVcIq?pe)}pb*|YHJ4zd#n)B{mU8-%)L+fJmI?>t`4g{FSUCTrtN$A_}ww0sHK3t{&k2nX`T<RxvD1ax4VVG7J@<=5tzSh1!BQ3nx<^uoC$@AZmURyTi(ot#&lh`Cc3#=ij^XV!{W<Lb&rd%DQk&Cd%-Db)fTmwIE`QAH=?pi6+wlL>!|1Dy(;`2B<Q$crXGXf$c`@%Gon1kz3cZ@`<-LwF*Gzk>IQsn9X44T+xXI0Fx%Ie)OewEjtwhyqfu;aQZ*v@#e@`+7hP=_LxgOiVOeILvYz61tkh&ae)-}5?~)P<61)8wA7;fotedTwDHZquP(Hs9AqxU0K)h8WhYejx=Pn@FzvW9td9G#AdIzS;rwx1QPZZgDPNiGypiIbNF+7Ij~;A|(GsR5$~^G+K*u%zR0m8OaN9c(NI%-j$l&T4{kOKZ9`g9(5wmA{3ska)PTJb<4*iYp<#Y2}^o*bCkN$O4}QDT%?+Z-GgbYNaxklg|{91?u*?G>_Tlz=p$v!nDsV<fj3pD-uM@8Hd8u_8@`)F30V6TfgEjf8b)0UR*2a;kAVnMO@pDuT&big8@&IHi;dJbnB5ujfDc}u%Z^I_arFcUq{U`lVo}Y&CaarwCkj?8qh+@z<(1>fnaHWJ+}q?nPOCx3^egF?S93;b9-HePHN_j>nFjL4=nl}-W41#S%GI?u{Y$|BSC|=2sltD=$~oQaYIgd@OXBxYO%y5xZWi*Y(w*yQe+a@3HK^$)0!3m^;|rw$a<wA1{8K~joGu3b!BXuM9WS|O9hLb++6)r?Ev`SS2C$vz7lGZdL0cTw*J@Bf*n*vhz2*_T`6l0-u9W7&Dx9b?z)t9@T3vEjl)WHSzM$!eOHAA9d=rBp*NgH>8zx%r^p%ITtD8~H#gB~hq^eh+B5Xt9*39+udw+h*sL(dzGDdcUo(#zdY8YnY8{rBz34rr8C^6iRX$ZJd;$#w6TH?Z4FGnslX4fQA!+Mru1x}!*!K-EjhXKh~eQi$>;hg`BI-E7Vc`apHyMoWO8j+lFHn<UNATsN|VLxbq>R(8_f)*l1{@AR6&wF}<z$E_cak)%VySj<R)+mxGG@qa8Uq1Z~Mwcl4rl$*b93$kJOx{i0{Ob!gAQvYAsd6lp?LWPIHp<&?k{Q^#bZ-i`=@5i)VHhFRU3X^x9nH2ht?)<8C8*!Y%$zuEG2qJhcq%uH`Chkh#c-s4x|Jh*@Z8$AYVbq{u^#k~Vllg>35)xEJW2Nd`WX|qnGA}=op`j&whBi?eJAo%{FeI;xf9tAe|v%D6FGb3{et3xjbOE|HER-li94lDD}ra*89ATmW3uI=oFq+>9nHwAr;6PRjN-KkD{1ILz;2aT4iQ{83F?NYE5$oc5nysOO1iYqnkM;%*za%i9mDguST3YLM9T8`A%hvLoH9n&X#$&6#z>66!7@=&{IWbfYy^|^K(-SEnbc~O(r+gjJB<hZx8GcROv6Jb%d7nuXEX^-i>I+L-yKnfqbG(y2$!FDgT`OjnZ6h?KE8hwXXiU$*fhARChXW$#ao?$^=J0#uC~0GnZX+ft`R{U)1rR_Pi{tAhu9N(LS?f<#RGH&sS1|GtzC-FYJacyCC%1;YUQ5fkv8q0Ky)Ams}QOjt|Hz@T5SzLfiRim6C(J=$lV97dzqnSXdhEi{~aHiQdr_%{&uoKKm-25%?l0~%jErRjO?qZcr3nhzv^O)u{0wBZtQx%j~@w%NhNZZC=<PlbI@(U{yes_d(DA+4<XYGl`CmxzXPq4z2-F-sVW)B+6Sy*``_uo-OG%B*s)|0%*q!`v}rP^ib5U2IQ3)EC|Pt68nrlf<)+h-^27k**kQ}4g?~9Ji<4;rogn+HQq(x(F~+sI<m^K|gXFj3{aRsdmZTJ>n(x-Bu5p0Q)~$crRqtyE<bKv7sMyE-cQqcb?K&+%Wlt~RriN9U@>CkIiRzYeW`e)#ca)10RCHkr)5)wzM7(?{Ts&NSZuXA-V8490I1`vIO#}U(G=HZj-(DB3ZqhoAnt#yzLW63R=0cW&`>t$YzFCw>wlP?jIhv%Qi{aHyegHn^c%(~Wwt^)!rxAlJQC^@uZbSj=p-R(?E~q^dmE4-v<C_2L_1cp)7@!T0p^F!g3a_6pzZTC$E~2B(x>HVLgH8Ls+O2!E-#DY)$Bzu|jfpPskfgG!wM<;^u?OJ7hzytiWFh&&YzV^bdUPuf?|o2DT8qj^6#SZpFsI+?7-R~ynXeoy^6wml(#J$u>On>Y;`pjZC0yrj+hq&QlN0fbUX88{;QvN@3vc#8pJ>DtT@MgocjG;CUiu197EPbD1|S~(N#{oXHeQ0CFxU0;GM7_B_%_lZpdxU?-YA-qwL$X4=|y3(K+2z`;&Kd*a9HdTV#BYz{C<l8)`lqd<zE~QnzIl@%HJObg90FIEAq&HOop^l4c<H9LlshXB{VRe<dW*RGd1iVOPY+#2_aIF9_h3e|M)z_M_uVODj7$>>IhsUcen;X0`t-Jekn}B!}V!bs!L7yWUg>@44xL3wp~-_mli7}C6t&iv_DXWmcX6v)$l-Zo4dxucki<$-}xn;4c~YD6l1vk*z@&MG=exkJfe>^%c~KGB9_!_UQl-GaShu{0{6M%!>b1PN3X6Qk0wf_KtZIMkrhW_tGdD&L2Ip|grPLhW#Y%RQ_re`2p7l}yVNH*W-(;|l}SUp<~hMj^F~I<I@5mp4yT?D&xI8qeItnKV??GUrGepSq9WeNIbUS++N7|^9&>C)$JaYzoG+B}87tZ~sktPklVgg)PE1`o<jzZqu3~*)+UJ_SE4<rynB>+s1sZ4?Lc(uTF24Id4g(w^*}r{9+=+PJ{)TJ5)3b0t*$$Vvsr2tsnHS@%G9nj)Yo1nOleFWCDqSd_X!xwzq~)*dutx;4!I&LJDaFbELj9~6O^d3V6gqkigDxMoQ3c8bWqZUfm{XH&<mWYRYu_4{#1vucu`0hg;GQ*_@K@aAvc(vTIO~?1zq~#(e+aW%mq%h~TowqkD3=y761beXg!n_1w=!gOyNbMcajgZ%&_GQ02F`2RO}M*0az;o#PSoqGo7V!5N;%fU?x@fp%}%DuN^Cm12!Xb;J|a9j@vAf4GoDZ?kK6klF*1`NC&iH`{k$~jJcM-#%fPYtiMH?k7ZGFbkT!;>r5(Iycvwvmr?g{P>bh=^MS{alM!v~iRUw>O7&~0Nq73eq6#=`cVq~`ZAjL~e88?5a$1|~RI~|5gQUYM+L^yvdD|PsGqbplZCpbQ+erkWKZ%2ntn*`BOl7+eCo*tA?OGczyk%IR?jP&&kyQ6+U9L*;LKa6A(^MtZ<tl!>m-p{CqqvF`o<$2~r&AV41A5~LcIgNHR2w-oL4kPt91uU*>V5h`|0I%%k;UmfZsSx)MKqzv1HbeJ5?J#kJZnbQb0}T_{WHc{gMwUuWo*mie45YuQcv)O1g$MUwc04W1WDt!zQYOegwnVM?&S^oDnkEqrRN{x(auyWP<dlyXR`@%i+5C(T$V5VZ7?C;teG+#UYUHHind7u~P^;4;rE3_kgZ!HA`ORQgv=d=kuXQ=P=l)bPim-)|y88@IEX1;2g!etU&d{$l=RW+bJq>i6<Qz>=;s<js7O?I8LjC8#8cfeITq5R!95Ysd+dnt|pegLB_e?xfPhhpu@*rmZ8@K9TO;i0Dz~E5BjaeA+>})hAg}VyR%2kF2?XrYOuXRA+UiuGZ0Ny}7QY{^@c7{dG_oXa|R3-jP>_*WVK?!M-zURiUg*;6VReXI~vcgI%>o#iRxzruQ@L<~R%E*WFFLM@=hrS)^9zKnJU3f8dibSJMMZT7VK0EQxt_Eq#1JU%Gc&6@?j_<8@)?T6_8$uF)Z!xGPf+$#c&)GPP4-JTsg1b;JVHR{6h?SSw&!wxUB$JDQV&yN(Nn=u7ZDb#VbV<@runLDf{@*Mfq&lI-&h9<#K5*C;*fPumzR2>LHgwcM3L%~vFA!#76PFp1=}{7R^&5%iMy-bPRmo-f*90TTe#{PY09iju0yA7dGT#YcD3pe(xlIG5fdn~|vYEiuxZRA({F8!e*W<vTXVE$i3Ok2;2%IsZaIKonygcoA6~R-9s!yqO-K5Z}L#SF><j~;RSJx}rDS~CHCq-@I6}3hF)p9*ZbQ|Z&FtL25HLRUV)zwPANWuSGOp(>6%9q-bwGn?uaYor~sKHB*RRx@lTA7YUc?7p+j5@n2BQu!v|BEXvEOxTTp|Ps*cnOgHo1c3A&x;Wz+2ZuKIHW=4RUmf|)5W>uhFLDt;37ZJ=$0(Lk}Dg?Q0O}mnm1dB%gEvlh1BnNsp%Df>@o?V!(H0g)3r8>(A7|t`N0aXNc#3~1vM=JB<P!SUdSrEs|?LyQ{yLfPtZ<LzYEQr-__Oa*>QzY$zs}}q9;+yHPo%}&2L0X=S96n!Mg*_6%I^{!0Rlx&O*vwX>zg&_pv2`boW=kUa2CyybrzguC9&V*nkx{zfcvy7FKy9`SfUq&ZPw!Yp*$ptEr{L0>QFMWxqF8=bzgWY&km`?cCyOuf)V~P?^`0)Vt7*c%4EFW*Tr!)q!lY=ZEzm2)}zBrQ*!E(1KWA-KrdtzRlK{+n(K@@j=5n+QgD0!iVbuBxAKz-UoJ62kXrVzagI_f|Qm=77tVzfa~>k3{2*hdItrWUVH^p<h9Zf?22<jy0Pfvaa*bbG&x1_rb6EDXP|qIB8mrMuh$CRBsmkU$Rm(J>3(YsNz_nZ7hKe{)0Z?0L<jueqSTGaUN1sS0z%k<-{s(nwWa*q?JUD=MdWSbuS7ntvLHBbT1h-*UKcyxjby|;oz&EhT8nV5UpiYUUMtp&q|>?owv<N+zo|<zuG*ouljocw(B#L(!=$H6=i+DUV-phcr%Bn>6Eg(;2|JKD1f>9CdPlVn%KFb24b<hWY;0jbQQ>~J9KSmU$!bmFnA~30<oVh$WXvKi8IMVzMMMQ~n-N8<cyS}YL4M06*g3)R9f9tS2?1%ZfDxivdLHB1Nj4UhU_jO4Q~LS$Ega@lfm%3uXHU>awYHDn0i3Q8p_yBaDc$@M{LeY9-IB67V#?A9V6r8U2yM4NFddti^#j6eJFe^_|2N*LsuZHn_=`Kp$+QQDSwb%=Rm!^0ee&|y;zy;-v%+PE1>u11;Bx*~ZlxkR<%5$(Cn<>d+6fh^|I_ylR^(3vYC{;)82_X#btkRb6fu2{&Vp}v|5+02D@6n4cYyRteyc<DD0v3UNxZ}g3|5;X_L2<C2*&_qY)pOj1T^9f%~5pw^eg*kz_+?ac}2TV1oh8+{+jL#K~MdioV>=$ZD6dwOW-u}i+xy@q!QInsP?4ZT#z2EN0>4OEim#)64|oTSi_P_$xlFZ<6Z8RbzK{{NH!%}DgXP}C<yi)mj`P9ngoOcOGrI;PAZddMsQv^GxUsY>|qy(3leXCJzDMIUQr#KFqsdCd7PSI<#VSxb78I@1A9OP1<UdVRbx!kd2FWm`-Bwedw*IV?r}dtyuE5d)fGZ;Beaa_b%x&pvM~Gji8(+`FO{j<#>!rMmB}Ofmu7n;vd3Bi%@EET3Fi<{@$f_P)__b@A6z>vV*UluLA?2p*Ww3Bg4U-YfzCk;6ncE@N?2ss$wqcu!79&f9j{kryp&e3iY^ui>4f(JU!4S)s{~)0SUMmn_re^P+BDN?OGB0O>j>+zHu4-$Agi_6+(dZ|jftRSze-7`zcKvj5rq(uml}IvSf5_u&D0`3SQDZXIO)X^r10J9?w|0_l*ypCuDukA6C&eGtJtRz^}xy*HE(eWwS75QZC;oO{<zb`X5rrMIvWd<zs^EZ%tzf@Iv%%BLE98FE|hmmQ2gcE10h2wodFk1;~p72SYwUV^N^06nFzi3<ciB|9c-%J8zmS(^o#&VS3d+fK=KuV49qm2hCtJyrklR9PdDVRub<u<oZuP7t?^&4!oV*v!V0fJ@2)_d%XsbFC1Cifg)bznV!Gn|XaY2OgT1~7Z$Ao2x(|w#w*U9a$GBroZG??8H-6%LSG?iGgog&bPrLd>suXYEay9`gw!OWTLIniUZgQOP$QTU15m;!cER}LUd798@Z=w*Ozm@kB4*oDekDNv9S9t$v=C<ih`9MhxgLb6VSbZAJNYq0b&7cTRiDk_@9f8^<OCP!+kg=d&BOCv%q6wuAp0~|q1e}UDM>LM^3$CC#a%%Lr&8gJ7u*->u?%x8kt14&edDnBwi|(huNLM0QZpf!v`7)z9Gfxpn<hbQs`ktfp#5^erqh?254HOW$7XUxFNHE8Ab6xk1oEgtWnTe_@h>+v?u*WT2;NhYNwvwo3h|fK+zbA<R*j&@!5J$y*oAyYKx(T)~NYU%U+md=<ehB{*gXxv4iG_3()|2rP!TiP(M%`LZOCJXzjvY!fBFGUeQhtos?|Fxy)0ja2(NI1(7Uldu6fS@aqG1#79=EfO5y8~11o<6giCc<s64XZklUF)&TH=?qBhWNX9R-C94iD6*YOM*eLkt%F*BA{;d{=+a&*;WJMX_i~bhl9bxJj_N$L<0@tE+!rCGBAx4@qU_4KF(g6?KjFM6CSA9y-`g*VK&GXewvJ=ssg+VR$9)Ar7RbX>tmlqNrlIRht>n&VzkOY#rv&P5`)8Fg_0d7aNfiQw)|Bcu-W^e^13mP~SnvPH_odeu5HhN0Fpg&kW29rBDZ^_i!R<_-#mm<$9J-203$VK}y}034EB*+C)YH=&lFuKBsi>ik0}uip(dPVyI79L!jZiAt}$Htv@Ct)Q%ws^inmqzI*Hati0O5Wz%7NbK4>ssPK`Y{iAmzwyX?y0-Lb|5{0R(Te?1EpD(slIMp5`uOM;=?;h#$L-h^45Z0)b<2e~|c94EK<^3+2EGxdMZ>)J7k9W<X>@gFNq?*_xRG7L2F&Ohr=Y)4CSx`+GHU+`vWRbrENC~{A5a4G)*~LkV$@=M2G$rkWxZLAjE?-|VEKdTcPf^J-m#3D?4=>YyDCzj7sHiQT4XHT(5D!%?mHoKJs|>oe@7oSqHF8ZlQ;=iuH5In&5(p?9qcJT_!E;l&VNk=(pODd?%XbgL9DOzxKa9<Uu#xm$vC)Mt65Wt5zMgMhUR4P|)3}vWa{lag;=+4fU9D&(J}R^`fBow8u4?FC@SP|+$9)Bh+I*;GQ@#g$*)Ij-N07B+VIH%_FEJ$M=MPR+Y1!Rb!(~*bJCXtFMk)-#)}*HE?ndv;)9op<ynt<_!m8C{sd3XDEvLOc6(7%EV_((2QABhS&hjqso8CVxEM~k=OQP_`(b&r|YKd&u!LcLrx;Gj^6=q1ELQ~QMtJ|*Rj`Q-8Ip|cIlSe&6lATL|?IUv8|G9p?zK)bf*Jce*bI1dmC!1B;8ReAAg78iUqP(RVQyqlSFc9!m%ksRz>-~USM9gmXJ|Z-1_6q2@s2^A>4}n-0*39N#$~3kQF}d&(kR1mM$CqY0O`0+nwmnGihgw$m#=Q8Wi$kUGQc@Lk#<d3|i65xQ_N#UIIS)p18rDP16Q3kDIzfoGIw(|qR&qHTf>iWu3?!gc8w_D~DN=>gabLHSNHjFn$XDGq)a0b`1<YQMMe7gs)_?=F2T_63K5xcz*HK7rfbmO3?Tn@mT*O(GZNGS8_pMMI*~*edP^hO@k77LY%P(tN^iVe+kDy`7Kwa<EqL0Xm+!HkKN#~2MED_Guad=^jp!hdiH6WRPd4&RBqc@q}uK^3mb1`5O1rmZYRbju+^ifO*#0q#o>`{Gw2nDrNBgjcVUq?Pg2?cv8BH=-z^t}vc!^p4Bn4$=F8|pj%^dRv%8%?ZGuH%uo-Z6eidVClLAUQ7?yZH+i(ILqQeo1;Ps;%{1BN{)kYWFvY>sw$S>a-+hSJ+L7xDY%;a=tei#TP7qD_LnFT7N02FXlI{iE4V3EsBK{#Me))XwkC%G?5zfI&trlbflKp5{C7}_5H$KH_vr*=Xa6(IvEwr#JPcZD1c!3i#Rm0>-xfN`+?krB|{PI0??RMK53zmn4?6Tk^@-z<ye&t&z2@<cS@lRdMRfttT=duF*lNf1kmJHe52v&i(Yn0M0RVi!G1eR?8pd5U1FkuJmd>X)%=q0qSB=^0V#?<BfI$EGDZJZbOH#!f5OlKP!~YZ|2k<@9sX55T;3ducHDZFj$i;p%C_t9N<al~=^YQr5uu1exkpFN2tvAt*Hx&A{hu;#$O&0jlka5Y`9l|fbtr=28<eE`r>hr1`tjKpl;Z1VQxmpSk8H3=8Mn|90^*KsC0%*$ohFLVXsXg<@&ce0GUlfLI)`J%D21Wzid8K&P~_QHLeR5-tq+O2=NFRFD1ME>_uCD2xD(t`iHbB&swX>FUmF7(DX6F6+!{ohnp<9ed3c=oiZ$H{=o1);O1TS92~^efYLge5ToAxY8e2EF?u+}nMg{t#_645?w%LI7c5)^LJ+f!ZkZvb6(~PG3hy11Ap<ZTGu8-3e<rWC2xZ76j%6Ce<mOGS&HZJno1A@lAlne^3H&CkV=sRV@rMZwcjG{{IY{sZ2#((?}Hn<7&@Cb-)g`~?cBK(wZM8sOc)K@EUXY)QVM{-o^nK}(Nfg4Fevo&SPkv>&4W+m*(g5c;NAk~fsF!)Bf{ez>;N08~tQKGB%O;ZD*S?>LTLtAxaL8Ha|Je09sO0a8LHDULEGa1)am5NRygUzR7mwzI!p4W1Fh3r*b_qhGy5<SlBM<6)H=Q{hM<L^nFvwxb?*+eg7dt3mY6>9!|L8V<7Xd(n{Sb^DapuDtPeTdCTHPpvOZo61TRjjHrFn5+%+su{=QQ-EQ?LLTa@IL_TMHz!V#3K3`)hvGRg$`iAWUXWf&+i?|((Gx-A{*Kb4==*Hd%yV6Sg+8t27{{Til$az(GDmRlOJ^+mYcW^G2*<q!XTOgT%}WL(FXkImga5naOtq#4V#X^YbH9b@)*>__@C*sKt4fTY0&PIAqgJmzW-Z9y$y60ktI(!8T_{bM(XRV!)+Q*oTK;xoDl<_+Mf2$OUofygeSf+Pl~Y_;!vc5Ec0q*CwY6o73EbTHJtp&Ezw=>B)9EbV`LL2`&hrEtXJi6(qy*fn>?c{IR^*R0~l3=T^W?IliQ_UBcjYNX>uKHYJM5h2yAL*#nDGs)BaZ?lj+uNRQ}_#?;P$1?Ac@y7!za^am7(RE}bc{E{fqE@f(oo$N8kjjW^kd8f_b0pL-8#9xy^pfV46B=UqR<#a=L7+;W?J*59-f`@nvSEphw+qX!@#0hI+}E_QKQ=9M7I6zzAP%*P*ou<FTrMLLA9F&aW9Px+&W^lmTzS3pnL)EG5nToI3Q_9xSxb3r=a&a(>yC*yM^lWeP*sneXK>m5w!*9;CX9MI9ei%qIq$xUI)f2IxmuGV4j4_jH{9eQVKa=rPqs)Xp}q&y&z7OJPO_^r2n>Vo1m<7ZLXiyC-8#!zw5IU1a0dEXJ*_^zRO#CYDD{U84Ci=OtnwRgFM6|mH#x~l);6>r2W!WoK&X`pYcd;Atk5MS1PGuZ`BxKE$AKE>f|SuDsX55r>_Tm|wLkYZc}`pOo~)1KqbW*w5bwh1(C_?8z<kR@zNXB$F@?1!r4C3e&){sq2BqO%iuPd@y7>P0D>_W;!o!HJ9M?)wWc@{5dE$2n^!wjMTS;Gm1HnNuy-qe_R*lIfXZ8nAI6bozp-K6sBA4H1H1A|eTO<)^FRnuB?K{OQuMX{RUV2h}rqs_WOsG5l~$!bU3#=OkPJ>*NiqPxLIIUe8RzbMR)}Pd){*h1q5HJ0ovp2JszC###0aa`8^LZ;#=qVF#?RIP<v;v&+d(YatJv)@qctr=MT8AB$H5RrU9R4{IS6>unJr{)MoHXTf*@)TTl`xV|q>z6gN#DBf`y)rhRu=&OR$2TKt_c;Q^cdqh>_rdLgcizhsBv(i*<pMpxA{tBLr>2yFwOdy8<WYSm}#Z2_j00F#qvX2A+6b-TAvBYQl0ssI200dcD')))
\ No newline at end of file
diff --git a/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_seed314.log b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_seed314.log
new file mode 100644
index 0000000000..dceb88ec42
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_seed314.log
@@ -0,0 +1,162 @@
+bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by bash)
+bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by bash)
+W0422 04:48:49.578000 305108 site-packages/torch/distributed/run.py:852] 
+W0422 04:48:49.578000 305108 site-packages/torch/distributed/run.py:852] *****************************************
+W0422 04:48:49.578000 305108 site-packages/torch/distributed/run.py:852] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+W0422 04:48:49.578000 305108 site-packages/torch/distributed/run.py:852] *****************************************
+NCCL version 2.27.5+cuda12.9
+Hyperparameters:
+  adam_eps: 1e-08
+  adam_wd: 0.02
+  beta1: 0.9
+  beta2: 0.95
+  compressor: brotli
+  data_dir: ./data/
+  datasets_dir: ./data/datasets/fineweb10B_sp8192
+  distributed: True
+  ema_decay: 0.9965
+  embed_bits: 8
+  embed_clip_sigmas: 20.0
+  embed_lr: 0.6
+  embed_wd: 0.085
+  embedding_dim: 512
+  enable_looping_at: 0.35
+  etlb_clip: 3.0
+  etlb_enabled: False
+  etlb_lr: 0.05
+  etlb_steps: 5
+  eval_seq_len: 2048
+  eval_stride: 64
+  gptq_calibration_batches: 64
+  gptq_reserve_seconds: 12.0
+  grad_accum_steps: 1
+  grad_clip_norm: 0.3
+  head_lr: 0.008
+  is_main_process: True
+  iterations: 20000
+  ln_scale: True
+  local_rank: 0
+  logfile: logs/3f29c864-515d-4002-bff1-c1bd284a61cd.txt
+  logit_softcap: 30.0
+  loop_end: 5
+  loop_start: 3
+  matrix_bits: 6
+  matrix_clip_sigmas: 12.85
+  matrix_lr: 0.022
+  max_wallclock_seconds: 600.0
+  min_lr: 0.0
+  mlp_mult: 4.0
+  mlp_mult_last: 4.25
+  mlp_mult_last_n: 0
+  model_dim: 512
+  model_dir: .
+  model_path: ./final_model.pt
+  muon_backend_steps: 4
+  muon_beta2: 0.95
+  muon_momentum: 0.99
+  muon_momentum_warmup_start: 0.92
+  muon_momentum_warmup_steps: 1500
+  muon_row_normalize: True
+  muon_wd: 0.095
+  num_heads: 8
+  num_kv_heads: 4
+  num_layers: 11
+  num_loops: 2
+  parallel_residual_start: 7
+  qk_gain_init: 5.25
+  quantized_model_path: ./final_model.int6.ptz
+  rank: 0
+  rope_base: 10000.0
+  rope_dims: 16
+  rope_train_seq_len: 2048
+  run_id: 3f29c864-515d-4002-bff1-c1bd284a61cd
+  scalar_lr: 0.02
+  seed: 314
+  skip_gates_enabled: True
+  sliding_window_enabled: True
+  tie_embeddings: True
+  tied_embed_init_std: 0.005
+  tied_embed_lr: 0.03
+  tokenizer_path: ./data/tokenizers/fineweb_8192_bpe.model
+  train_batch_tokens: 786432
+  train_files: ./data/datasets/fineweb10B_sp8192/fineweb_train_*.bin
+  train_log_every: 500
+  train_seq_len: 2048
+  ttt_chunk_tokens: 32768
+  ttt_enabled: True
+  ttt_epochs: 3
+  ttt_lr: 0.01
+  ttt_momentum: 0.9
+  val_batch_tokens: 524288
+  val_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_*.bin
+  val_loss_every: 4000
+  vocab_size: 8192
+  warmdown_frac: 0.72
+  warmup_steps: 20
+  world_size: 8
+  xsa_last_n: 11
+train_shards: 128
+val_tokens: 40540160
+model_params:35990616
+gptq:reserving 12s, effective=588000ms
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+warmup_step: 1/20
+warmup_step: 2/20
+warmup_step: 3/20
+warmup_step: 4/20
+warmup_step: 5/20
+warmup_step: 6/20
+warmup_step: 10/20
+warmup_step: 20/20
+loop_warmup:enabled encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+loop_warmup_step: 1/20
+loop_warmup_step: 2/20
+loop_warmup_step: 3/20
+loop_warmup_step: 4/20
+loop_warmup_step: 5/20
+loop_warmup_step: 6/20
+loop_warmup_step: 10/20
+loop_warmup_step: 20/20
+0/20000 val_loss: 8.9992 val_bpb: 3.4839
+1/20000 train_loss: 9.0000 train_time: 0.0m tok/s: 8307347
+2/20000 train_loss: 9.3837 train_time: 0.0m tok/s: 4170147
+3/20000 train_loss: 9.4641 train_time: 0.0m tok/s: 3524396
+4/20000 train_loss: 9.0572 train_time: 0.0m tok/s: 3295882
+5/20000 train_loss: 8.6021 train_time: 0.0m tok/s: 3170754
+500/20000 train_loss: 3.3305 train_time: 0.9m tok/s: 7573129
+1000/20000 train_loss: 3.1782 train_time: 1.7m tok/s: 7621375
+1500/20000 train_loss: 3.0894 train_time: 2.6m tok/s: 7631524
+layer_loop:enabled step:1999 frac:0.350 encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+2000/20000 train_loss: 3.4685 train_time: 3.4m tok/s: 7614484
+2500/20000 train_loss: 3.0538 train_time: 4.7m tok/s: 6952992
+3000/20000 train_loss: 2.9393 train_time: 6.0m tok/s: 6570796
+3500/20000 train_loss: 2.9595 train_time: 7.3m tok/s: 6320580
+4000/20000 train_loss: 2.9037 train_time: 8.5m tok/s: 6146381
+4000/20000 val_loss: 2.8621 val_bpb: 1.1080
+4495/20000 val_loss: 2.8093 val_bpb: 1.0876
+stopping_early: wallclock_cap train_time: 587638ms step: 4495/20000
+peak memory allocated: 39202 MiB reserved: 39406 MiB
+ema:applying EMA weights
+pre-quantization post-ema val_loss:2.80641025 val_bpb:1.08644886 eval_time:6634ms
+Serialized model: 135615997 bytes
+Code size: 19062 bytes
+GPTQ:collecting Hessians from calibration data...
+GPTQ:collected 67 Hessians in 12.9s
+Quantized weights:
+  gptq (int6) layout=raw: blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight
+  gptq (int6) layout=transpose: blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight
+  gptq (int8) layout=raw: tok_emb.weight
+  passthrough (float16): blocks.attn.q_gain, blocks.attn_scale, blocks.mlp_scale, blocks.resid_mix, final_norm.weight, skip_gates, skip_weights, smear.gate
+Serialized model quantized+brotli: 15964613 bytes
+Total submission size quantized+brotli: 15983675 bytes
+quantized val_loss:2.83414481 val_bpb:1.09718578 eval_time:8831ms
+quantized_sliding_window val_loss:2.79123035 val_bpb:1.08057225 eval_time:92720ms
+ttt:start chunks=1238 ttt_lr=0.01 ttt_epochs=3
+quantized_ttt val_loss:2.78788941 val_bpb:1.07927887 eval_time:320664ms
diff --git a/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_seed42.log b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_seed42.log
new file mode 100644
index 0000000000..48a0bd8eb8
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_seed42.log
@@ -0,0 +1,162 @@
+bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by bash)
+bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by bash)
+W0422 04:25:40.659000 303281 site-packages/torch/distributed/run.py:852] 
+W0422 04:25:40.659000 303281 site-packages/torch/distributed/run.py:852] *****************************************
+W0422 04:25:40.659000 303281 site-packages/torch/distributed/run.py:852] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+W0422 04:25:40.659000 303281 site-packages/torch/distributed/run.py:852] *****************************************
+NCCL version 2.27.5+cuda12.9
+Hyperparameters:
+  adam_eps: 1e-08
+  adam_wd: 0.02
+  beta1: 0.9
+  beta2: 0.95
+  compressor: brotli
+  data_dir: ./data/
+  datasets_dir: ./data/datasets/fineweb10B_sp8192
+  distributed: True
+  ema_decay: 0.9965
+  embed_bits: 8
+  embed_clip_sigmas: 20.0
+  embed_lr: 0.6
+  embed_wd: 0.085
+  embedding_dim: 512
+  enable_looping_at: 0.35
+  etlb_clip: 3.0
+  etlb_enabled: False
+  etlb_lr: 0.05
+  etlb_steps: 5
+  eval_seq_len: 2048
+  eval_stride: 64
+  gptq_calibration_batches: 64
+  gptq_reserve_seconds: 12.0
+  grad_accum_steps: 1
+  grad_clip_norm: 0.3
+  head_lr: 0.008
+  is_main_process: True
+  iterations: 20000
+  ln_scale: True
+  local_rank: 0
+  logfile: logs/0a66a03b-c068-49f4-896f-ab41d064b781.txt
+  logit_softcap: 30.0
+  loop_end: 5
+  loop_start: 3
+  matrix_bits: 6
+  matrix_clip_sigmas: 12.85
+  matrix_lr: 0.022
+  max_wallclock_seconds: 600.0
+  min_lr: 0.0
+  mlp_mult: 4.0
+  mlp_mult_last: 4.25
+  mlp_mult_last_n: 0
+  model_dim: 512
+  model_dir: .
+  model_path: ./final_model.pt
+  muon_backend_steps: 4
+  muon_beta2: 0.95
+  muon_momentum: 0.99
+  muon_momentum_warmup_start: 0.92
+  muon_momentum_warmup_steps: 1500
+  muon_row_normalize: True
+  muon_wd: 0.095
+  num_heads: 8
+  num_kv_heads: 4
+  num_layers: 11
+  num_loops: 2
+  parallel_residual_start: 7
+  qk_gain_init: 5.25
+  quantized_model_path: ./final_model.int6.ptz
+  rank: 0
+  rope_base: 10000.0
+  rope_dims: 16
+  rope_train_seq_len: 2048
+  run_id: 0a66a03b-c068-49f4-896f-ab41d064b781
+  scalar_lr: 0.02
+  seed: 42
+  skip_gates_enabled: True
+  sliding_window_enabled: True
+  tie_embeddings: True
+  tied_embed_init_std: 0.005
+  tied_embed_lr: 0.03
+  tokenizer_path: ./data/tokenizers/fineweb_8192_bpe.model
+  train_batch_tokens: 786432
+  train_files: ./data/datasets/fineweb10B_sp8192/fineweb_train_*.bin
+  train_log_every: 500
+  train_seq_len: 2048
+  ttt_chunk_tokens: 32768
+  ttt_enabled: True
+  ttt_epochs: 3
+  ttt_lr: 0.01
+  ttt_momentum: 0.9
+  val_batch_tokens: 524288
+  val_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_*.bin
+  val_loss_every: 4000
+  vocab_size: 8192
+  warmdown_frac: 0.72
+  warmup_steps: 20
+  world_size: 8
+  xsa_last_n: 11
+train_shards: 128
+val_tokens: 40540160
+model_params:35990616
+gptq:reserving 12s, effective=588000ms
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+warmup_step: 1/20
+warmup_step: 2/20
+warmup_step: 3/20
+warmup_step: 4/20
+warmup_step: 5/20
+warmup_step: 6/20
+warmup_step: 10/20
+warmup_step: 20/20
+loop_warmup:enabled encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+loop_warmup_step: 1/20
+loop_warmup_step: 2/20
+loop_warmup_step: 3/20
+loop_warmup_step: 4/20
+loop_warmup_step: 5/20
+loop_warmup_step: 6/20
+loop_warmup_step: 10/20
+loop_warmup_step: 20/20
+0/20000 val_loss: 8.9972 val_bpb: 3.4831
+1/20000 train_loss: 8.9978 train_time: 0.0m tok/s: 8317597
+2/20000 train_loss: 9.3658 train_time: 0.0m tok/s: 4212043
+3/20000 train_loss: 9.4016 train_time: 0.0m tok/s: 3614480
+4/20000 train_loss: 8.9865 train_time: 0.0m tok/s: 3376482
+5/20000 train_loss: 8.5536 train_time: 0.0m tok/s: 3250553
+500/20000 train_loss: 3.3237 train_time: 0.9m tok/s: 7584248
+1000/20000 train_loss: 3.1794 train_time: 1.7m tok/s: 7625963
+1500/20000 train_loss: 3.0880 train_time: 2.6m tok/s: 7635377
+layer_loop:enabled step:1999 frac:0.350 encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+2000/20000 train_loss: 3.4243 train_time: 3.4m tok/s: 7613347
+2500/20000 train_loss: 3.0577 train_time: 4.7m tok/s: 6949713
+3000/20000 train_loss: 2.9371 train_time: 6.0m tok/s: 6567548
+3500/20000 train_loss: 2.9634 train_time: 7.3m tok/s: 6318439
+4000/20000 train_loss: 2.9053 train_time: 8.5m tok/s: 6144500
+4000/20000 val_loss: 2.8635 val_bpb: 1.1085
+4493/20000 val_loss: 2.8109 val_bpb: 1.0882
+stopping_early: wallclock_cap train_time: 587655ms step: 4493/20000
+peak memory allocated: 39202 MiB reserved: 39406 MiB
+ema:applying EMA weights
+pre-quantization post-ema val_loss:2.80790659 val_bpb:1.08702814 eval_time:6558ms
+Serialized model: 135615997 bytes
+Code size: 19062 bytes
+GPTQ:collecting Hessians from calibration data...
+GPTQ:collected 67 Hessians in 12.9s
+Quantized weights:
+  gptq (int6) layout=raw: blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight
+  gptq (int6) layout=transpose: blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight
+  gptq (int8) layout=raw: tok_emb.weight
+  passthrough (float16): blocks.attn.q_gain, blocks.attn_scale, blocks.mlp_scale, blocks.resid_mix, final_norm.weight, skip_gates, skip_weights, smear.gate
+Serialized model quantized+brotli: 15966752 bytes
+Total submission size quantized+brotli: 15985814 bytes
+quantized val_loss:2.83554832 val_bpb:1.09772912 eval_time:8898ms
+quantized_sliding_window val_loss:2.79265733 val_bpb:1.08112468 eval_time:92468ms
+ttt:start chunks=1238 ttt_lr=0.01 ttt_epochs=3
+quantized_ttt val_loss:2.78937898 val_bpb:1.07985553 eval_time:322004ms
diff --git a/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_seed999.log b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_seed999.log
new file mode 100644
index 0000000000..bdff3f23c1
--- /dev/null
+++ b/records/track_10min_16mb/2026-04-22_SP8192_3LayerRecur_ParResid_QK525_LegalTTT_Vgated/train_seed999.log
@@ -0,0 +1,162 @@
+bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by bash)
+bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by bash)
+W0422 05:12:01.064000 306935 site-packages/torch/distributed/run.py:852] 
+W0422 05:12:01.064000 306935 site-packages/torch/distributed/run.py:852] *****************************************
+W0422 05:12:01.064000 306935 site-packages/torch/distributed/run.py:852] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+W0422 05:12:01.064000 306935 site-packages/torch/distributed/run.py:852] *****************************************
+NCCL version 2.27.5+cuda12.9
+Hyperparameters:
+  adam_eps: 1e-08
+  adam_wd: 0.02
+  beta1: 0.9
+  beta2: 0.95
+  compressor: brotli
+  data_dir: ./data/
+  datasets_dir: ./data/datasets/fineweb10B_sp8192
+  distributed: True
+  ema_decay: 0.9965
+  embed_bits: 8
+  embed_clip_sigmas: 20.0
+  embed_lr: 0.6
+  embed_wd: 0.085
+  embedding_dim: 512
+  enable_looping_at: 0.35
+  etlb_clip: 3.0
+  etlb_enabled: False
+  etlb_lr: 0.05
+  etlb_steps: 5
+  eval_seq_len: 2048
+  eval_stride: 64
+  gptq_calibration_batches: 64
+  gptq_reserve_seconds: 12.0
+  grad_accum_steps: 1
+  grad_clip_norm: 0.3
+  head_lr: 0.008
+  is_main_process: True
+  iterations: 20000
+  ln_scale: True
+  local_rank: 0
+  logfile: logs/7bd830a2-7a71-4d7c-939b-89954cf9dc5e.txt
+  logit_softcap: 30.0
+  loop_end: 5
+  loop_start: 3
+  matrix_bits: 6
+  matrix_clip_sigmas: 12.85
+  matrix_lr: 0.022
+  max_wallclock_seconds: 600.0
+  min_lr: 0.0
+  mlp_mult: 4.0
+  mlp_mult_last: 4.25
+  mlp_mult_last_n: 0
+  model_dim: 512
+  model_dir: .
+  model_path: ./final_model.pt
+  muon_backend_steps: 4
+  muon_beta2: 0.95
+  muon_momentum: 0.99
+  muon_momentum_warmup_start: 0.92
+  muon_momentum_warmup_steps: 1500
+  muon_row_normalize: True
+  muon_wd: 0.095
+  num_heads: 8
+  num_kv_heads: 4
+  num_layers: 11
+  num_loops: 2
+  parallel_residual_start: 7
+  qk_gain_init: 5.25
+  quantized_model_path: ./final_model.int6.ptz
+  rank: 0
+  rope_base: 10000.0
+  rope_dims: 16
+  rope_train_seq_len: 2048
+  run_id: 7bd830a2-7a71-4d7c-939b-89954cf9dc5e
+  scalar_lr: 0.02
+  seed: 999
+  skip_gates_enabled: True
+  sliding_window_enabled: True
+  tie_embeddings: True
+  tied_embed_init_std: 0.005
+  tied_embed_lr: 0.03
+  tokenizer_path: ./data/tokenizers/fineweb_8192_bpe.model
+  train_batch_tokens: 786432
+  train_files: ./data/datasets/fineweb10B_sp8192/fineweb_train_*.bin
+  train_log_every: 500
+  train_seq_len: 2048
+  ttt_chunk_tokens: 32768
+  ttt_enabled: True
+  ttt_epochs: 3
+  ttt_lr: 0.01
+  ttt_momentum: 0.9
+  val_batch_tokens: 524288
+  val_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_*.bin
+  val_loss_every: 4000
+  vocab_size: 8192
+  warmdown_frac: 0.72
+  warmup_steps: 20
+  world_size: 8
+  xsa_last_n: 11
+train_shards: 128
+val_tokens: 40540160
+model_params:35990616
+gptq:reserving 12s, effective=588000ms
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+/bin/bash: /miniconda3/envs/swift/lib/libtinfo.so.6: no version information available (required by /bin/bash)
+warmup_step: 1/20
+warmup_step: 2/20
+warmup_step: 3/20
+warmup_step: 4/20
+warmup_step: 5/20
+warmup_step: 6/20
+warmup_step: 10/20
+warmup_step: 20/20
+loop_warmup:enabled encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+loop_warmup_step: 1/20
+loop_warmup_step: 2/20
+loop_warmup_step: 3/20
+loop_warmup_step: 4/20
+loop_warmup_step: 5/20
+loop_warmup_step: 6/20
+loop_warmup_step: 10/20
+loop_warmup_step: 20/20
+0/20000 val_loss: 8.9978 val_bpb: 3.4833
+1/20000 train_loss: 8.9982 train_time: 0.0m tok/s: 8332439
+2/20000 train_loss: 9.4184 train_time: 0.0m tok/s: 4180949
+3/20000 train_loss: 9.4671 train_time: 0.0m tok/s: 3581756
+4/20000 train_loss: 9.0713 train_time: 0.0m tok/s: 3333454
+5/20000 train_loss: 8.6839 train_time: 0.0m tok/s: 3204096
+500/20000 train_loss: 3.3351 train_time: 0.9m tok/s: 7574571
+1000/20000 train_loss: 3.1891 train_time: 1.7m tok/s: 7603063
+1500/20000 train_loss: 3.0961 train_time: 2.6m tok/s: 7617686
+layer_loop:enabled step:1995 frac:0.350 encoder:[0, 1, 2, 3, 4, 5, 3, 4] decoder:[5, 3, 4, 5, 6, 7, 8, 9, 10]
+2000/20000 train_loss: 3.2130 train_time: 3.4m tok/s: 7600469
+2500/20000 train_loss: 3.0593 train_time: 4.7m tok/s: 6941685
+3000/20000 train_loss: 2.9376 train_time: 6.0m tok/s: 6562554
+3500/20000 train_loss: 2.9632 train_time: 7.3m tok/s: 6315088
+4000/20000 train_loss: 2.9032 train_time: 8.5m tok/s: 6142400
+4000/20000 val_loss: 2.8630 val_bpb: 1.1083
+4495/20000 val_loss: 2.8098 val_bpb: 1.0878
+stopping_early: wallclock_cap train_time: 587643ms step: 4495/20000
+peak memory allocated: 39202 MiB reserved: 39406 MiB
+ema:applying EMA weights
+pre-quantization post-ema val_loss:2.80726428 val_bpb:1.08677949 eval_time:6514ms
+Serialized model: 135615997 bytes
+Code size: 19062 bytes
+GPTQ:collecting Hessians from calibration data...
+GPTQ:collected 67 Hessians in 12.9s
+Quantized weights:
+  gptq (int6) layout=raw: blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight
+  gptq (int6) layout=transpose: blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight
+  gptq (int8) layout=raw: tok_emb.weight
+  passthrough (float16): blocks.attn.q_gain, blocks.attn_scale, blocks.mlp_scale, blocks.resid_mix, final_norm.weight, skip_gates, skip_weights, smear.gate
+Serialized model quantized+brotli: 15967586 bytes
+Total submission size quantized+brotli: 15986648 bytes
+quantized val_loss:2.83533793 val_bpb:1.09764767 eval_time:8919ms
+quantized_sliding_window val_loss:2.79261234 val_bpb:1.08110726 eval_time:92643ms
+ttt:start chunks=1238 ttt_lr=0.01 ttt_epochs=3
+quantized_ttt val_loss:2.78905564 val_bpb:1.07973035 eval_time:319222ms