diff --git a/records/track_10min_16mb/2026-04-20_SP8192_Progressive3LayerRecurrence_ParResid_QK525_LegalTTT/README.md b/records/track_10min_16mb/2026-04-20_SP8192_Progressive3LayerRecurrence_ParResid_QK525_LegalTTT/README.md new file mode 100644 index 0000000000..39777a4521 --- /dev/null +++ b/records/track_10min_16mb/2026-04-20_SP8192_Progressive3LayerRecurrence_ParResid_QK525_LegalTTT/README.md @@ -0,0 +1,113 @@ +# Record: SP8192 + Progressive 3-Layer Recurrence + Parallel Residuals + QK-Gain 5.25 + Legal TTT + +**val_bpb = 1.0806** (3-seed mean, std 0.0011) | **~15.99 MB** | **8xH100 SXM** + +## 3-Seed Results + +| Seed | Sliding BPP | **TTT BPP** | Artifact | +|------|-------------|-------------|----------| +| 42 | 1.0818 | **1.0805** | 15,992,388 | +| 999 | 1.0831 | **1.0818** | 15,996,018 | +| 1337 | 1.0810 | **1.0796** | 15,989,841 | +| **Mean** | **1.0820** | **1.0806** | **15,992,749** | +| **Std** | **0.0009** | **0.0011** | | + +Current merged record in repo: **1.0810 BPP**. Delta: **-0.0004 BPP**. + +## Key Change + +This submission keeps the merged `2026-04-09` SP8192 record stack intact and changes only the recurrence schedule. + +Baseline recurrence schedule: + +- full 3-layer recurrence activates at `frac=0.35` + +This submission uses a progressive schedule: + +- phase 1 at `frac=0.35`: one extra recurrence pass +- phase 2 at `frac=0.55`: full 3-layer recurrence + +The hypothesis is that the strongest recurrence stack benefits from a smoother transition into deeper virtual depth rather than a single hard switch. + +## Key Techniques + +1. **SP8192 + GPTQ SDClip** — int6 matrices (`k=12.85`), int8 embeddings (`k=20.0`), Brotli-compressed GPTQ state +2. **Progressive 3-Layer Depth Recurrence** — layers `3..5`, with partial recurrence at `0.35` and full recurrence at `0.55` +3. **Parallel Residuals** (`layers 7+`) — attention and MLP read from the same pre-residual input +4. **QK-Gain 5.25** — learnable per-head query scaling +5. **Legal Score-First TTT** — SGD (`lr=0.005`, momentum `0.9`), 3 epochs per 32K-token chunk, score-before-update ordering +6. **Tuned Hyperparameters** — WD `0.095`, matrix LR `0.022`, EMA `0.9965`, warmdown `0.72` +7. **Packed LZMA wrapper** — keeps code overhead low enough for the full artifact to fit under 16 MB + +## Architecture + +11L x 512d x 8H / 4KV, MLP 4x, LeakyReLU(0.5)^2, Partial RoPE (16 dims), layerwise LN scale, tied embeddings, logit softcap `30.0`, skip gates enabled. + +Progressive recurrence schedules the looped `3..5` segment in two phases: + +- phase 1 encoder: `[0,1,2,3,4,5,3]` +- phase 1 decoder: `[4,5,6,7,8,9,10]` +- phase 2 encoder: `[0,1,2,3,4,5,3,4]` +- phase 2 decoder: `[5,3,4,5,6,7,8,9,10]` + +Parallel residuals start at layer 7. + +## Training + +MuonEq-R optimizer (row-normalized Muon, Newton-Schulz 5 steps), AdamW for embeddings/scalars, max wallclock `600s`, GPTQ reserve `12s`. Training stops at the wallclock cap around `4699-4711` steps depending on seed. + +## Quantization + +Full-Hessian GPTQ with SDClip, int6 matrices and int8 token embeddings. Byte-shuffle plus Brotli-11 compression. All three seeds fit under `16,000,000` bytes with the packed code wrapper. + +## TTT + +Score-first, chunk-based SGD adaptation at eval time: + +- chunk size `32K` tokens +- score each chunk under `torch.no_grad()` before any update +- train for 3 epochs on already-scored chunk tokens +- cosine LR decay across chunks +- gradient clipping at `1.0` + +## Compliance + +Per Issue #1017 Track B: + +- strictly causal sliding-window scoring +- normalized softmax over the full vocabulary +- score-before-update TTT ordering +- single-pass token scoring +- no SLOT +- no ETLB +- no n-gram cache +- all artifacts under 16 MB on all 3 seeds +- training under 600s on all 3 seeds + +## Reproduction + +```bash +pip install brotli sentencepiece +pip install flash_attn_3 --no-deps --find-links https://windreamer.github.io/flash-attention3-wheels/cu128_torch291/ +MATCHED_FINEWEB_REPO_ID=kevclark/parameter-golf python3 data/cached_challenge_fineweb.py --variant sp8192 + +SEED=42 QK_GAIN_INIT=5.25 PARALLEL_RESIDUAL_START=7 \ +ENABLE_LOOPING_AT=0.35 LOOP_PHASE2_AT=0.55 \ +TTT_ENABLED=1 TTT_LR=0.005 TTT_EPOCHS=3 COMPRESSOR=brotli \ +torchrun --standalone --nproc_per_node=8 train_gpt.py +``` + +## Credits + +- **@clarkkev** — SP8192 + GPTQ embeddings + SDClip + MuonEq-R base stack +- **@dexhunter** — 3-layer depth recurrence and legal TTT framework on SP8192 +- **@abaybektursun** — score-first TTT framework +- **@Robby955** — parallel residuals on SP8192 +- **@msisovic** — parallel residual concept +- **@X-Abhishek-X** — hyperparameter tuning for the strong SP8192 family + +## Included Files + +- `README.md` +- `submission.json` +- `train_gpt.py` diff --git a/records/track_10min_16mb/2026-04-20_SP8192_Progressive3LayerRecurrence_ParResid_QK525_LegalTTT/submission.json b/records/track_10min_16mb/2026-04-20_SP8192_Progressive3LayerRecurrence_ParResid_QK525_LegalTTT/submission.json new file mode 100644 index 0000000000..ea91b23333 --- /dev/null +++ b/records/track_10min_16mb/2026-04-20_SP8192_Progressive3LayerRecurrence_ParResid_QK525_LegalTTT/submission.json @@ -0,0 +1,37 @@ +{ + "author": "wisebreadloaf", + "github_id": "wisebreadloaf", + "name": "SP8192 + Progressive 3-Layer Recurrence + Parallel Residuals + QK-Gain 5.25 + Legal TTT", + "date": "2026-04-20", + "track": "10min_16mb", + "val_bpb": 1.08061, + "val_bpb_std": 0.00112, + "seeds": [42, 999, 1337], + "seed_results": { + "42": {"val_bpb": 1.08051, "artifact_bytes": 15992388}, + "999": {"val_bpb": 1.08177, "artifact_bytes": 15996018}, + "1337": {"val_bpb": 1.07955, "artifact_bytes": 15989841} + }, + "hardware": "8xH100 80GB SXM", + "pytorch_version": "2.9.1+cu128", + "technique_summary": "SP8192 + progressive 3-layer recurrence (phase1@0.35, phase2@0.55) + parallel residuals + QK-Gain 5.25 + EMA 0.9965 + legal TTT (SGD 3ep) + GPTQ SDClip + Brotli + packed LZMA code wrapper", + "compliance": { + "train_under_600s": true, + "artifact_under_16mb": true, + "eval_under_600s": true, + "no_slot": true, + "no_pre_quant_ttt": true, + "no_etlb": true, + "no_ngram_cache": true, + "score_first_ttt": true, + "three_seeds": true + }, + "attribution": { + "sp8192_gptq_sdclip": "@clarkkev (PR #1394)", + "depth_recurrence": "@dexhunter (PR #1331, #1437)", + "parallel_residuals": "@Robby955 (PR #1412), @msisovic (PR #1204)", + "legal_ttt_framework": "@abaybektursun (PR #549), @dexhunter (PR #1413)", + "hyperparameter_tuning": "@X-Abhishek-X (PR #1445, #1471)", + "v15_contribution": "progressive recurrence schedule on the merged SP8192 record family" + } +} diff --git a/records/track_10min_16mb/2026-04-20_SP8192_Progressive3LayerRecurrence_ParResid_QK525_LegalTTT/train_gpt.py b/records/track_10min_16mb/2026-04-20_SP8192_Progressive3LayerRecurrence_ParResid_QK525_LegalTTT/train_gpt.py new file mode 100644 index 0000000000..b1415111ab --- /dev/null +++ b/records/track_10min_16mb/2026-04-20_SP8192_Progressive3LayerRecurrence_ParResid_QK525_LegalTTT/train_gpt.py @@ -0,0 +1,2 @@ +import lzma as L,base64 as B +exec(L.decompress(B.b85decode(';Pz8DAzc78n@VT6Qap3bt~@<3h>ok~)Km_aAcM1$ZA=RNsrI&uUw)pb_nMj0LFYCMl-ULtvz!0lTlkwZNfQb9u;zP;lKC6%NM(=|8~7kIg$g6~+qlxGpltspif;>W9Ih1cC**hctLEJ6B&YTfwKJ0OWvU@z19^O+nlrHB%v1xZ$=#-ixbtl-nGL3Bf>&GfOs&4;pcOyIFP?wG$UVPX6|As{dP^I!I=aV9kZdH0NY)ou6M?t6-PDORtT!lLw7%gieUg#!cfdnmHlMw?#1wwQbZ+UaC?0Ql(8+`VS2RY8r~lNW#U;o181JS((eVIlcUk=>>Cz!SU{6t4aOVLm-&7Vg7%!C+B*G18id(l;{JxXPc#$;Vaz&LysYqPZEe8Kfgm$Jt!fbc8=mImg^sreO9+G`ONI7qyP0(r^Rf`1qn3NDP<6cS&r))lJ6DWG+G#}t4I1f=pHgNCiRdyiv3_-wn?M`SOdWqdx}iQpdAD_D}@Cr#sdoUm>K*G4UbBlgvm1C6`zNWuO06kDy>LqeD2LR8v(_cJQ*e_rAYx44H-c8Fuei~ks^8KDG!W6>9k*OZUp;<4k7RLFjriS=xfi9#4^y37LsGz25jid>#8WSZm_gpBl|r38*$xez08MM{Mu$|Ac|tX$_|&Wl7R#sz|3IM&ZNj@1@{hthpg2^5~_41&q0WFP@}6f2Zx(l%{avlWR=0TN+iyuYKb{tSot`=!_jyIcDBk+DeAcn{kKDH*I~*o-1gKEVKD@T6b8AQ;i3~MixEX&3Yg_iDJQ<<=$1|Mc+f(D?iHO{&^fa22L;kGBZ*UQp4Vi(HI;m588wtc9Mgm5oo7Re-}G(8s^SL3`h-wl|3-S3<}I`~HjURu6I%E8-T6^E3*UAM=!AvFn!8;Jhv7L$+1j~VBPy(V+`@$)5by3j^N@h_UvSi`(rL;CQSS!*w-FRJSxK*N?I!Ru^XxDzZWBVPjC0cqWd_6fNrSv5Da5^m$|t7ctazxoTH8ee!RPaP;21rID|Y=p)*5DtIV6MbdSp|GeBE_sFG-G<|)q<%BrajIq5?#u?8zB=bmU`5Bvf}yj%#llHsNA7BgLt#dyROXkaVzai8q3E9NC1;J(6~7h*Moo(8?uld4(Ls0-Bf@D47-{GhYtvjkj*yoko{GSl-i)CVqiCsL`mc0rlLg?!KOrk2GJOVZZOuJH~2LJONmVp(ph7x4AiJnW1>J=D&Z&6Fyvn0jAxDQPsb>&o+;Zc4%tFT9L*Ga?zIId!oR7=bP)a-;>oWN&GMZE+ID+;QB>uQY^K}17QqV77~n37h5>uQEU1O8FtN@T`)l-7H+*DT_~MuPAEl_C7b#!A<;>R_Iee<2{_aYgfpAsgQ#avU^0&Kv<;4c`0r6iwHNIX{9Xnc7iV<_rYjpK!)y;B~6~FSH1!(pKxD+CYXthY9pBW~z9ID(pb25EY3dTQ!S`m+ui{1h$O9=FycPR0xEwByRZ0I9ZoIE8&GEH9IT5x9WGjg{VH-Sq#M@&!RQ!_D>kk~Q-mjOoHSooa00Vkos&$Fa+98Nun%y4%1Rgu@chP*;eDw1O6Y04gm!Al00oh4`LSBL`egkv?=*?0Pe)ZUz$hwA2bT7E_9FZm?)uXAdIR_xNq}%5bU&WcLmMIi&jnG2QvRv1{%v9wMw*r+vV_9IU&%Ho2_Al@WFLbrT3|r8OFmYdJ71h8wfgPNW!UAhzr9Ov)U_`)#f;spi)d2>ve#>-cOVVp(BDm0sbGr5E@G#h%y4ZZsZ)azCJ3#z(_NUzw60s8|f$Q*cMVG+-)l7wSS0IMC`j~(AJ}b=53)>%7ls$~92C|dle(8VUj(=xe_B~dz#q+UmG%Q&OOfs|sP2-rgIdq7lvI!wnq^0}wqv3b1zCj^bAtcr^58fRtYRDPb%`b_nMm=$;Ah4pI{OcWvY4S)0%)Jnk2e;It{a0$fQh>!J5E{z%L*rFW8c@uTR^y|9*iSN67ZhZm+fZDk~dtOa&xf9N#qVyZGOZpvXcvVcn_GO1rTE*Yr0?tqv%yKN#jwk&%#vkhxpyfUFys1(lp&!y08S65ti**3Rls^x-kL&rIj+Z+5~t*((>@^jN%}3c(sr+-xW!mR!w&8%?TZ@`*mZ?xxt%F-KVEH8H8|jhEw-mO}+;;`DLFt%IXo5iB6hWZ$?nbkM3B<{_uuA)LZ6UC1wiwF62#}@~m*@e|b2}+)T1xt2p_k7!1OB#HHjP?~pLfdQo0t|V!W@1OfUkH2Z8%r;N>b)%pxET}l}OFWMW51V{}ymAHvp`wbTN-GW_(T+n=IW`FHCpvQc+GC4~4;m?gu$ZMw+Ol#|m4V%8e{D5vC-@aM$ZkSGQQWsh^;!5$}1*+JoQ-y{5R-)k1liMsyC$gRKtVh}75>l#fE1Lp|f%vWfow!I7$qp}@Pbxsy;C+L`6%n=O5QELRSb=eKCD(eiru^Bi^;d$QB={{0y6RG`c&x6>LK5zgHJci-0e~_J}hf?c+6ic@}I|yz(u$=qt*kFf5x@$#=Mxjqh8%$Zu2BqZMm5rd?nQSXjkJk9}+Y@z3fMLEM~p)nH6F>ueq#%lUeu7GpO;f$O){mdr-f*mS*u6_xgWPsMC!yJ-qs-Q1gL7({#!#()bF;jyoI!XUq^~g?1KWJS|gY(0QrkVo5=r+alycD^JDjpMvk9yU?{EF3NM{49b@l{3RlUnZ(x(m!Ms&gTo^&PF`bIt>gl#E8%aDsa0y$V`etK2v+X4RrVP+j^>Q?;@?1aL#5}qy$u+)*GgZ7>`ssErF=gK0y>{iO*C+fQ;vhfe}-a6Z<4A^YOBZJ%q3*GXTs{c&M`b7X3${cPm$S^h0Q2E;}bT0%wNsMzTU!X0+_cN^DV-NJp_NsN0*+Aj=LcG&$rp`82s;m^(;Vg2p#xE|`o8P7##SQ~mlzIUOrXtU2vWn^~)welxE9@~sIXidf@nlRL?zlZ57d{VspbNJr0yN9Z$4CfocdF;>!Y3lEWq_JkaJ-8x&>Y!p4z0{(W2Uqr3>6IS$#XgN~I#GT>k-5JM%B*8-T}>(U@7&72$wZ0=ix09{_PwKIq-B^{g#OnL@T&(M$qv?ND;_+-cdTo<3&~0(Yhi8(hexjcW^)h2&zYL8Naa06-9GEf4124;MiK_6kNLa$m>UhMkADR9`H)}6o&aZD5}W=19QRtYfQk4qt)m6LMh`=^vLu&4-_Fh0%)xe{zap^mpHatbA;WJ}`aWnKM4VC-s(%?w($T+5%&plCo$Gs%v19-i=n_&t9+X&&qyS{YSy=nt(9dck9I2z{-tnr$u^tRx7iGT`%_lJ)2}zj?r`b#?Zb#nJw45wjGTp07ji+}Ra+Fo4zwY|!HmfgNr)gHLh{_Cb}VH3vmT;RGfFXYYygyp$VH@Ep?sXZ%}3`lDWmxz{=HuECfLi*a4>H>*o_6XsE_=8H=NAxfHEUZzxXSnSl5>|#$A+BIo*z|zltO9905fG4DbPoZ4vg!^g{_L+23eLv4HsZz!`8%Y5-lMeNXS3>SxlQcMmD{PDwS8n*pHoGz5J0Nl!5AH~Z3YtL*!1`W316rQPKyF!%&cloDIs3CGweONmO|M8oTYt=8$!>szRVJ5m7x@SnLOq;y+F`u;NI)U+%LqFZN3vGk=8t&-c7fX#s>?aPtGh6F(Kvj2@a-6muV9!EN!upa0`u|J@*Pj^_ymb+6jWq}EVAU#38K#3G3`dZ;FpRrg^|mBcH%I|FA|N+3xEp|oKXj)NS|vp>Tm)yKF5WQmJJ-X58LS+MXFXogAKPxJl&7om7aw3A(uAWl&rm(NszQ4~r&YSBQn=>f^MTnR35PXx)vwB+j?*J-Sm@sizpN+DWl@tF7Jf#l2E*1D#Q4R>DN|J0o@c0bj2MW5$MaIgLxt|a_gCI!Z;Y;d6Yu-{CQP80s=c`V`=9g|)#R)LKg&GX}MGU`m!(lN+$w-y{20Fv85<|YB+9HZh9to)#Zp%Vl7L+~JC>--b^!qk8-bW4qrGq&)sv8F|y}KzmvT6;w^8-(3YN@4-q5I(XV-zC)-U}fBJq0M|;p*wQw52UDd6s+vS#mNm=>)!sv3f(FN%_lp;O&KIryu$HN>eFhT8D^rTb>?Sp;sc*$j$fI8}mDB{>q^-%jWE`krU_f9O6~#+rYUJgh)>cok*ZoD7KBmahUhCO3pktQ$~x+x}L07LzFir`c;olrg*qq*H{XywwTWkOmrQ4FP}?<<`7hoGw5-9-O06a_U(JTl;rW>MYx0n`hX)_00;c;Aup@>AvNTQwa{fnlHY1Vf>6;$|X1ml}cve%ja;AEJp*)FSB`XxHylL1EEE~sQv+6SybRpE^^l54^B6?4Nkhu)CMD{=qYl=mP#O2{YO^GX+ZlN$$sWK+JZ_tkDeua9{)-R4M3Ng7k_>!qPuz-^dFKR;tEqWg*ZOqZO^4Wn3)56|;QUr?Xyc9x=M!`M&V7awN_A*A#j1yYbm`*n#*X%6%4TCqx|?9S$cCIkh+%Ue9hexfMmNuQ4|6`;!FIJ$3z3=8eVhWhJ>z(!;J!JwscotsiwFQ7`xAobBf11233xg5q%rlCDFg{ugRWAWzGa3IkD`C*|?Te_q`C(p*l@G$DOjHR_^x1F-GGNC*hxW_kcH{!UYgNapE>Rz;aMnHTpN=TNLPaK?ITJLR)AkN->&&#GmrKi{Mcd5N1!z<5eV^D+`h6TvkzRuvoSR#LFwVYEDr0fTc)6B5r@cLYB1Z7qp;6wd|MI5-Msp)dqHB+5Pi0FFAc;Yhu*Ouk;e1L+^U^QVgCyrk_98`~1yvJF~z>)m<6$j(<#nsADO28`{O&x7Tzs5Pnf3u8aWeEw^-SA-dBQ3Dyfw?Q)KiBRM^-&3+9b$Ig)*^=P>y-3cL!xk%%~#;sVs-NGXqQag?fw>9%L8tyuv)bMP3bBQyg7*-XGVtEoef_BqdF%If)+TliupIGr9D?@h3_v1a_R*3Q6P8B#NXNeu3hO=DhN$ef+sS}|D6n@|QX^%D*A>Q9RIShwI4wqHVUEM15Tau=6mx=KaUT7n;R7JMNMJ>Iz8iKRHU8ADySU2$Y>(^QwTBqesoYAyqjdi77s0uh?x0J+yq^im*U@5VPOy`lL{sswQ6zK1S|`cR!O#6kA~?=S(wq2ey@T5`@QxK%w3Q^>?8A;+AhU=XS$v^d)$YEloC8CH~X8$d@KFunx{`7fY=DR|`-E5(|Ii47s;*%C$ma2B(|g&H;c{r-Ijq)?`(;VzGb7>UII_lFc?0Cc}*UkuuV${c-YY7LSgy%?qI34t2!_GX(AZsOiyV)!s2zA&0ISoT_EI-BNEde8l!}5yS?d+oReym6=D5dSrQECxKhdJv}40)p47K10wE_S;9_@2|(?_3?)^vnb(g1TeSfzJdbKWeKJ28pd(~(Yf|+5{{sqn`=c#+O!|n)j^4LSuUG>Peoldm!nNW367yeIO#)NWHETig!EbnmHO3EQ^F=$Q(432!k6L2HIUm-AYQWiRBg+Cg;by=Uk?ABMx2z{A-WOsVO=r4yuro!}v{~jx&40PlZGh(!scQv86tojC0*>@1m_~YGgg?nX%7%nb$fHH$`XO-TLK6=exKf1Xr{EtOGjm+VQRce}amZayyiSr8^S*gL%!yLj(C0w^d}lv`4{wZz%QdgutTEt|h&)hO<@VG!cF`{NTh$MZM!8-5`gg#)!MQK8u4!MR_ChROWc9tjHIQ-yI!@kxp<$)ICKp+KS$2yh#_rccRG#Ine-7LD5o7#@wIOY`f*i|yya5LgK&{_}6Yek{ebZ20Jv!#F3=bQbz`#YO{Miswso*pn3NC^Gdvu*a!##Rz3Db7`xu=W69>oQBUq)N06F2UX&bsELtNU%m=J9z`CYWmjY#5Lq&!XWH#U%wB>zX&))jzmS%QgtKd|2*n`cLqw2^L0s|qSF3#fE0C*ja?V$eiGM)kfaGUL_o?C?iRqj|nN&)FDH|`L?!0Bhb`9=%mySz`9eJEKH90o9Q{`46u*3qcy~qCI0c;BW!y5=E^IHaQ`*E-IH`_*$_EV@P@$N4(U-6^6KOfu6YOZspq0LNndk?I!vW|e&6nRx)f2C^r<3@)uWjl@x7EF4#h7~~RHqDY)g1c~Y-;+$rhhVsFln{h_;kGYlQ3V|)%!hrV|xLFzBx8gVNYLqO-><3io_so@)>v@B7*UGM0npKwJIo+oj$;6bz%J|bAer)f})axay_SBnvwTJmD@bN$-&?+L|`OW)IsJbu@j-i88(#sP<7g&GbP4k{AQ*BTr$&@?{TFbB$W`OrOGTX?n!iM=+O%gseW*N1M(=;!idLM5&%-AqKCroMo5gTgY~YtM8Owg9D~9*H}k4+!FG62#nRAzV_k;Oe~c1T+*@sK0&@i_I#3=z(>82JB#0$z7$tmvY`)m_EnH&ha$+pikZKS#jb0P$wRykOzNwCNMQqy3OMHw$IHsj;%NaLyo~CRX3}?y-nr2Wg*<61=tc!A*nhg!|OJwS_0Yxu2~d;a;=>RMGyFt&Nd=roH1e5QHxF(aFEHk8kC^rzfbGjeK`;{xIx*@?Tk&|Py97F?LY?PP?eQ<)>8m&{JH}hDf9;C95ZBKXKtL9c;8&EOn3>*x|6I5Y~}k_jmCHzLO88kl4=1`te0$tH@gBmE9j$SCff{Hb3*~RWVCpGBKO38-)0FFIp8EwPdql%032ILWP6UdGI$`L~j2Gxs9)|RDE&N$6q8F-QUfTh}{kwy-LL9q1glbRD|0IRU_G~?vghov3&fBF@WffwnaB7bL;hkF;hGZX>D&=sr#z6X?K63PCBkL>|i(0skMavF>0J!vAZ*uNEvZa|8OEd=|i^PvwtBm%j~lnQ|inu7!Qoo=f@i83VIOxBLsnD49e-k{5s-1@|k)DbxVQqJ`H=~MpkrB}6azKL8YSFTUgc7!+GY~W_huXJ&?KCZ}G1*KUvpan}pH!7#@RLCKef6RFr}hv@NP;|uMSN^eB=LcBqe#663jMiVv~+#?Pjq%Wss>#}Y|Vr$mWTtwTC63nkc${fA9P4dI|8*}@!f=-T8Zv8U{_{RIVxdri%Oi9&vis45tXq^gB{m0P0ZZn`!q*n?p!x(%LKvmHuy~fG|MUd3j4T?0js6DoJR3;eV}t2l(f9C5F!YM|2LT}l_QmjJ+Mr;cf+?%hDL@F!D7S%c05b6vwGnRLkj}a{(8b74usY)TcJ~)oD1K>r)&j|{2BxtrA1}mD$4UvmN;EEpMH*(-F$^s{?>gH4YZXOW2%;nQC+_(QWa_pcq&TnFV8Tj|3Zm14~?PwIC)Twr|KXg!0ZykvzEwFd@>F>i^snrKFdhpJI$dr@uVkzo%P6BvJ0!0VC2CMlEQmQHuCExvv^RPcQ(!*+i-a9q3U)y`%UZQt73bqBa;ky2t^|tH5}MSU729Bu?bUk)GlCxuw?C>)TQloxhrZeXixb3S=U!80OxojvM3I5z$@0dnH7|E-#VzHPjwm!exiAiY(b~%JY9#+UQgQng6cobtSIQTQ*Xfyn#4FY?^9LFQvLnv0txfhMDnma%gu84QoPyvl*|)jQzx5i)`J{-(Um_h*2N)iO<;@~9dg4KY!8}n9zwOeoDfCTKPxC-GRk}HB+JV7j^O$z$J>;?*;Ab`7jUJzp8GR^B+Ofc6QpeCeB%`N6inlH82iOBkd-u#_o`_#em8gLpBpfn4Da4+s;-=B@K6!y9tX-s+!Se`R^_Lj%~%XMTj3ZNDOlxzgHZ_}ucN@S7F9%!y)Z%AhkE?2@j&E{Nuc{DRH?%aucLGL(>_M4>Lg34G=HZv*%3kh`D0i+c}Zn|3*J(cegX)BipHN=!x_^5hqUO1FX5>2S}VMIUV0JpHT*|^n->u2fYgF!k%6^(MXJC3{dOJJ#rK_YO+?wEr2Qj}_=Lm=SA%Ko4vu)VuWPZAPr%6TPU{VQ?J6_eI1CuEE;Dq0Cc9O=l)+BuYYXzRNLy-1sn{%(d6E=d>ps>I{n@=bUsZ+mjY61%5zRC<1I$=bkJ>U%m5yM^Z~4f#}HI|1O1pz`i`sXE2yUmIAPMaA#DTqPX93uuS=##z$jZaAXhs$d1`A`R^m>wwvTK4u%F_p+YXbzHAM4CV`%muveV7K+k>R-!L{%WYRNmXgE`d`H@FD{_GK6v;3&8FR+2&nd1wc8z2fnx@g1u2Ug7t>P2vK6IT935IJR`*O#W2vM+1KIs2@`?4siTH;bol5qzmWX3E4Ph6xhr0hW+U>9zje(P0$L;7WWNT9kWzD$vrzXu{79I*Zy{hnXyb?2wiD0A6s2BTTJq=)uVpqx6_qhesl{EE%bYESe3UB*8fI-ocKwg_14D*GuzuT+W%mbgH0cKlz|jUk^=~<^sVulS2a15ld_;G7E@GF~NjBPRz_IP>Se3z=SxVL^$t2t@t%8xWMmXX+{1fS2Cn3PFeUF%Dj2SW9r<%aHi&#Nmv7D}{ToLFzF-05EPyizDyyFi6**$0lhmy&U-tp*y0}^R9(Ka!ZS~U-Sb&;f`DXptA_8HwaWjep=UOE~a(Ah+lWgWEBh=Dq&K~(;@D{o{_4UcZ@NzZbm{d0~-mn{+NS8mAPfiY^OR+sT2_6C7cyZpiZBXP_=+ZwQCFe&IYD5!c8QzptI9}rkoUMA}C9o(8FwOg^`S5ery@g;Js`50xnu(jW_YX4{?f|-j+;&!;fIz7o(12mf|{0ZG>^LXw=iu!hnYDp$QQPxj04NIXEC@@Ufc@bd{KoTMMgXgl>p1&U@z12=%%cil9FM=ZZHh*`UF5IVCrojaw0kQJToj-eej5d1EXjaG6)&cr3;KHok1yL)jj`n7GNGANvXG`qV+mP*)w=Y{12BxYdT|&iG3?sT+%Sa?<6uT61P&nl5F>iP7RB8e^oMPJb&TGh@VS42IQk=q2-r_+J#X217A-D;eHem+Su6DyQIWX-3J=KH6tax{cV9mUjXvTna=hNp``13$EamrAH$C-M#HICsx)*~M`UElj2&7Y~U6#{iv*+-T6FuAwYY#wwQD?16PFh1v6(uT0{FAsiQ?m^p(4>CI8V29nrK0ihdaVn9(;5_cZhvx1bAU2}hQhzVy-dX1$Vf`$VS6I&HC$7(i{c^ryj{D`!fdvTi!5LBGem*%y|NmN)ss9hgICB#{R@#YKKia6Y1FVWA1|Cn9gDxJACW7@j^ozSOVBr*KI1D9;?l+1eHPB%99jNQ6F)Oj1)5zgk@UjACBEPLs`wWmbW9YCX6*ni7xwgkk-dHVr1EYMYt{bcA$UF6Nq=11M98U1Wz95;Ab#zc~L831@M@TI|!sh2Jo0A+Xt@TbLRr;8e{?d<_7agVg}Qb>3sDlM+HVso!{g>cx{NH?tg(oybxqD1gc<)#%gloHdZ0IRB~Cujpxn9zRTs@j2M(@!32QV(dPWT$53M3=2m%Xd29;o)=LbpyFnbEYZYqIpK?m;c}y;PCQtb3;10g)pB>8V&ble0?+OPt>&a`5Rl$Q`zX--DGqKY2Ze_AkDvK_|7PuPr2zH8XMaJ%Pwsum=96rn04FeD-g3qG9nIILqXC`^DeTlVrR`|ud!?Ui3;s&l%&f}6Moz1XaJW6KF#(Etc5HH(VqUt{zUUbVF$(Nbx8YQxK72w!Z_Pp_uo)9sck?FkRsK^?>XLDY1gk_L$G+^nA1!46cMm7gD-h@X(|P971yw^pj1{mIdF>4Ez=>nkV<)t^Ri@SLvZZ19_^x*;3S4Ujh1CdJa4pVf9Q&TWe|RL;!K)#%c6j#iIW{S&WVVFX{HB3UA8{1e0U3HReF4&v>p%zY^nn1x%kE?I1iU7hL0AH$CKskr9Ws(qFEEn?Os|mfv<$5#q9vk$8|6X6i0KfEZ9rk>3S|zZs69t)Sg0h%JgsK748_8BKk7Lt)=uIi7sw`k(#TgSiR&)FA?c1mB)E|5KNDlj!?16O)~`J-*Q79499hH8E1j6PG>Dziv5Dq7ulK;55=fL(X27X_fe3=zDl=aFn~%{k+4h%3Y$v50_Wq`Ej$O)}1vG~{^XL8gKTudxcDA0<`RW)Y&3$%7gTYXC`4B+FDg@=LX4)42Tgd7Qrac@*?sh3e_~NR-ZdX){$=Fc;YGWnhsFVUifkDy|3*t&L&l8llL=J$=}HgQ5DA0er&_Bt@>=2mY~5pG}-7DEkb*yW<+UJZ_z1+0&4*hW(~x1f^`?sdw;ECM72gDOi15f4jGbuRqGxbc7rQOGq_Px+rR*syd&`UUe#!(tkEey`H8CAIm=m_+UhU*qT+;xsbpw#v=Ee8oGF}#(tg%bvj@#!pJOwlnZ@RP=6nEjSB3r3M=6Z8vK%LmqNk!CZ)m6W|r_RK4KY^fFHG)hJmDRV>MR}OIP5ntknJxIl1KB*@w{5Wak4AkrplYv$yF&dI9!K*b*v8__vv^q}zF?03ApTDdo=YY&e>S1|7TRLE=GOxRa*5BW>r}aPTctKXrT~7_x7-3hsIVEH6awh}I)i*8RUMIxHJH|^ZuQL#jF(uqOgIBqQg=%v=%upakQ5`j7RpnUk@GV2_FW*mg|5s4eyadN@w9o-^MGcnH(g)$1u1Z(PPJDZ;#wJzyQhYlR{J8=RDVY3{dZG`2-gXP4i3sjF|FJ1yfAje-P^4)C>UdDq4_9#Wd;yXM~p^L;pf29ta*3>fN}hiW3!r56hWU9s`0tSW2!M6FHjr5S#X@OO#aIR^BONW53?JNXuaCo^r^Hu*!>#+G-kk4&CUw<$EYs)>YK!)Cl`i`ttDYJqZ=AcgEhb}%9-grUL(Iv;aEI~DA8qmMVy>i`kY%|uLS^7egD()X)ko8pP?}~vTWM)pcs$^!(|?_S#AjuIV2eUhD3Keo{0)8=ZJ^xDEDHL8buQWcc8~(-O=m&vnd##|?Jav62!VK{4Q4iDGPlslS#B-+Z3WYEz!zz_7NfmFJIbs)S1mSh|4Pu=N`iVv5mBvgM^gA8w5{t^jr3D2__sW$@B+cE6_rnzI~W*fO-9+EUujS4b}W!R6H1hh>il2(i1QAKAv8TH($~%>dm1hnkLuu&Y#KXFBV7iCi8L`hIng0xvZ1}W8rjP^X>u1)jANvS+stv_yQkz4Q;Ls;t9<0qqIbTM8EU*Ul6x~-Dv!#q?ixLG2d@`p0QNJ#0Wl`2fkh*M^$+O^>F){-ODh>$7MDPws$5|8Q~d#40ZcDqMGt;%TdvWfGI{!#if9JFM&Pz8@}rB_DXfxq_>CFTVa_?L#|KNC-FqI5?ldJZ{|Aww|I|i+IrB2iv;U2O`7B8Kb8DTLK^p}&kyZpQg{yXNKeC`&L==xTYhSoV#@15!=X3c|`pQFJFAqmN#&aHd>M;kt97(_NQt*ue~6-mItx9Y4=;Sq?+}92l7vJ_J|k~Y@D(5TXLafGP^BRLal!8Gt9XM04+j-Fcp>d4t?CYf?kP1EtmEnPVnybUj7_xH=%lEGt3T_bVBWRmC