Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Record: QK-Gain 5.5 + SP8192 + 3-Layer Recurrence + Parallel Residuals + Legal TTT

**val_bpb = 1.0810** (3-seed mean, std 0.0005) | **< 16 MB** | 8xH100 SXM

## 3-Seed Results

| Seed | **TTT BPB** | Artifact |
|------|-------------|----------|
| 42 | **1.0804** | 15,994,470 |
| 314 | **1.0812** | 15,993,777 |
| 999 | **1.0814** | 15,991,277 |
| **Mean** | **1.0810** | |
| **Std** | **0.0005** | |

## Key Change

**QK_GAIN_INIT=5.5** (up from 5.25). The monotonic improvement trend in query-key gain scaling continues past 5.25. This extends the finding from PR #1394 (@clarkkev) which documented improvement from 4.0 to 5.25.

## Base Architecture

Built on the SOTA foundation from:
- **@clarkkev** — SP8192 + GPTQ SDClip + MuonEq-R + depth recurrence (PR #1394)
- **@dexhunter** — 3-layer depth recurrence (PR #1331, #1437), legal TTT on SP8192 (PR #1413)
- **@abaybektursun** — Score-first TTT framework (PR #549)
- **@Robby955** — Parallel residuals on SP8192 (PR #1412)
- **@msisovic** — Parallel residuals concept (PR #1204)
- **@X-Abhishek-X** — Hyperparameter tuning (PR #1445, #1471)

## Architecture

11L x 512d x 8H / 4KV, MLP 4x, LeakyReLU(0.5)^2, Partial RoPE (16/64 dims), layerwise LN scale, tied embeddings, logit softcap=30.0. Depth recurrence: layers 3-5 loop (num_loops=2, activated at frac=0.35). Parallel residuals from layer 7. Skip gates (sigmoid-gated U-Net connections). XSA on all layers.

## Training

~4600 steps in ~588s on 8xH100 SXM. EMA decay 0.9965. Warmdown frac 0.72. WD=0.095. MuonEq-R (row-normalized, Newton-Schulz 5 steps).

## Quantization

Full-Hessian GPTQ with SDClip: int6 for attention/MLP matrices, int8 for token embeddings. Brotli-11 compression.

## TTT (Test-Time Training)

Legal score-first TTT: SGD (lr=0.005, momentum=0.9), 3 epochs per 32K-token chunk, cosine LR decay. Each chunk scored under `torch.no_grad()` before any SGD update. Each token scored exactly once.

## Compliance

Per Issue #1017 (Track B — legal eval-time adaptation):
- Condition 1 (Causality): Sliding-window eval is strictly causal
- Condition 2 (Normalized distribution): Standard softmax over full vocab
- Condition 3 (Score before update): Each chunk scored under torch.no_grad() before SGD update
- Condition 4 (Single pass): Each token scored exactly once
- All artifacts under 16,000,000 bytes on all 3 seeds
- Training under 600s on all 3 seeds (~588s actual)
- Eval (sliding + TTT) under 600s on all 3 seeds
Comment on lines +52 to +54
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compliance claim that all artifacts are under 16,000,000 bytes is contradicted by the results in this folder (e.g., artifacts are ~16.02MB). Since the repo defines the cap as decimal 16,000,000 bytes, this submission is currently over the size limit; please reduce model+code size to <= 16,000,000 and update the compliance bullets accordingly.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit 6fdc0c6. All three seeds re-run with LZMA-compressed self-extracting wrapper. Artifact sizes now: seed 42 = 15,994,470, seed 314 = 15,993,777, seed 999 = 15,991,277 — all under 16,000,000 bytes. README and submission.json updated to match.


## Reproduction

```bash
pip install brotli sentencepiece
pip install flash_attn_3 --no-deps --find-links https://windreamer.github.io/flash-attention3-wheels/cu128_torch291/
MATCHED_FINEWEB_REPO_ID=kevclark/parameter-golf python3 data/cached_challenge_fineweb.py --variant sp8192

SEED=42 QK_GAIN_INIT=5.5 TTT_ENABLED=1 TTT_LR=0.005 TTT_EPOCHS=3 \
torchrun --standalone --nproc_per_node=8 train_gpt.py
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"val_bpb_mean": 1.08102,
"val_bpb_std": 0.00051,
"seeds": {
"42": {"val_bpb": 1.08044, "artifact_bytes": 15994470},
"314": {"val_bpb": 1.08120, "artifact_bytes": 15993777},
"999": {"val_bpb": 1.08141, "artifact_bytes": 15991277}
},
"hardware": "8xH100 80GB SXM",
"training_time_seconds": 588,
"eval_method": "sliding_window + legal_ttt",
"key_change": "QK_GAIN_INIT=5.5 (up from 5.25)",
"base": "SP8192 + 3-Layer Recurrence + Parallel Residuals + Legal TTT (PR #1394, #1331, #1412, #549)",
"author": "G3sparky (Gavin Saunders)"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
import lzma as L,base64 as B
exec(L.decompress(B.b85decode(";Jw{5fL#DIn@VT6Qap3bt~@<3h>ok~)Km^%<bI~`7~^@P9dNt*OmJtouSV|m@^}~LJOY#qcoM@9BaGiY8ypPvdJq=NbK}E`t%**OWHq5Yg*w`LcO2`&Ki8P4h{e@}LyJM^e5fwE@n-bff1Ph*mUliW#yCOV*=I+v<n|QZ$y02ziZN=i)3}Qx?Dtm=+{LAgGTC@~>c^ys%R{D_%yAk9-_tV7^coUOo3$w>`(`ci)t`2F7>r>Ltx>>S2CRw|7ov>Wn1e~_!RLQ=%V9g?)G3yPsu%SBy!lj1PaC-x%dDmCDOZ^r^!)+WWz}ejKXTJ#^U6Ra!};QocHHXQC+4UM!QQ!-N5Xd|%~a(9)bTYIO+>B~8~@lqmri%^qEkQUy074Rh6w7V_#^s9J-3BNA`G;qyR$LYcI?e+loZVWi~B$n=TKFp{%SeHYp{oNWh;U@Ahk8M2$OU%K8B$lb*dRQXd-GR_@*KAZdRdwSd#v=LSq1v@Puul=a7WXDmh1^kBj}Y2XlER!D2E{&{%lV(hz$#n5%+%sk&Q}>{y0xpRgiQQBJeVV0hy8UD3ntyo@(Pv+K7^zVRDt4bah<BW;bd+Cav&<Xg-~v4E$T<1_zFr^A}Ais)|vX*pO`3FxNShvEi59luzir1VmzF1ek|5CGkl?5qc}ko)L1|6lu6I`m+*bf%)mk)RQ$_JmfE%eS#TV<mEIvI}8;<$6J^=w~cJs5Q7=n#vHa4v1R&1_9=;Q2@QfQORDBW6CkyC4%tXHv%r3D5rF;6FZ>(r8kfsZThb+H1)~K-lIr4`|V#-2R>G7pP*N!fwWd&Dq8C)y=NrG_U_Oz6Q?+@ok1?(VJ5?ZT~&}C4Ks38WRB>3i=I!}H-8qq=&yKJ;tbpwwn~lAseD^q1C*u5T;l<jmq^*|gW0-BBpM5(&T>KQtF;?zv@u0f36%6SXU~txi3v5iSPK*`fNE9531KaQDL`zTPF$MX4U(-3sY-&?>QJe)giBQzpor7H)AZ#4=Hn#`AoAL7tT){&bw(fgz|eQRt`#6-<>;m*+&$!nf|od6&lVKYYHuOoNgZU_L>E@!O%__mlt=);Hwdc43+CM?sh5y+my3XSVYMO8F1pXuq$fvTU<$mpDjr>Lm){DeV)>4AKAhA?jxjH<-3yYQ<QU|mI~;=9eDwNwPw;)v(Mu^%YrcdcwEH`*s$RtDvffGRv||Tctzy{1@`75RzWmpHgSnSab&b&Fyos-0wdom9#6ey3CnhkwfuHfVJc+7~(p~gN6Q?zV1K>#5qz+4c`Utifny+Ydmr4?c_z60#9@FU+U1&O$Lfg$WrX7gCj50O1t`1A`k04LVr;^*~{|@(TS5>#TAjL(B`umc8bVA$bS|F?^2A7E}z7IIgZlY(8Ex#K+nLh0vzlKK=74U!g+sX4T?e3_^_7XB1A(HB{pYd{vHYcak_P3DZ2LAB20wAP+C_9p7R|0}wA=p~JFi&xD8H}n(LxCc5rcmw<AY&of601+p3__P4;S)9wD!!2(*apEWmyI^3Nlf>F`!s(tSf_<O3Y4@Lb^pR}8!JWi5F-x7_`)oM#pVlXHEPzi0_w{=OcHSWtLTLJZ%v~xfE-VLIhk+V3}!ek?mRs^iiZfQr<Cz8wcNzPD3j5B{~Al3+P&zpBEN%rOTMb?hCP$lA?OeffC?6d+83}cL8pBK3e!YNJ~9kmX-Ex)_GbEil+L!j=<aZeLrs`Qx2fCt`V(`tk{k`tP!^_a&cJLbp-5%L!|FGej~o0jPXI6XZYF*KMbV3}lTx@Ln<Lua-O#Rd6MFf|hg{hiGOq{aNC;vw8&q5AhpiiicR}mCe^oxpwpei+d|T&qL>l_TXk(cPZJ_z`)iV4#r^gzawYQ%HE1iaUF=(KAcKXE%%6Hx0i;?;p1w#dN7!-y!(2GUw()t4|BXt%+05bu$y<wAnZSUY=-zn2daV7XQ!FRQ&E39a_e-evNQeH$%?k$j{<f2ReW`))eJS=r+^AO%o{S-|mcA#-J)~|Na8W7N&x7Y%qc}W|PQU3N_33v}9rHTp1gPXKFiZ=ghykdcGL*?F!Cgp^gGOHsjlG6z&`N$_rfTirGgHnbxUmK`ZYAYjsz^O;`V+8FnR!3RjxgtUMj}@iffiyK*40=kBdOKbvmxC&OtL?AD%tI-f#Ue!e@rkDfVbz!EEW?KyfWIX?60i{``J&VX2%Z3u0DFujf-mL7MLYzyLu&oz@SWnnB8as3Q2~X{m^08~{^CZm!I=WZQ(|Ri`w4QD6oqUIp&ghyb1r3)XQIiwz*99LjnA%97e+<(Dy<%Le~b6b^n_z}O<Z)T6;z(P@UO8JGf2q~fNKR$4R3eV1RzaSj7^{@Y+^a&@GhCY(7b`{jL2ry%;_^Q!l>ea+{f!deuk%(g-o}&XEEWm!lO+1^On#i#4rhP{bDYb9ZnbGd5n{*P->hZxI<{=3c-92I#g*mTey8O>cuw%hdwjB!#=GH_0?hY|Lf@L$0Qp+k03PROh)o-cMOQ!b>qfvPNJLTVvCBX24BGgI=_|35Bd%&Vq&!LWECF4!1&J?@uRfe=N2mi{l-S0aW101I*cY_A&2R~zfij0IZST;@;xJYti>{)weL@A2ZOGrq(U-ibnWz0BL;s!=S<i~82|F>;`!K6@M501z-dU(OqY0?!!!Tk6Z{9!iH*tDZsjYG`J8UEqJz~7cNEPh1A#iz_$2*Nxo;S{UehRA^{Mf3GV^9hBEKSL(iD!=aKpjYCTqmhYA|h4zASL-v?9UWx8tzm#N5eHo2h3w*`(kHM2e}vDviz3$~a(Y#jlJL?*}m9&(Oqs1+CNUy7g~Z#lRN#>g9{7u~tot;|0qTu4G4xwdXk3eT+l1l$)Vq%}j^^1b(jIvF|OcNb1Jz&)>b)qGiC5P7yS}AvC}VDK<woZNv4t;T(e%=S+yy9Wl!Mk?)7e{+73*u$1ks?HX%Es<Jb3p=`hTp!8!8l%0)~O(9avW&Tp9s8lf<<!52RHDrrWVny4z`ftyF2ZJ6}qhUWU$@+y3>ORD$#^Ydjg!zDuM#$J+k<}O|o#9dvGrp)*yShv3->joMiF%~orV4^0cl9F!@VqwD`5f<QPLctdikXXI0rQ$mAAaeO)6}Zwp1jH%fdXVoxoZ`z5lV>jekV@3E`STlX!=JDxbOQiv?Jp)$Xy|Z>g)@q_QYKopPeu&ghhPNw0y&{j?$GHwDQoztHvVU)a0ca6}7{#3^KK1uTcBkMSF$IDQp#Nhy>JTHPK2w+%N#FZ(D)=sw?BkfduBK{Owa(SkBq^_S*|NP@DI(VEWNqETjYsZ@bch5-dlWjP>|)xv+AknhsqQ!j3!=TT%CYvl>o#XU4AApoVBJ;db<uv@;Zle>=W0m0#FH7by8-Z*V~$!QptJuPqLkH-bA#`L?*g#-60qO9x7)rWh{~YY77{NX_v_!Mc-#`(n{>OHy|HxotTLyFAdqCe^bQsveKyNdxf%^ECJD<Wa?}FX&QnOVoQL$TZ#&f##R{pUuK(_{#}CVvVm7@+&m{N8R2nI$9!ALHZPv*Rk%b#hP48Q>w1jQaV{3doP1nC-IuYoJBS)BjwI+@*WRZpBQq--|WAHx2WWVue@lE`*T9AY1=3wKyIT}9Ss;d##=nZ>!%19!lx_0W91se3dXzq5oIE}=)Lf4xkby*McKg=z&Qh>gK5l~kV<Y3G|XD^O|uI3K$}GY)tl)92}xD#u_|>7t^lQK_s8TXQqiwiAz+UT7qOmRWvI~~2y<G+o%s+*8rsYGCml^rm)<LgsXmv>t~5_%swZRW#&SB~-uWRFRPsi5WZAzJp1&o@T%9?d1EUEpyP1v5zSN9`Nzg4<9J>%D?ZP~-T(dwGEKqZMPuhgN<|K<!knseUP=32Qh=k%@&WeMpwkJyP5WK?vq@D7&kY*+seMo3rb+KxP@Zkyra8qm)C>igW>x{H0%t_&ya;8v^0F=-)sK*R85LA|5>|ZYqA#XmVbFc92&H9WjeO5C!FX%LsiXg6}0#l(Pg(=6hjd7H_7$6NLIA+rCa;GE_3R75D#&J(7=>z|LN$87?M}UpavJo5jeYlJy>3UxeQ{duojamIjZmWv|*!Tjr5rT-K7C#w~_vZ!oIz>O;(D%nYcBK)`IjO`=SDZo*4vJ4V2bFcFg(2@0lt|4uUCPbO&N6^dv4y}sPBwT(0$|M|*?y;Jv@#8^JCr!hxD0=c#R8ALJkOUZ5;?_TS5GI^kyB>q;{eo<-=N|;JU?~G80$0+y}Bn>nRaoX5bq_lK8&2G2D0K(N6U_xX}HirikYywzHoCpo)+j^d}t`9sXluV$o6?ewHe5Ui+m5Y9oyhGHXI2OTu~#~ow24E&_|NZmvkjEEo{?lrj>I+3}kwNN$<|WFHD7&hT*J`96C?gGpoC>Df4aU8P&s$90m&Ugy{5AY@?hVDTc{$QAjsoHqS{ck6snhl_)o^474tl{Idqaq1M4gTm}}RWDGq`oxuuW;}<=ge*lXX+3Fk7GOExz3(~A%nd`>nKrLxi-U((%)yeb9`|*{}XN<!aZ;K4$uq0Oaz(HwoaYiOebn91@RynR!R8j7s;7Hra4}H!6j$8JPBT!r&nU1}{`9xQ4xJ`pVq(u7*0xR)<BH-?0@E)`61eEbr2+TPRO!dQhz6=IyRVaDDLEI15s(e*LMiSoTYZ)%g@+%W997F?^5x*mgOyrJBo>04z>c7Ok#4FmP|M+baT*Fuu4_Vg~%D*h%0S&xmIOVhF)nXWYKj;F_kSpCA=E?|IY3TP731i4AmJ9{uIj}nvkZA~PCqrx=X%FiI(pX*UhQoq9-g$`cDVZp7ZcaGt$}M)fe&uhR9-a*yxklx#6Au8ICI}z@KWrk3Obx%(^tG4C1D@?bgq2jBnZ(O?j&jyR!8J6j%~%zfEtIAPDKu$5hd4V~`Wco06Vlpvp}HuyKK}fz6t3mD@K4unNqV2DlC&6KFhx<%Ai%h1TodO4gAJ51=G}q7(kO9N(i4X$<eW-!*R9YA;7)@J*L9oP=fK!D6|+fcPENd)?Uh;;D^?V=^lDx@B&4gKK``fb35<plFL>8MWnd^UE!-v`^1{5*9tbIpjgUqjwfygmDW~97EWp8~n;>1xM9pw3FxHUd_tgGkMlT)SNNWx(w9cg>9zUZzX{Q^w*d#U194JF-wwlAz0yZ)tH+4r_JXO{=3ej6LQd~!naPgtZ;`)wmc&a4dppCSVTiQcc8UfugJge8Gerop`ck3K6;T*^kZp<9Hf-jlYfu+Ncv+`5i&JOwCg|NE;{ox`VY;Ub%E~4C%G@CLJi@0X_co$N5u2iD|I6taml4pwulJ)p!O69folAr>W$wc_@9v9VvU6N+O#a=gX;BimpVU*u)q2XscX=lx^Zn-BN-;_}UkjSaA)uW&+!Y^2T_W8ydgx(`~YE%eEmgcn-wY0id8}v=bSq#xRdSY89OVSts;`59NKaqnBJzq!px7xcxQkwC!%nzq;ACIq57xa9CNC~WPR=_N|sDZ*mo*d={3!J1&)iMAr6Ji^XPO(S$Grm|<fw5+?L~eAHeAbwmOWw>U4{YYnUtbS1j`5tgw6*^XZGHcn{g#{1)Vn2E4|r#ek2MWQ9#>rzQ3zDgm9txa-#PSX0d6dM`yebz%Twh(oT!pmA+0Mo^hZ(f*S=n@m;h!I4LZqoenOsP%;j#RNdY2pvZFx4)uHo@w&tRGaMg#!x08$i%Dckl;@(Bn`nq>l-xa+7FYu&~4Y<w4YkQu%DC`x>q@Vdt+c}5e4r$M9h_8xqG>aHp&?q8)HWa0Sz{jdpt`RF2TQ#ak`F0Ku`0Z2b&|K}-j9!Ag!ByLopEz?X#Iz-hH-XI{L8;k+U1StEfZI9`UEgh8v|1fNGF!KlPDs)l0lp4YCF?m_QdcJRn{uyNT<tv(XvN3Q$usCUQMQPBAr(r!9GoxzDxD$8giB^ipm=QSOMrmo2|~`WMQD6#wk8V{@d4E^0=<9y1;)!H0-b%jS!&}s$c`c%#yqM0P6bK(eh4|=KIQ1#uuTTdW8X=~5cnI1Xg{T-P}=H_!t(4`KrHAMMa}8a^T=5L+o^Q+-&a;8iKn|MVWYVH!K>lYhoL&(q5rP2Z9SU>2tYy;UDp^wa%Fg~Hhk(2S`|te$Gx+Ii0m3i){{7ug{x+aM|ib=hj3X>-y&H^d1>Yk&CE~y^DBjF0EgaKtEaKI7Wky8h&Y@UJ7P7{NwNv$D(&at=BZzc4W-VudsUAV!}qh{Bx~xqASW>QiwS&Pp?nj~MhW!&p&ZK!3XYPt|0r=y$NiXqT5r|a3t~awh`ELbbKti#lGw4_Z}(|4CO9mS3kjLy9es(hZKys_D+DbXrMuaf<R7ECuCc``XbNXj>)dQf+fd=>LNl`QmMe|5HAZu3&M|Qc$e&*iiiM1kpmZL3f%xon_T*7vFZe9~Mf+S+JioV)e9`9pu;^&wln&a4*Ffme0YR3eyDzaPl4eJOEcxo}+_<$hvv>BL)>?w7w?zq|h-nRo`f!5Z^Z0EYf3QZs73GD`>p#=LV@%sLW3~V7UBob6G;zP|6$TYwVbdcqN#-p-y330KXa?YLk+7|%B|s655R+FA)bcA;K2bp!effB5mS5M{$`4RUF)D-}S2zDLqE^vwi-W%}HG7E`-9#iZY98C2*wag^L8FGhTg>4~1t<927~iA%U4&gdRuPFnqSz#Pp^sW|GtL`42h!3vJLPs{Yk+FgK!+g~a3qqe(;LMP&qfS#hT%mKhNhcUPsWZ`K`DcL?sHamF=Hhu*!?d1u{s#h*t!VqsV3y_mGfSdb^JScFk8jUdO?g(o(2gC%_^uh1$<-K*o~`R)Y+;0K`JnLDY6s#;ZWB1vt<!42|CAdo@qaF&WNqL(1q`9x@WZs4=&f^2FP5u!K#I<Vv3;TVszC!BIc(+apK;;U41_qd_w7i6^n9**N*@iEb)_x{QQ1@lka-;!7IY}I8x=Oa$WGlOg81lKk4@Y8af5?hU+DCwpk*Fc<FOpLP(K$4d~Ns^W&1>nry0W&135=PkgRY7sTp#l@E7$0*w4Kr<2ke+_dq6&O6JW$*9x7%6WHy4`^yxw#Al;YVTb<VV-Mu@9r37=E`e#uVZlaC%s#Gcr?rD(q(p)OSLJK`vJ*d=};9~C#_8YOgxIv&j3f&Y*~uTDDmRa!#J}XYUc^bJb&jTe1A_s&r&85yBMlRi^|$*qDyP)y|m!f7~L@N9AsB@$Ut#ICLclm6{ggs&QQ@@81!$X&@+zKkQq(imrNNx0_}e8gB75FgqrV-rTF+UBUlaLsPfE@@F}Ip(PR^lXW!89kvOOeGKK=#<Q@S4Y5Pl&7~CWdN;xfb7t18SRixp=XcD6=A)f?7Z0?0mAa@g-Tt@73YTtuRrf}Pq2}-<O8@CD=_C1N&%?dT?wH1|mKOOQ89r`v1YYh295gB11aT)w8Wyu;8ZG0pvm?(7>3CPy~Z;@}r_c~jG3g)%+iPOR2*zg^<$yYfn!U2MNM&RjuDjCBg+-)-Z5SNK$@YUY3LX2>lQFN54`UZRmO7GTUTn(e0S)DjFC0qZn#`W~1Tlo=z>)mCXdAK+UBFc44FEMl2e8`+5f&30d-K)8VmW}JZU`gXNq<|eO?7R^m97LpKwJUX;MC3lNB6M=aQZ4k|KAi4}mlh$J>ZI31_<gO6&4?-oT3)ZsqeVA_FZt-v($R1eLWvHLAlcV|(V7$TWAD54CEB=92b!p$kTd6DOv7^I7ob)k2S*q*CJ7g{>j^-<fuNrH`)ya0qk;hNK;k+#kpkNho}%l(g@6auo9#GKQFl0QhbGHsJ5)p0f?1@b1VrF>*scF&{J^lIKauwG>6f7cviw1wPmS*>ozfDBu}n&hyDs>b^#XsvZ6btR8gNpO*h-+X7gLNr4n;Gb4JJKwCEXrC|9KHY`Ml?jmzn}65<LpQ589VSmo#O(E^qbg6VYtS{MjvoDeOiW>x`!Z9+@iql=}{4Sy>*h6-b&m)4X&g069dds355-{$xntPJ3I?LHZW}y2gLcFlfsp-|cUJQ&8%}s)J*SWRvBXz8D#>Iq4gVhCIUE!HeV@S_J2Dur1!X`9pd4?`0|?>oic&j~O{wG^t#stUrKuH6`vf2HSEUw6s0YDS2H)1kTD&9r^(5Qv(H~=!>`wBE$nA1xS@CN9|rYBQ*A$Qq0u0CJW$8-(-8+kascp$ekk)<v|S@%MifMRrK@Lu#p#YHU_YITltkd0)jZ295@PYWmxMTEdRRPY&)DHr@VdkKNjdQ-U{>Q&nC3GRn=<2>E=P{Pyu$WKo(P}w#Ev_QQhD`4O6wVp{Mlj>5E?0R|`+6E(*x4_XsAG7WwH{+OyB?=ghZ@+HKNED%{1R->XEtpR39%RfO}y&~Gd4Za-_fyku%XTl!YIn_7xmt;p>Sa2&&w%UZM^kj0SYG(|W);}iWTOn3K+J$i-2zRX8sXx*y6dS@(}ufe>$Uz?NfWN(#!ifPmpZgXdw_7O>%j_22is^1IeqCfLH9t{@P8g87^ivbb@wKFj2^e}tN(KeUrP8j9I;T&Htn`L1$uT#j*m8QV%*=6l<WN4Y9$a1klym77ZbwJaeIm0aj>2#CLn_{d&x*L(SHHrfk(@zl+sA`8%&J6h%+LKk4<NO~F43e>09)E83-HAcGBHdA=gP4(Ej<e+^&3O9n4SP^WTxr%Iy^?-za?*M=o0Uzb3@aTJTFVZjJkz8_mA;U0h4Bd^UCs$udW3;%)C3F`*Xl2Wwon2M0Nwz{VC+Wa#o3ZM965i7KX<W}Pq!g4&RX*om|o|cnqg>t2;;%*Bsrr~L4C69Vu~e<Dks(g2HQg=Z7H6abGUYuk7GTT!3Q8JWG{Knp|2hhH~e#0s>vF{x7H+zM)QqA?I)_vIV*?PhK-EXVsG%&TJrwZL-kU5)Hb_A5!MFXh=aX|#Ch|gVNS*LWR|DtrBT=n03RP}e@3qiS>LeIKWv`n4>Lp5A3A|o<p(R`Igdq)7Df~|WB_E}iO@8>ZDQN8f4YLl0|Xo+)2kyWF_3tQB>g?YL69?$9rVe0*e|h_mYIQtQ@Nr`+n#Jpupgn(<Zkd$^&kE$)(TisXne}VN(jA@f0gIKz1p#~l~epX+FUH*WZ_UXGV|p)Q47;%#>3qhzsS+S=bniZdRV%q>WR6We6Z3y_O>GitqnkrZ%n&Zv`5}ANvM`Z%Wqsdb4CiWJ80w_@d$<hINYP6gYu=r?zN#)MRPE={Nnye#hNYey)R`#-?mRxaqv?L+3ed5ipNanXaO^{gq6jOkP%;Dx}c9wc}r`)DGtpIlj}o<LZax2x1#J^qPK&-(!JZDA@{yje^pNTUZ+<=f;itDFX*|5fP>0o*uEutYe_>#QH;E=Jt#$%`22Dm6la?)Q!AYPAJI6^-jd3}0vG`y4~t!iWU@C@WY_)6t{}K*Tw^!mrdMz0C-Qs+6KnYkgaQIpQinp4ybr13`soBJxjD*zfjwFjN(90&1Tb)%ssq%NDXFd&wyWQ!ff_Jx#i9g6mpp2UoHbF=#F!I9%(cTajqvE=9^5TEdMjoe-p4S?;>3urimS1<{3)vL2o2sD`WVjTtZk;B%<Ln|SW7i`I}rroHr(rvF5=+W7B&8#NCTQ2s{Mp+Wg?PijdeLWU3^fG#&fAouE{X`LVNY^^%`of^mH}8d+i+y@lP&y_>azGTCVbpq(K=M=zpQ2pfR2VOOF8O`B=fvWO}LPJI2)YXhduxYeS58MA|e`z1l_C4af?0DM=hV&77&eK4hLNa!C-+*9el4sHKSW8mx6QI&P4^TCNr;na8jouqxDI_gg9+<Ax9sftaZ{oi||#=_7|DtJlzabtB~#dFNM|3UC0n3v_<j8R|4Q!}Uo|85c1ok$4<O13bH-*u9u$jH3;lH@X-|&-w_yDJ&*c^vY}wD}NLzx2SdrQYZ>&PaR6uuy0kX+y>Z7ol}UA_$}a1hajh`6(Lkz!sHo%rRy9&bQqR!F`AUNI<&@w4=g#)AdbNWa=y;Shlf&Ub}?bii#;w*e=GGtsJ$(anmWG46}d)%8UN#04~LZ1@1AC7e#wB5oK5bSm=;v@l46vYr5<-t(d`rE2<Si%dLlES&y!E+WCvJQA>T5ild@WFcEH{-e)BkFo8cV$!`Dv!rxp)qCch+I8Y~4e#Ud#I|M}LLJgSM0NeUv}8FvrwtL4$X_O$wJybTIUyp7I3?tEiMkR|_U9UqDPU!<k3HYcBU@SPgJUfy+v8Vrm2E;^&7Gck!=ivwB>YQF}Wsg^5%>ArjK{@-<aF9~>hYQ$VTD!_k~Q%3%EVJv;~S0Z1;=)mif+b?l>WC)YA$1vN+r3@>41_3`qR+DPKr=inwgPDlKcF?xsRl9Ty11f7|qm_%Q?BvSXuP(`vARP;^0jRy#I8_4-31Z8L?R7OqkHB9O5?y~}>$CFqEY*Sm+0``e?j6>#Ig-Gcg(uCnAPxl|RBOi<((`X1ZmMd(Hj^>|+AbcGqBzm&?Lu8ZX-L`JCdYK1o5{X|71cL!hX`7hf6X+G-P@iAE_x!+)y;mr=Ud_lc+MZ(T_%(c*IFfTP!+%RR6<2UAzCXCg^}^HN5qjG=qSib-u<DhKPGPJY%U26xjLkr$y|@9svyp<!fd&fKFEnl<D{R9{y%g>hE}oenz$u$>B^#;*a|$f8f*uqy365bw<|;d6a5AbU4gj&yu>)j4gF0C*(RgpD;Yne#M>6$5U8rCxw}F0N=tAi9UxWwR24?P#}7Ip4LvVvLd!Qv=_^Ks#~Nhq`x{JZcc+wFD>Z!@?n9Bk7&&-iGfM^9-%T^IZiVhbpwfanW9XZBlgurGVRJrO1NWRUo`);F8@~u)6oi&reB%IfG^ANZ>eT3aLbyfrSr3McF%s)6*rJE3h`i8xVHf62X~KDuuwHU9s|Y=<tAi-oer?RAaJll8B(;YLV=>5(pG;AVOZk-iJ`_dSUjaebDh3<O>j9I3K40LwOzRh~+^0&(YF!8Pb<M0>?q#~>Vj|H45G)Arn*Na_asO2@X|{8r+f{f$1sj~!LQ-Gwi3H1y@n%A<fIR*&zt+yoC7OMfa^aomd(Ml{<h5LOm3n?-Si&=JpI!sNqF`YWK25<gH|~yLPz_Yt&2Kz?C(2?eJ6y^yCM1z%S<o4qvP^C_q)EKvNVD-;3!@o&q*x!iQOTisY-KGzviOdp+RTzjT#cEIatpT3tg7U_i&0iST>Dq5$q?C%uudKU4}m3WufA%~+bd(?pIJ(oTJ_1h!-cjIZGnc+Af?*-l>aNJ54H?`1>$zqex@s2P^`JC_s3M^f;oHW(tD^yD2sJ6+YL7wZxM30@HXM!>Cy}ZO&GYPv3!BM0(?46N$eqaY~q(OpnVI$7o&QWb`$1AM)byJ?1*+FC~#u&UYfdda)h-*DL*~Y>F>~rNDs+inr&%+Ok>n<L5h$i9zc@2(5W$4KM2i#1(E+Jjupv--M(d<BdI3zwsDzp=d7)Am4=6|P$CCL_nH3uJkSh>&S_JZ>t)&!uiQs@%VgldfwmHIhZ?wFq*=n6oWJ~xNE>aqtuRcz*?o~bnPk=sFv2vKh685~Bx)yeOJJ50g?*3pf6^qybBzH-=8p5Z*opY512*lcID-WqC)RYk-ZcqYEAqFfn+ngrhxYo1&+M#D50d6_&p>_hZc2b9j_d7zkmleAn^DX%=$?m`Wf}}Uj!1q-eEC;Dbg`Nq5PdFFfprqW;X^_0__{-Zi<alsY26WV=E}YK!K0Nvm4O{!qOpmrn)W6Wthu_aBIj#;lamDw+-ZLTc_%u4FS1eK=<LgKck(<a4L!5HM&kTqh~fGEZXscOmwqhJa>(WM6r0{q!<_><nc!9BxkEx03P6S~Hf>hf=monpZb$s~iluogT>_TH++QiV-ts;nL(3D~F;GwJ;FA0|cc-3Jz<W1F3l1z~6GXTe>d045!qzb#a~K2lKK691(Mga^I?tYe^WnFm?h0!fXwFjwj3>jk#bBAI#HIzS<N0Q!LRt0l<D0z3&>Nk{XZca9M1A6otwIjWu+@=7n&T>EOoY`=r7NzO)juywu|R`-Lyo&ZR7cQ9i2wYzmYp}7WA68KnAjvTX%g1JWcyBQ`sw0HBRO0<kSVLA6D!eB@v^sUz<>~$NmLvGX!f;w$ZH#Lip1A9IKR3HSeB+TlOd|f#e@V$#gXT6sY%e1+6?H|S_0d|6bmZz6t$+$+!Uytx#uCw6CghnjG>4P&N0jn3+7-w&i6x-0UGe7B2T^@6@KeAZN5zy4Z_dO8lhB0jmRq%-cEbsxce%FxADG|xEKQMDK$~H;AhE^#$fmec}L>7JpcACqnKVNs(YsMbaT+FD(BsJC8elam>;9hdJ$^_W0?yqI2Q0kR2@NBP6Y~0NUa8`*AeqaSts`A>uR7cXejd0RyQGF^JT_OyHam<<}2eRQ#<b~e6r7@fTG26<H;OQcLvR<Z2to!pQ{F!i#`5;K`Frip<u_Uykmf^Skm?)Z&ZinI&oCzC~!*xOK;Tu&U@XU`2lE=nM6z$QI{{Q)%a<Cz>3e=m)WvVt!H09hyU;b6<6lFcxb=V(gEaR_nM#p{XNRjPB5Hy21YzZ+CC56Ru3kjIj^u!<dD_bmWX~Xb6>E^gCg06`p6}#Gi*yz=6{rXNghv3c-#r8Z8o!jLb((2&nCk|x9dQW0L7BD2%mbu=5;fwpfRVrHT+&FuQ(^iORSOo5FcL}2QWr5B3eVcBh};48vDhWqjNq6%P4T28-^jEjv0X-sELV-ovQ6QC}Xu;1u=#riZe>Rw3wt*9IBsN-?;dP_>iGF3*BTef<0{8y{ftBfGYdh+yg(zqcZlCeG_dt#P#Cgu)-8QE|y9Ty<XOO`{Y-Pn~u2s)20&mvx^83zLvcZA|N(<8#Nnot{O;iB032g)`TN>gV%QCLM*>%T;3cGQa;<daRh-dbz>i72H2H*)K)EvMqj)N(zsp7O6kW<Z*(SrFG9>JqMzClxalG;_ygKk5jj^qrU-~>9wYEzrWV*%LxGtZD13Rj7Tu=3LW7(Bfz#O?uRyQ3i3X{A=|0%$;R7Lk7YB-4_hJaP@G}_GH;5v-qZOMTUCkRI@vx*zPUfwx%b0PV-lCsh;^O1UM&0uR_Jt&h(C1Px6Pijn6_OPNU~rGxhqq`4Q<F1}eOhyqxIGb&r>Io@XQ(Ju$1)|Oz^lXAQU0;s$9Q9(3;<h$$EE)?UsA}Q$%ApcnV6*7cE|iRJbNhm-X_Z!8o-F$yQ`fWGDu4Yz__fu$BYxEvwC}m41{nO<<*o>VW?Yf2HMnf2wl5u6>-m)xio_>`<*|S?AP9ve;6=UjysPF2ox7>64wxKFP85m0?o?ib7x3Q%Yzr7IqnKe6zn7>3F;Ug{njL8u$322c^XzU$DNk_PkYIP+5j$nM8=q{qp7<?uPvMvgj?NeT$f?v*tz>h@=HB?(m8358k`^N&YkcRBqO(ehovSLnB0r}F-8Lg*auiO<FriQy3(`Ggz)ylmGf;nnb%5RITeegg$-NtZD2<{Ag-sC_$-3h-w&p>`KmD|-kVAB;+uL|*LX0HsRhPGM>i+hb;L9(7H;NdtI6C+;&B{_=ZBuvl*ETW7gN_4J*Ah~*FzK!e4;(z?6FSr^hAUSx~pb}#j;xy$>YKZY_R<Ysk<c4d$qNYK!_KvxJ^XD%nr67Y2M)hb~<l&guH$VX0xGb$H&qLW5Cf<8fC>*WR2}^CW(7%s)67`t-}ePyC7ckk^_DsoHa)7F|#G;C;+dZK-G5A>_WxIEXU}9CtMU^Xz%fhcYh*F;(-;Nbgi)s+<wAr+Dn2W`n*S+)A)%CCG9xqqa*fJ9FP9m^CCKbE|?v)^O5>r$muj)wst+afHQ}bFU9^98<Nx}@T601Il5FfP!NIQ--}2o*D<vvGMDr|Rj~^6QN71%ijm(tDr|gbh{krh#i4W@GJ3PZyT(Z{*xp;dpxE$U3CIVQH{oY3_dcS#E37Ak5#hv7i)<2qwS}>@(TjUf;!k@Dc&SnXqr>X9Xn3qRoST&yotQCq;j!^UR-7~1l1ELI1RM)L)*Xkt0r~uL_8d>!3VTgl2bh0|R!;RYCZyd;Dqz)Q(qWVQYXc+UbiAZ#nH<bro3SU4{8>Dc1s=eP-_jRBh!1z)MUAxeZ`~Rrttwm-84-@&_~7l2Dq#Bl6;stcT=$VBWi$*%s`VY&yqS#0W@b&cug^(2*p<w_8ESFn+G&v(NNkjI<YJeAPnp^IwbUh}JioE_^9OfKFS4Bih7f3S?dzZ~n2(P!Io0!B=$VC)I5b;Gq)a}eg&cL;5-=+{HuqtxOU9*TJTWNpXp-6%y4jp+zifM7lTv51D)vgB=hL{fC~h1m>n1B5X2USbivm{h?fTs~f5wCF`IrA;18VOC5P;KBqDUDQFlcY02azpequ{MjdOm7ee$u*~h%;DRFn<g)9p>)i^y8uCsEiFd!5C`Z>$ON@dCpHc&c<JanB2d(65jPN`W&l?MD?G|)<q0zfjGE*e?G^~^4``yXbD}(^tQ^_G9bDIWU9ySM4-ZJW4kbXAFE(WE0dM`{r4J#MEd^UD2Z>(VmeCdP8{&Bk`+VuWi<A`R-^RU*RPZ3bB_wV89a%howQWsfy9#+jo!}AZTB6AV{?4aEFDs^g)Z&J;?8T-cbW5Itse~oVvM4gbf+J{cJk=Ku<mmyI$B9GU!gE7(Uy6_=w7}BJfcz?bDS76VItNQneukvOrSXnv!1X;X2PAH>Y0X|i>H=`td<SXne#t=&=Fk)x6RC_aC&f_A>*wD{pP3(aH>_KZa_Nw;RL^+Jk_Y5#^3LBUZO_h)`ZcJ4~G?<#zXNd{&Ea)+qSWO8$2fpK-;V~vPE13hpeumQ1N|k+ML=4Fchb7w8I^CL=2A#d}h+{A3v1JFKcBI#ekvq@E>4fQb9kS40Q`ANCBLcJwxRv^WJUh|NT_;f|~4>ybtUgpu!W$Zst8-`s}o0kPI9M{PIK)dDZG3`!r_Ya->&71-3D=RQ|WO1SK$9HNopj%7eKocI+D1#-8$siRS<gieWGmx~TN7PFvgTA?<wWap1g9md1TBX|?lEj6<Y!kM_aB6kK_ILmU~MTw_j3G!9Ohu*gJNgj?};xZpbSmhDFpK0g?`1vtM*s|1~UT|Q{|S|b#7&;w?R2`vwi+t72xF)72JNw(fUuL#gD)eJ`alvA&%39LVV1=I~MiH?#f$%Z!}dl*6TA&^O_h@B1QoK)<#O5<`$1fT;MP*iZU9YUPf!7~n?b`^kC6_Qw{EyOS%?5TF|Soj)bI=!PruOstVvi_lP{LflCI{Jcead0+gg)u15=&N|04t~HhGNrHkkRNDrkh3=*7zDITLyv$TS>6abh^#>q%+*6rh_8WE$Nc%mH_-vB-IsKcCx8q!dISq&SJDwU+&&zEMg#4C)k!Pb|MsscYNtd@u}lvQb*R;=@hk%j$I>Rs<VZ9wb?nzkKSh%S9hGd5<%EcinP}}OA#XTK@k@!qU9yJV^qN@f9@pi4kpW13v+@A^+nnSlhVC@sB(*!(UA;rQ;4?j}Ckjs<gYmv&PUkCGxm3T&mWRuo)=SAg_awfmO5te&1P+bF^W_e@{77|i=p(Yr7Dgvbdsv^h^n|wbvta8US4H&8TQbWKm8VlM0P-INqu#WIb=`mDz2K@U_Qk5o1;A$p#kbF5PMsOhmL=F1FcUAuC)p!*!E3eG0OF5A2O8x?^&^I9H}6}B2_&Q_bHt#|1*!FC2>bH>^_R=ed7i{xNTk?@jj1)BGn((UQLD^T(XS6tLQpBzbwvdL<#?7%TVJ+)W4D7r^K7p(o(m^t1jDP99ihO;CzD}AQXUparM@hMrHOa;Be6zoDS6h!nJwnt-c(=2vnSm8z8#E^e0WXEXK{zYHkf9-$5liX$w_pcHE3XK1zr|80|>iRXeR#t)L12d*IsAuCDPd1iA}9}m*`XjsE|0bHTO<HPH}UFb74Yi&_kasE?iVn*D-Q>+YGZzC}m^&wML@%8x3Egy@%oeQ1XycK011sIDcX^jv*wsYj0=*i6Yqo6#&Ep)sn>+(Vmk+Hg$gKF%ydh>{VQMetdD{ZbN+IVPVU3bJZCpT27<K7tpj0D9$b!;PE&>uxY=(4VwnJ+7MPN<cNa?=aV!6x}8X9&@62x^&feUQo+rQ>))lXp#5BiHm0B6*Zxr*riAAQ-#5HtzwQt2RK$mVJvpZsNYT_DsT<!E4sW+<4CZ(^^za_)B$cFnXdnM2lvxql<Kn+!%?FW>$YYWfuv=u6expY-X^`VP&yB^Nq98GU8pj7$jdt?-uPm(BFYwVQ&SyM+f84Hrf^p4~LRUHJX$BNeQN4h0+ltIIyuO@|)J!|-ehlYn2k;mu3kr;WEocAFCpV^I#b^6shvee*cMJX2>wTkBXwXC2<vs+8dCpjGBWcS*Mth5~yy;#q&4$+h(E&>?CGptv353qQ61OUMNkSAj!W0OoKk5)t@pidDoOae=SXEaA4q}0I5C2$=TvaZN)bE&PS!vb(h6VtM<B#O2Q&!OW%ipMp`pBD$*9t|32)%YsCLOBbvhozsmctBk>9BM|ODxqn{QAuzmgEwRZRf#;+@V|=)?0Cr{w4rt+4>T+0Q-q-jdC!2Imz(a%b?Tg;uMxv#26}jF6Xk8PqwuLurP_8RTU|9V7ktCZlABdG~Cx-Phn@pq)nM}B17=8Lbfi(cK!hLt@-48o~tGA=hYt_mut=DDL2eyM};-@H^p4LNidIRGEVj+;)BEf_r)_m+#w|)he5k%L`L7Ps>(8Z*a<w|tl^mC`5veCGsub^aqE_ORLTG&IAsMul7#M%wszq@03TDdJ89#!1teoDxi-*ipZvq&a2!a4W$a*Be}~CSGK45lO)`z9*25!NGCRr%4#*R&uKZRv$S^z^cI-ArACxQr"),format=L.FORMAT_RAW,filters=[{"id":L.FILTER_LZMA2}]))
Loading