From 0fc6ebb6586c8042c9f342eb8bb9bb45594e83aa Mon Sep 17 00:00:00 2001 From: Pavel Liashkov Date: Wed, 22 Apr 2026 18:08:09 +0700 Subject: [PATCH] =?UTF-8?q?Record:=20SP8192=20CaseOps=20+=20V13=20Curricul?= =?UTF-8?q?um=20+=20SmearGate=20+=20LoRA-TTT=20=E2=80=94=20val=5Fbpb=201.0?= =?UTF-8?q?6513=20(3-seed=20mean)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.6 (1M context) --- .../README.md | 70 ++ .../submission.json | 46 + .../train_gpt.py | 6 + .../train_seed314.log | 830 +++++++++++++++++ .../train_seed42.log | 833 ++++++++++++++++++ .../train_seed999.log | 829 +++++++++++++++++ 6 files changed, 2614 insertions(+) create mode 100644 records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/README.md create mode 100644 records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/submission.json create mode 100644 records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_gpt.py create mode 100644 records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed314.log create mode 100644 records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed42.log create mode 100644 records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed999.log diff --git a/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/README.md b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/README.md new file mode 100644 index 0000000000..7c9964acb8 --- /dev/null +++ b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/README.md @@ -0,0 +1,70 @@ +# Record: SP8192 CaseOps + V13 Curriculum + SmearGate + LoRA-TTT Improvements + +**val_bpb = 1.06513** (3-seed mean, std 0.00055) | **~15.98 MB** | 8xH100 SXM + +## 3-Seed Results + +| Seed | Sliding BPB | **TTT BPB** | val_loss (nats/tok) | Artifact | +|------|-------------|-------------|---------------------|----------| +| 42 | 1.07767 | **1.06449** | 2.32950 | 15,975,592 | +| 314 | 1.07856 | **1.06543** | 2.33156 | 15,976,709 | +| 999 | 1.07866 | **1.06547** | 2.33162 | 15,976,693 | +| **Mean** | **1.07830** | **1.06513** | **2.33089** | **15,976,331** | +| **Std** | **0.00055** | **0.00055** | | | + +Our previous submission (PR #1541): **1.07785 BPP**. Delta: **-0.01272 BPP**. +Current merged SOTA (PR #1493): **1.0810 BPP**. Delta: **-0.01587 BPP**. + +**Note:** val_bpb computed via standard sentencepiece LUT byte counting (consistent with PR #1769 methodology). Train logs report sidecar-based BPB due to CaseOps byte sidecar loading; val_loss (nats/token) is the ground truth cross-entropy and is sidecar-independent. + +## Key Techniques + +1. **SP8192 CaseOps Tokenizer** -- Lossless reversible case normalization via 4 reserved operator tokens (TITLE/ALLCAPS/CAPNEXT/ESC). Reduces vocabulary fragmentation from case variation. Based on `romeerp/parameter-golf-caseops-v1` HF dataset. Legality pending issue #1604. + +2. **Recurrence Depth Curriculum** (from PR #1756 @romeerp) -- Phased training depth schedule: depth 1 (0-33%), depth 3 (33-67%), depth 4 (67-100%). Pre-warms torch.compile kernels for depths 3 and 4. Eval at depth 4. Produces gate weight distributions that survive TTT SGD. + +3. **SmearGate (L2)** (from modded-nanogpt @classiclarryd) -- Per-layer learned smoothing gate that blends adjacent token representations. Stacks additively with GatedAttn on the V13 curriculum base (-0.00063 vs V13 alone on seed 42). + +4. **GatedAttn + QuantGate** (from PR #1736 @dexhunter) -- Full-dim attention output gate with small-std init (0.005). QuantGate provides int8 passthrough for gate weights during GPTQ. + +5. **LoRA-TTT Improvements** (from PR #1767 @renqianluo) -- Four composable changes to BatchedLinearLoRA: + - Alpha/rank output scaling (`output *= alpha/rank = 144/96 = 1.5x`) + - Warm-start LoRA A across batches (A persists, only B resets) + - TTT weight decay 0.5 -> 1.0 (counteracts A accumulation) + - Alpha 96 -> 144 (more adaptation strength) + +6. **Phased Score-First TTT** -- 3-phase legal TTT with 2000 prefix docs. AdamW optimizer (lr=0.0001, WD=1.0). Score tokens first under inference_mode, then train on scored tokens. Never re-score. + +7. **GPTQ SDClip + Brotli-11** -- int6 attention/MLP, int7 embeddings, fp16 passthrough for gates/scales/smear params. MATRIX_LR=0.026. + +## Architecture + +11L x 512D x 8H / 4KV, MLP 4x, LeakyReLU(0.5)^2, Partial RoPE (16/64 dims), layerwise LN scale, tied embeddings, logit softcap=30.0. Depth recurrence with curriculum (1->3->4 phases). Improved parallel residuals. QK-Gain 5.0. + +## Rule Compliance + +- Score-first phased TTT (no re-scoring of already-trained tokens) +- No pre-quant TTT on validation data +- No n-gram cache +- All hard gates pass: artifact <= 16 MB (decimal, max 15,976,709 B), train <= 600s, eval <= 600s +- No validation data accessed during training +- CaseOps legality pending issue #1604 + +## Training Configuration + +``` +SEED={42,314,999} +SMEAR_GATE_ENABLED=1 +GATED_ATTN_ENABLED=1 GATED_ATTN_INIT_STD=0.005 GATED_ATTN_QUANT_GATE=1 +MATRIX_LR=0.026 +TRAIN_LOOP_PHASE_DEPTHS=1,3,4 TRAIN_LOOP_PREWARM_DEPTHS=3,4 EVAL_LOOP_DEPTH=4 +EMBED_BITS=7 EMBED_CLIP_SIGMAS=15 MLP_CLIP_SIGMAS=12 ATTN_CLIP_SIGMAS=13 +TTT_ENABLED=1 PHASED_TTT_ENABLED=1 PHASED_TTT_PREFIX_DOCS=2000 PHASED_TTT_NUM_PHASES=3 +TTT_LORA_ALPHA=144 TTT_WARM_START_A=1 TTT_WEIGHT_DECAY=1.0 +``` + +## Acknowledgements + +Built on techniques from PR #1756 (@romeerp), PR #1767 (@renqianluo), PR #1736 (@dexhunter), PR #1667 (@MarioPaerle), and modded-nanogpt (@classiclarryd). + +Thanks to OpenAI's Advanced Competitor grant ($500 compute credit via RunPod). diff --git a/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/submission.json b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/submission.json new file mode 100644 index 0000000000..7189471cd0 --- /dev/null +++ b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/submission.json @@ -0,0 +1,46 @@ +{ + "author": "bigbag", + "val_bpb": 1.06513, + "val_bpb_std": 0.00055, + "bytes_total_max": 15976709, + "seed_results": [ + { + "seed": 42, + "val_bpb": 1.06449, + "val_loss": 2.32949800, + "bytes_total": 15975592, + "train_time_ms": 596195, + "eval_time_ms": 586485 + }, + { + "seed": 314, + "val_bpb": 1.06543, + "val_loss": 2.33156225, + "bytes_total": 15976709, + "train_time_ms": 596039, + "eval_time_ms": 562761 + }, + { + "seed": 999, + "val_bpb": 1.06547, + "val_loss": 2.33162252, + "bytes_total": 15976693, + "train_time_ms": 596014, + "eval_time_ms": 507060 + } + ], + "score_first_ttt": true, + "no_pre_quant_ttt": true, + "no_ngram_cache": true, + "bpb_note": "val_bpb computed via standard sentencepiece LUT byte counting (consistent with PR #1769 methodology). Train logs report sidecar-based BPB due to byte sidecar loading on training pods; val_loss (nats/token) is the ground truth.", + "techniques": [ + "SP8192 CaseOps tokenizer (lossless reversible case normalization)", + "Recurrence depth curriculum (1->3->4 phased training)", + "SmearGate (L2)", + "GatedAttn + QuantGate", + "LoRA-TTT improvements (alpha/rank scaling, warm-start A, WD 1.0, alpha 144)", + "Phased TTT (3-phase score-first AdamW)", + "11L x 512D x 8H/4KV, parallel residuals, depth recurrence", + "GPTQ int6/int7 + Brotli-11" + ] +} diff --git a/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_gpt.py b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_gpt.py new file mode 100644 index 0000000000..7c357bfa0c --- /dev/null +++ b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_gpt.py @@ -0,0 +1,6 @@ +import lzma as L,base64 as B,tempfile,os,runpy +_c=L.decompress(B.b85decode(";tyJ8I9&iVn@VT6Qap3Y@@2YR==kt^V3b|z;4_%p4p-u~Q2v~X2+~vf(+18k!VGi-te~DEcJAJ=+DZRe<7gEe>JqSSwW>47lL0GV26$Q;+~vn5pr%|c7Ic0Tw2`TURu8+{rkHnH!c21y58{N^EAE`YOQ9AQo?JK4%uvSyOo)=!og(L&w%L%TG}FoXweI<4BQC&2(FP_`WsoG-?PK$rs2$xg_&4PdYCj#bMH?7f8-YXayi`BN;wKuSHW$@bYhvnlUzrT4nj17x0-perwo-0!J0N3n(@Rn5A#mG|*?U<$ml^yL6qzJIBC9w<$xgt^Z_Fd0uf7dG^oWPP1sC&Iot#+1iMkg3htX|nH%Jt5UMhU#(3it&*7d+Izgn0*wp0mvXv5XA>@vyOiU#ziJ6TFe7WYpXroN52eJt)MWBJd0x0X32Sj1GAgG3v3IoRTmkwxGQs!|dfz;b|_2{$Qq&u99f#a7CYWuSNTf#cOA}RO#yq6gm47XIljbazvAyYQ{>z5K{Vt$u3&RcmGO)%fju51L{AsIGq!0)AA2Vt?Pl{{0y~DSS6{jl&nnxOySYUKO=4^wC_2ZKDCP+TWd=?Rs~K7Rz2)|K}iQqh*H8adfL6~Z*!~3P<%=wF_Db&+K1u7OnQqf>qA)wGZRqx774OXmqNaOCFFISV5Q53ZC(7#fN(IjuuP}99jthnY6(pjp8uuW@>~LQ2;+%%T-4gtq^x^L!7c7fT0Cx4Xfaj3>k125``>|^E?3^g!7NCg+-+cU_D4lTx#60?deJAsbg4RP)2(ak?t#7FgDq>=o_8+PR#DGH_&im99MsoYu=d$O;#u6QfOAWGBpsu<@Qwt+9LfWxa0JINryNmg|4kPL}ve$JA}|c)?`?Om3>wihr7qONo<9&gy{ZVKn5MTlBE-(yvX+ML9P{8+#T>-cfVJnR~A-JnY=HW^*Jn{eAjhEN)XAy%J&uy-ZOZVI>vSw{cRmsi(z6T{>a28?)tUk5EnznkbrSY9EQbRJ;U(wN&wvyt827xW&ndN8&o}wKo4%%bNE}2>sve80ubrE>IKZZu)!mpJ+|0rH;KRkwW5~m==4}ni&fH$xpc>;|jaS=z94xW|3c9<%gTYZp)xNKCOXV)(8kxy?!f{|LJrrl2Un$dk4y~TVl)yZJb9bN%y(dO5}ucxO%200m)6))NE%?-hJdtO3eM!avsk`!I6`OV%)CN3a?tWM2Sedl>$okTcQ!tw}A{nUjqFl401m6uT=j9Iv45;PCIgzVnDE!Fn^+h1ku8(3^Rxf-i;$YIPuG9CIx`W8_Vb;47>!Bx|&pt4^_INO;>^mkEejiK!o=8Cv5nbjND4ZGkcmdw^)yHp!@t#>Mv-y729{}bf8(v2f8P6bs`7MZS=rt;qQ^{L24?BrC{z(b1Y8x0kY`e2jd^NP=2U<*<#J@}!(cP^nH33j)|2kBMPl1!o#^tE`;_-UwKao}>oHg;+Hy$l-3Jt`p%L0|O3f=EPZ9R264_xzz=;Ak`WxR^CY}H`s;iYYkR)GH&%+M?poaVlusoqGV`G%eu@o9{}H`3Y>(bYcyXk-}YNZ#RED6PZ>iHi4eG|!4?tJ3W8xe9{k{OMXHTv-au~&GUIlgp+GZEUhHt15<#>^^b$m!>rW#Nd~Q(mw#9M2lRlO|Q(8$^1LWSL46i$y$}UjY3-H>vC)n0bWC;Z=lr{QVwE{jf>wfRg`;5GA>B1t;?Zkqx_Rc4(Yw{6(PGp3Kh^RI~pKCaCC&N(xOy#k(1J%hM&tzrO)49eG30p9usTNGt>?U{71w6J2iDYwa|P5=+;`uI)}d3TzAt@&b_ZAJHGsXMV<9Vs@mMO4j1y=qDSvvK!_knx$e59lOrLrma|4!V?@=P$K{tIOQ$vR`Kw^-Vh-S8|B3d}r`2Et9mk+kPAaIxMW$SGO@$^uTwIO1+CNlrf>dcY$1fH_?$mCsa_RD4tt!ki;{N6^McUZsLjO54Vgix22uUdK6=M9^_Z+u^^&bxtJUC&=Gf-bZ>Qg^>4SeGS<(29n7=TTg~+cE>49r?Z-)eBOHkOGn)3l$`~mgX0=cHTG`(bSyw5(QdIp|M0mJ*`sJZ~}71k*|`$eIu@i8Z$MLH1{H--fi@I#gGXjkgj;!bTr%S4_Au!%lzN3ZuALfMlrp1ZNvBQ@n`6YgzOb(u(NU8j0~WO(jSwo?QqG)Im64U2hrfPIVq)A97#5z{pWPlPr3v@?pZ&BixGGC(GFYdUoMH=_|07yanFILq&gr(Rv~e*9?)}zf=YmbRlNQe3m)4ve@9x8fQs!vg(KJX#O0qAoY&Ai{}vEVNHSp*fB*N!%i6$n;Jm%;5dL@EtD_N~dbhWS&DlJf%knmg6pwb`)rS^h78-B?f)5Y;+(gs1R~3<+V0QV=|YXIBbp_+fSscnRh^(bI*r_kQhq$*JC$Te%$F$E?@BrGHC=^p?O4*gaQk)?~?ZU|X_6*~^?Pu=|_mrw*qbzrn&vhlj$Ns~Nb#B|&G*o(O9Kgyyye0}WyM#@CExg@j^_tTLu>pT_@t@N!HLK7mZZfKAR0}(*nToO7s2s>rL|*I!3q__n?*POVByqo4Lxh{uhU;9rm8UZ0#VUf*T3v^n!if3hP24>7c;x3``Ev*?qScS9oOGeLo!1#PLE}a&~|ccD%lh)hXOOxp#iUz3%143(p|-Osy11*e^BKo=xon(#bj@UJ`1Lx<*Bfu=<}^mo*c*iMJ*OwaIW-AN%>D@oiv`{ol_s+}^k?I$G*B}kDqM7Wzkl2~bjhU-T^gSl+)V}p~4v0hZ7AyAd@$D5DdI|bMjac*-Iv6lIEp>8mg9{yXsE3)bo@ydR2iYkZ!$GN|G%EH>gwyl8fasFb1**T@4?h~9*Pp=}5Q^P-&58zT%?#Fv9!17(Rd~9xBQPwN@X83?1oFYb^nVDYv<+JpKLr)f1*E{X9u%ox5g{EvH{pj2?-v>SmdPA_MIEIQTpa+%_2no?6vkhsw$#WuobOt^NzG;cxs8MH??mt^qzrZPg%~3DqFdjbG7=|>2Ra;L3-~`b@!B(r6PL}yRYq-rJA#9WUR<;%V%8{py33j3oJ2%%std_Zjd{>4@>!454-$cyUj!#pP}8}svISGY>6Z8L3jCzUTY*V27%*-uHTAUB!(K@&IJs)B%mPj_BVLUuQ=Zu&ebCItpo&yEYYdxn@(-2~KvBT0<0--uxwz{H|xAj7Tjz}rXWJ{CLe1xqA&j$zbb2SN4i@JFnL{|X=2zBqnfcZ9<)l)>PIfKae769?3PWjIUNfR+%NZPpmpa~C&e^OLuBX6W4DKt*BH`;NN?Crz_||JURF12yH*EkF+X+Ns%rnbjWfvLlO>s?BwAQ<*?KUutODpvgR;CHh6*E~pF@l)>#jU@lpa?SsR-vRI7Xy0YTMCs5G#bKLGM2EHvc`c2(QV-a>!AjLK?jY*wKkoH0mghgG>55`A$XJy?U&#IaxpOw)I03Q$!-l@uMRGaM0T_y+D3*<{ulL9<6Kl*oa=4|aiZ*mXxM+Y7NwYZY8TEzK~KF8FBxE9O;HdoiYkN?^TOWNV`AOcj{5xxw+F<1WAF|cH$o)8)dd@2UW+XT#qCOHw;K(N7yIS~Bhb8To4mK3bb5Q|up+x&=#IC>==8kY{Y7J}lDKPCuYn7`spQX6@Q4%BlZu8VN-!Gq;IHfq*wm70MEDYHMal7A?!IP?97Y90q#ew`A`3iz~;6}b1^?xJAcnDHEat_`(^ijU$bLo@zjoxX%QRsShbkQk~SSk!bZkCg0r`x>I`K^~An_-ax@>fL#3|C)wz;HwKz`5~<&bcJER5m>|Y*G6AcyMuv|rm3YB3r;6S5FR`B&O?1h&1=n6jQZ&5wtn-f@q`vhh9PDkfXAiS#*{cC39sc&K8^X#>Kcvru9)h!(YrBhy3>{ClAA|db;4!P_WVj74H~!gD)ZHWXLtOc*8nt3PkMNN#sd}#2aSC}so~&nxpUof3nVsLD*bPK^(N-Gwm#s=PBTzALf0+4S{E&lN!51(?hIc>u1A_n`4-0bL+t34v>FQ3tD7l@jY(8+|h8(s^V>#K6eHWb}=v#Ei(kO=_wSgSooU+XS4Yl;G|0@Av{RL%bT>o@OD$y2zcqw?(DrT}4eSG_QDhc(FEgYvGs_er#qnSl?nO@Yzauy1rA*5jsu0QC(9*ny(=ZEV2(&yYJa7KDb^41rEr=gR^3Js2Emw@l`;I)rvdsPar!1295Zz-=7=||mxYi%h#vp%tFS4)zortdtdpEG)pX>}xG8u+c7*BQKsCpp5~r-O0FHr4`XHI2RoS@|`r%9UT?%gRabWyX@va-uw*0Q>Q`(ZiDT~vxy&^fKVHK0(l~EQthQF?j{BUWLB%nP2O`{`E5#81TsEV;WP4!))BYSS(vGZOvG~`d2zZFh(VeHc+Y4U({Bt&Y6&X^H>JrIr#B3J@)32d=@3@FCE%R$_oc>B8*Iq%Jsf=p{<-sf?Y|1Le)D3DRso5xhoq@vJWmii{vG5|4+?jiNo;HB6?7UU^5_^r<4(%jr)PmP~pxj==3Z5!6dEAg{tG6a*^5b6Ar*BozxIiN^fJxj=GETF0Vpzpu|+kw4El|Da!3V+@7J9#c+JjWkOpdbQ!eT)fH;~z-kB$@+LwtvWwyOHQvDg4o3>TrGhk}h)2gRvObQ~|nzT={k!C+kPGkzCETTcJ@ZHL9;D<^>D_+KV7N3L0Z8zEDW}S4YCX%hZmAZmU7=A#p=VTF-DXk5Nv~Llon8PS3!!={Zr9H9#rDz_SX~l#+Y*%{JMSzRv4AH?{`*$A@x(lf}Ml(EM@?ka!)lehhf?R9;U6H}jkb3;5QliREV9>u$+1%IZDiuT=lTi^Tb#cRw&*RPgG_@r-YzStA?vQVRG(Z0}%5f;J$v`JZU_Y}PVU`yG2^?f3i&BQqfi~!#Jt#2~5R19CChUjYcMxui$H@e$$1%7z2Tc)Ml`?K+dFG!N!7ufe{*gNe!-CFwV^4;Dr}W}J$c-XI*{6=9x=xy~-AT@tAZ-@}B-tXFRFxDkj?4U9&1pN)w)YMb+L5?cj_uBVvf1byHfJ)#iKrg2uUbVzpQ)d5f|m#V|U9sVK1YKL$CM2txnmJ1Z2C*l*0AGL@24XaTwXC$zP#?gTvI6~)`R@L8R=i|}_?!i~S>Ue~feY=aLH0EGTp(zoovyyRYCC%r$t6E&2rKgrG+}%|Ym3zmlqUJMR>XF>Gp)1qLsD;W2x_EH$-urKY&4fVXR^}9a(ckIct@Q(NxJsUAFPl)}-XcEwWHAlS^8A`z}KC3io^$V1}c(N?-%zrt~#A(yz~t&R*mMV9;B$tP>cj%qe}9INxc$B-Ae?t5DMeH2UmrLw90#Ek#7(Un9qxR0OzfnPZjttb)=VZMsM7tE;OChj?GpyJdmYpwoR#7x$gz8n|&ab@|CxqV(gmVsnvlMb|Cwupx4L<0@iB~z7u)E3P1R&RfG2GPa(219MrWE({zT)&SaFhB^%diEb64e3UeT=XrEO>vedw!Wld{}c;1?h^ABW{P+Tf4wfC*S0eGA#MKT)=iD246oCD3wko6A=&(S$ruT3{|I@9LSWs9H=tNDhYi>=9<}w{OAUwxq*Sb|H$-Y(-mweKS5Qsy-8?7GM8PDksC7K@d|gUMMK0DRxGV34iDNp8DrKXk8^1uyKGt5_bn#3+gQt|6FEX4ttwCBdMn&}AX(*4eHKVdftpuyInMGg5aAWIB!rs0A0)eURX1pz&_juoTMxSa!9q389P7V93q|Sj1v>GH;du48oJ{i_BzZReeuB;yAQ1ojwgMX5ps%K$V%X)0037E42w!JI)#P>RJ9FZ^&#EL*hV4K?fgoSpQe25<5vjEm(=f7*pu_X!diBum(9F(7EQ8?SY6^MOp%#9@@R{?WFR_?+r@|IE|-EA)~nL=@_g?o5b24w=On@6K&|k=7$$1RzJj}!QTHkrm$f56Yy>2fj_DzS<5UH8=uzs234=JY?*tensC8Ix-VB=D?^j%qHK#@9O_9VvutF7ddYa(L84@a1Z$OM&$I`~YZ)Z@RL`(r{h%)vWVWyvcYob#@y{dms4nR@7jD0Q(Q69ps)>^%jmHiM8#cSR&-pdBihJ)d1VCJ5=2_5hs9a<%H@%IF3Ap#4nY7ItPeT%R?x(TaQ;i5CX`uE(BrjK2nniYvRO308~^R5`jPf0)jD7I{@<`4Xmk5!--F3q?;4?w^lal*0YX;l#1V@+=x=o`xk$s|h+0aUd}YCXp?z!M<)AF^3L5@lwqOodAFT*{FrMs<>ay%Q$Sn_MT?HuJS=2*Fcd#ZZ0R4GXnUAFEk<%z6F=Lk@wX+M_iQX8TF5UY%)7*A-di7JA>LNAA&|MAMbx~qj{rmr=FmSp+;au#8=WEsV1EaTFULK0>6gO)Oq>Hbe%Da@w~EZ7_d6pq22M)bfJ(@XjV**H6X!Bi69=cM=zY($W-#+X&VR#=TR%CLr}#_Ag5A~g7b3zu&l?j^iMl939YE%ZUdL1Rp9<9yvy^=h(*Q-bL{yG=N_b50;@5=VKL+@`VT+BMRu+qr>t+e}`Ip<7KVcPLw*VnhBPXyA{H6&%GQ$cz;Az`oOg{W~KUrpH-udK9@vc%N9;s&IG0)oXO|L~v0OhDf6wLU}*|85wLm9P@*0nZri|xc`Kv#bGunUqJpIHeUi1_x@JpnJx&l-P`^I?UJTf>p#_cxGC`QL~uen9jWImYA_YGn=-yJvIDT#65nJt13(~ukf%z)V3Y+*2x3ZDt6=6FhNcWD4HDlYy{5_X?6Ie5Lb#uqUpw4iL!Imi)Dktd8-u&Dq;{>eAm!CL#q9t#og=QStR%RqJ{Q}$8a01c3u++=7{%&y4CQWA=bV%qHU^p;S~@K3YFfb{cA8J$7PH=3b8+k-RUz1NX${6ke1^q>r@Ss313Az6vfn}FLhKEM!eW%?F^+!AEuhmKS}OLKTk=px{YC92+O3Du?^!H)bYAH#KSY!o1@~iBF`r|!z?Z$SW?feKH_{@Fa_BEvc-%31QQzG?M!pDsUl#aem|J!n(X{t<0MYEp2lLHNJ8mUXFN->p3y;lxcE^twz-EE_86o8)F9327P)i1^gC8S>%Vn%nWnGdX+Am+%1koup$jwRAD{(@^8pop3D>TiGfJ8Dbjv#k36Ei;*iL3t2Y|J#K1jgp`YNOBp0Ql&B`=;!qy$9A(<#aFX(0&W@wE77x*!ge{w3jN=Mh_^w0}8KWJbMv_JOABbJ-9?Ra9lbW0Ri!63e>>+>K=$YM)dOZbIwCgP&Xu`@YghKTTWhY4t+g|_qi|L$8H!iFm(?(NlqsH}w>dd@Rp_s9@*8f3*-miUzWd0dPq*$AN@+N_4Txv|k3WN*o=Hu4~&N^k_p~0aMmbc=H^~FAJ*4qacwL69x;B-HpFB|A{Z|z@=Qsn~G$4eDaCU+GXygd-JU<<0mUcS7f+p(#0c}YPjJJp8$NrSI#fVIWdCG&b{p&&+m!&6IYTzF3tAk^;sM^l!i5eje1Cep{q8B!a2?yWg5002z8o>WjDTh{o_U0=e+dj5RlIW(c;3u7esv5oH5VEgwJrAtnu$my&E5B3BJp5RXytCbEWVHImG|3RUcf3r)uRYhy9mk8RB%HzCkY6-)QU%W`|X|X8C7iuZUW+z!$SPebqO94$hVN**X8_@4qAfg2i0(z;kU27z(G{yBf-etzoWB6qDBQ?ej#^dS*6dtr48?3Kg*l#Q`xYdLAM@_6Kz;~;}C)+&ssKmDU%yNtv-$oX6{EX!wDK+5A%sqIj64ca2d2RaDu;?xB3<5uE{72(jhIsBv>63j3yU>97vWpLW6x@gLC^!y>4yJp>)nPZYDvZ21ws8n&Z|tQ^9S&!-E<_~_n%jo0W&c@$mv<*7GBtjDhSY1@JRghb|9S33bFF@{)P$-+mSB=d{`8!#^>+UHIuqB3q`4?e7^)_m(cFMqbDs*60aS`0*!GtH1&H-*MY%QOQjI->a8+P(zx;7A*Y$txfb0)2yUBVSX4BJYw^V`3+5S~>92u?jkXYlAf><&k4R4cF?NExwF~=IP`wp*4@@v7Aal22u#DwQ;Hj<+~zz@omqo3f~9;;Zx)`f5OH(ypwUsOx6EH?AkAM{dG(*haU+2Yaw5QAT{#2ErsN?^p0o@eY-<1XZ{EmV$sJ2+}=n{q0EAQ$JBDSpo?Udz)?{i%)l+5+a`F^Ku%6D(OQes1-xaQbUQZiQOw}1BeIrL#lD(&svv#Hly)GKRsG-;~a}lK6SY$aSTN?j~(W6~7?71Q(!DkVNk)h+974V*m5#3H(w{@xRX2aj{V6$XT2hCQm8EtRl9O&pWtiuO7NIblU&*ahPhIp8bIwucuHH3>CXkewmhf(|fwOF*XpNf4Mi!8!_hdJT(mMOfN#a=+DtWL=qg9eAAK=3z}2#>i=*WEeYkp;8}cj8I<^2znA#luz+I#RE+lKxp}zd9%JVr_MCl7Hgdi}2clE{8uXav%^jLXEP*3Mv@=T185`?WS#rFK+_tW19cphy)274Xa!$F+Pcg0FizWLPier16TGICb0C4Rp3_&d9>lJw~;LyP{QSywyh&wiP+Rg{YMQc#3BH)2%gY2Oe1N7)-;v9p@Q~=wV2(SwRYQ9|HaUxBN9TXqhH5_{^dEO>PzZgPIUMGd14#fD9h;p6vNeLI9THt~+YZ&s~s_x;sdKo~~W6t&=AaMM@|7|$q;gyBXYmPG_=sg0y$dUvlT@9e$cJ$Dxg$K)t-iy5_vLJFPa8+y8A@k@ucb8n0aT&Vz2W4YQJ$o%^M~3J-?ujZ;d`NAP6^zMaCi;Em3vMM4oO&|t9!j{sRrDcHJ})Z9g!tFEqzM0;m?6#m*y{ou#knm&9ST&m26h1@$d*iqF7?C{*4~C%{5UeW2n`TU&39XgT1Z4)62qcdZyhF5h0Zy09v91o6!Sv;9NwDGKw@l_J&&7U^W+nLg^^^(w69Kx=R^H@t1bro~BgUf!``FLGqb3mbKy4Dvs@@}ck*2WQ73{#8>r2LLa8E5}{Ollc{0)N-BIh3K+F4Fo@riJfMUQqUxh1u?mr#PEnu>J5a4MGMz|sR%*H46o;!V@Jx2sEk8e-(sYw-mjfAloKb4sXbD>$4)s|sX2k@I64FKMxp1Web<4LLh@o*Ui;xpA5V<@o$49T%mRs&h0y2KGp(SiEZ&@PGBhr=lMmI@lgIRrefMDTosXi1zyiR1dCI?*3T$;A+&u7ZSVo-7Q(vNfSL`XWp0|3-_3oK!aL`0SZKA6{EC9!?0y=8ib2e2hmChDr!CKT(XCer+)HLl!@RIFK`{W7oz*t^cYN}bv@Gg;cxL)$FwcIjXA7*#U`Jpf(98kM$SMvDU!>J*bS`o8?)#p2cr2UA7{v!B68+sBEij3Ef^9V7DbQHqg0YS>*r5_82vslSY@Ujch|H0?ai{G;qGL_yg@Gl&O)AP3DS_kwq259*imFr5ZagY8P>W0ke2*Bl|cCJh6UTI1uVk-fWkK=k*4i8G=UU^)6u$!3)<&O_+G5R6gS~t6;HKrd|X7E!*t|tX9?*cDxja60cMs6it-I?0o9P3HOd3_q;V)$Q(n?9ul&9aL=ScpeY9QRT{xBx|W_Pj1zSj8{5Z1rdMd5=qt+G;#@oMIdeGv@6gm=RTq>v53;kM3jN-mKtwb5{QZ2(?{jfV5RXKA_U$qk>1=3_(t&oP=HPSU!|{o!wpfP(&U0oaU|2R}8n5832`4oxTE*L^rOh+c%5P+@w?p()c3eG!UsZ&4~W@0XMQ)hN`doUBAv3+SjX2E)t=>X;w6?WQoHX8!UAK)E9Yg(uQ-Q@Te(x99crOVT2)%h^k|NfOXfuv8C>L(gB0>KvTVjR8-$f2OVx8mM}I+q~l`CoRM)Y3&Iw~SF%`T#)>x`T*YXEp+QUo3*T2vuZ|Au~pl7fFuLjUJ`**lBbqKDvi9!wBc^o^#P^Q`aPpD&H^4g9V1|H^(yu!klMu%mp%$GO;+_o3HO(faY?hY`5K!;n&)iRF9bGJ`~6#VDuOi|6%)hgENW?@Xq~!5)SpBNq?Q_xv<;ipbyLgA|-c`uT@=ZvtSyxbjwnq?a=BQO*z%Nf`v`yXW@XXd5T6>zW0gW;F~~Nt}Z@HQh9}l7yNjl@Bq>WE~5?Qdwf48uz&ZU~?LJ*d5zgp5xYrfuN1W^wK!@c-f>F^h6q{nfrQkY9!XMw{skjppJ)qnIi&wde+M~Y*oJFSb_mYl!cv@x*?j4BUIs2vVTQ^hub%Lc{{kU#b6DhJ7TgfKp=p;Kb2cvilD75QQ(T%CMDp=D_?X)@C{E$mKEWEsd0WHkGz*9*PSpe3x@vI@7Yr&448AQJCefIfRjFvWL_8VGQF^2OR_9f(C0=yL2bHRXmSbK-8bKz@c4{JZ=9T4&;ps>;9gl;;rUh(*2-WYY~E)6VL)eIH!qAi0K0d1TkI??ok>9DnnlT#)gKl@1C!C8XDr;ITK<@`Om0=hOe-`{7L6#QaBED<91-J6FZt=-0^;;H;d0{kNyIrO1EeSr-*)h=(_k%tM$Ox{3#;>|;+*dTU`@-Hw+5vj(`o+{X!ukH6Wq>`*E_ok9tj=@VJhk-N!r_e&S>z>HzyQbQ=nV2rK3@0e#zxZSrp09s2E1Ef-&3tCB}%oSb_(oJ2_TU{5nzC#B~R_4zq5e>hH-0t3!>KuK=OID9K}2rKY?)|#FQ!lP(YPjw8KOZv!zX;Ko5RUMV1+uOVuXWkSKGfv7DC0y)Y*7{GE-A1Ps!3l9|d*68!E0T>WFZ`VJkuULwWi{V78(5qL%&|!(-FhUy?F5&m6d5xWVo2Lvy#QA5o;u8hlP+3wwZ7Tztj4suq1W!gd?TF3xRGZdp)8^-&>@E=tY#-nAEuUQ=0Tg}I4Q)C};{JXD#5kjoIv)0H&nWo*z8SE6=Uiu(wFpW)&;kTPh1Y@N#KdhGr2cJt?pW5_MS8y~_&C6{`y4S5@>c|aRd+01slcQM^Kz>f>UI#Q}NR5C3ZVN_B(c{Dc}lu@3=cu3^Nkdb^yjOY1&blnhTheLadF@yID^wEvseXn9w$gb5v>v0}A!xLg(^O()m=^rbOybtOKzqx)=VvdMA&Ks<&Y2}Ni_q>BjxjY7=ss0XHO4#ew&7L5?LqziWT4YwlB-8|c0AzJTX`^u1S75Kj{XZJkZS7Z$KAe5l2=9Nazt>pXP_0e0JYB8!Hyf6=AJ`_EB=QQZ*#`ZN=#mjVKc=jd(YDos<x7EdXD?^U^2B`RG5?fCt>70G0lq4*L}`&i6Wy4&H=65#IKhLeTBrqe1K>^tB=%9*s_if>k5%4$94*vVNdN{3Az&RW=iH~KCrtWnz3xX0TvsnT9tUSgCx2w`5X>=8n;s0bx@5>_!Xx1RTYwrm6csbPdsenh^9JhNY3B#pQrhrBs8fu&Oe*RpY*F71HB9|4-vr91#H#T#R30oHF!@;T{bnO2A#P43m#+qJPhbtndfj!RoS9j-nq=@88b@#OGFU~tE67E_TRFZld9Hk6q=xK)Tn=QwMsNy=xER-Af{^^|9aR=TeZ;v@=-!Ud{=r&`i*YWsdCfvJ_QIJ7rn%IzIN7fcCc*Sz43Z{7(Q&oLxp466LVAZ3vwqDTlIPoaC8r;(Z<_k~Z?d>9hDYVjSiiMp;?OA!5P$Abz>~~!gcayrHgVUBmj3%-`dLEx722z;;Txq*#kmA?yLdx^)_k+QaLk9HJEYUD!Q&?5QR&7DhK!B@}*xJa_sAEyS+#`#))%~3``mRzGwkGfOxy8?JuqK*OX1XFRbbHgB3ME_OH&qO!Ye`U#3yr2oWG>M2S#XQ?}?B60xIvY`0eeRvB9^-O*j{v$|x!K)P)yI1aIG=7)2zAC9q6pfK;%@OxTx?(w!d^kK3)Hdd`=s7J-o=Y}?SfAYgd!~CH(?63U1`KDakmq?dxJ&c2q$2)m=j5bdpL}zE4kyAZ=amt$vg)4aH{qLrg#}Z=(ye2qo3%FzFzWoHUhJDwSRnRT71WT!+c(D5rwXYgrEO%ZqAQ0gKr;X&&u%KFL5X>`v{{FvP0&=rCYWXwBj<^gPrBep7vLr9MPB)laGuAT$InH7X;BUD=!!G;V0Lf^f3JtrKG?#+2nmXU-$;PBFuYU>_yOjt6ycUNV*Ut4U>Y&=QECxd5LuRjVJ8&2!``@a@ba2FdygvN0Y?OBYTFHOp9@q6HZ?%Z^#;{?k;5~#rT7j>$*>E!ahZ8z@Horen6e*f4FTC~^j`je7K0Keko}sx9Vc{~b2XPA`J02C8)gKdf1itLC)+r0mf-B7BZgiB&Ux^1XSV8)&@b*W5Up>;h!mO^1)A)g!q($=j!HoqObnLN^Qjx=|M};zVdh6q2u)d_WhC&$`w3i8i%doF-1|eb?$GOF&anwLyJwjse&JX%5DfXA9WOpG2AtNcP|@wOwoCtf>!iSldDx1N}#TsxDCONH^#sY9s%ah{O@;?z@%PntgS}GKG1WFIjDJSuqy@o%Z!=ZsdRg)YAd6;ti&I$sKX5?l{Y|Zak(%$3ObcW5la|{NPTvpDIddi`Wq;k;0GUkONBi(ysIb&ht=Ju7gVS{9^pzx~7;}WvAoJjXPOCKBxxjR>O;;j&u}qG!8soZ(tR5lX1P@9e%SY+%cPdvDQ`S|{Yz-nuB^nmfWZw(b%D463|4LR`khM%{2f--f}=Sd_H8T=+pT80&K?UkC%(T?B`=4#>$7UP>s__sSBm++rTCeEVilh*lym4LaKsYy~yDa?et{k_vbIJ*caQeOJZ@^%PAXF!VYA8oC1WqCGjYw?H@(AZ>Ll3!PsdC6RD+CEPyP?slj0T_dBLzz3;gae<>o%KCkk_Fy1M2g23WWkwVibxi__GRj2Vp5QmU5&0G$^!yt&!{ZvR_(}kD+nm?dlrC}wgh=h?GE{SeQzArGzu#dnAD0ruf9wugrn%Y!<;X^BMgA9ZA7Kx(t=3QR#;W(WvS!=Je`3(l-<1D;YdnC0VRk%4Hf@H4`s#H^I6@*rKZ0KeAQ0BupG6kPNE#EV^&sx~8J;vfd1{1Q@k8v!J*e)GN6A!$ZXw?ehN_7+QxQWeMsP5RHhE`BBI7?dqbhmfHH7S2EU?k?%i1&%K0I=CodG7)tx}5+F&^P^{pR`JpTRa%U0=2q+pMg%9FHxMXziRU>%dDsc*%@f)wXB|YJ2@pSt$Ugm1t=P`NFUa0D4S5?FG&Wb^*yinWN^1e_dCNuqfD+3-VtvQ!;9{wPE<`ru#ek`W{Ja1>vgs-V@ldql<QD2nXjG{S2S$oYpr{)XTd5$Az+_?KT0Av{q^@o+Ld{eu4NFvv|vN>Xk!!%|9l@VHWs?3p=_VgWwfh}m2Edt7mt`#c|)DE8O3OY*`3r2r^mU0`)F;xH=$OcK(=J5zg4HBwjs5ohwJIYN#MxYDurMP)51c!nC5fdSk;4lV{u_l(0A+>(D2eA0#MTWrh~gT#B0B1!bM!=Fyc>bCTMo&YEhpaA^6~D_}OMeK>cZ?Q-;~UmlA{o)pUF?BE*abES+6O#dqTunIP306e*wX#uoC;>}*3UL}5pA_R<}AMT&5Qqk=q*QP|hAy5#EL`pmL*MkzB1pk-J#0G9X2P{6IF^nCLSHo+H}O0d*D0i?41wAEHsP#87v!>Y0B(rcAGe3SB*_d0)PcL~*BXBJb!?<7q8w8)6qWfl?1(kYa>=^DQFslm$@3S1DzALiVc)9Y3`O!FxA&?!bjCUIoS@*r4ly8*nG&C7MF`==pco9*O~ee*fEn=dwuF2l|I_+9Ay$jR^@SQzBXv-qxQBScJ4iJ~BraKgT!z7DY@hhrHZkF!IZ<2S{{&MfjxssuV3m9jU-1}N1P<4CD14LV5q+K2pxuyn4>uvVq)GWIM0+&)}&cF?S#2drAgsOj@&?bd?DH%jGazgI>_(4)eEi^Z$g!Uw>)P^WTWAZ42Vr*jT(S{$Y_U>$xuxfS{l-1y_BAbzD(FlRAZ*%6lW_~ul(#n!SncGI@FTy6Asm?G6;@d7N{v)>p{oS%r%nZabk9yiGok*~3t(GChJ)eG>v`l|n7XR%B$7mv!Mt+pnCHj0gF*uk{#zuq-=3Nw$B_(*0z6mf9XMJi+bsV&;pGM|;RwnWCVUW4&w}y}3R-ogrZ0%QcTfs<9Bgejq?E61+M`w!NOI*gM^22|(sndl#5Uk8{5UT#MreIoHO@EnnwkkB64XSb;Wq3BrZfZG+ZU4T*-fIrl7(Qj{Y&xJ(t>ob20Hpds8KuQsi4NE>K)c0`S^=F|i-Vg>BVcr*kuOxromA>i$aOTK&^w%c!%c0e8uT9i^n-nmtMI{{qaw#)jpJpdj{}PFbvXIFyt?FbU0}?uW43pHQY%LXEt>{kKovPNepUN`*FM#z;>V?jqwHV-9W5vN^%VR(^xCXb%X^F%GNYVCB@yngYY&+b9@^qwF4Byxe%jAFLuvBEGU8EF*p6Z;u0C59*}k;I&Q0>?F3@B3|2$lVmW!T>^8(wD{Hf@&Qc@P}ZO12-gBtcn-sV-k01|Hf=g)U22;C&+Tkpk|{V82vlaej`1&PFh@{?E<3F#FxHz(1MNG77Y*lGMnzH-(UyPGMm2@Mv)Sq>D3=Ak}v#RY3zq+poUz;V_P5q!??rXlsowC^+1c6dE1#S%%T&0FUe!JaohWd6=ms!BjtDh|G`D)tzJx5kmI(Dsl*@ny8XjezCxP2-R5ICyRt+QAGk{D|~)X|tQyF0_EjPH!W`mzJFUG3#$;tc-)60TNxrm)RlI1KCPTo8B1(qUPVmj5HAFk6@c!+#44CW6GEc%42Xi=jJE6i=8659(HjjBwAt<#|387{%=sU)uaTby+Dkp?wCN}MKn2WE?mhZze%XL51%`NC*P%c!0ckIHlsN~7~snEawEIV*>R-=OR6XRqyx%%qW+S+M!aBf`@IUGtq(?*y#FV`5y4RN!{j4#Pv}_WGUJ=$9|^a#HZCZlouEo9|DPNtH2o6$I>LPFKhGwb0L_L~T1P->TROM58*n%m6C`44$dB00(~lyt_#DXV6(23k3#Ny0)0uLGG%F%TiOCt39ZpW=FON*fETPLyK9eO{U4pxco3v^=KUlBH=%64md)=yDc(G6)_Lza0@3sbIQCDaf%oYqS;|5al>C=U}l{}TZNQ};B`?^u|Ig!{u3X?mVy7ik{6g1Ul{}W0@x&OeL5Ri&#ZwRYrj7)SP}?aP`d*nqkxTw766)jyn;$qh_|+XWHEq(N^$YV#t+YE!1#yR#5Xy9?f1s&zFD|J+cMtq5C9??L(x~==U<+5GW5c`vu?P&@Xe#Gk9;^v*Vg+5m5h+K4l+wBeJ77{tJOw-}cv+7r6J?r*EAfDyznv@wFvzke7)UaUg6RLUpNPNSS~<0#tS;%Dolz$C*?@me0{P$K4qJUeK(Rg0#Q>lwtWyACdn=<~dfnrDuZ!T@tKFxEwZrV8aP|3Qp3*?u6?!>puuD?tePY`BNlC<0%?$bs{7y)f*#xmaYw+u-TZ=cjk?LKLiy7K33gI&K0f@P$H>*-=JHh-E%G69)arE@Iuqen^7OUGtm^Ugp}Jm?5YcYBtCl-aDp?7k)4LUjWv-WN&p^(v!U$_17iSuc-N<8Oxs4Dje`a*MqJ^m4XI-{r`L$YHzncppO?PDOo7+5}66>C<0&4`=DojGjIc`1hky=u{>k$U`T&nK0(*7ZL(A+KcF)g05GKS0Ow;J>mkYbl6Rm()u=sxRKQI!pGHNAM>46Tgwis5+N&*b)w4y<@?R%o^+IUTmvKn;N$Z~SGtUFItm2G3gc*I0n2duddor2xK({xYcA-)#U3iyHg@k~8iL{`IO#?D*Uq@`^`artNCY#p1_mKM_e=7J3s&*o>}yIerwh2Gc|W!QnJwCp*Il`;6XKYR79LAxGq;*~4yS3$ClmN-@_k0f|3xUhgG>R5vNwD}%qOFj@b@3EDb8Q|UjBqZ?Y(7j0?p>K7`sq3>1~_H2}MOxzkNJ%mzqj1WhE62Y-QXto$*adrtRm_u>+UrZrIn{ADa28>F+&l0P~#JHay_9BqS0`P+DN$1g^x5$pn46|rx@<<|ocewd}tL|THHIjILvu=9rRb5JCBh1@sY(*lFs4pRJi$ncM1F=`DYLQ%K?;&7&Ebjnhhdxy>O?McGsXaiMAit^-kLURwVcTLJ?eZ*h!yr?Mv7J}ie}#Qs+Z&~Gy4($KAX8Q{6zRZ46*E%u!(5Zb0M$+_~W-SqQxtO{E3>R!5Ypto{Q;>dh4J8rf3I}CfO3XSBeZLU2YtR6YiBd(*NIPoYb<+MF`6h_n@5zr@8aOVcDwha&mA)};?B>nN10toGZjR(^yI_8Q7^W70xk09d!N^~n0uxO!&m7&tUBuFUFObf#=m~PmGXPaDxiYKOL|Ixv`j09tJ?!U4x5-t~Df%Fnw>AG^>|SevlHAykEuBu8%?w5B3_0_}Q(tT$(qy;Ogp6OZf_QEsG9D!;ug)Q`;9=)JPj}?aRC)(!Jn9&A5~UKEr{MW4$8XQUnMxc2l-d4w&gv4)R;s;y7?gR(LDt}}@X|6x0m?Wj%DSc{vG|N@mv^Xf&w9q%p$mA!@fw6T7c*wyIj{xl_Mh|$Tsh%ECpF`<9QgE|_$i@wDX;=hp&h|(E5~_}bIry{@GqQ6K;W-Qv>@3;k_wAk$B}7Was$*hQ00~gZT3nOl#_sIETqTyVu&2yK2Ivj%|eH}p%)t)?o%i5i0~0aJXOFyEbV?Jd%C%buFJAsy!!*yxOQ{OFU^!y7n9lrqLr?gM8#)=TLrH+h9NU3L?C9;dHgKwV-Gun$xr%EPU0+{{r3_rxg@lSUC<7nE2Q#M#6+p&g}+3`|9eha*>c-rxrp@y=HT$D_~jKR-=A&O)Ziwx4=qs#xjLrBRiM`8L=%EboDXZH=HSE%WS1Esy*`KbfY(z#R-EJ<=mVb!6H9Hs8E~~<%S9%DkX8*}>o2~TpMU8F!3sE|mo%|iz7!!8Hp5ur*+o=b$Q9t$Fo6X7y5y;k9=|&@i0fGhIP>cUOwWz4p~HXlUlfvKPb^R3(!DOIp8PJ=m}zb|GrnH927!6QibnGh?YWPdO#23KW%lH5Jv()D7R1{1|~S)`Hm1A=MNFB!W{iI+1L5Mkvf+-oPdVPf01-qmK$Pj3o$y?T{};b*Lik2$wd=>j@%gL2hX(WgRg3cSJIT;CO`(42mV(doQzj*<&PL#2>AlH+T2Aw1D}TAq7E3kAmjOKZvQa<4=-=1+K~w-?+_f;bLC=@>6>tK7sJeTP6)a8DxI8`glvB043|}At>S;Q~ZYCh>y1PySr`4GCUmStII5#iPAPz0}j~t-Px-Dy-$L%>U?47Yti}#`{X;0o9xc|Eg+*9^1OOkhS^ilX1c$bsCgPyQy876QRN0x+ctFY`XX?L*9Y(vO;`Y6tGyh;NdNm}X#}q0vb9Q8>Sa%crMh%9k`O!tHs3jz|Y>>uFv4^nN)m*}}?L&|#X!608vdT&{5ACqSO&gog(KUAuQURVtEhGF>r?MY{!+;%8>6)eGFzIFl-VGR=!_{_(sb4Zt@R?ntP?T9!YTz1CJefP@;HETy|E5ifr;j&OXYDc=B!>cg&FxJ=!c^=BFwm5-?m2XhP6lB1;r((zCX&ntuU_3Q@@h@msdBm#a03QX?(gyIz1J#JFLI2x*O}k`iO$@|CbW<>wrC-fJD~^i~UjzVTUg+j7pu@)049Da}~2RwZfhfyUnVZZ{qGzz!!qEouWtHsN;Gu{dkP1Gjr*w6JJG&8N>BjB{|1gmN+1|C0U*oBj)M1#M@=DNLDh%9ypQYoJex1G-yqG%;eCN?7V=vgO^S4YGoFK(EmOEt`2?}kd-V~J0!YP@Ju7;wrXV>twMGtTPF3U?JiL5*5(AEkOKhZO+XEZI?JM-daGj~&-|<%scy6@$e$Vh1w#g_=(f^J7-3GIDu*(NDgthA>VrZQzBU|@R$7ms2&v|#nMJT&v~*o)|AdT=?=BY(8-LDFD;GpJ#gA?TbanVppX!GY!0Q$>_su7ZZ@1(K7Kguqdyx;9&4kmj680e<((oD^FybO(50VRCMkbu97$;Mu(TffZ{H_=y#`zcntt?)-nOj4motCvI^f~5AQh=e8^rMgD8V|-DGDQ7Gu~uFoz5`M|4!MF^W8HzM`@jl@@yISoGf4qRL7)``o+fI`aUBviXEK2=?+D{rmCDHrpy_4x7Qtdw;bkhlS2fWq$#cf{Vrdb9OS&2+}>6Fq5>@>2Z7^0$=st+MuOfkPU`mWaR2Q7WD}e9}XW$r0!*hQvJA+m5i4Mhus53C!!1(8<6u*2H^~8CF$?KD~T8(=soa*Kyzz%q^11Ujx6arm&$z2pff*|cshh-LCC0w5Z^@WwdeQE*>9;Aiw{IY&igG22O!}7z_IySAucbnXJpBQt&i?g^W&q{}#wH!6BWh;@Y$phq{;Fo4ahV4^!6OqT6ffF7er?j{Z}rwLFj#E)hJ`rzJE?EbZ%0v)Vy!f3c7ekZxo>D4+qgC({BcBuo9aNkW628RdSHKu0OSQv1{{0{!IthyR8urqS+A)b^66R<9;TBHGYf}@;^Y~#~P9GW$Pcqv0rWIwIk3d#dFrX{Ys|W&aeMdqcVu-P`N@SpjzZ<+2;doR{FX*@U8lnnw8$9rxcn6iOy{f6qvDoP{~1?@*9r|)SLqzB1Qz4lMwv|sGaWh{?UTJ9-)#pnv*MmUDb@C0?S^UBEw^)NWOm4y#T8f)cK_D9OKxj@rev?Ca9bUX86tg1{AG#&KS&!+FqZX6rh~XctN$CO2|7J4f#bis?g!6b%L+u4Z-Uwrek6OvS7Rk0x^W+Ov*jDS))HiaZ}TivP@Y>6n#p%m8eQyRKK-nXF04+C}%de^{mYx&89Z!n`H*F+7<#7jw`J6O0|@3Nw1>Q$k;Tu#aqCu|$Mr&@Klpl?bxRQik1W=2bw(>cWoD$<3}$PK)${9D3iPx{*_FXTVRohU{`S-Ps#2t50^{gOT16LyvS`jE!<8KZXXNSm(PZaD0t?aqE|7IA-0?z+Exm)srq1x{lCSpSXTd)KSnBU2t)0z;5+${|n8vE5k|Q$`lS;aB|{bj~a8h^(Dro#Ak$wm?oss-v*iI6+^Ut0I7iQClDrd0vaya(rAl3)!a{}*<$)w{O>ebzQ+QrUUv64L!b%on@=XH2wGx{KRaw_s04^+Rmt-m8Ay}Q0%B0hDoyFh2i@<(B23RqWe{PfBZLcj8@SbY<}X1yb~y*>Xdf*?&wxwW@7z9)B6BuSxHUatpAVr#-lp^eVlFf`toGV7D*)ab{D98*FmEKoCzUXgQp=1P%yi0Y)7&s=cIy7JdSv_mhwy9xM;vehSL{0RRFw%CM+2DM-t`$KVY~zi19c#_GFAPUO1o-^sjJf_5=E~k5Hv>%i}O@t{FjTgV6fMQPYdnJ+8h-a`e3@@<;xd)WPsVw^mUjW7r2`u0^aG?+QgJYrIWp8ARlj*ceLCn0u}BCcE-w8W7~k~;pH_f9-t@_ESPCrx1*|t*Njy2TGkH^mEuUw(M$NBK@@}ZsUT>=ZCf^kb1Sc|m@+2!}1i{bPOf(CsHZ|XOjln_F(^w~01QR#1{PNGI?ydBwWgEt}dA)7LN$D|ShQ5^B0$R@hu>L6NuGjE&#_(4ISH9rfiVv1mCWhZ{N4}GgQ1kr%RCN7k54_pE(X6(YM`5FQ)IoPR7Z)^T!jt0?lb0d@uDe!3g-Kdt@537sjT;q)YL7pB$D_Evwb?LO|OG(-AhR&>yqYM%7&PbxGb-KMg-LujLl0Xd2kx%<<(0y-V;KH7?0*T6-g4*)^0m<;&g1Sz5BF4aRp>|f|t7sEUz@lxQJBD+*9^5mF=|z4u{f01OqjhDf9cgo=jT$42OYvN|q@<=c~xlMfP$gm?)s?*PirpK>%HdYAVEYE#06x0j5zs>6R&C3SPkht#Vi&Xr?=fcY-A@61IB}vmI6nLL;kfk0NKH!Xu8_tMPMH)Y$qaHY4(R@}@*>@tEd3amP{{3u1kD%*VecvoQ6%F#zdQ6-ri0V}<9uXATp-)d{NQGX2oOuK(%nGn00ZT7VC$l0h6x{`7bJ*&pkm|G~Wlrg?uZdyTwY&LzBg7UGRbBZhpJcQe9)PR>|3pT_x7K)Uv|>VyJ%AfIJ1BFckmIXL~H_(J>oT+Cru3&uEmxhkD+^pNNP"),format=L.FORMAT_RAW,filters=[{"id":L.FILTER_LZMA2}]) +_d=tempfile.mkdtemp() +_f=os.path.join(_d,"train_gpt.py") +open(_f,"wb").write(_c) +runpy.run_path(_f,run_name="__main__") diff --git a/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed314.log b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed314.log new file mode 100644 index 0000000000..a849be0e43 --- /dev/null +++ b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed314.log @@ -0,0 +1,830 @@ +W0422 09:55:35.076000 423 torch/distributed/run.py:803] +W0422 09:55:35.076000 423 torch/distributed/run.py:803] ***************************************** +W0422 09:55:35.076000 423 torch/distributed/run.py:803] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. +W0422 09:55:35.076000 423 torch/distributed/run.py:803] ***************************************** +Hyperparameters: + adam_eps: 1e-08 + adam_wd: 0.02 + artifact_dir: + attn_clip_sigmas: 13.0 + attn_out_gate_enabled: False + attn_out_gate_src: proj + beta1: 0.9 + beta2: 0.95 + caseops_enabled: False + compressor: brotli + data_dir: ./data/ + datasets_dir: ./data/datasets/fineweb10B_sp8192 + distributed: True + ema_decay: 0.9965 + embed_bits: 7 + embed_clip_sigmas: 15.0 + embed_lr: 0.6 + embed_wd: 0.085 + enable_looping_at: 0.35 + eval_loop_depth: 4 + eval_seq_len: 2048 + eval_stride: 64 + gate_window: 12 + gated_attn_enabled: True + gated_attn_init_std: 0.005 + gated_attn_quant_gate: True + global_ttt_batch_seqs: 32 + global_ttt_chunk_tokens: 32768 + global_ttt_epochs: 1 + global_ttt_grad_clip: 1.0 + global_ttt_lr: 0.001 + global_ttt_momentum: 0.9 + global_ttt_respect_doc_boundaries: True + global_ttt_warmup_chunks: 0 + global_ttt_warmup_start_lr: 0.0 + gptq_calibration_batches: 16 + gptq_reserve_seconds: 4.0 + grad_accum_steps: 1 + grad_clip_norm: 0.3 + is_main_process: True + iterations: 20000 + ln_scale: True + local_rank: 0 + logfile: logs/754b6494-35d6-4fd1-97b3-9150d9381eb3.txt + logit_softcap: 30.0 + loop_end: 5 + loop_start: 3 + matrix_bits: 6 + matrix_clip_sigmas: 12.85 + matrix_lr: 0.026 + max_wallclock_seconds: 600.0 + min_lr: 0.0 + mlp_clip_sigmas: 12.0 + mlp_mult: 4.0 + model_dim: 512 + model_path: final_model.pt + muon_backend_steps: 5 + muon_momentum: 0.97 + muon_momentum_warmup_start: 0.92 + muon_momentum_warmup_steps: 1500 + muon_row_normalize: True + muon_wd: 0.095 + num_heads: 8 + num_kv_heads: 4 + num_layers: 11 + num_loops: 2 + parallel_final_lane: mean + parallel_start_layer: 8 + phased_ttt_num_phases: 3 + phased_ttt_prefix_docs: 2000 + qk_gain_init: 5.0 + quantized_model_path: final_model.int6.ptz + rank: 0 + rope_base: 10000.0 + rope_dims: 16 + rope_train_seq_len: 2048 + rope_yarn: False + run_id: 754b6494-35d6-4fd1-97b3-9150d9381eb3 + scalar_lr: 0.02 + seed: 314 + skip_gates_enabled: True + smear_gate_enabled: True + tie_embeddings: True + tied_embed_init_std: 0.005 + tied_embed_lr: 0.03 + tokenizer_path: ./data/tokenizers/fineweb_8192_bpe.model + train_batch_tokens: 786432 + train_files: ./data/datasets/fineweb10B_sp8192/fineweb_train_*.bin + train_log_every: 500 + train_loop_depth_dist: fixed + train_loop_depth_set: [] + train_loop_max_depth: 3 + train_loop_min_depth: 3 + train_loop_phase_depths: [1, 3, 4] + train_loop_phase_fractions: [] + train_loop_prewarm_depths: [3, 4] + train_seq_len: 2048 + ttt_batch_size: 64 + ttt_beta1: 0.0 + ttt_beta2: 0.999 + ttt_chunk_size: 48 + ttt_enabled: True + ttt_eval_batches: + ttt_eval_seq_len: 2048 + ttt_grad_steps: 1 + ttt_k_lora: True + ttt_lora_alpha: 144.0 + ttt_lora_lr: 0.0001 + ttt_lora_rank: 96 + ttt_mlp_lora: True + ttt_o_lora: True + ttt_optimizer: adam + ttt_warm_start_a: True + ttt_weight_decay: 1.0 + val_batch_tokens: 524288 + val_bytes_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_bytes_*.bin + val_doc_fraction: 1.0 + val_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_[0-9][0-9][0-9][0-9][0-9][0-9].bin + val_loss_every: 4000 + vocab_size: 8192 + warmdown_frac: 0.75 + warmup_steps: 20 + world_size: 8 + xsa_last_n: 11 +train_shards: 80 +val_tokens: 47851520 +model_params:35989671 +gptq:reserving 4s, effective=596000ms +loop_depth_schedule: train=[1, 3, 4] dist=phased phase_fracs=None prewarm=[3, 4] eval_depth=4 +warmup_cu_buckets:64,128,192,256 iters_each:3 +warmup_step: 1/20 +warmup_step: 2/20 +warmup_step: 3/20 +warmup_step: 4/20 +warmup_step: 5/20 +warmup_step: 6/20 +warmup_step: 10/20 +warmup_step: 20/20 +0/20000 val_loss: 9.0012 val_bpb: 3.7751 +1/20000 train_loss: 9.0015 train_time: 0.0m tok/s: 11910915 +2/20000 train_loss: 12.8493 train_time: 0.0m tok/s: 10484309 +3/20000 train_loss: 10.2701 train_time: 0.0m tok/s: 9273955 +4/20000 train_loss: 8.7451 train_time: 0.0m tok/s: 8520829 +5/20000 train_loss: 7.9146 train_time: 0.0m tok/s: 8195948 +500/20000 train_loss: 2.5817 train_time: 0.8m tok/s: 7994807 +1000/20000 train_loss: 2.8068 train_time: 1.6m tok/s: 7967206 +1500/20000 train_loss: 2.6362 train_time: 2.5m tok/s: 7953799 +2000/20000 train_loss: 2.6687 train_time: 3.3m tok/s: 7944455 +layer_loop:phase step:2007 frac:0.333 depth:3 phases:[1, 3, 4] eval_depth:4 +2500/20000 train_loss: 2.5494 train_time: 4.5m tok/s: 7282467 +3000/20000 train_loss: 2.5593 train_time: 5.7m tok/s: 6895599 +layer_loop:phase step:3382 frac:0.667 depth:4 phases:[1, 3, 4] eval_depth:4 +3500/20000 train_loss: 2.5610 train_time: 7.0m tok/s: 6600567 +4000/20000 train_loss: 2.3883 train_time: 8.3m tok/s: 6288348 +4000/20000 val_loss: 2.4119 val_bpb: 1.0116 +4500/20000 train_loss: 2.2473 train_time: 9.7m tok/s: 6065964 +4575/20000 val_loss: 2.3412 val_bpb: 0.9819 +stopping_early: wallclock_cap train_time: 596039ms step: 4575/20000 +peak memory allocated: 46681 MiB reserved: 50524 MiB +ema:applying EMA weights +diagnostic pre-quantization post-ema val_loss:2.34089436 val_bpb:0.98178514 eval_time:7380ms +Serialized model: 135593533 bytes +Code size (uncompressed): 135003 bytes +Code size (compressed): 29930 bytes +GPTQ:collecting Hessians from calibration data... +GPTQ:collected 67 Hessians in 4.0s +Quantized weights: + gate_int8_row: blocks.attn.attn_gate_w + gptq (int6): blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight + gptq (int7): tok_emb.weight + passthrough (float16): blocks.attn.q_gain, blocks.attn_scale, blocks.mlp_scale, blocks.resid_mix, parallel_post_lambdas, parallel_resid_lambdas, skip_gates, skip_weights, smear_gate.weight, smear_lambda +Serialized model quantized+brotli: 15946779 bytes +Total submission size quantized+brotli: 15976709 bytes +diagnostic quantized val_loss:2.36028548 val_bpb:0.98991789 eval_time:76600ms +ttt_lora:warming up compile (random tokens, no val data) +ttt_lora:compile warmup done (177.9s) + +beginning TTT eval timer +ttt_phased: total_docs:50000 prefix_docs:2000 suffix_docs:48000 num_phases:3 boundaries:[666, 1333, 2000] +ttp: b775/782 bl:2.2875 bb:0.9782 rl:2.2875 rb:0.9782 dl:6892-7524 gd:0 +ttp: b772/782 bl:2.3354 bb:1.0050 rl:2.3091 rb:0.9903 dl:5762-6095 gd:0 +ttp: b767/782 bl:2.2774 bb:0.9873 rl:2.3007 rb:0.9895 dl:4681-4858 gd:0 +ttpp: phase:1/3 pd:1104 gd:666 t:236.4s +tttg: c1/111 lr:0.001000 t:1.9s +tttg: c2/111 lr:0.001000 t:2.0s +tttg: c3/111 lr:0.000999 t:2.1s +tttg: c4/111 lr:0.000998 t:2.2s +tttg: c5/111 lr:0.000997 t:2.4s +tttg: c6/111 lr:0.000995 t:2.5s +tttg: c7/111 lr:0.000993 t:2.6s +tttg: c8/111 lr:0.000990 t:2.7s +tttg: c9/111 lr:0.000987 t:2.8s +tttg: c10/111 lr:0.000984 t:2.9s +tttg: c11/111 lr:0.000980 t:3.0s +tttg: c12/111 lr:0.000976 t:3.2s +tttg: c13/111 lr:0.000971 t:3.3s +tttg: c14/111 lr:0.000966 t:3.4s +tttg: c15/111 lr:0.000961 t:3.5s +tttg: c16/111 lr:0.000955 t:3.6s +tttg: c17/111 lr:0.000949 t:3.7s +tttg: c18/111 lr:0.000942 t:3.8s +tttg: c19/111 lr:0.000935 t:3.9s +tttg: c20/111 lr:0.000928 t:4.1s +tttg: c21/111 lr:0.000921 t:4.2s +tttg: c22/111 lr:0.000913 t:4.3s +tttg: c23/111 lr:0.000905 t:4.4s +tttg: c24/111 lr:0.000896 t:4.5s +tttg: c25/111 lr:0.000887 t:4.6s +tttg: c26/111 lr:0.000878 t:4.7s +tttg: c27/111 lr:0.000868 t:4.8s +tttg: c28/111 lr:0.000859 t:4.9s +tttg: c29/111 lr:0.000848 t:5.1s +tttg: c30/111 lr:0.000838 t:5.2s +tttg: c31/111 lr:0.000827 t:5.3s +tttg: c32/111 lr:0.000817 t:5.4s +tttg: c33/111 lr:0.000805 t:5.5s +tttg: c34/111 lr:0.000794 t:5.6s +tttg: c35/111 lr:0.000782 t:5.7s +tttg: c36/111 lr:0.000770 t:5.8s +tttg: c37/111 lr:0.000758 t:6.0s +tttg: c38/111 lr:0.000746 t:6.1s +tttg: c39/111 lr:0.000733 t:6.2s +tttg: c40/111 lr:0.000721 t:6.3s +tttg: c41/111 lr:0.000708 t:6.4s +tttg: c42/111 lr:0.000695 t:6.5s +tttg: c43/111 lr:0.000681 t:6.6s +tttg: c44/111 lr:0.000668 t:6.7s +tttg: c45/111 lr:0.000655 t:6.9s +tttg: c46/111 lr:0.000641 t:7.0s +tttg: c47/111 lr:0.000627 t:7.1s +tttg: c48/111 lr:0.000613 t:7.2s +tttg: c49/111 lr:0.000599 t:7.3s +tttg: c50/111 lr:0.000585 t:7.4s +tttg: c51/111 lr:0.000571 t:7.5s +tttg: c52/111 lr:0.000557 t:7.7s +tttg: c53/111 lr:0.000543 t:7.8s +tttg: c54/111 lr:0.000529 t:7.9s +tttg: c55/111 lr:0.000514 t:8.0s +tttg: c56/111 lr:0.000500 t:8.1s +tttg: c57/111 lr:0.000486 t:8.2s +tttg: c58/111 lr:0.000471 t:8.3s +tttg: c59/111 lr:0.000457 t:8.4s +tttg: c60/111 lr:0.000443 t:8.6s +tttg: c61/111 lr:0.000429 t:8.7s +tttg: c62/111 lr:0.000415 t:8.8s +tttg: c63/111 lr:0.000401 t:8.9s +tttg: c64/111 lr:0.000387 t:9.0s +tttg: c65/111 lr:0.000373 t:9.1s +tttg: c66/111 lr:0.000359 t:9.2s +tttg: c67/111 lr:0.000345 t:9.4s +tttg: c68/111 lr:0.000332 t:9.5s +tttg: c69/111 lr:0.000319 t:9.6s +tttg: c70/111 lr:0.000305 t:9.7s +tttg: c71/111 lr:0.000292 t:9.8s +tttg: c72/111 lr:0.000279 t:10.0s +tttg: c73/111 lr:0.000267 t:10.1s +tttg: c74/111 lr:0.000254 t:10.2s +tttg: c75/111 lr:0.000242 t:10.3s +tttg: c76/111 lr:0.000230 t:10.4s +tttg: c77/111 lr:0.000218 t:10.5s +tttg: c78/111 lr:0.000206 t:10.6s +tttg: c79/111 lr:0.000195 t:10.7s +tttg: c80/111 lr:0.000183 t:10.9s +tttg: c81/111 lr:0.000173 t:11.0s +tttg: c82/111 lr:0.000162 t:11.1s +tttg: c83/111 lr:0.000152 t:11.2s +tttg: c84/111 lr:0.000141 t:11.3s +tttg: c85/111 lr:0.000132 t:11.5s +tttg: c86/111 lr:0.000122 t:11.6s +tttg: c87/111 lr:0.000113 t:11.7s +tttg: c88/111 lr:0.000104 t:11.8s +tttg: c89/111 lr:0.000095 t:11.9s +tttg: c90/111 lr:0.000087 t:12.0s +tttg: c91/111 lr:0.000079 t:12.2s +tttg: c92/111 lr:0.000072 t:12.3s +tttg: c93/111 lr:0.000065 t:12.4s +tttg: c94/111 lr:0.000058 t:12.5s +tttg: c95/111 lr:0.000051 t:12.6s +tttg: c96/111 lr:0.000045 t:12.7s +tttg: c97/111 lr:0.000039 t:12.9s +tttg: c98/111 lr:0.000034 t:13.0s +tttg: c99/111 lr:0.000029 t:13.1s +tttg: c100/111 lr:0.000024 t:13.2s +tttg: c101/111 lr:0.000020 t:13.3s +tttg: c102/111 lr:0.000016 t:13.4s +tttg: c103/111 lr:0.000013 t:13.6s +tttg: c104/111 lr:0.000010 t:13.7s +tttg: c105/111 lr:0.000007 t:13.8s +tttg: c106/111 lr:0.000005 t:13.9s +tttg: c107/111 lr:0.000003 t:14.0s +tttg: c108/111 lr:0.000002 t:14.1s +tttg: c109/111 lr:0.000001 t:14.3s +tttg: c110/111 lr:0.000000 t:14.4s +ttpr: phase:1/3 t:253.8s +ttp: b757/782 bl:2.2901 bb:0.9797 rl:2.2989 rb:0.9879 dl:3550-3633 gd:0 +ttpp: phase:2/3 pd:1808 gd:1333 t:370.6s +tttg: c1/185 lr:0.001000 t:0.1s +tttg: c2/185 lr:0.001000 t:0.2s +tttg: c3/185 lr:0.001000 t:0.3s +tttg: c4/185 lr:0.000999 t:0.4s +tttg: c5/185 lr:0.000999 t:0.5s +tttg: c6/185 lr:0.000998 t:0.6s +tttg: c7/185 lr:0.000997 t:0.7s +tttg: c8/185 lr:0.000996 t:0.8s +tttg: c9/185 lr:0.000995 t:0.9s +tttg: c10/185 lr:0.000994 t:1.0s +tttg: c11/185 lr:0.000993 t:1.2s +tttg: c12/185 lr:0.000991 t:1.3s +tttg: c13/185 lr:0.000990 t:1.4s +tttg: c14/185 lr:0.000988 t:1.5s +tttg: c15/185 lr:0.000986 t:1.6s +tttg: c16/185 lr:0.000984 t:1.8s +tttg: c17/185 lr:0.000981 t:1.9s +tttg: c18/185 lr:0.000979 t:2.0s +tttg: c19/185 lr:0.000977 t:2.1s +tttg: c20/185 lr:0.000974 t:2.3s +tttg: c21/185 lr:0.000971 t:2.4s +tttg: c22/185 lr:0.000968 t:2.5s +tttg: c23/185 lr:0.000965 t:2.6s +tttg: c24/185 lr:0.000962 t:2.7s +tttg: c25/185 lr:0.000959 t:2.8s +tttg: c26/185 lr:0.000955 t:3.0s +tttg: c27/185 lr:0.000952 t:3.1s +tttg: c28/185 lr:0.000948 t:3.2s +tttg: c29/185 lr:0.000944 t:3.3s +tttg: c30/185 lr:0.000940 t:3.5s +tttg: c31/185 lr:0.000936 t:3.6s +tttg: c32/185 lr:0.000932 t:3.7s +tttg: c33/185 lr:0.000927 t:3.8s +tttg: c34/185 lr:0.000923 t:3.9s +tttg: c35/185 lr:0.000918 t:4.1s +tttg: c36/185 lr:0.000913 t:4.2s +tttg: c37/185 lr:0.000908 t:4.3s +tttg: c38/185 lr:0.000904 t:4.4s +tttg: c39/185 lr:0.000898 t:4.5s +tttg: c40/185 lr:0.000893 t:4.7s +tttg: c41/185 lr:0.000888 t:4.8s +tttg: c42/185 lr:0.000882 t:4.9s +tttg: c43/185 lr:0.000877 t:5.0s +tttg: c44/185 lr:0.000871 t:5.1s +tttg: c45/185 lr:0.000865 t:5.3s +tttg: c46/185 lr:0.000860 t:5.4s +tttg: c47/185 lr:0.000854 t:5.5s +tttg: c48/185 lr:0.000847 t:5.6s +tttg: c49/185 lr:0.000841 t:5.7s +tttg: c50/185 lr:0.000835 t:5.9s +tttg: c51/185 lr:0.000829 t:6.0s +tttg: c52/185 lr:0.000822 t:6.1s +tttg: c53/185 lr:0.000816 t:6.2s +tttg: c54/185 lr:0.000809 t:6.3s +tttg: c55/185 lr:0.000802 t:6.5s +tttg: c56/185 lr:0.000795 t:6.6s +tttg: c57/185 lr:0.000788 t:6.7s +tttg: c58/185 lr:0.000781 t:6.8s +tttg: c59/185 lr:0.000774 t:6.9s +tttg: c60/185 lr:0.000767 t:7.1s +tttg: c61/185 lr:0.000760 t:7.2s +tttg: c62/185 lr:0.000752 t:7.3s +tttg: c63/185 lr:0.000745 t:7.4s +tttg: c64/185 lr:0.000738 t:7.5s +tttg: c65/185 lr:0.000730 t:7.6s +tttg: c66/185 lr:0.000722 t:7.8s +tttg: c67/185 lr:0.000715 t:7.9s +tttg: c68/185 lr:0.000707 t:8.0s +tttg: c69/185 lr:0.000699 t:8.1s +tttg: c70/185 lr:0.000691 t:8.2s +tttg: c71/185 lr:0.000683 t:8.3s +tttg: c72/185 lr:0.000675 t:8.5s +tttg: c73/185 lr:0.000667 t:8.6s +tttg: c74/185 lr:0.000659 t:8.7s +tttg: c75/185 lr:0.000651 t:8.8s +tttg: c76/185 lr:0.000643 t:9.0s +tttg: c77/185 lr:0.000635 t:9.1s +tttg: c78/185 lr:0.000627 t:9.2s +tttg: c79/185 lr:0.000618 t:9.3s +tttg: c80/185 lr:0.000610 t:9.4s +tttg: c81/185 lr:0.000602 t:9.6s +tttg: c82/185 lr:0.000593 t:9.7s +tttg: c83/185 lr:0.000585 t:9.8s +tttg: c84/185 lr:0.000577 t:9.9s +tttg: c85/185 lr:0.000568 t:10.0s +tttg: c86/185 lr:0.000560 t:10.2s +tttg: c87/185 lr:0.000551 t:10.3s +tttg: c88/185 lr:0.000543 t:10.4s +tttg: c89/185 lr:0.000534 t:10.5s +tttg: c90/185 lr:0.000526 t:10.6s +tttg: c91/185 lr:0.000517 t:10.7s +tttg: c92/185 lr:0.000509 t:10.9s +tttg: c93/185 lr:0.000500 t:11.0s +tttg: c94/185 lr:0.000491 t:11.1s +tttg: c95/185 lr:0.000483 t:11.2s +tttg: c96/185 lr:0.000474 t:11.3s +tttg: c97/185 lr:0.000466 t:11.5s +tttg: c98/185 lr:0.000457 t:11.6s +tttg: c99/185 lr:0.000449 t:11.7s +tttg: c100/185 lr:0.000440 t:11.8s +tttg: c101/185 lr:0.000432 t:11.9s +tttg: c102/185 lr:0.000423 t:12.1s +tttg: c103/185 lr:0.000415 t:12.2s +tttg: c104/185 lr:0.000407 t:12.3s +tttg: c105/185 lr:0.000398 t:12.4s +tttg: c106/185 lr:0.000390 t:12.5s +tttg: c107/185 lr:0.000382 t:12.7s +tttg: c108/185 lr:0.000373 t:12.8s +tttg: c109/185 lr:0.000365 t:12.9s +tttg: c110/185 lr:0.000357 t:13.0s +tttg: c111/185 lr:0.000349 t:13.1s +tttg: c112/185 lr:0.000341 t:13.2s +tttg: c113/185 lr:0.000333 t:13.4s +tttg: c114/185 lr:0.000325 t:13.5s +tttg: c115/185 lr:0.000317 t:13.6s +tttg: c116/185 lr:0.000309 t:13.7s +tttg: c117/185 lr:0.000301 t:13.8s +tttg: c118/185 lr:0.000293 t:13.9s +tttg: c119/185 lr:0.000285 t:14.1s +tttg: c120/185 lr:0.000278 t:14.2s +tttg: c121/185 lr:0.000270 t:14.3s +tttg: c122/185 lr:0.000262 t:14.4s +tttg: c123/185 lr:0.000255 t:14.5s +tttg: c124/185 lr:0.000248 t:14.7s +tttg: c125/185 lr:0.000240 t:14.8s +tttg: c126/185 lr:0.000233 t:14.9s +tttg: c127/185 lr:0.000226 t:15.0s +tttg: c128/185 lr:0.000219 t:15.2s +tttg: c129/185 lr:0.000212 t:15.3s +tttg: c130/185 lr:0.000205 t:15.4s +tttg: c131/185 lr:0.000198 t:15.5s +tttg: c132/185 lr:0.000191 t:15.7s +tttg: c133/185 lr:0.000184 t:15.8s +tttg: c134/185 lr:0.000178 t:15.9s +tttg: c135/185 lr:0.000171 t:16.0s +tttg: c136/185 lr:0.000165 t:16.2s +tttg: c137/185 lr:0.000159 t:16.3s +tttg: c138/185 lr:0.000153 t:16.4s +tttg: c139/185 lr:0.000146 t:16.5s +tttg: c140/185 lr:0.000140 t:16.6s +tttg: c141/185 lr:0.000135 t:16.8s +tttg: c142/185 lr:0.000129 t:16.9s +tttg: c143/185 lr:0.000123 t:17.0s +tttg: c144/185 lr:0.000118 t:17.1s +tttg: c145/185 lr:0.000112 t:17.2s +tttg: c146/185 lr:0.000107 t:17.4s +tttg: c147/185 lr:0.000102 t:17.5s +tttg: c148/185 lr:0.000096 t:17.6s +tttg: c149/185 lr:0.000092 t:17.7s +tttg: c150/185 lr:0.000087 t:17.8s +tttg: c151/185 lr:0.000082 t:17.9s +tttg: c152/185 lr:0.000077 t:18.1s +tttg: c153/185 lr:0.000073 t:18.2s +tttg: c154/185 lr:0.000068 t:18.3s +tttg: c155/185 lr:0.000064 t:18.4s +tttg: c156/185 lr:0.000060 t:18.5s +tttg: c157/185 lr:0.000056 t:18.7s +tttg: c158/185 lr:0.000052 t:18.8s +tttg: c159/185 lr:0.000048 t:18.9s +tttg: c160/185 lr:0.000045 t:19.0s +tttg: c161/185 lr:0.000041 t:19.1s +tttg: c162/185 lr:0.000038 t:19.3s +tttg: c163/185 lr:0.000035 t:19.4s +tttg: c164/185 lr:0.000032 t:19.5s +tttg: c165/185 lr:0.000029 t:19.6s +tttg: c166/185 lr:0.000026 t:19.7s +tttg: c167/185 lr:0.000023 t:19.8s +tttg: c168/185 lr:0.000021 t:20.0s +tttg: c169/185 lr:0.000019 t:20.1s +tttg: c170/185 lr:0.000016 t:20.2s +tttg: c171/185 lr:0.000014 t:20.3s +tttg: c172/185 lr:0.000012 t:20.4s +tttg: c173/185 lr:0.000010 t:20.5s +tttg: c174/185 lr:0.000009 t:20.7s +tttg: c175/185 lr:0.000007 t:20.8s +tttg: c176/185 lr:0.000006 t:20.9s +tttg: c177/185 lr:0.000005 t:21.0s +tttg: c178/185 lr:0.000004 t:21.1s +tttg: c179/185 lr:0.000003 t:21.2s +tttg: c180/185 lr:0.000002 t:21.3s +tttg: c181/185 lr:0.000001 t:21.5s +tttg: c182/185 lr:0.000001 t:21.6s +tttg: c183/185 lr:0.000000 t:21.7s +tttg: c184/185 lr:0.000000 t:21.8s +ttpr: phase:2/3 t:395.5s +ttp: b746/782 bl:2.4206 bb:0.9894 rl:2.3134 rb:0.9881 dl:2884-2943 gd:0 +ttp: b745/782 bl:2.2452 bb:0.9418 rl:2.3063 rb:0.9831 dl:2842-2883 gd:0 +ttpp: phase:3/3 pd:2448 gd:2000 t:415.6s +tttg: c1/250 lr:0.001000 t:0.1s +tttg: c2/250 lr:0.001000 t:0.2s +tttg: c3/250 lr:0.001000 t:0.3s +tttg: c4/250 lr:0.001000 t:0.4s +tttg: c5/250 lr:0.000999 t:0.5s +tttg: c6/250 lr:0.000999 t:0.6s +tttg: c7/250 lr:0.000999 t:0.7s +tttg: c8/250 lr:0.000998 t:0.8s +tttg: c9/250 lr:0.000997 t:1.0s +tttg: c10/250 lr:0.000997 t:1.1s +tttg: c11/250 lr:0.000996 t:1.2s +tttg: c12/250 lr:0.000995 t:1.3s +tttg: c13/250 lr:0.000994 t:1.4s +tttg: c14/250 lr:0.000993 t:1.5s +tttg: c15/250 lr:0.000992 t:1.7s +tttg: c16/250 lr:0.000991 t:1.8s +tttg: c17/250 lr:0.000990 t:1.9s +tttg: c18/250 lr:0.000989 t:2.0s +tttg: c19/250 lr:0.000987 t:2.1s +tttg: c20/250 lr:0.000986 t:2.3s +tttg: c21/250 lr:0.000984 t:2.4s +tttg: c22/250 lr:0.000983 t:2.5s +tttg: c23/250 lr:0.000981 t:2.6s +tttg: c24/250 lr:0.000979 t:2.7s +tttg: c25/250 lr:0.000977 t:2.8s +tttg: c26/250 lr:0.000975 t:3.0s +tttg: c27/250 lr:0.000973 t:3.1s +tttg: c28/250 lr:0.000971 t:3.2s +tttg: c29/250 lr:0.000969 t:3.3s +tttg: c30/250 lr:0.000967 t:3.4s +tttg: c31/250 lr:0.000965 t:3.6s +tttg: c32/250 lr:0.000962 t:3.7s +tttg: c33/250 lr:0.000960 t:3.8s +tttg: c34/250 lr:0.000957 t:3.9s +tttg: c35/250 lr:0.000955 t:4.0s +tttg: c36/250 lr:0.000952 t:4.1s +tttg: c37/250 lr:0.000949 t:4.2s +tttg: c38/250 lr:0.000947 t:4.4s +tttg: c39/250 lr:0.000944 t:4.5s +tttg: c40/250 lr:0.000941 t:4.6s +tttg: c41/250 lr:0.000938 t:4.7s +tttg: c42/250 lr:0.000935 t:4.8s +tttg: c43/250 lr:0.000931 t:4.9s +tttg: c44/250 lr:0.000928 t:5.0s +tttg: c45/250 lr:0.000925 t:5.2s +tttg: c46/250 lr:0.000922 t:5.3s +tttg: c47/250 lr:0.000918 t:5.4s +tttg: c48/250 lr:0.000915 t:5.5s +tttg: c49/250 lr:0.000911 t:5.6s +tttg: c50/250 lr:0.000907 t:5.7s +tttg: c51/250 lr:0.000904 t:5.9s +tttg: c52/250 lr:0.000900 t:6.0s +tttg: c53/250 lr:0.000896 t:6.1s +tttg: c54/250 lr:0.000892 t:6.2s +tttg: c55/250 lr:0.000888 t:6.3s +tttg: c56/250 lr:0.000884 t:6.4s +tttg: c57/250 lr:0.000880 t:6.5s +tttg: c58/250 lr:0.000876 t:6.7s +tttg: c59/250 lr:0.000872 t:6.8s +tttg: c60/250 lr:0.000868 t:6.9s +tttg: c61/250 lr:0.000863 t:7.0s +tttg: c62/250 lr:0.000859 t:7.1s +tttg: c63/250 lr:0.000855 t:7.3s +tttg: c64/250 lr:0.000850 t:7.4s +tttg: c65/250 lr:0.000846 t:7.5s +tttg: c66/250 lr:0.000841 t:7.6s +tttg: c67/250 lr:0.000836 t:7.7s +tttg: c68/250 lr:0.000832 t:7.8s +tttg: c69/250 lr:0.000827 t:8.0s +tttg: c70/250 lr:0.000822 t:8.1s +tttg: c71/250 lr:0.000817 t:8.2s +tttg: c72/250 lr:0.000812 t:8.3s +tttg: c73/250 lr:0.000807 t:8.4s +tttg: c74/250 lr:0.000803 t:8.5s +tttg: c75/250 lr:0.000797 t:8.7s +tttg: c76/250 lr:0.000792 t:8.8s +tttg: c77/250 lr:0.000787 t:8.9s +tttg: c78/250 lr:0.000782 t:9.0s +tttg: c79/250 lr:0.000777 t:9.1s +tttg: c80/250 lr:0.000772 t:9.2s +tttg: c81/250 lr:0.000766 t:9.4s +tttg: c82/250 lr:0.000761 t:9.5s +tttg: c83/250 lr:0.000755 t:9.6s +tttg: c84/250 lr:0.000750 t:9.7s +tttg: c85/250 lr:0.000745 t:9.8s +tttg: c86/250 lr:0.000739 t:9.9s +tttg: c87/250 lr:0.000733 t:10.1s +tttg: c88/250 lr:0.000728 t:10.2s +tttg: c89/250 lr:0.000722 t:10.3s +tttg: c90/250 lr:0.000717 t:10.4s +tttg: c91/250 lr:0.000711 t:10.5s +tttg: c92/250 lr:0.000705 t:10.6s +tttg: c93/250 lr:0.000699 t:10.7s +tttg: c94/250 lr:0.000694 t:10.9s +tttg: c95/250 lr:0.000688 t:11.0s +tttg: c96/250 lr:0.000682 t:11.1s +tttg: c97/250 lr:0.000676 t:11.2s +tttg: c98/250 lr:0.000670 t:11.3s +tttg: c99/250 lr:0.000664 t:11.4s +tttg: c100/250 lr:0.000658 t:11.6s +tttg: c101/250 lr:0.000652 t:11.7s +tttg: c102/250 lr:0.000646 t:11.8s +tttg: c103/250 lr:0.000640 t:11.9s +tttg: c104/250 lr:0.000634 t:12.0s +tttg: c105/250 lr:0.000628 t:12.1s +tttg: c106/250 lr:0.000622 t:12.2s +tttg: c107/250 lr:0.000616 t:12.3s +tttg: c108/250 lr:0.000610 t:12.5s +tttg: c109/250 lr:0.000603 t:12.6s +tttg: c110/250 lr:0.000597 t:12.7s +tttg: c111/250 lr:0.000591 t:12.8s +tttg: c112/250 lr:0.000585 t:12.9s +tttg: c113/250 lr:0.000579 t:13.0s +tttg: c114/250 lr:0.000572 t:13.2s +tttg: c115/250 lr:0.000566 t:13.3s +tttg: c116/250 lr:0.000560 t:13.4s +tttg: c117/250 lr:0.000554 t:13.5s +tttg: c118/250 lr:0.000547 t:13.6s +tttg: c119/250 lr:0.000541 t:13.7s +tttg: c120/250 lr:0.000535 t:13.8s +tttg: c121/250 lr:0.000528 t:13.9s +tttg: c122/250 lr:0.000522 t:14.0s +tttg: c123/250 lr:0.000516 t:14.1s +tttg: c124/250 lr:0.000509 t:14.2s +tttg: c125/250 lr:0.000503 t:14.3s +tttg: c126/250 lr:0.000497 t:14.3s +tttg: c127/250 lr:0.000491 t:14.5s +tttg: c128/250 lr:0.000484 t:14.6s +tttg: c129/250 lr:0.000478 t:14.6s +tttg: c130/250 lr:0.000472 t:14.8s +tttg: c131/250 lr:0.000465 t:14.9s +tttg: c132/250 lr:0.000459 t:14.9s +tttg: c133/250 lr:0.000453 t:15.1s +tttg: c134/250 lr:0.000446 t:15.2s +tttg: c135/250 lr:0.000440 t:15.3s +tttg: c136/250 lr:0.000434 t:15.4s +tttg: c137/250 lr:0.000428 t:15.4s +tttg: c138/250 lr:0.000421 t:15.6s +tttg: c139/250 lr:0.000415 t:15.7s +tttg: c140/250 lr:0.000409 t:15.8s +tttg: c141/250 lr:0.000403 t:15.8s +tttg: c142/250 lr:0.000397 t:15.9s +tttg: c143/250 lr:0.000390 t:16.1s +tttg: c144/250 lr:0.000384 t:16.2s +tttg: c145/250 lr:0.000378 t:16.3s +tttg: c146/250 lr:0.000372 t:16.4s +tttg: c147/250 lr:0.000366 t:16.5s +tttg: c148/250 lr:0.000360 t:16.5s +tttg: c149/250 lr:0.000354 t:16.7s +tttg: c150/250 lr:0.000348 t:16.8s +tttg: c151/250 lr:0.000342 t:16.9s +tttg: c152/250 lr:0.000336 t:17.0s +tttg: c153/250 lr:0.000330 t:17.2s +tttg: c154/250 lr:0.000324 t:17.3s +tttg: c155/250 lr:0.000318 t:17.3s +tttg: c156/250 lr:0.000312 t:17.5s +tttg: c157/250 lr:0.000306 t:17.5s +tttg: c158/250 lr:0.000301 t:17.7s +tttg: c159/250 lr:0.000295 t:17.8s +tttg: c160/250 lr:0.000289 t:17.9s +tttg: c161/250 lr:0.000283 t:18.0s +tttg: c162/250 lr:0.000278 t:18.1s +tttg: c163/250 lr:0.000272 t:18.2s +tttg: c164/250 lr:0.000267 t:18.2s +tttg: c165/250 lr:0.000261 t:18.4s +tttg: c166/250 lr:0.000255 t:18.5s +tttg: c167/250 lr:0.000250 t:18.6s +tttg: c168/250 lr:0.000245 t:18.7s +tttg: c169/250 lr:0.000239 t:18.8s +tttg: c170/250 lr:0.000234 t:18.9s +tttg: c171/250 lr:0.000228 t:19.0s +tttg: c172/250 lr:0.000223 t:19.1s +tttg: c173/250 lr:0.000218 t:19.2s +tttg: c174/250 lr:0.000213 t:19.2s +tttg: c175/250 lr:0.000208 t:19.4s +tttg: c176/250 lr:0.000203 t:19.5s +tttg: c177/250 lr:0.000197 t:19.5s +tttg: c178/250 lr:0.000193 t:19.7s +tttg: c179/250 lr:0.000188 t:19.7s +tttg: c180/250 lr:0.000183 t:19.9s +tttg: c181/250 lr:0.000178 t:20.0s +tttg: c182/250 lr:0.000173 t:20.1s +tttg: c183/250 lr:0.000168 t:20.2s +tttg: c184/250 lr:0.000164 t:20.3s +tttg: c185/250 lr:0.000159 t:20.4s +tttg: c186/250 lr:0.000154 t:20.5s +tttg: c187/250 lr:0.000150 t:20.6s +tttg: c188/250 lr:0.000145 t:20.7s +tttg: c189/250 lr:0.000141 t:20.8s +tttg: c190/250 lr:0.000137 t:20.9s +tttg: c191/250 lr:0.000132 t:21.0s +tttg: c192/250 lr:0.000128 t:21.1s +tttg: c193/250 lr:0.000124 t:21.2s +tttg: c194/250 lr:0.000120 t:21.3s +tttg: c195/250 lr:0.000116 t:21.4s +tttg: c196/250 lr:0.000112 t:21.5s +tttg: c197/250 lr:0.000108 t:21.6s +tttg: c198/250 lr:0.000104 t:21.7s +tttg: c199/250 lr:0.000100 t:21.8s +tttg: c200/250 lr:0.000096 t:21.9s +tttg: c201/250 lr:0.000093 t:22.0s +tttg: c202/250 lr:0.000089 t:22.1s +tttg: c203/250 lr:0.000085 t:22.2s +tttg: c204/250 lr:0.000082 t:22.3s +tttg: c205/250 lr:0.000078 t:22.4s +tttg: c206/250 lr:0.000075 t:22.5s +tttg: c207/250 lr:0.000072 t:22.6s +tttg: c208/250 lr:0.000069 t:22.7s +tttg: c209/250 lr:0.000065 t:22.7s +tttg: c210/250 lr:0.000062 t:22.9s +tttg: c211/250 lr:0.000059 t:22.9s +tttg: c212/250 lr:0.000056 t:23.0s +tttg: c213/250 lr:0.000053 t:23.2s +tttg: c214/250 lr:0.000051 t:23.3s +tttg: c215/250 lr:0.000048 t:23.4s +tttg: c216/250 lr:0.000045 t:23.4s +tttg: c217/250 lr:0.000043 t:23.6s +tttg: c218/250 lr:0.000040 t:23.7s +tttg: c219/250 lr:0.000038 t:23.7s +tttg: c220/250 lr:0.000035 t:23.9s +tttg: c221/250 lr:0.000033 t:24.0s +tttg: c222/250 lr:0.000031 t:24.1s +tttg: c223/250 lr:0.000029 t:24.2s +tttg: c224/250 lr:0.000027 t:24.3s +tttg: c225/250 lr:0.000025 t:24.4s +tttg: c226/250 lr:0.000023 t:24.4s +tttg: c227/250 lr:0.000021 t:24.6s +tttg: c228/250 lr:0.000019 t:24.7s +tttg: c229/250 lr:0.000017 t:24.8s +tttg: c230/250 lr:0.000016 t:24.9s +tttg: c231/250 lr:0.000014 t:25.0s +tttg: c232/250 lr:0.000013 t:25.1s +tttg: c233/250 lr:0.000011 t:25.2s +tttg: c234/250 lr:0.000010 t:25.3s +tttg: c235/250 lr:0.000009 t:25.4s +tttg: c236/250 lr:0.000008 t:25.5s +tttg: c237/250 lr:0.000007 t:25.6s +tttg: c238/250 lr:0.000006 t:25.6s +tttg: c239/250 lr:0.000005 t:25.8s +tttg: c240/250 lr:0.000004 t:25.9s +tttg: c241/250 lr:0.000003 t:26.0s +tttg: c242/250 lr:0.000003 t:26.0s +tttg: c243/250 lr:0.000002 t:26.2s +tttg: c244/250 lr:0.000001 t:26.3s +tttg: c245/250 lr:0.000001 t:26.3s +tttg: c246/250 lr:0.000001 t:26.5s +tttg: c247/250 lr:0.000000 t:26.6s +tttg: c248/250 lr:0.000000 t:26.7s +tttg: c249/250 lr:0.000000 t:26.7s +ttpr: phase:3/3 t:444.4s +ttp: b736/782 bl:2.2560 bb:0.9731 rl:2.3020 rb:0.9823 dl:2526-2550 gd:1 +ttp: b735/782 bl:2.3974 bb:1.0110 rl:2.3094 rb:0.9845 dl:2495-2526 gd:1 +ttp: b722/782 bl:2.3565 bb:0.9764 rl:2.3124 rb:0.9840 dl:2163-2185 gd:1 +ttp: b717/782 bl:2.2644 bb:0.9481 rl:2.3096 rb:0.9819 dl:2070-2088 gd:1 +ttp: b706/782 bl:2.4115 bb:0.9968 rl:2.3147 rb:0.9827 dl:1898-1910 gd:1 +ttp: b703/782 bl:2.3495 bb:0.9527 rl:2.3163 rb:0.9812 dl:1859-1872 gd:1 +ttp: b693/782 bl:2.3477 bb:0.9702 rl:2.3176 rb:0.9808 dl:1746-1757 gd:1 +ttp: b685/782 bl:2.3073 bb:0.9509 rl:2.3172 rb:0.9796 dl:1665-1675 gd:1 +ttp: b680/782 bl:2.2914 bb:0.9467 rl:2.3163 rb:0.9784 dl:1618-1628 gd:1 +ttp: b671/782 bl:2.3165 bb:0.9657 rl:2.3163 rb:0.9780 dl:1544-1552 gd:1 +ttp: b658/782 bl:2.2667 bb:0.9446 rl:2.3148 rb:0.9770 dl:1452-1459 gd:1 +ttp: b651/782 bl:2.3997 bb:0.9736 rl:2.3172 rb:0.9769 dl:1406-1411 gd:1 +ttp: b644/782 bl:2.3734 bb:0.9740 rl:2.3187 rb:0.9768 dl:1362-1367 gd:1 +ttp: b634/782 bl:2.3920 bb:0.9765 rl:2.3205 rb:0.9768 dl:1302-1308 gd:1 +ttp: b626/782 bl:2.3205 bb:0.9504 rl:2.3205 rb:0.9761 dl:1260-1265 gd:1 +ttp: b618/782 bl:2.4179 bb:0.9926 rl:2.3227 rb:0.9765 dl:1216-1221 gd:1 +ttp: b610/782 bl:2.2646 bb:0.9311 rl:2.3215 rb:0.9755 dl:1177-1182 gd:1 +ttp: b602/782 bl:2.3852 bb:0.9724 rl:2.3227 rb:0.9755 dl:1141-1146 gd:1 +ttp: b598/782 bl:2.3681 bb:0.9813 rl:2.3236 rb:0.9756 dl:1124-1129 gd:1 +ttp: b588/782 bl:2.3324 bb:0.9666 rl:2.3238 rb:0.9754 dl:1081-1086 gd:1 +ttp: b579/782 bl:2.3571 bb:0.9594 rl:2.3244 rb:0.9751 dl:1044-1048 gd:1 +ttp: b572/782 bl:2.3221 bb:0.9566 rl:2.3243 rb:0.9748 dl:1017-1021 gd:1 +ttp: b564/782 bl:2.2979 bb:0.9415 rl:2.3239 rb:0.9743 dl:990-993 gd:1 +ttp: b558/782 bl:2.3843 bb:0.9798 rl:2.3248 rb:0.9744 dl:968-972 gd:1 +ttp: b550/782 bl:2.3775 bb:0.9810 rl:2.3256 rb:0.9745 dl:943-946 gd:1 +ttp: b542/782 bl:2.3372 bb:0.9643 rl:2.3258 rb:0.9743 dl:918-921 gd:1 +ttp: b534/782 bl:2.3330 bb:0.9617 rl:2.3259 rb:0.9741 dl:893-896 gd:1 +ttp: b526/782 bl:2.3309 bb:0.9493 rl:2.3259 rb:0.9738 dl:869-872 gd:1 +ttp: b518/782 bl:2.2566 bb:0.9331 rl:2.3251 rb:0.9733 dl:846-850 gd:1 +ttp: b510/782 bl:2.3967 bb:0.9939 rl:2.3259 rb:0.9735 dl:823-826 gd:1 +ttp: b502/782 bl:2.3266 bb:0.9506 rl:2.3259 rb:0.9733 dl:802-804 gd:1 +ttp: b494/782 bl:2.3293 bb:0.9729 rl:2.3260 rb:0.9733 dl:780-783 gd:1 +ttp: b486/782 bl:2.4188 bb:1.0015 rl:2.3270 rb:0.9736 dl:761-764 gd:1 +ttp: b478/782 bl:2.3504 bb:0.9893 rl:2.3272 rb:0.9737 dl:742-744 gd:1 +ttp: b470/782 bl:2.3627 bb:0.9758 rl:2.3276 rb:0.9738 dl:724-726 gd:1 +ttp: b462/782 bl:2.3440 bb:0.9598 rl:2.3277 rb:0.9736 dl:706-708 gd:1 +ttp: b454/782 bl:2.4030 bb:1.0032 rl:2.3284 rb:0.9739 dl:689-691 gd:1 +ttp: b446/782 bl:2.3076 bb:0.9877 rl:2.3282 rb:0.9740 dl:672-674 gd:1 +ttp: b438/782 bl:2.3175 bb:0.9670 rl:2.3281 rb:0.9740 dl:655-657 gd:1 +ttp: b430/782 bl:2.3983 bb:0.9693 rl:2.3287 rb:0.9739 dl:640-642 gd:1 +ttp: b422/782 bl:2.3125 bb:0.9897 rl:2.3286 rb:0.9740 dl:624-626 gd:1 +ttp: b414/782 bl:2.2170 bb:0.9268 rl:2.3277 rb:0.9737 dl:609-611 gd:1 +ttp: b406/782 bl:2.3243 bb:0.9795 rl:2.3277 rb:0.9737 dl:593-595 gd:1 +ttp: b398/782 bl:2.2579 bb:0.9240 rl:2.3272 rb:0.9733 dl:579-581 gd:1 +ttp: b385/782 bl:2.4214 bb:0.9959 rl:2.3278 rb:0.9735 dl:555-557 gd:1 +ttp: b376/782 bl:2.3304 bb:0.9616 rl:2.3278 rb:0.9734 dl:540-542 gd:1 +ttp: b368/782 bl:2.3767 bb:1.0092 rl:2.3282 rb:0.9737 dl:527-528 gd:1 +ttp: b365/782 bl:2.3466 bb:0.9548 rl:2.3283 rb:0.9735 dl:522-524 gd:1 +ttp: b351/782 bl:2.3673 bb:0.9911 rl:2.3285 rb:0.9736 dl:498-499 gd:1 +ttp: b343/782 bl:2.2273 bb:0.9527 rl:2.3279 rb:0.9735 dl:486-488 gd:1 +ttp: b335/782 bl:2.3766 bb:0.9835 rl:2.3282 rb:0.9736 dl:474-476 gd:1 +ttp: b329/782 bl:2.2984 bb:0.9877 rl:2.3280 rb:0.9736 dl:465-466 gd:1 +ttp: b319/782 bl:2.4093 bb:0.9971 rl:2.3285 rb:0.9738 dl:450-451 gd:1 +ttp: b311/782 bl:2.3549 bb:0.9874 rl:2.3286 rb:0.9738 dl:438-439 gd:1 +ttp: b303/782 bl:2.4079 bb:1.0025 rl:2.3290 rb:0.9740 dl:426-427 gd:1 +ttp: b295/782 bl:2.2797 bb:0.9692 rl:2.3288 rb:0.9740 dl:414-415 gd:1 +ttp: b287/782 bl:2.4158 bb:1.0039 rl:2.3292 rb:0.9741 dl:402-403 gd:1 +ttp: b279/782 bl:2.3260 bb:0.9994 rl:2.3292 rb:0.9742 dl:391-392 gd:1 +ttp: b271/782 bl:2.3867 bb:1.0280 rl:2.3294 rb:0.9745 dl:380-382 gd:1 +ttp: b263/782 bl:2.4015 bb:0.9923 rl:2.3297 rb:0.9745 dl:370-371 gd:1 +ttp: b255/782 bl:2.3834 bb:1.0033 rl:2.3300 rb:0.9747 dl:360-361 gd:1 +ttp: b247/782 bl:2.3570 bb:0.9978 rl:2.3301 rb:0.9748 dl:350-351 gd:1 +ttp: b239/782 bl:2.3902 bb:1.0060 rl:2.3303 rb:0.9749 dl:340-341 gd:1 +ttp: b231/782 bl:2.3111 bb:0.9878 rl:2.3302 rb:0.9749 dl:330-331 gd:1 +ttp: b223/782 bl:2.3483 bb:1.0237 rl:2.3303 rb:0.9751 dl:321-322 gd:1 +ttp: b215/782 bl:2.4018 bb:1.0037 rl:2.3306 rb:0.9752 dl:312-313 gd:1 +ttp: b207/782 bl:2.3632 bb:1.0263 rl:2.3307 rb:0.9754 dl:303-304 gd:1 +ttp: b198/782 bl:2.4071 bb:0.9762 rl:2.3309 rb:0.9754 dl:294-295 gd:1 +ttp: b186/782 bl:2.4343 bb:1.0310 rl:2.3313 rb:0.9755 dl:280-281 gd:1 +ttp: b176/782 bl:2.3251 bb:1.0154 rl:2.3312 rb:0.9757 dl:270-271 gd:1 +ttp: b167/782 bl:2.5359 bb:1.0403 rl:2.3318 rb:0.9758 dl:262-263 gd:1 +ttp: b159/782 bl:2.4886 bb:1.0503 rl:2.3323 rb:0.9761 dl:254-255 gd:1 +ttp: b152/782 bl:2.3943 bb:1.0373 rl:2.3325 rb:0.9762 dl:247-248 gd:1 +ttp: b144/782 bl:2.3636 bb:1.0124 rl:2.3325 rb:0.9763 dl:239-240 gd:1 +ttp: b137/782 bl:2.4244 bb:1.0435 rl:2.3328 rb:0.9765 dl:233-233 gd:1 +ttp: b124/782 bl:2.3963 bb:1.0488 rl:2.3329 rb:0.9767 dl:220-222 gd:1 +ttp: b113/782 bl:2.5716 bb:1.0542 rl:2.3335 rb:0.9768 dl:210-211 gd:1 +ttp: b105/782 bl:2.4259 bb:1.0431 rl:2.3337 rb:0.9770 dl:203-204 gd:1 +ttp: b97/782 bl:2.4704 bb:1.0604 rl:2.3340 rb:0.9772 dl:196-197 gd:1 +ttp: b90/782 bl:2.4879 bb:1.0968 rl:2.3343 rb:0.9774 dl:190-190 gd:1 +ttp: b82/782 bl:2.5114 bb:1.0732 rl:2.3347 rb:0.9776 dl:183-183 gd:1 +ttp: b73/782 bl:2.5507 bb:1.1248 rl:2.3351 rb:0.9778 dl:174-175 gd:1 +ttp: b63/782 bl:2.5411 bb:1.0923 rl:2.3354 rb:0.9780 dl:166-166 gd:1 +ttp: b53/782 bl:2.5268 bb:1.0894 rl:2.3358 rb:0.9782 dl:156-157 gd:1 +ttp: b43/782 bl:2.5171 bb:1.1079 rl:2.3360 rb:0.9784 dl:146-147 gd:1 +ttp: b34/782 bl:2.6331 bb:1.1009 rl:2.3365 rb:0.9786 dl:137-138 gd:1 +ttp: b26/782 bl:2.6020 bb:1.1575 rl:2.3369 rb:0.9788 dl:129-130 gd:1 +ttp: b15/782 bl:2.6580 bb:1.1270 rl:2.3373 rb:0.9790 dl:115-117 gd:1 +ttp: b6/782 bl:2.7294 bb:1.1162 rl:2.3377 rb:0.9792 dl:99-101 gd:1 +quantized_ttt_phased val_loss:2.33156225 val_bpb:0.97791756 eval_time:562761ms +total_eval_time:562.8s diff --git a/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed42.log b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed42.log new file mode 100644 index 0000000000..c73c11f2c2 --- /dev/null +++ b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed42.log @@ -0,0 +1,833 @@ +W0422 04:53:28.892000 298 torch/distributed/run.py:803] +W0422 04:53:28.892000 298 torch/distributed/run.py:803] ***************************************** +W0422 04:53:28.892000 298 torch/distributed/run.py:803] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. +W0422 04:53:28.892000 298 torch/distributed/run.py:803] ***************************************** +Hyperparameters: + adam_eps: 1e-08 + adam_wd: 0.02 + artifact_dir: + attn_clip_sigmas: 13.0 + attn_out_gate_enabled: False + attn_out_gate_src: proj + beta1: 0.9 + beta2: 0.95 + caseops_enabled: False + compressor: brotli + data_dir: ./data/ + datasets_dir: ./data/datasets/fineweb10B_sp8192 + distributed: True + ema_decay: 0.9965 + embed_bits: 7 + embed_clip_sigmas: 15.0 + embed_lr: 0.6 + embed_wd: 0.085 + enable_looping_at: 0.35 + eval_loop_depth: 4 + eval_seq_len: 2048 + eval_stride: 64 + gate_window: 12 + gated_attn_enabled: True + gated_attn_init_std: 0.005 + gated_attn_quant_gate: True + global_ttt_batch_seqs: 32 + global_ttt_chunk_tokens: 32768 + global_ttt_epochs: 1 + global_ttt_grad_clip: 1.0 + global_ttt_lr: 0.001 + global_ttt_momentum: 0.9 + global_ttt_respect_doc_boundaries: True + global_ttt_warmup_chunks: 0 + global_ttt_warmup_start_lr: 0.0 + gptq_calibration_batches: 16 + gptq_reserve_seconds: 4.0 + grad_accum_steps: 1 + grad_clip_norm: 0.3 + is_main_process: True + iterations: 20000 + ln_scale: True + local_rank: 0 + logfile: logs/1ccd2180-6d96-4213-9b68-034667bed0fd.txt + logit_softcap: 30.0 + loop_end: 5 + loop_start: 3 + matrix_bits: 6 + matrix_clip_sigmas: 12.85 + matrix_lr: 0.026 + max_wallclock_seconds: 600.0 + min_lr: 0.0 + mlp_clip_sigmas: 12.0 + mlp_mult: 4.0 + model_dim: 512 + model_path: final_model.pt + muon_backend_steps: 5 + muon_momentum: 0.97 + muon_momentum_warmup_start: 0.92 + muon_momentum_warmup_steps: 1500 + muon_row_normalize: True + muon_wd: 0.095 + num_heads: 8 + num_kv_heads: 4 + num_layers: 11 + num_loops: 2 + parallel_final_lane: mean + parallel_start_layer: 8 + phased_ttt_num_phases: 3 + phased_ttt_prefix_docs: 2000 + qk_gain_init: 5.0 + quantized_model_path: final_model.int6.ptz + rank: 0 + rope_base: 10000.0 + rope_dims: 16 + rope_train_seq_len: 2048 + rope_yarn: False + run_id: 1ccd2180-6d96-4213-9b68-034667bed0fd + scalar_lr: 0.02 + seed: 42 + skip_gates_enabled: True + smear_gate_enabled: True + tie_embeddings: True + tied_embed_init_std: 0.005 + tied_embed_lr: 0.03 + tokenizer_path: ./data/tokenizers/fineweb_8192_bpe.model + train_batch_tokens: 786432 + train_files: ./data/datasets/fineweb10B_sp8192/fineweb_train_*.bin + train_log_every: 500 + train_loop_depth_dist: fixed + train_loop_depth_set: [] + train_loop_max_depth: 3 + train_loop_min_depth: 3 + train_loop_phase_depths: [1, 3, 4] + train_loop_phase_fractions: [] + train_loop_prewarm_depths: [3, 4] + train_seq_len: 2048 + ttt_batch_size: 64 + ttt_beta1: 0.0 + ttt_beta2: 0.999 + ttt_chunk_size: 48 + ttt_enabled: True + ttt_eval_batches: + ttt_eval_seq_len: 2048 + ttt_grad_steps: 1 + ttt_k_lora: True + ttt_lora_alpha: 144.0 + ttt_lora_lr: 0.0001 + ttt_lora_rank: 96 + ttt_mlp_lora: True + ttt_o_lora: True + ttt_optimizer: adam + ttt_warm_start_a: True + ttt_weight_decay: 1.0 + val_batch_tokens: 524288 + val_bytes_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_bytes_*.bin + val_doc_fraction: 1.0 + val_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_[0-9][0-9][0-9][0-9][0-9][0-9].bin + val_loss_every: 4000 + vocab_size: 8192 + warmdown_frac: 0.75 + warmup_steps: 20 + world_size: 8 + xsa_last_n: 11 +train_shards: 80 +val_tokens: 47851520 +model_params:35989671 +gptq:reserving 4s, effective=596000ms +loop_depth_schedule: train=[1, 3, 4] dist=phased phase_fracs=None prewarm=[3, 4] eval_depth=4 +warmup_cu_buckets:64,128,192,256 iters_each:3 +warmup_step: 1/20 +warmup_step: 2/20 +warmup_step: 3/20 +warmup_step: 4/20 +warmup_step: 5/20 +warmup_step: 6/20 +warmup_step: 10/20 +warmup_step: 20/20 +0/20000 val_loss: 9.0071 val_bpb: 3.7776 +1/20000 train_loss: 9.0072 train_time: 0.0m tok/s: 11789099 +2/20000 train_loss: 12.7502 train_time: 0.0m tok/s: 10945858 +3/20000 train_loss: 10.1465 train_time: 0.0m tok/s: 9823759 +4/20000 train_loss: 8.6337 train_time: 0.0m tok/s: 9393060 +5/20000 train_loss: 7.8612 train_time: 0.0m tok/s: 9145480 +500/20000 train_loss: 2.5756 train_time: 0.8m tok/s: 8124248 +1000/20000 train_loss: 2.8038 train_time: 1.6m tok/s: 8104437 +1500/20000 train_loss: 2.6385 train_time: 2.4m tok/s: 8095896 +2000/20000 train_loss: 2.6650 train_time: 3.2m tok/s: 8092338 +layer_loop:phase step:2043 frac:0.333 depth:3 phases:[1, 3, 4] eval_depth:4 +2500/20000 train_loss: 2.5472 train_time: 4.4m tok/s: 7457303 +3000/20000 train_loss: 2.5609 train_time: 5.6m tok/s: 7045679 +layer_loop:phase step:3438 frac:0.667 depth:4 phases:[1, 3, 4] eval_depth:4 +3500/20000 train_loss: 2.5675 train_time: 6.8m tok/s: 6754369 +4000/20000 train_loss: 2.3942 train_time: 8.2m tok/s: 6423543 +4000/20000 val_loss: 2.4178 val_bpb: 1.0141 +4500/20000 train_loss: 2.2533 train_time: 9.5m tok/s: 6188424 +4648/20000 val_loss: 2.3389 val_bpb: 0.9810 +stopping_early: wallclock_cap train_time: 596195ms step: 4648/20000 +peak memory allocated: 46681 MiB reserved: 50524 MiB +ema:applying EMA weights +diagnostic pre-quantization post-ema val_loss:2.33843408 val_bpb:0.98075329 eval_time:8065ms +Serialized model: 135593533 bytes +Code size (uncompressed): 135003 bytes +Code size (compressed): 29930 bytes +GPTQ:collecting Hessians from calibration data... +GPTQ:collected 67 Hessians in 3.9s +Quantized weights: + gate_int8_row: blocks.attn.attn_gate_w + gptq (int6): blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight + gptq (int7): tok_emb.weight + passthrough (float16): blocks.attn.q_gain, blocks.attn_scale, blocks.mlp_scale, blocks.resid_mix, parallel_post_lambdas, parallel_resid_lambdas, skip_gates, skip_weights, smear_gate.weight, smear_lambda +Serialized model quantized+brotli: 15945662 bytes +Total submission size quantized+brotli: 15975592 bytes +diagnostic quantized val_loss:2.35834176 val_bpb:0.98910269 eval_time:91703ms +ttt_lora:warming up compile (random tokens, no val data) +ttt_lora:compile warmup done (187.5s) + +beginning TTT eval timer +ttt_phased: total_docs:50000 prefix_docs:2000 suffix_docs:48000 num_phases:3 boundaries:[666, 1333, 2000] +ttp: b775/782 bl:2.2867 bb:0.9779 rl:2.2867 rb:0.9779 dl:6892-7524 gd:0 +ttp: b774/782 bl:2.2965 bb:0.9808 rl:2.2914 rb:0.9793 dl:6447-6872 gd:0 +ttp: b769/782 bl:2.3357 bb:0.9957 rl:2.3035 rb:0.9838 dl:5097-5309 gd:0 +ttpp: phase:1/3 pd:1104 gd:666 t:253.0s +tttg: c1/111 lr:0.001000 t:1.5s +tttg: c2/111 lr:0.001000 t:1.6s +tttg: c3/111 lr:0.000999 t:1.7s +tttg: c4/111 lr:0.000998 t:1.8s +tttg: c5/111 lr:0.000997 t:1.9s +tttg: c6/111 lr:0.000995 t:1.9s +tttg: c7/111 lr:0.000993 t:2.0s +tttg: c8/111 lr:0.000990 t:2.1s +tttg: c9/111 lr:0.000987 t:2.2s +tttg: c10/111 lr:0.000984 t:2.2s +tttg: c11/111 lr:0.000980 t:2.3s +tttg: c12/111 lr:0.000976 t:2.4s +tttg: c13/111 lr:0.000971 t:2.5s +tttg: c14/111 lr:0.000966 t:2.5s +tttg: c15/111 lr:0.000961 t:2.6s +tttg: c16/111 lr:0.000955 t:2.7s +tttg: c17/111 lr:0.000949 t:2.8s +tttg: c18/111 lr:0.000942 t:2.9s +tttg: c19/111 lr:0.000935 t:2.9s +tttg: c20/111 lr:0.000928 t:3.0s +tttg: c21/111 lr:0.000921 t:3.1s +tttg: c22/111 lr:0.000913 t:3.2s +tttg: c23/111 lr:0.000905 t:3.2s +tttg: c24/111 lr:0.000896 t:3.3s +tttg: c25/111 lr:0.000887 t:3.4s +tttg: c26/111 lr:0.000878 t:3.5s +tttg: c27/111 lr:0.000868 t:3.5s +tttg: c28/111 lr:0.000859 t:3.6s +tttg: c29/111 lr:0.000848 t:3.7s +tttg: c30/111 lr:0.000838 t:3.8s +tttg: c31/111 lr:0.000827 t:3.8s +tttg: c32/111 lr:0.000817 t:3.9s +tttg: c33/111 lr:0.000805 t:4.0s +tttg: c34/111 lr:0.000794 t:4.1s +tttg: c35/111 lr:0.000782 t:4.1s +tttg: c36/111 lr:0.000770 t:4.2s +tttg: c37/111 lr:0.000758 t:4.3s +tttg: c38/111 lr:0.000746 t:4.4s +tttg: c39/111 lr:0.000733 t:4.4s +tttg: c40/111 lr:0.000721 t:4.5s +tttg: c41/111 lr:0.000708 t:4.6s +tttg: c42/111 lr:0.000695 t:4.7s +tttg: c43/111 lr:0.000681 t:4.7s +tttg: c44/111 lr:0.000668 t:4.8s +tttg: c45/111 lr:0.000655 t:4.9s +tttg: c46/111 lr:0.000641 t:5.0s +tttg: c47/111 lr:0.000627 t:5.1s +tttg: c48/111 lr:0.000613 t:5.1s +tttg: c49/111 lr:0.000599 t:5.2s +tttg: c50/111 lr:0.000585 t:5.3s +tttg: c51/111 lr:0.000571 t:5.4s +tttg: c52/111 lr:0.000557 t:5.4s +tttg: c53/111 lr:0.000543 t:5.5s +tttg: c54/111 lr:0.000529 t:5.6s +tttg: c55/111 lr:0.000514 t:5.7s +tttg: c56/111 lr:0.000500 t:5.7s +tttg: c57/111 lr:0.000486 t:5.8s +tttg: c58/111 lr:0.000471 t:5.9s +tttg: c59/111 lr:0.000457 t:6.0s +tttg: c60/111 lr:0.000443 t:6.0s +tttg: c61/111 lr:0.000429 t:6.1s +tttg: c62/111 lr:0.000415 t:6.2s +tttg: c63/111 lr:0.000401 t:6.3s +tttg: c64/111 lr:0.000387 t:6.4s +tttg: c65/111 lr:0.000373 t:6.4s +tttg: c66/111 lr:0.000359 t:6.5s +tttg: c67/111 lr:0.000345 t:6.6s +tttg: c68/111 lr:0.000332 t:6.7s +tttg: c69/111 lr:0.000319 t:6.7s +tttg: c70/111 lr:0.000305 t:6.8s +tttg: c71/111 lr:0.000292 t:6.9s +tttg: c72/111 lr:0.000279 t:7.0s +tttg: c73/111 lr:0.000267 t:7.0s +tttg: c74/111 lr:0.000254 t:7.1s +tttg: c75/111 lr:0.000242 t:7.2s +tttg: c76/111 lr:0.000230 t:7.3s +tttg: c77/111 lr:0.000218 t:7.3s +tttg: c78/111 lr:0.000206 t:7.4s +tttg: c79/111 lr:0.000195 t:7.5s +tttg: c80/111 lr:0.000183 t:7.6s +tttg: c81/111 lr:0.000173 t:7.6s +tttg: c82/111 lr:0.000162 t:7.7s +tttg: c83/111 lr:0.000152 t:7.8s +tttg: c84/111 lr:0.000141 t:7.9s +tttg: c85/111 lr:0.000132 t:7.9s +tttg: c86/111 lr:0.000122 t:8.0s +tttg: c87/111 lr:0.000113 t:8.1s +tttg: c88/111 lr:0.000104 t:8.2s +tttg: c89/111 lr:0.000095 t:8.2s +tttg: c90/111 lr:0.000087 t:8.3s +tttg: c91/111 lr:0.000079 t:8.4s +tttg: c92/111 lr:0.000072 t:8.5s +tttg: c93/111 lr:0.000065 t:8.6s +tttg: c94/111 lr:0.000058 t:8.6s +tttg: c95/111 lr:0.000051 t:8.7s +tttg: c96/111 lr:0.000045 t:8.8s +tttg: c97/111 lr:0.000039 t:8.9s +tttg: c98/111 lr:0.000034 t:8.9s +tttg: c99/111 lr:0.000029 t:9.0s +tttg: c100/111 lr:0.000024 t:9.1s +tttg: c101/111 lr:0.000020 t:9.2s +tttg: c102/111 lr:0.000016 t:9.2s +tttg: c103/111 lr:0.000013 t:9.3s +tttg: c104/111 lr:0.000010 t:9.4s +tttg: c105/111 lr:0.000007 t:9.5s +tttg: c106/111 lr:0.000005 t:9.5s +tttg: c107/111 lr:0.000003 t:9.6s +tttg: c108/111 lr:0.000002 t:9.7s +tttg: c109/111 lr:0.000001 t:9.8s +tttg: c110/111 lr:0.000000 t:9.8s +ttpr: phase:1/3 t:265.0s +ttp: b757/782 bl:2.2900 bb:0.9797 rl:2.3014 rb:0.9831 dl:3550-3633 gd:0 +ttp: b754/782 bl:2.2934 bb:0.9718 rl:2.3003 rb:0.9816 dl:3345-3397 gd:0 +ttpp: phase:2/3 pd:1808 gd:1333 t:397.7s +tttg: c1/185 lr:0.001000 t:0.1s +tttg: c2/185 lr:0.001000 t:0.2s +tttg: c3/185 lr:0.001000 t:0.2s +tttg: c4/185 lr:0.000999 t:0.3s +tttg: c5/185 lr:0.000999 t:0.4s +tttg: c6/185 lr:0.000998 t:0.5s +tttg: c7/185 lr:0.000997 t:7.8s +tttg: c8/185 lr:0.000996 t:7.9s +tttg: c9/185 lr:0.000995 t:8.0s +tttg: c10/185 lr:0.000994 t:8.1s +tttg: c11/185 lr:0.000993 t:8.2s +tttg: c12/185 lr:0.000991 t:8.2s +tttg: c13/185 lr:0.000990 t:8.3s +tttg: c14/185 lr:0.000988 t:8.4s +tttg: c15/185 lr:0.000986 t:8.5s +tttg: c16/185 lr:0.000984 t:8.6s +tttg: c17/185 lr:0.000981 t:8.6s +tttg: c18/185 lr:0.000979 t:8.7s +tttg: c19/185 lr:0.000977 t:8.8s +tttg: c20/185 lr:0.000974 t:8.9s +tttg: c21/185 lr:0.000971 t:8.9s +tttg: c22/185 lr:0.000968 t:9.0s +tttg: c23/185 lr:0.000965 t:9.1s +tttg: c24/185 lr:0.000962 t:9.2s +tttg: c25/185 lr:0.000959 t:9.3s +tttg: c26/185 lr:0.000955 t:9.3s +tttg: c27/185 lr:0.000952 t:9.4s +tttg: c28/185 lr:0.000948 t:9.5s +tttg: c29/185 lr:0.000944 t:9.6s +tttg: c30/185 lr:0.000940 t:9.7s +tttg: c31/185 lr:0.000936 t:9.7s +tttg: c32/185 lr:0.000932 t:9.8s +tttg: c33/185 lr:0.000927 t:9.9s +tttg: c34/185 lr:0.000923 t:10.0s +tttg: c35/185 lr:0.000918 t:10.1s +tttg: c36/185 lr:0.000913 t:10.1s +tttg: c37/185 lr:0.000908 t:10.2s +tttg: c38/185 lr:0.000904 t:10.3s +tttg: c39/185 lr:0.000898 t:10.4s +tttg: c40/185 lr:0.000893 t:10.5s +tttg: c41/185 lr:0.000888 t:10.5s +tttg: c42/185 lr:0.000882 t:10.6s +tttg: c43/185 lr:0.000877 t:10.7s +tttg: c44/185 lr:0.000871 t:10.8s +tttg: c45/185 lr:0.000865 t:10.8s +tttg: c46/185 lr:0.000860 t:10.9s +tttg: c47/185 lr:0.000854 t:11.0s +tttg: c48/185 lr:0.000847 t:11.1s +tttg: c49/185 lr:0.000841 t:11.2s +tttg: c50/185 lr:0.000835 t:11.2s +tttg: c51/185 lr:0.000829 t:11.3s +tttg: c52/185 lr:0.000822 t:11.4s +tttg: c53/185 lr:0.000816 t:11.5s +tttg: c54/185 lr:0.000809 t:11.6s +tttg: c55/185 lr:0.000802 t:11.6s +tttg: c56/185 lr:0.000795 t:11.7s +tttg: c57/185 lr:0.000788 t:11.8s +tttg: c58/185 lr:0.000781 t:11.9s +tttg: c59/185 lr:0.000774 t:12.0s +tttg: c60/185 lr:0.000767 t:12.0s +tttg: c61/185 lr:0.000760 t:12.1s +tttg: c62/185 lr:0.000752 t:12.2s +tttg: c63/185 lr:0.000745 t:12.3s +tttg: c64/185 lr:0.000738 t:12.4s +tttg: c65/185 lr:0.000730 t:12.4s +tttg: c66/185 lr:0.000722 t:12.5s +tttg: c67/185 lr:0.000715 t:12.6s +tttg: c68/185 lr:0.000707 t:12.7s +tttg: c69/185 lr:0.000699 t:12.7s +tttg: c70/185 lr:0.000691 t:12.8s +tttg: c71/185 lr:0.000683 t:12.9s +tttg: c72/185 lr:0.000675 t:13.0s +tttg: c73/185 lr:0.000667 t:13.1s +tttg: c74/185 lr:0.000659 t:13.1s +tttg: c75/185 lr:0.000651 t:13.2s +tttg: c76/185 lr:0.000643 t:13.3s +tttg: c77/185 lr:0.000635 t:13.4s +tttg: c78/185 lr:0.000627 t:13.5s +tttg: c79/185 lr:0.000618 t:13.5s +tttg: c80/185 lr:0.000610 t:13.6s +tttg: c81/185 lr:0.000602 t:13.7s +tttg: c82/185 lr:0.000593 t:13.8s +tttg: c83/185 lr:0.000585 t:13.9s +tttg: c84/185 lr:0.000577 t:13.9s +tttg: c85/185 lr:0.000568 t:14.0s +tttg: c86/185 lr:0.000560 t:14.1s +tttg: c87/185 lr:0.000551 t:14.2s +tttg: c88/185 lr:0.000543 t:14.3s +tttg: c89/185 lr:0.000534 t:14.3s +tttg: c90/185 lr:0.000526 t:14.4s +tttg: c91/185 lr:0.000517 t:14.5s +tttg: c92/185 lr:0.000509 t:14.6s +tttg: c93/185 lr:0.000500 t:14.7s +tttg: c94/185 lr:0.000491 t:14.7s +tttg: c95/185 lr:0.000483 t:14.8s +tttg: c96/185 lr:0.000474 t:14.9s +tttg: c97/185 lr:0.000466 t:15.0s +tttg: c98/185 lr:0.000457 t:15.1s +tttg: c99/185 lr:0.000449 t:15.1s +tttg: c100/185 lr:0.000440 t:15.2s +tttg: c101/185 lr:0.000432 t:15.3s +tttg: c102/185 lr:0.000423 t:15.4s +tttg: c103/185 lr:0.000415 t:15.5s +tttg: c104/185 lr:0.000407 t:15.5s +tttg: c105/185 lr:0.000398 t:15.6s +tttg: c106/185 lr:0.000390 t:15.7s +tttg: c107/185 lr:0.000382 t:15.8s +tttg: c108/185 lr:0.000373 t:15.9s +tttg: c109/185 lr:0.000365 t:15.9s +tttg: c110/185 lr:0.000357 t:16.0s +tttg: c111/185 lr:0.000349 t:16.1s +tttg: c112/185 lr:0.000341 t:16.2s +tttg: c113/185 lr:0.000333 t:16.3s +tttg: c114/185 lr:0.000325 t:16.3s +tttg: c115/185 lr:0.000317 t:16.4s +tttg: c116/185 lr:0.000309 t:16.5s +tttg: c117/185 lr:0.000301 t:16.6s +tttg: c118/185 lr:0.000293 t:16.7s +tttg: c119/185 lr:0.000285 t:16.7s +tttg: c120/185 lr:0.000278 t:16.8s +tttg: c121/185 lr:0.000270 t:16.9s +tttg: c122/185 lr:0.000262 t:17.0s +tttg: c123/185 lr:0.000255 t:17.0s +tttg: c124/185 lr:0.000248 t:17.1s +tttg: c125/185 lr:0.000240 t:17.2s +tttg: c126/185 lr:0.000233 t:17.3s +tttg: c127/185 lr:0.000226 t:17.4s +tttg: c128/185 lr:0.000219 t:17.4s +tttg: c129/185 lr:0.000212 t:17.5s +tttg: c130/185 lr:0.000205 t:17.6s +tttg: c131/185 lr:0.000198 t:17.7s +tttg: c132/185 lr:0.000191 t:17.8s +tttg: c133/185 lr:0.000184 t:17.8s +tttg: c134/185 lr:0.000178 t:17.9s +tttg: c135/185 lr:0.000171 t:18.0s +tttg: c136/185 lr:0.000165 t:18.1s +tttg: c137/185 lr:0.000159 t:18.2s +tttg: c138/185 lr:0.000153 t:18.2s +tttg: c139/185 lr:0.000146 t:18.3s +tttg: c140/185 lr:0.000140 t:18.4s +tttg: c141/185 lr:0.000135 t:18.5s +tttg: c142/185 lr:0.000129 t:18.6s +tttg: c143/185 lr:0.000123 t:18.6s +tttg: c144/185 lr:0.000118 t:18.7s +tttg: c145/185 lr:0.000112 t:18.8s +tttg: c146/185 lr:0.000107 t:18.9s +tttg: c147/185 lr:0.000102 t:18.9s +tttg: c148/185 lr:0.000096 t:19.0s +tttg: c149/185 lr:0.000092 t:19.1s +tttg: c150/185 lr:0.000087 t:19.2s +tttg: c151/185 lr:0.000082 t:19.3s +tttg: c152/185 lr:0.000077 t:19.4s +tttg: c153/185 lr:0.000073 t:19.4s +tttg: c154/185 lr:0.000068 t:19.5s +tttg: c155/185 lr:0.000064 t:19.6s +tttg: c156/185 lr:0.000060 t:19.7s +tttg: c157/185 lr:0.000056 t:19.8s +tttg: c158/185 lr:0.000052 t:19.8s +tttg: c159/185 lr:0.000048 t:19.9s +tttg: c160/185 lr:0.000045 t:20.0s +tttg: c161/185 lr:0.000041 t:20.1s +tttg: c162/185 lr:0.000038 t:20.2s +tttg: c163/185 lr:0.000035 t:20.2s +tttg: c164/185 lr:0.000032 t:20.3s +tttg: c165/185 lr:0.000029 t:20.4s +tttg: c166/185 lr:0.000026 t:20.5s +tttg: c167/185 lr:0.000023 t:20.5s +tttg: c168/185 lr:0.000021 t:20.6s +tttg: c169/185 lr:0.000019 t:20.7s +tttg: c170/185 lr:0.000016 t:20.8s +tttg: c171/185 lr:0.000014 t:20.9s +tttg: c172/185 lr:0.000012 t:20.9s +tttg: c173/185 lr:0.000010 t:21.0s +tttg: c174/185 lr:0.000009 t:21.1s +tttg: c175/185 lr:0.000007 t:21.2s +tttg: c176/185 lr:0.000006 t:21.3s +tttg: c177/185 lr:0.000005 t:21.3s +tttg: c178/185 lr:0.000004 t:21.4s +tttg: c179/185 lr:0.000003 t:21.5s +tttg: c180/185 lr:0.000002 t:21.6s +tttg: c181/185 lr:0.000001 t:21.7s +tttg: c182/185 lr:0.000001 t:21.7s +tttg: c183/185 lr:0.000000 t:21.8s +tttg: c184/185 lr:0.000000 t:21.9s +ttpr: phase:2/3 t:421.7s +ttp: b748/782 bl:2.3243 bb:0.9899 rl:2.3028 rb:0.9825 dl:2992-3039 gd:0 +ttpp: phase:3/3 pd:2448 gd:2000 t:449.1s +tttg: c1/250 lr:0.001000 t:0.1s +tttg: c2/250 lr:0.001000 t:0.2s +tttg: c3/250 lr:0.001000 t:0.2s +tttg: c4/250 lr:0.001000 t:0.3s +tttg: c5/250 lr:0.000999 t:0.4s +tttg: c6/250 lr:0.000999 t:0.5s +tttg: c7/250 lr:0.000999 t:0.6s +tttg: c8/250 lr:0.000998 t:0.6s +tttg: c9/250 lr:0.000997 t:0.7s +tttg: c10/250 lr:0.000997 t:0.8s +tttg: c11/250 lr:0.000996 t:0.9s +tttg: c12/250 lr:0.000995 t:0.9s +tttg: c13/250 lr:0.000994 t:1.0s +tttg: c14/250 lr:0.000993 t:1.1s +tttg: c15/250 lr:0.000992 t:1.2s +tttg: c16/250 lr:0.000991 t:1.2s +tttg: c17/250 lr:0.000990 t:1.3s +tttg: c18/250 lr:0.000989 t:1.4s +tttg: c19/250 lr:0.000987 t:1.5s +tttg: c20/250 lr:0.000986 t:1.6s +tttg: c21/250 lr:0.000984 t:1.6s +tttg: c22/250 lr:0.000983 t:1.7s +tttg: c23/250 lr:0.000981 t:1.8s +tttg: c24/250 lr:0.000979 t:1.9s +tttg: c25/250 lr:0.000977 t:2.0s +tttg: c26/250 lr:0.000975 t:2.0s +tttg: c27/250 lr:0.000973 t:2.1s +tttg: c28/250 lr:0.000971 t:2.2s +tttg: c29/250 lr:0.000969 t:2.3s +tttg: c30/250 lr:0.000967 t:2.3s +tttg: c31/250 lr:0.000965 t:2.4s +tttg: c32/250 lr:0.000962 t:2.5s +tttg: c33/250 lr:0.000960 t:2.6s +tttg: c34/250 lr:0.000957 t:2.7s +tttg: c35/250 lr:0.000955 t:2.7s +tttg: c36/250 lr:0.000952 t:2.8s +tttg: c37/250 lr:0.000949 t:2.9s +tttg: c38/250 lr:0.000947 t:3.0s +tttg: c39/250 lr:0.000944 t:3.0s +tttg: c40/250 lr:0.000941 t:3.1s +tttg: c41/250 lr:0.000938 t:3.2s +tttg: c42/250 lr:0.000935 t:3.3s +tttg: c43/250 lr:0.000931 t:3.3s +tttg: c44/250 lr:0.000928 t:3.4s +tttg: c45/250 lr:0.000925 t:3.5s +tttg: c46/250 lr:0.000922 t:3.6s +tttg: c47/250 lr:0.000918 t:3.7s +tttg: c48/250 lr:0.000915 t:3.7s +tttg: c49/250 lr:0.000911 t:3.8s +tttg: c50/250 lr:0.000907 t:3.9s +tttg: c51/250 lr:0.000904 t:4.0s +tttg: c52/250 lr:0.000900 t:4.0s +tttg: c53/250 lr:0.000896 t:4.1s +tttg: c54/250 lr:0.000892 t:4.2s +tttg: c55/250 lr:0.000888 t:4.3s +tttg: c56/250 lr:0.000884 t:4.4s +tttg: c57/250 lr:0.000880 t:4.4s +tttg: c58/250 lr:0.000876 t:4.5s +tttg: c59/250 lr:0.000872 t:4.6s +tttg: c60/250 lr:0.000868 t:4.7s +tttg: c61/250 lr:0.000863 t:4.7s +tttg: c62/250 lr:0.000859 t:4.8s +tttg: c63/250 lr:0.000855 t:4.9s +tttg: c64/250 lr:0.000850 t:5.0s +tttg: c65/250 lr:0.000846 t:5.0s +tttg: c66/250 lr:0.000841 t:5.1s +tttg: c67/250 lr:0.000836 t:5.2s +tttg: c68/250 lr:0.000832 t:5.3s +tttg: c69/250 lr:0.000827 t:5.4s +tttg: c70/250 lr:0.000822 t:5.4s +tttg: c71/250 lr:0.000817 t:5.5s +tttg: c72/250 lr:0.000812 t:5.6s +tttg: c73/250 lr:0.000807 t:5.7s +tttg: c74/250 lr:0.000803 t:5.7s +tttg: c75/250 lr:0.000797 t:5.8s +tttg: c76/250 lr:0.000792 t:5.9s +tttg: c77/250 lr:0.000787 t:6.0s +tttg: c78/250 lr:0.000782 t:6.1s +tttg: c79/250 lr:0.000777 t:6.1s +tttg: c80/250 lr:0.000772 t:6.2s +tttg: c81/250 lr:0.000766 t:6.3s +tttg: c82/250 lr:0.000761 t:6.4s +tttg: c83/250 lr:0.000755 t:6.4s +tttg: c84/250 lr:0.000750 t:6.5s +tttg: c85/250 lr:0.000745 t:6.6s +tttg: c86/250 lr:0.000739 t:6.7s +tttg: c87/250 lr:0.000733 t:6.7s +tttg: c88/250 lr:0.000728 t:6.8s +tttg: c89/250 lr:0.000722 t:6.9s +tttg: c90/250 lr:0.000717 t:7.0s +tttg: c91/250 lr:0.000711 t:7.1s +tttg: c92/250 lr:0.000705 t:7.1s +tttg: c93/250 lr:0.000699 t:7.2s +tttg: c94/250 lr:0.000694 t:7.3s +tttg: c95/250 lr:0.000688 t:7.4s +tttg: c96/250 lr:0.000682 t:7.4s +tttg: c97/250 lr:0.000676 t:7.5s +tttg: c98/250 lr:0.000670 t:7.6s +tttg: c99/250 lr:0.000664 t:7.7s +tttg: c100/250 lr:0.000658 t:7.8s +tttg: c101/250 lr:0.000652 t:7.8s +tttg: c102/250 lr:0.000646 t:7.9s +tttg: c103/250 lr:0.000640 t:8.0s +tttg: c104/250 lr:0.000634 t:8.1s +tttg: c105/250 lr:0.000628 t:8.1s +tttg: c106/250 lr:0.000622 t:8.2s +tttg: c107/250 lr:0.000616 t:8.3s +tttg: c108/250 lr:0.000610 t:8.4s +tttg: c109/250 lr:0.000603 t:8.4s +tttg: c110/250 lr:0.000597 t:8.5s +tttg: c111/250 lr:0.000591 t:8.6s +tttg: c112/250 lr:0.000585 t:8.7s +tttg: c113/250 lr:0.000579 t:8.8s +tttg: c114/250 lr:0.000572 t:8.8s +tttg: c115/250 lr:0.000566 t:8.9s +tttg: c116/250 lr:0.000560 t:9.0s +tttg: c117/250 lr:0.000554 t:9.1s +tttg: c118/250 lr:0.000547 t:9.1s +tttg: c119/250 lr:0.000541 t:9.2s +tttg: c120/250 lr:0.000535 t:9.3s +tttg: c121/250 lr:0.000528 t:9.4s +tttg: c122/250 lr:0.000522 t:9.4s +tttg: c123/250 lr:0.000516 t:9.5s +tttg: c124/250 lr:0.000509 t:9.6s +tttg: c125/250 lr:0.000503 t:9.7s +tttg: c126/250 lr:0.000497 t:9.8s +tttg: c127/250 lr:0.000491 t:9.8s +tttg: c128/250 lr:0.000484 t:9.9s +tttg: c129/250 lr:0.000478 t:10.0s +tttg: c130/250 lr:0.000472 t:10.1s +tttg: c131/250 lr:0.000465 t:10.1s +tttg: c132/250 lr:0.000459 t:10.2s +tttg: c133/250 lr:0.000453 t:10.3s +tttg: c134/250 lr:0.000446 t:10.4s +tttg: c135/250 lr:0.000440 t:10.5s +tttg: c136/250 lr:0.000434 t:10.5s +tttg: c137/250 lr:0.000428 t:10.6s +tttg: c138/250 lr:0.000421 t:10.7s +tttg: c139/250 lr:0.000415 t:10.8s +tttg: c140/250 lr:0.000409 t:10.8s +tttg: c141/250 lr:0.000403 t:10.9s +tttg: c142/250 lr:0.000397 t:11.0s +tttg: c143/250 lr:0.000390 t:11.1s +tttg: c144/250 lr:0.000384 t:11.1s +tttg: c145/250 lr:0.000378 t:11.2s +tttg: c146/250 lr:0.000372 t:11.3s +tttg: c147/250 lr:0.000366 t:11.4s +tttg: c148/250 lr:0.000360 t:11.5s +tttg: c149/250 lr:0.000354 t:11.5s +tttg: c150/250 lr:0.000348 t:11.6s +tttg: c151/250 lr:0.000342 t:11.7s +tttg: c152/250 lr:0.000336 t:11.8s +tttg: c153/250 lr:0.000330 t:11.8s +tttg: c154/250 lr:0.000324 t:11.9s +tttg: c155/250 lr:0.000318 t:12.0s +tttg: c156/250 lr:0.000312 t:12.1s +tttg: c157/250 lr:0.000306 t:12.2s +tttg: c158/250 lr:0.000301 t:12.2s +tttg: c159/250 lr:0.000295 t:12.3s +tttg: c160/250 lr:0.000289 t:12.4s +tttg: c161/250 lr:0.000283 t:12.5s +tttg: c162/250 lr:0.000278 t:12.5s +tttg: c163/250 lr:0.000272 t:12.6s +tttg: c164/250 lr:0.000267 t:12.7s +tttg: c165/250 lr:0.000261 t:12.8s +tttg: c166/250 lr:0.000255 t:12.8s +tttg: c167/250 lr:0.000250 t:12.9s +tttg: c168/250 lr:0.000245 t:13.0s +tttg: c169/250 lr:0.000239 t:13.1s +tttg: c170/250 lr:0.000234 t:13.2s +tttg: c171/250 lr:0.000228 t:13.2s +tttg: c172/250 lr:0.000223 t:13.3s +tttg: c173/250 lr:0.000218 t:13.4s +tttg: c174/250 lr:0.000213 t:13.5s +tttg: c175/250 lr:0.000208 t:13.5s +tttg: c176/250 lr:0.000203 t:13.6s +tttg: c177/250 lr:0.000197 t:13.7s +tttg: c178/250 lr:0.000193 t:13.8s +tttg: c179/250 lr:0.000188 t:13.9s +tttg: c180/250 lr:0.000183 t:13.9s +tttg: c181/250 lr:0.000178 t:14.0s +tttg: c182/250 lr:0.000173 t:14.1s +tttg: c183/250 lr:0.000168 t:14.2s +tttg: c184/250 lr:0.000164 t:14.2s +tttg: c185/250 lr:0.000159 t:14.3s +tttg: c186/250 lr:0.000154 t:14.4s +tttg: c187/250 lr:0.000150 t:14.5s +tttg: c188/250 lr:0.000145 t:14.5s +tttg: c189/250 lr:0.000141 t:14.6s +tttg: c190/250 lr:0.000137 t:14.7s +tttg: c191/250 lr:0.000132 t:14.8s +tttg: c192/250 lr:0.000128 t:14.9s +tttg: c193/250 lr:0.000124 t:14.9s +tttg: c194/250 lr:0.000120 t:15.0s +tttg: c195/250 lr:0.000116 t:15.1s +tttg: c196/250 lr:0.000112 t:15.2s +tttg: c197/250 lr:0.000108 t:15.2s +tttg: c198/250 lr:0.000104 t:15.3s +tttg: c199/250 lr:0.000100 t:15.4s +tttg: c200/250 lr:0.000096 t:15.5s +tttg: c201/250 lr:0.000093 t:15.6s +tttg: c202/250 lr:0.000089 t:15.6s +tttg: c203/250 lr:0.000085 t:15.7s +tttg: c204/250 lr:0.000082 t:15.8s +tttg: c205/250 lr:0.000078 t:15.9s +tttg: c206/250 lr:0.000075 t:15.9s +tttg: c207/250 lr:0.000072 t:16.0s +tttg: c208/250 lr:0.000069 t:16.1s +tttg: c209/250 lr:0.000065 t:16.2s +tttg: c210/250 lr:0.000062 t:16.2s +tttg: c211/250 lr:0.000059 t:16.3s +tttg: c212/250 lr:0.000056 t:16.4s +tttg: c213/250 lr:0.000053 t:16.5s +tttg: c214/250 lr:0.000051 t:16.6s +tttg: c215/250 lr:0.000048 t:16.6s +tttg: c216/250 lr:0.000045 t:16.7s +tttg: c217/250 lr:0.000043 t:16.8s +tttg: c218/250 lr:0.000040 t:16.9s +tttg: c219/250 lr:0.000038 t:16.9s +tttg: c220/250 lr:0.000035 t:17.0s +tttg: c221/250 lr:0.000033 t:17.1s +tttg: c222/250 lr:0.000031 t:17.2s +tttg: c223/250 lr:0.000029 t:17.2s +tttg: c224/250 lr:0.000027 t:17.3s +tttg: c225/250 lr:0.000025 t:17.4s +tttg: c226/250 lr:0.000023 t:17.5s +tttg: c227/250 lr:0.000021 t:17.6s +tttg: c228/250 lr:0.000019 t:17.6s +tttg: c229/250 lr:0.000017 t:17.7s +tttg: c230/250 lr:0.000016 t:17.8s +tttg: c231/250 lr:0.000014 t:17.9s +tttg: c232/250 lr:0.000013 t:17.9s +tttg: c233/250 lr:0.000011 t:18.0s +tttg: c234/250 lr:0.000010 t:18.1s +tttg: c235/250 lr:0.000009 t:18.2s +tttg: c236/250 lr:0.000008 t:18.3s +tttg: c237/250 lr:0.000007 t:18.3s +tttg: c238/250 lr:0.000006 t:18.4s +tttg: c239/250 lr:0.000005 t:18.5s +tttg: c240/250 lr:0.000004 t:18.6s +tttg: c241/250 lr:0.000003 t:18.6s +tttg: c242/250 lr:0.000003 t:18.7s +tttg: c243/250 lr:0.000002 t:18.8s +tttg: c244/250 lr:0.000001 t:18.9s +tttg: c245/250 lr:0.000001 t:18.9s +tttg: c246/250 lr:0.000001 t:19.0s +tttg: c247/250 lr:0.000000 t:19.1s +tttg: c248/250 lr:0.000000 t:19.2s +tttg: c249/250 lr:0.000000 t:19.3s +ttpr: phase:3/3 t:470.5s +ttp: b737/782 bl:2.3252 bb:0.9577 rl:2.3046 rb:0.9804 dl:2550-2583 gd:1 +ttp: b735/782 bl:2.3934 bb:1.0093 rl:2.3112 rb:0.9826 dl:2495-2526 gd:1 +ttp: b725/782 bl:2.3272 bb:0.9658 rl:2.3122 rb:0.9815 dl:2232-2254 gd:1 +ttp: b715/782 bl:2.3656 bb:0.9567 rl:2.3150 rb:0.9801 dl:2036-2053 gd:1 +ttp: b709/782 bl:2.4528 bb:1.0118 rl:2.3217 rb:0.9817 dl:1937-1952 gd:1 +ttp: b699/782 bl:2.4233 bb:0.9812 rl:2.3261 rb:0.9817 dl:1814-1824 gd:1 +ttp: b695/782 bl:2.3482 bb:0.9890 rl:2.3269 rb:0.9820 dl:1769-1779 gd:1 +ttp: b680/782 bl:2.2867 bb:0.9448 rl:2.3255 rb:0.9806 dl:1618-1628 gd:1 +ttp: b677/782 bl:2.3177 bb:0.9579 rl:2.3252 rb:0.9798 dl:1595-1601 gd:1 +ttp: b666/782 bl:2.4198 bb:0.9856 rl:2.3282 rb:0.9800 dl:1507-1514 gd:1 +ttp: b660/782 bl:2.3814 bb:0.9692 rl:2.3297 rb:0.9797 dl:1466-1474 gd:1 +ttp: b652/782 bl:2.2549 bb:0.9397 rl:2.3277 rb:0.9786 dl:1411-1419 gd:1 +ttp: b643/782 bl:2.3605 bb:0.9524 rl:2.3285 rb:0.9779 dl:1356-1362 gd:1 +ttp: b634/782 bl:2.3928 bb:0.9769 rl:2.3301 rb:0.9778 dl:1302-1308 gd:1 +ttp: b625/782 bl:2.4198 bb:0.9769 rl:2.3321 rb:0.9778 dl:1255-1260 gd:1 +ttp: b617/782 bl:2.3214 bb:0.9420 rl:2.3319 rb:0.9770 dl:1211-1216 gd:1 +ttp: b609/782 bl:2.2795 bb:0.9402 rl:2.3308 rb:0.9763 dl:1172-1177 gd:1 +ttp: b602/782 bl:2.3876 bb:0.9734 rl:2.3319 rb:0.9762 dl:1141-1146 gd:1 +ttp: b598/782 bl:2.3665 bb:0.9807 rl:2.3326 rb:0.9763 dl:1124-1129 gd:1 +ttp: b590/782 bl:2.3139 bb:0.9704 rl:2.3322 rb:0.9762 dl:1089-1093 gd:1 +ttp: b582/782 bl:2.3585 bb:0.9561 rl:2.3327 rb:0.9758 dl:1056-1060 gd:1 +ttp: b568/782 bl:2.3653 bb:0.9907 rl:2.3332 rb:0.9761 dl:1004-1007 gd:1 +ttp: b566/782 bl:2.3034 bb:0.9509 rl:2.3327 rb:0.9757 dl:997-1001 gd:1 +ttp: b558/782 bl:2.3840 bb:0.9797 rl:2.3335 rb:0.9757 dl:968-972 gd:1 +ttp: b547/782 bl:2.3429 bb:0.9708 rl:2.3336 rb:0.9757 dl:934-937 gd:1 +ttp: b539/782 bl:2.3435 bb:0.9581 rl:2.3338 rb:0.9754 dl:909-912 gd:1 +ttp: b535/782 bl:2.3870 bb:0.9600 rl:2.3345 rb:0.9752 dl:896-899 gd:1 +ttp: b527/782 bl:2.3524 bb:0.9508 rl:2.3347 rb:0.9749 dl:872-875 gd:1 +ttp: b515/782 bl:2.3481 bb:0.9626 rl:2.3349 rb:0.9747 dl:838-841 gd:1 +ttp: b507/782 bl:2.3029 bb:0.9472 rl:2.3345 rb:0.9744 dl:814-817 gd:1 +ttp: b502/782 bl:2.3219 bb:0.9487 rl:2.3343 rb:0.9741 dl:802-804 gd:1 +ttp: b494/782 bl:2.3277 bb:0.9722 rl:2.3343 rb:0.9741 dl:780-783 gd:1 +ttp: b486/782 bl:2.4134 bb:0.9992 rl:2.3351 rb:0.9744 dl:761-764 gd:1 +ttp: b478/782 bl:2.3458 bb:0.9874 rl:2.3352 rb:0.9745 dl:742-744 gd:1 +ttp: b469/782 bl:2.3355 bb:0.9456 rl:2.3352 rb:0.9742 dl:721-724 gd:1 +ttp: b457/782 bl:2.2579 bb:0.9420 rl:2.3345 rb:0.9739 dl:695-697 gd:1 +ttp: b449/782 bl:2.4284 bb:0.9849 rl:2.3353 rb:0.9740 dl:678-680 gd:1 +ttp: b441/782 bl:2.3459 bb:0.9602 rl:2.3354 rb:0.9739 dl:662-664 gd:1 +ttp: b435/782 bl:2.3290 bb:0.9458 rl:2.3354 rb:0.9736 dl:648-651 gd:1 +ttp: b427/782 bl:2.2646 bb:0.9732 rl:2.3348 rb:0.9736 dl:634-636 gd:1 +ttp: b420/782 bl:2.3664 bb:0.9700 rl:2.3350 rb:0.9736 dl:620-622 gd:1 +ttp: b415/782 bl:2.2923 bb:0.9666 rl:2.3347 rb:0.9736 dl:611-613 gd:1 +ttp: b407/782 bl:2.2786 bb:0.9554 rl:2.3343 rb:0.9734 dl:595-597 gd:1 +ttp: b399/782 bl:2.2911 bb:0.9464 rl:2.3340 rb:0.9732 dl:581-582 gd:1 +ttp: b386/782 bl:2.3460 bb:0.9977 rl:2.3341 rb:0.9734 dl:557-559 gd:1 +ttp: b378/782 bl:2.4381 bb:0.9788 rl:2.3348 rb:0.9734 dl:544-545 gd:1 +ttp: b370/782 bl:2.3771 bb:0.9936 rl:2.3350 rb:0.9736 dl:530-532 gd:1 +ttp: b361/782 bl:2.3579 bb:1.0014 rl:2.3352 rb:0.9737 dl:515-517 gd:1 +ttp: b354/782 bl:2.3160 bb:0.9798 rl:2.3351 rb:0.9738 dl:503-504 gd:1 +ttp: b346/782 bl:2.3845 bb:0.9892 rl:2.3354 rb:0.9739 dl:491-492 gd:1 +ttp: b338/782 bl:2.3726 bb:1.0058 rl:2.3356 rb:0.9740 dl:478-480 gd:1 +ttp: b333/782 bl:2.4378 bb:0.9969 rl:2.3361 rb:0.9742 dl:471-472 gd:1 +ttp: b325/782 bl:2.3542 bb:0.9891 rl:2.3362 rb:0.9743 dl:459-461 gd:1 +ttp: b317/782 bl:2.3137 bb:0.9600 rl:2.3361 rb:0.9742 dl:446-448 gd:1 +ttp: b309/782 bl:2.4187 bb:1.0157 rl:2.3365 rb:0.9744 dl:435-437 gd:1 +ttp: b298/782 bl:2.4308 bb:1.0124 rl:2.3370 rb:0.9746 dl:418-420 gd:1 +ttp: b290/782 bl:2.3492 bb:0.9799 rl:2.3371 rb:0.9746 dl:406-407 gd:1 +ttp: b282/782 bl:2.3234 bb:0.9810 rl:2.3370 rb:0.9746 dl:395-396 gd:1 +ttp: b274/782 bl:2.3111 bb:0.9728 rl:2.3369 rb:0.9746 dl:384-385 gd:1 +ttp: b271/782 bl:2.3834 bb:1.0265 rl:2.3371 rb:0.9748 dl:380-382 gd:1 +ttp: b263/782 bl:2.3967 bb:0.9904 rl:2.3373 rb:0.9749 dl:370-371 gd:1 +ttp: b255/782 bl:2.3697 bb:0.9976 rl:2.3375 rb:0.9750 dl:360-361 gd:1 +ttp: b247/782 bl:2.3561 bb:0.9974 rl:2.3375 rb:0.9751 dl:350-351 gd:1 +ttp: b239/782 bl:2.3932 bb:1.0073 rl:2.3377 rb:0.9752 dl:340-341 gd:1 +ttp: b226/782 bl:2.3775 bb:1.0071 rl:2.3379 rb:0.9753 dl:324-325 gd:1 +ttp: b218/782 bl:2.4657 bb:1.0214 rl:2.3383 rb:0.9755 dl:315-316 gd:1 +ttp: b210/782 bl:2.2702 bb:0.9797 rl:2.3381 rb:0.9755 dl:306-307 gd:1 +ttp: b203/782 bl:2.4381 bb:1.0191 rl:2.3384 rb:0.9756 dl:299-300 gd:1 +ttp: b195/782 bl:2.4292 bb:1.0287 rl:2.3387 rb:0.9758 dl:290-291 gd:1 +ttp: b186/782 bl:2.4303 bb:1.0293 rl:2.3390 rb:0.9760 dl:280-281 gd:1 +ttp: b178/782 bl:2.3552 bb:1.0034 rl:2.3391 rb:0.9761 dl:272-273 gd:1 +ttp: b170/782 bl:2.3808 bb:1.0190 rl:2.3392 rb:0.9762 dl:264-265 gd:1 +ttp: b162/782 bl:2.4177 bb:1.0214 rl:2.3394 rb:0.9763 dl:256-257 gd:1 +ttp: b154/782 bl:2.4770 bb:1.0900 rl:2.3398 rb:0.9766 dl:249-250 gd:1 +ttp: b146/782 bl:2.4662 bb:1.0662 rl:2.3401 rb:0.9768 dl:241-242 gd:1 +ttp: b136/782 bl:2.4315 bb:1.0432 rl:2.3403 rb:0.9770 dl:232-233 gd:1 +ttp: b129/782 bl:2.3914 bb:1.0326 rl:2.3405 rb:0.9771 dl:225-226 gd:1 +ttp: b122/782 bl:2.4194 bb:1.0392 rl:2.3406 rb:0.9773 dl:219-219 gd:1 +ttp: b113/782 bl:2.5625 bb:1.0504 rl:2.3412 rb:0.9774 dl:210-211 gd:1 +ttp: b105/782 bl:2.4122 bb:1.0372 rl:2.3413 rb:0.9775 dl:203-204 gd:1 +ttp: b98/782 bl:2.6104 bb:1.1175 rl:2.3419 rb:0.9778 dl:197-198 gd:1 +ttp: b89/782 bl:2.4931 bb:1.0531 rl:2.3422 rb:0.9780 dl:189-190 gd:1 +ttp: b81/782 bl:2.4848 bb:1.0310 rl:2.3425 rb:0.9781 dl:182-183 gd:1 +ttp: b74/782 bl:2.4900 bb:1.0566 rl:2.3427 rb:0.9782 dl:175-176 gd:1 +ttp: b65/782 bl:2.4740 bb:1.0568 rl:2.3430 rb:0.9784 dl:167-169 gd:1 +ttp: b60/782 bl:2.4801 bb:1.0728 rl:2.3432 rb:0.9785 dl:163-164 gd:1 +ttp: b52/782 bl:2.6943 bb:1.1404 rl:2.3438 rb:0.9788 dl:155-156 gd:1 +ttp: b42/782 bl:2.4750 bb:1.0840 rl:2.3440 rb:0.9789 dl:145-146 gd:1 +ttp: b33/782 bl:2.5969 bb:1.1088 rl:2.3443 rb:0.9791 dl:136-137 gd:1 +ttp: b24/782 bl:2.4679 bb:1.0539 rl:2.3445 rb:0.9792 dl:127-128 gd:1 +ttp: b16/782 bl:2.6452 bb:1.1457 rl:2.3449 rb:0.9794 dl:117-118 gd:1 +ttp: b8/782 bl:2.8177 bb:1.1923 rl:2.3454 rb:0.9796 dl:103-105 gd:1 +quantized_ttt_phased val_loss:2.32949800 val_bpb:0.97705175 eval_time:586485ms +total_eval_time:586.5s diff --git a/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed999.log b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed999.log new file mode 100644 index 0000000000..895f9f67df --- /dev/null +++ b/records/track_10min_16mb/2026-04-22_SP8192_CaseOps_V13_L2_LoRA_TTT/train_seed999.log @@ -0,0 +1,829 @@ +W0422 10:31:46.847000 139940 torch/distributed/run.py:803] +W0422 10:31:46.847000 139940 torch/distributed/run.py:803] ***************************************** +W0422 10:31:46.847000 139940 torch/distributed/run.py:803] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. +W0422 10:31:46.847000 139940 torch/distributed/run.py:803] ***************************************** +Hyperparameters: + adam_eps: 1e-08 + adam_wd: 0.02 + artifact_dir: + attn_clip_sigmas: 13.0 + attn_out_gate_enabled: False + attn_out_gate_src: proj + beta1: 0.9 + beta2: 0.95 + caseops_enabled: False + compressor: brotli + data_dir: ./data/ + datasets_dir: ./data/datasets/fineweb10B_sp8192 + distributed: True + ema_decay: 0.9965 + embed_bits: 7 + embed_clip_sigmas: 15.0 + embed_lr: 0.6 + embed_wd: 0.085 + enable_looping_at: 0.35 + eval_loop_depth: 4 + eval_seq_len: 2048 + eval_stride: 64 + gate_window: 12 + gated_attn_enabled: True + gated_attn_init_std: 0.005 + gated_attn_quant_gate: True + global_ttt_batch_seqs: 32 + global_ttt_chunk_tokens: 32768 + global_ttt_epochs: 1 + global_ttt_grad_clip: 1.0 + global_ttt_lr: 0.001 + global_ttt_momentum: 0.9 + global_ttt_respect_doc_boundaries: True + global_ttt_warmup_chunks: 0 + global_ttt_warmup_start_lr: 0.0 + gptq_calibration_batches: 16 + gptq_reserve_seconds: 4.0 + grad_accum_steps: 1 + grad_clip_norm: 0.3 + is_main_process: True + iterations: 20000 + ln_scale: True + local_rank: 0 + logfile: logs/7a47dc1f-fd32-4ea2-80a9-878af21efe8c.txt + logit_softcap: 30.0 + loop_end: 5 + loop_start: 3 + matrix_bits: 6 + matrix_clip_sigmas: 12.85 + matrix_lr: 0.026 + max_wallclock_seconds: 600.0 + min_lr: 0.0 + mlp_clip_sigmas: 12.0 + mlp_mult: 4.0 + model_dim: 512 + model_path: final_model.pt + muon_backend_steps: 5 + muon_momentum: 0.97 + muon_momentum_warmup_start: 0.92 + muon_momentum_warmup_steps: 1500 + muon_row_normalize: True + muon_wd: 0.095 + num_heads: 8 + num_kv_heads: 4 + num_layers: 11 + num_loops: 2 + parallel_final_lane: mean + parallel_start_layer: 8 + phased_ttt_num_phases: 3 + phased_ttt_prefix_docs: 2000 + qk_gain_init: 5.0 + quantized_model_path: final_model.int6.ptz + rank: 0 + rope_base: 10000.0 + rope_dims: 16 + rope_train_seq_len: 2048 + rope_yarn: False + run_id: 7a47dc1f-fd32-4ea2-80a9-878af21efe8c + scalar_lr: 0.02 + seed: 999 + skip_gates_enabled: True + smear_gate_enabled: True + tie_embeddings: True + tied_embed_init_std: 0.005 + tied_embed_lr: 0.03 + tokenizer_path: ./data/tokenizers/fineweb_8192_bpe.model + train_batch_tokens: 786432 + train_files: ./data/datasets/fineweb10B_sp8192/fineweb_train_*.bin + train_log_every: 500 + train_loop_depth_dist: fixed + train_loop_depth_set: [] + train_loop_max_depth: 3 + train_loop_min_depth: 3 + train_loop_phase_depths: [1, 3, 4] + train_loop_phase_fractions: [] + train_loop_prewarm_depths: [3, 4] + train_seq_len: 2048 + ttt_batch_size: 64 + ttt_beta1: 0.0 + ttt_beta2: 0.999 + ttt_chunk_size: 48 + ttt_enabled: True + ttt_eval_batches: + ttt_eval_seq_len: 2048 + ttt_grad_steps: 1 + ttt_k_lora: True + ttt_lora_alpha: 144.0 + ttt_lora_lr: 0.0001 + ttt_lora_rank: 96 + ttt_mlp_lora: True + ttt_o_lora: True + ttt_optimizer: adam + ttt_warm_start_a: True + ttt_weight_decay: 1.0 + val_batch_tokens: 524288 + val_bytes_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_bytes_*.bin + val_doc_fraction: 1.0 + val_files: ./data/datasets/fineweb10B_sp8192/fineweb_val_[0-9][0-9][0-9][0-9][0-9][0-9].bin + val_loss_every: 4000 + vocab_size: 8192 + warmdown_frac: 0.75 + warmup_steps: 20 + world_size: 8 + xsa_last_n: 11 +train_shards: 80 +val_tokens: 47851520 +model_params:35989671 +gptq:reserving 4s, effective=596000ms +loop_depth_schedule: train=[1, 3, 4] dist=phased phase_fracs=None prewarm=[3, 4] eval_depth=4 +warmup_cu_buckets:64,128,192,256 iters_each:3 +warmup_step: 1/20 +warmup_step: 2/20 +warmup_step: 3/20 +warmup_step: 4/20 +warmup_step: 5/20 +warmup_step: 6/20 +warmup_step: 10/20 +warmup_step: 20/20 +0/20000 val_loss: 8.9974 val_bpb: 3.7736 +1/20000 train_loss: 8.9984 train_time: 0.0m tok/s: 12181919 +2/20000 train_loss: 12.9141 train_time: 0.0m tok/s: 9188989 +3/20000 train_loss: 10.2566 train_time: 0.0m tok/s: 8558847 +4/20000 train_loss: 8.7368 train_time: 0.0m tok/s: 8163186 +5/20000 train_loss: 7.9172 train_time: 0.0m tok/s: 8014646 +500/20000 train_loss: 2.5817 train_time: 0.8m tok/s: 7991097 +1000/20000 train_loss: 2.8052 train_time: 1.6m tok/s: 7961084 +1500/20000 train_loss: 2.6425 train_time: 2.5m tok/s: 7941294 +2000/20000 train_loss: 2.6702 train_time: 3.3m tok/s: 7935814 +layer_loop:phase step:2005 frac:0.334 depth:3 phases:[1, 3, 4] eval_depth:4 +2500/20000 train_loss: 2.5426 train_time: 4.5m tok/s: 7270130 +3000/20000 train_loss: 2.5592 train_time: 5.7m tok/s: 6883910 +layer_loop:phase step:3378 frac:0.667 depth:4 phases:[1, 3, 4] eval_depth:4 +3500/20000 train_loss: 2.5606 train_time: 7.0m tok/s: 6589658 +4000/20000 train_loss: 2.3860 train_time: 8.4m tok/s: 6278697 +4000/20000 val_loss: 2.4111 val_bpb: 1.0112 +4500/20000 train_loss: 2.2429 train_time: 9.7m tok/s: 6054906 +4570/20000 val_loss: 2.3409 val_bpb: 0.9818 +stopping_early: wallclock_cap train_time: 596014ms step: 4570/20000 +peak memory allocated: 46670 MiB reserved: 50440 MiB +ema:applying EMA weights +diagnostic pre-quantization post-ema val_loss:2.34060593 val_bpb:0.98166417 eval_time:7218ms +Serialized model: 135593533 bytes +Code size (uncompressed): 135003 bytes +Code size (compressed): 29930 bytes +GPTQ:collecting Hessians from calibration data... +GPTQ:collected 67 Hessians in 4.1s +Quantized weights: + gate_int8_row: blocks.attn.attn_gate_w + gptq (int6): blocks.attn.c_k.weight, blocks.attn.c_q.weight, blocks.attn.c_v.weight, blocks.attn.proj.weight, blocks.mlp.fc.weight, blocks.mlp.proj.weight + gptq (int7): tok_emb.weight + passthrough (float16): blocks.attn.q_gain, blocks.attn_scale, blocks.mlp_scale, blocks.resid_mix, parallel_post_lambdas, parallel_resid_lambdas, skip_gates, skip_weights, smear_gate.weight, smear_lambda +Serialized model quantized+brotli: 15946763 bytes +Total submission size quantized+brotli: 15976693 bytes +diagnostic quantized val_loss:2.36051854 val_bpb:0.99001564 eval_time:13088ms +ttt_lora:warming up compile (random tokens, no val data) +ttt_lora:compile warmup done (107.2s) + +beginning TTT eval timer +ttt_phased: total_docs:50000 prefix_docs:2000 suffix_docs:48000 num_phases:3 boundaries:[666, 1333, 2000] +ttp: b781/782 bl:2.1617 bb:0.9673 rl:2.1617 rb:0.9673 dl:17258-30330 gd:0 +ttpp: phase:1/3 pd:1104 gd:666 t:224.9s +tttg: c1/111 lr:0.001000 t:0.3s +tttg: c2/111 lr:0.001000 t:0.4s +tttg: c3/111 lr:0.000999 t:0.5s +tttg: c4/111 lr:0.000998 t:0.7s +tttg: c5/111 lr:0.000997 t:0.8s +tttg: c6/111 lr:0.000995 t:0.9s +tttg: c7/111 lr:0.000993 t:1.0s +tttg: c8/111 lr:0.000990 t:1.1s +tttg: c9/111 lr:0.000987 t:1.2s +tttg: c10/111 lr:0.000984 t:1.3s +tttg: c11/111 lr:0.000980 t:1.4s +tttg: c12/111 lr:0.000976 t:1.5s +tttg: c13/111 lr:0.000971 t:1.6s +tttg: c14/111 lr:0.000966 t:1.7s +tttg: c15/111 lr:0.000961 t:1.8s +tttg: c16/111 lr:0.000955 t:1.9s +tttg: c17/111 lr:0.000949 t:2.1s +tttg: c18/111 lr:0.000942 t:2.2s +tttg: c19/111 lr:0.000935 t:2.3s +tttg: c20/111 lr:0.000928 t:2.4s +tttg: c21/111 lr:0.000921 t:2.5s +tttg: c22/111 lr:0.000913 t:2.6s +tttg: c23/111 lr:0.000905 t:2.7s +tttg: c24/111 lr:0.000896 t:2.8s +tttg: c25/111 lr:0.000887 t:2.9s +tttg: c26/111 lr:0.000878 t:3.0s +tttg: c27/111 lr:0.000868 t:3.2s +tttg: c28/111 lr:0.000859 t:3.3s +tttg: c29/111 lr:0.000848 t:3.4s +tttg: c30/111 lr:0.000838 t:3.5s +tttg: c31/111 lr:0.000827 t:3.6s +tttg: c32/111 lr:0.000817 t:3.7s +tttg: c33/111 lr:0.000805 t:3.8s +tttg: c34/111 lr:0.000794 t:3.9s +tttg: c35/111 lr:0.000782 t:4.0s +tttg: c36/111 lr:0.000770 t:4.1s +tttg: c37/111 lr:0.000758 t:4.3s +tttg: c38/111 lr:0.000746 t:4.4s +tttg: c39/111 lr:0.000733 t:4.5s +tttg: c40/111 lr:0.000721 t:4.6s +tttg: c41/111 lr:0.000708 t:4.7s +tttg: c42/111 lr:0.000695 t:4.8s +tttg: c43/111 lr:0.000681 t:4.9s +tttg: c44/111 lr:0.000668 t:5.0s +tttg: c45/111 lr:0.000655 t:5.1s +tttg: c46/111 lr:0.000641 t:5.3s +tttg: c47/111 lr:0.000627 t:5.4s +tttg: c48/111 lr:0.000613 t:5.5s +tttg: c49/111 lr:0.000599 t:5.6s +tttg: c50/111 lr:0.000585 t:5.7s +tttg: c51/111 lr:0.000571 t:5.8s +tttg: c52/111 lr:0.000557 t:5.9s +tttg: c53/111 lr:0.000543 t:6.0s +tttg: c54/111 lr:0.000529 t:6.2s +tttg: c55/111 lr:0.000514 t:6.3s +tttg: c56/111 lr:0.000500 t:6.4s +tttg: c57/111 lr:0.000486 t:6.5s +tttg: c58/111 lr:0.000471 t:6.6s +tttg: c59/111 lr:0.000457 t:6.8s +tttg: c60/111 lr:0.000443 t:6.9s +tttg: c61/111 lr:0.000429 t:7.0s +tttg: c62/111 lr:0.000415 t:7.1s +tttg: c63/111 lr:0.000401 t:7.2s +tttg: c64/111 lr:0.000387 t:7.3s +tttg: c65/111 lr:0.000373 t:7.5s +tttg: c66/111 lr:0.000359 t:7.6s +tttg: c67/111 lr:0.000345 t:7.7s +tttg: c68/111 lr:0.000332 t:7.8s +tttg: c69/111 lr:0.000319 t:7.9s +tttg: c70/111 lr:0.000305 t:8.0s +tttg: c71/111 lr:0.000292 t:8.1s +tttg: c72/111 lr:0.000279 t:8.2s +tttg: c73/111 lr:0.000267 t:8.4s +tttg: c74/111 lr:0.000254 t:8.5s +tttg: c75/111 lr:0.000242 t:8.7s +tttg: c76/111 lr:0.000230 t:8.8s +tttg: c77/111 lr:0.000218 t:8.9s +tttg: c78/111 lr:0.000206 t:9.0s +tttg: c79/111 lr:0.000195 t:9.1s +tttg: c80/111 lr:0.000183 t:9.2s +tttg: c81/111 lr:0.000173 t:9.4s +tttg: c82/111 lr:0.000162 t:9.5s +tttg: c83/111 lr:0.000152 t:9.6s +tttg: c84/111 lr:0.000141 t:9.7s +tttg: c85/111 lr:0.000132 t:9.8s +tttg: c86/111 lr:0.000122 t:9.9s +tttg: c87/111 lr:0.000113 t:10.0s +tttg: c88/111 lr:0.000104 t:10.2s +tttg: c89/111 lr:0.000095 t:10.4s +tttg: c90/111 lr:0.000087 t:10.5s +tttg: c91/111 lr:0.000079 t:10.6s +tttg: c92/111 lr:0.000072 t:10.7s +tttg: c93/111 lr:0.000065 t:10.8s +tttg: c94/111 lr:0.000058 t:10.9s +tttg: c95/111 lr:0.000051 t:11.1s +tttg: c96/111 lr:0.000045 t:11.2s +tttg: c97/111 lr:0.000039 t:11.3s +tttg: c98/111 lr:0.000034 t:11.4s +tttg: c99/111 lr:0.000029 t:11.5s +tttg: c100/111 lr:0.000024 t:11.6s +tttg: c101/111 lr:0.000020 t:11.8s +tttg: c102/111 lr:0.000016 t:11.9s +tttg: c103/111 lr:0.000013 t:12.0s +tttg: c104/111 lr:0.000010 t:12.1s +tttg: c105/111 lr:0.000007 t:12.2s +tttg: c106/111 lr:0.000005 t:12.3s +tttg: c107/111 lr:0.000003 t:12.4s +tttg: c108/111 lr:0.000002 t:12.5s +tttg: c109/111 lr:0.000001 t:12.7s +tttg: c110/111 lr:0.000000 t:12.8s +ttpr: phase:1/3 t:239.8s +ttp: b757/782 bl:2.2897 bb:0.9796 rl:2.1796 rb:0.9691 dl:3550-3633 gd:0 +ttpp: phase:2/3 pd:1808 gd:1333 t:314.6s +tttg: c1/185 lr:0.001000 t:0.1s +tttg: c2/185 lr:0.001000 t:0.2s +tttg: c3/185 lr:0.001000 t:0.3s +tttg: c4/185 lr:0.000999 t:0.4s +tttg: c5/185 lr:0.000999 t:0.5s +tttg: c6/185 lr:0.000998 t:0.6s +tttg: c7/185 lr:0.000997 t:0.7s +tttg: c8/185 lr:0.000996 t:0.8s +tttg: c9/185 lr:0.000995 t:0.9s +tttg: c10/185 lr:0.000994 t:1.0s +tttg: c11/185 lr:0.000993 t:1.1s +tttg: c12/185 lr:0.000991 t:1.2s +tttg: c13/185 lr:0.000990 t:1.4s +tttg: c14/185 lr:0.000988 t:1.5s +tttg: c15/185 lr:0.000986 t:1.6s +tttg: c16/185 lr:0.000984 t:1.7s +tttg: c17/185 lr:0.000981 t:1.8s +tttg: c18/185 lr:0.000979 t:1.9s +tttg: c19/185 lr:0.000977 t:2.1s +tttg: c20/185 lr:0.000974 t:2.2s +tttg: c21/185 lr:0.000971 t:2.3s +tttg: c22/185 lr:0.000968 t:2.4s +tttg: c23/185 lr:0.000965 t:2.5s +tttg: c24/185 lr:0.000962 t:2.6s +tttg: c25/185 lr:0.000959 t:2.7s +tttg: c26/185 lr:0.000955 t:2.9s +tttg: c27/185 lr:0.000952 t:3.0s +tttg: c28/185 lr:0.000948 t:3.1s +tttg: c29/185 lr:0.000944 t:3.2s +tttg: c30/185 lr:0.000940 t:3.3s +tttg: c31/185 lr:0.000936 t:3.4s +tttg: c32/185 lr:0.000932 t:3.5s +tttg: c33/185 lr:0.000927 t:3.6s +tttg: c34/185 lr:0.000923 t:3.8s +tttg: c35/185 lr:0.000918 t:3.9s +tttg: c36/185 lr:0.000913 t:4.0s +tttg: c37/185 lr:0.000908 t:4.1s +tttg: c38/185 lr:0.000904 t:4.2s +tttg: c39/185 lr:0.000898 t:4.3s +tttg: c40/185 lr:0.000893 t:4.5s +tttg: c41/185 lr:0.000888 t:4.6s +tttg: c42/185 lr:0.000882 t:4.7s +tttg: c43/185 lr:0.000877 t:4.8s +tttg: c44/185 lr:0.000871 t:4.9s +tttg: c45/185 lr:0.000865 t:5.0s +tttg: c46/185 lr:0.000860 t:5.1s +tttg: c47/185 lr:0.000854 t:5.2s +tttg: c48/185 lr:0.000847 t:5.4s +tttg: c49/185 lr:0.000841 t:5.5s +tttg: c50/185 lr:0.000835 t:5.6s +tttg: c51/185 lr:0.000829 t:5.7s +tttg: c52/185 lr:0.000822 t:5.8s +tttg: c53/185 lr:0.000816 t:5.9s +tttg: c54/185 lr:0.000809 t:6.1s +tttg: c55/185 lr:0.000802 t:6.2s +tttg: c56/185 lr:0.000795 t:6.3s +tttg: c57/185 lr:0.000788 t:6.4s +tttg: c58/185 lr:0.000781 t:6.5s +tttg: c59/185 lr:0.000774 t:6.6s +tttg: c60/185 lr:0.000767 t:6.7s +tttg: c61/185 lr:0.000760 t:6.9s +tttg: c62/185 lr:0.000752 t:7.0s +tttg: c63/185 lr:0.000745 t:7.1s +tttg: c64/185 lr:0.000738 t:7.2s +tttg: c65/185 lr:0.000730 t:7.3s +tttg: c66/185 lr:0.000722 t:7.4s +tttg: c67/185 lr:0.000715 t:7.5s +tttg: c68/185 lr:0.000707 t:7.6s +tttg: c69/185 lr:0.000699 t:7.7s +tttg: c70/185 lr:0.000691 t:7.9s +tttg: c71/185 lr:0.000683 t:8.0s +tttg: c72/185 lr:0.000675 t:8.1s +tttg: c73/185 lr:0.000667 t:8.2s +tttg: c74/185 lr:0.000659 t:8.3s +tttg: c75/185 lr:0.000651 t:8.4s +tttg: c76/185 lr:0.000643 t:8.5s +tttg: c77/185 lr:0.000635 t:8.6s +tttg: c78/185 lr:0.000627 t:8.8s +tttg: c79/185 lr:0.000618 t:8.9s +tttg: c80/185 lr:0.000610 t:9.0s +tttg: c81/185 lr:0.000602 t:9.1s +tttg: c82/185 lr:0.000593 t:9.2s +tttg: c83/185 lr:0.000585 t:9.3s +tttg: c84/185 lr:0.000577 t:9.4s +tttg: c85/185 lr:0.000568 t:9.5s +tttg: c86/185 lr:0.000560 t:9.7s +tttg: c87/185 lr:0.000551 t:9.8s +tttg: c88/185 lr:0.000543 t:9.9s +tttg: c89/185 lr:0.000534 t:10.0s +tttg: c90/185 lr:0.000526 t:10.1s +tttg: c91/185 lr:0.000517 t:10.2s +tttg: c92/185 lr:0.000509 t:10.3s +tttg: c93/185 lr:0.000500 t:10.4s +tttg: c94/185 lr:0.000491 t:10.6s +tttg: c95/185 lr:0.000483 t:10.7s +tttg: c96/185 lr:0.000474 t:10.8s +tttg: c97/185 lr:0.000466 t:10.9s +tttg: c98/185 lr:0.000457 t:11.0s +tttg: c99/185 lr:0.000449 t:11.1s +tttg: c100/185 lr:0.000440 t:11.2s +tttg: c101/185 lr:0.000432 t:11.4s +tttg: c102/185 lr:0.000423 t:11.5s +tttg: c103/185 lr:0.000415 t:11.6s +tttg: c104/185 lr:0.000407 t:11.7s +tttg: c105/185 lr:0.000398 t:11.8s +tttg: c106/185 lr:0.000390 t:11.9s +tttg: c107/185 lr:0.000382 t:12.0s +tttg: c108/185 lr:0.000373 t:12.1s +tttg: c109/185 lr:0.000365 t:12.3s +tttg: c110/185 lr:0.000357 t:12.4s +tttg: c111/185 lr:0.000349 t:12.5s +tttg: c112/185 lr:0.000341 t:12.6s +tttg: c113/185 lr:0.000333 t:12.7s +tttg: c114/185 lr:0.000325 t:12.8s +tttg: c115/185 lr:0.000317 t:13.0s +tttg: c116/185 lr:0.000309 t:13.1s +tttg: c117/185 lr:0.000301 t:13.2s +tttg: c118/185 lr:0.000293 t:13.3s +tttg: c119/185 lr:0.000285 t:13.4s +tttg: c120/185 lr:0.000278 t:13.5s +tttg: c121/185 lr:0.000270 t:13.6s +tttg: c122/185 lr:0.000262 t:13.7s +tttg: c123/185 lr:0.000255 t:13.8s +tttg: c124/185 lr:0.000248 t:14.0s +tttg: c125/185 lr:0.000240 t:14.1s +tttg: c126/185 lr:0.000233 t:14.2s +tttg: c127/185 lr:0.000226 t:14.3s +tttg: c128/185 lr:0.000219 t:14.4s +tttg: c129/185 lr:0.000212 t:14.5s +tttg: c130/185 lr:0.000205 t:14.6s +tttg: c131/185 lr:0.000198 t:14.7s +tttg: c132/185 lr:0.000191 t:14.8s +tttg: c133/185 lr:0.000184 t:15.0s +tttg: c134/185 lr:0.000178 t:15.1s +tttg: c135/185 lr:0.000171 t:15.2s +tttg: c136/185 lr:0.000165 t:15.3s +tttg: c137/185 lr:0.000159 t:15.4s +tttg: c138/185 lr:0.000153 t:15.5s +tttg: c139/185 lr:0.000146 t:15.7s +tttg: c140/185 lr:0.000140 t:15.8s +tttg: c141/185 lr:0.000135 t:15.9s +tttg: c142/185 lr:0.000129 t:16.0s +tttg: c143/185 lr:0.000123 t:16.1s +tttg: c144/185 lr:0.000118 t:16.2s +tttg: c145/185 lr:0.000112 t:16.3s +tttg: c146/185 lr:0.000107 t:16.5s +tttg: c147/185 lr:0.000102 t:16.6s +tttg: c148/185 lr:0.000096 t:16.7s +tttg: c149/185 lr:0.000092 t:16.8s +tttg: c150/185 lr:0.000087 t:16.9s +tttg: c151/185 lr:0.000082 t:17.0s +tttg: c152/185 lr:0.000077 t:17.1s +tttg: c153/185 lr:0.000073 t:17.2s +tttg: c154/185 lr:0.000068 t:17.3s +tttg: c155/185 lr:0.000064 t:17.5s +tttg: c156/185 lr:0.000060 t:17.6s +tttg: c157/185 lr:0.000056 t:17.7s +tttg: c158/185 lr:0.000052 t:17.8s +tttg: c159/185 lr:0.000048 t:17.9s +tttg: c160/185 lr:0.000045 t:18.0s +tttg: c161/185 lr:0.000041 t:18.1s +tttg: c162/185 lr:0.000038 t:18.2s +tttg: c163/185 lr:0.000035 t:18.4s +tttg: c164/185 lr:0.000032 t:18.5s +tttg: c165/185 lr:0.000029 t:18.6s +tttg: c166/185 lr:0.000026 t:18.7s +tttg: c167/185 lr:0.000023 t:18.8s +tttg: c168/185 lr:0.000021 t:18.9s +tttg: c169/185 lr:0.000019 t:19.0s +tttg: c170/185 lr:0.000016 t:19.1s +tttg: c171/185 lr:0.000014 t:19.2s +tttg: c172/185 lr:0.000012 t:19.4s +tttg: c173/185 lr:0.000010 t:19.5s +tttg: c174/185 lr:0.000009 t:19.6s +tttg: c175/185 lr:0.000007 t:19.7s +tttg: c176/185 lr:0.000006 t:19.8s +tttg: c177/185 lr:0.000005 t:20.0s +tttg: c178/185 lr:0.000004 t:20.1s +tttg: c179/185 lr:0.000003 t:20.2s +tttg: c180/185 lr:0.000002 t:20.3s +tttg: c181/185 lr:0.000001 t:20.4s +tttg: c182/185 lr:0.000001 t:20.5s +tttg: c183/185 lr:0.000000 t:20.6s +tttg: c184/185 lr:0.000000 t:20.7s +ttpr: phase:2/3 t:337.3s +ttp: b746/782 bl:2.4223 bb:0.9901 rl:2.2044 rb:0.9714 dl:2884-2943 gd:0 +ttp: b745/782 bl:2.2462 bb:0.9422 rl:2.2082 rb:0.9686 dl:2842-2883 gd:0 +ttpp: phase:3/3 pd:2448 gd:2000 t:357.4s +tttg: c1/250 lr:0.001000 t:0.1s +tttg: c2/250 lr:0.001000 t:0.2s +tttg: c3/250 lr:0.001000 t:0.3s +tttg: c4/250 lr:0.001000 t:0.4s +tttg: c5/250 lr:0.000999 t:0.5s +tttg: c6/250 lr:0.000999 t:0.6s +tttg: c7/250 lr:0.000999 t:0.9s +tttg: c8/250 lr:0.000998 t:1.0s +tttg: c9/250 lr:0.000997 t:1.1s +tttg: c10/250 lr:0.000997 t:1.2s +tttg: c11/250 lr:0.000996 t:1.3s +tttg: c12/250 lr:0.000995 t:1.5s +tttg: c13/250 lr:0.000994 t:1.6s +tttg: c14/250 lr:0.000993 t:1.7s +tttg: c15/250 lr:0.000992 t:1.8s +tttg: c16/250 lr:0.000991 t:2.0s +tttg: c17/250 lr:0.000990 t:2.1s +tttg: c18/250 lr:0.000989 t:2.2s +tttg: c19/250 lr:0.000987 t:2.3s +tttg: c20/250 lr:0.000986 t:2.4s +tttg: c21/250 lr:0.000984 t:2.5s +tttg: c22/250 lr:0.000983 t:2.7s +tttg: c23/250 lr:0.000981 t:2.8s +tttg: c24/250 lr:0.000979 t:2.9s +tttg: c25/250 lr:0.000977 t:3.0s +tttg: c26/250 lr:0.000975 t:3.1s +tttg: c27/250 lr:0.000973 t:3.3s +tttg: c28/250 lr:0.000971 t:3.4s +tttg: c29/250 lr:0.000969 t:3.5s +tttg: c30/250 lr:0.000967 t:3.6s +tttg: c31/250 lr:0.000965 t:3.7s +tttg: c32/250 lr:0.000962 t:3.8s +tttg: c33/250 lr:0.000960 t:4.0s +tttg: c34/250 lr:0.000957 t:4.1s +tttg: c35/250 lr:0.000955 t:4.2s +tttg: c36/250 lr:0.000952 t:4.3s +tttg: c37/250 lr:0.000949 t:4.4s +tttg: c38/250 lr:0.000947 t:4.5s +tttg: c39/250 lr:0.000944 t:4.7s +tttg: c40/250 lr:0.000941 t:4.8s +tttg: c41/250 lr:0.000938 t:4.9s +tttg: c42/250 lr:0.000935 t:5.0s +tttg: c43/250 lr:0.000931 t:5.1s +tttg: c44/250 lr:0.000928 t:5.2s +tttg: c45/250 lr:0.000925 t:5.4s +tttg: c46/250 lr:0.000922 t:5.5s +tttg: c47/250 lr:0.000918 t:5.6s +tttg: c48/250 lr:0.000915 t:5.7s +tttg: c49/250 lr:0.000911 t:5.8s +tttg: c50/250 lr:0.000907 t:6.0s +tttg: c51/250 lr:0.000904 t:6.1s +tttg: c52/250 lr:0.000900 t:6.2s +tttg: c53/250 lr:0.000896 t:6.4s +tttg: c54/250 lr:0.000892 t:6.5s +tttg: c55/250 lr:0.000888 t:6.6s +tttg: c56/250 lr:0.000884 t:6.7s +tttg: c57/250 lr:0.000880 t:6.8s +tttg: c58/250 lr:0.000876 t:7.0s +tttg: c59/250 lr:0.000872 t:7.1s +tttg: c60/250 lr:0.000868 t:7.2s +tttg: c61/250 lr:0.000863 t:7.3s +tttg: c62/250 lr:0.000859 t:7.4s +tttg: c63/250 lr:0.000855 t:7.6s +tttg: c64/250 lr:0.000850 t:7.7s +tttg: c65/250 lr:0.000846 t:7.8s +tttg: c66/250 lr:0.000841 t:7.9s +tttg: c67/250 lr:0.000836 t:8.0s +tttg: c68/250 lr:0.000832 t:8.1s +tttg: c69/250 lr:0.000827 t:8.2s +tttg: c70/250 lr:0.000822 t:8.4s +tttg: c71/250 lr:0.000817 t:8.5s +tttg: c72/250 lr:0.000812 t:8.6s +tttg: c73/250 lr:0.000807 t:8.7s +tttg: c74/250 lr:0.000803 t:8.8s +tttg: c75/250 lr:0.000797 t:9.0s +tttg: c76/250 lr:0.000792 t:9.1s +tttg: c77/250 lr:0.000787 t:9.2s +tttg: c78/250 lr:0.000782 t:9.3s +tttg: c79/250 lr:0.000777 t:9.4s +tttg: c80/250 lr:0.000772 t:9.5s +tttg: c81/250 lr:0.000766 t:9.7s +tttg: c82/250 lr:0.000761 t:9.8s +tttg: c83/250 lr:0.000755 t:9.9s +tttg: c84/250 lr:0.000750 t:10.0s +tttg: c85/250 lr:0.000745 t:10.1s +tttg: c86/250 lr:0.000739 t:10.2s +tttg: c87/250 lr:0.000733 t:10.3s +tttg: c88/250 lr:0.000728 t:10.5s +tttg: c89/250 lr:0.000722 t:10.6s +tttg: c90/250 lr:0.000717 t:10.7s +tttg: c91/250 lr:0.000711 t:10.8s +tttg: c92/250 lr:0.000705 t:10.9s +tttg: c93/250 lr:0.000699 t:11.1s +tttg: c94/250 lr:0.000694 t:11.2s +tttg: c95/250 lr:0.000688 t:11.3s +tttg: c96/250 lr:0.000682 t:11.4s +tttg: c97/250 lr:0.000676 t:11.6s +tttg: c98/250 lr:0.000670 t:11.7s +tttg: c99/250 lr:0.000664 t:11.8s +tttg: c100/250 lr:0.000658 t:11.9s +tttg: c101/250 lr:0.000652 t:12.0s +tttg: c102/250 lr:0.000646 t:12.1s +tttg: c103/250 lr:0.000640 t:12.3s +tttg: c104/250 lr:0.000634 t:12.4s +tttg: c105/250 lr:0.000628 t:12.5s +tttg: c106/250 lr:0.000622 t:12.6s +tttg: c107/250 lr:0.000616 t:12.7s +tttg: c108/250 lr:0.000610 t:12.9s +tttg: c109/250 lr:0.000603 t:13.0s +tttg: c110/250 lr:0.000597 t:13.1s +tttg: c111/250 lr:0.000591 t:13.2s +tttg: c112/250 lr:0.000585 t:13.3s +tttg: c113/250 lr:0.000579 t:13.5s +tttg: c114/250 lr:0.000572 t:13.6s +tttg: c115/250 lr:0.000566 t:13.7s +tttg: c116/250 lr:0.000560 t:13.8s +tttg: c117/250 lr:0.000554 t:13.9s +tttg: c118/250 lr:0.000547 t:14.0s +tttg: c119/250 lr:0.000541 t:14.1s +tttg: c120/250 lr:0.000535 t:14.3s +tttg: c121/250 lr:0.000528 t:14.4s +tttg: c122/250 lr:0.000522 t:14.5s +tttg: c123/250 lr:0.000516 t:14.6s +tttg: c124/250 lr:0.000509 t:14.7s +tttg: c125/250 lr:0.000503 t:14.9s +tttg: c126/250 lr:0.000497 t:15.0s +tttg: c127/250 lr:0.000491 t:15.1s +tttg: c128/250 lr:0.000484 t:15.2s +tttg: c129/250 lr:0.000478 t:15.4s +tttg: c130/250 lr:0.000472 t:15.5s +tttg: c131/250 lr:0.000465 t:15.6s +tttg: c132/250 lr:0.000459 t:15.7s +tttg: c133/250 lr:0.000453 t:15.8s +tttg: c134/250 lr:0.000446 t:15.9s +tttg: c135/250 lr:0.000440 t:16.1s +tttg: c136/250 lr:0.000434 t:16.2s +tttg: c137/250 lr:0.000428 t:16.3s +tttg: c138/250 lr:0.000421 t:16.5s +tttg: c139/250 lr:0.000415 t:16.6s +tttg: c140/250 lr:0.000409 t:16.7s +tttg: c141/250 lr:0.000403 t:16.8s +tttg: c142/250 lr:0.000397 t:16.9s +tttg: c143/250 lr:0.000390 t:17.1s +tttg: c144/250 lr:0.000384 t:17.2s +tttg: c145/250 lr:0.000378 t:17.3s +tttg: c146/250 lr:0.000372 t:17.4s +tttg: c147/250 lr:0.000366 t:17.6s +tttg: c148/250 lr:0.000360 t:17.7s +tttg: c149/250 lr:0.000354 t:17.8s +tttg: c150/250 lr:0.000348 t:17.9s +tttg: c151/250 lr:0.000342 t:18.0s +tttg: c152/250 lr:0.000336 t:18.2s +tttg: c153/250 lr:0.000330 t:18.3s +tttg: c154/250 lr:0.000324 t:18.4s +tttg: c155/250 lr:0.000318 t:18.5s +tttg: c156/250 lr:0.000312 t:18.6s +tttg: c157/250 lr:0.000306 t:18.7s +tttg: c158/250 lr:0.000301 t:18.9s +tttg: c159/250 lr:0.000295 t:19.0s +tttg: c160/250 lr:0.000289 t:19.1s +tttg: c161/250 lr:0.000283 t:19.2s +tttg: c162/250 lr:0.000278 t:19.3s +tttg: c163/250 lr:0.000272 t:19.5s +tttg: c164/250 lr:0.000267 t:19.6s +tttg: c165/250 lr:0.000261 t:19.7s +tttg: c166/250 lr:0.000255 t:19.8s +tttg: c167/250 lr:0.000250 t:19.9s +tttg: c168/250 lr:0.000245 t:20.1s +tttg: c169/250 lr:0.000239 t:20.2s +tttg: c170/250 lr:0.000234 t:20.3s +tttg: c171/250 lr:0.000228 t:20.4s +tttg: c172/250 lr:0.000223 t:20.5s +tttg: c173/250 lr:0.000218 t:20.6s +tttg: c174/250 lr:0.000213 t:20.8s +tttg: c175/250 lr:0.000208 t:20.9s +tttg: c176/250 lr:0.000203 t:21.0s +tttg: c177/250 lr:0.000197 t:21.1s +tttg: c178/250 lr:0.000193 t:21.2s +tttg: c179/250 lr:0.000188 t:21.4s +tttg: c180/250 lr:0.000183 t:21.5s +tttg: c181/250 lr:0.000178 t:21.6s +tttg: c182/250 lr:0.000173 t:21.7s +tttg: c183/250 lr:0.000168 t:21.8s +tttg: c184/250 lr:0.000164 t:21.9s +tttg: c185/250 lr:0.000159 t:22.1s +tttg: c186/250 lr:0.000154 t:22.2s +tttg: c187/250 lr:0.000150 t:22.3s +tttg: c188/250 lr:0.000145 t:22.4s +tttg: c189/250 lr:0.000141 t:22.5s +tttg: c190/250 lr:0.000137 t:22.6s +tttg: c191/250 lr:0.000132 t:22.7s +tttg: c192/250 lr:0.000128 t:22.9s +tttg: c193/250 lr:0.000124 t:23.0s +tttg: c194/250 lr:0.000120 t:23.1s +tttg: c195/250 lr:0.000116 t:23.2s +tttg: c196/250 lr:0.000112 t:23.3s +tttg: c197/250 lr:0.000108 t:23.4s +tttg: c198/250 lr:0.000104 t:23.6s +tttg: c199/250 lr:0.000100 t:23.7s +tttg: c200/250 lr:0.000096 t:23.8s +tttg: c201/250 lr:0.000093 t:23.9s +tttg: c202/250 lr:0.000089 t:24.0s +tttg: c203/250 lr:0.000085 t:24.2s +tttg: c204/250 lr:0.000082 t:24.3s +tttg: c205/250 lr:0.000078 t:24.4s +tttg: c206/250 lr:0.000075 t:24.5s +tttg: c207/250 lr:0.000072 t:24.6s +tttg: c208/250 lr:0.000069 t:24.7s +tttg: c209/250 lr:0.000065 t:24.9s +tttg: c210/250 lr:0.000062 t:25.0s +tttg: c211/250 lr:0.000059 t:25.1s +tttg: c212/250 lr:0.000056 t:25.2s +tttg: c213/250 lr:0.000053 t:25.3s +tttg: c214/250 lr:0.000051 t:25.4s +tttg: c215/250 lr:0.000048 t:25.6s +tttg: c216/250 lr:0.000045 t:25.7s +tttg: c217/250 lr:0.000043 t:25.8s +tttg: c218/250 lr:0.000040 t:25.9s +tttg: c219/250 lr:0.000038 t:26.0s +tttg: c220/250 lr:0.000035 t:26.1s +tttg: c221/250 lr:0.000033 t:26.3s +tttg: c222/250 lr:0.000031 t:26.4s +tttg: c223/250 lr:0.000029 t:26.5s +tttg: c224/250 lr:0.000027 t:26.6s +tttg: c225/250 lr:0.000025 t:26.7s +tttg: c226/250 lr:0.000023 t:26.8s +tttg: c227/250 lr:0.000021 t:27.0s +tttg: c228/250 lr:0.000019 t:27.1s +tttg: c229/250 lr:0.000017 t:27.2s +tttg: c230/250 lr:0.000016 t:27.3s +tttg: c231/250 lr:0.000014 t:27.4s +tttg: c232/250 lr:0.000013 t:27.5s +tttg: c233/250 lr:0.000011 t:27.7s +tttg: c234/250 lr:0.000010 t:27.8s +tttg: c235/250 lr:0.000009 t:27.9s +tttg: c236/250 lr:0.000008 t:28.0s +tttg: c237/250 lr:0.000007 t:28.1s +tttg: c238/250 lr:0.000006 t:28.2s +tttg: c239/250 lr:0.000005 t:28.3s +tttg: c240/250 lr:0.000004 t:28.5s +tttg: c241/250 lr:0.000003 t:28.6s +tttg: c242/250 lr:0.000003 t:28.7s +tttg: c243/250 lr:0.000002 t:28.8s +tttg: c244/250 lr:0.000001 t:28.9s +tttg: c245/250 lr:0.000001 t:29.1s +tttg: c246/250 lr:0.000001 t:29.2s +tttg: c247/250 lr:0.000000 t:29.3s +tttg: c248/250 lr:0.000000 t:29.4s +tttg: c249/250 lr:0.000000 t:29.5s +ttpr: phase:3/3 t:390.1s +ttp: b736/782 bl:2.2552 bb:0.9728 rl:2.2117 rb:0.9689 dl:2526-2550 gd:1 +ttp: b734/782 bl:2.2760 bb:0.9473 rl:2.2161 rb:0.9674 dl:2469-2495 gd:1 +ttp: b727/782 bl:2.2755 bb:0.9618 rl:2.2196 rb:0.9671 dl:2277-2305 gd:1 +ttp: b713/782 bl:2.2640 bb:0.9330 rl:2.2218 rb:0.9653 dl:2002-2017 gd:1 +ttp: b710/782 bl:2.2359 bb:0.9534 rl:2.2224 rb:0.9647 dl:1952-1966 gd:1 +ttp: b702/782 bl:2.4376 bb:1.0047 rl:2.2314 rb:0.9665 dl:1847-1858 gd:1 +ttp: b691/782 bl:2.4593 bb:0.9897 rl:2.2399 rb:0.9674 dl:1725-1737 gd:1 +ttp: b682/782 bl:2.3556 bb:0.9781 rl:2.2439 rb:0.9678 dl:1638-1646 gd:1 +ttp: b675/782 bl:2.3734 bb:0.9807 rl:2.2480 rb:0.9682 dl:1578-1586 gd:1 +ttp: b665/782 bl:2.3431 bb:0.9701 rl:2.2508 rb:0.9683 dl:1500-1507 gd:1 +ttp: b658/782 bl:2.2671 bb:0.9447 rl:2.2513 rb:0.9676 dl:1452-1459 gd:1 +ttp: b649/782 bl:2.2970 bb:0.9442 rl:2.2524 rb:0.9670 dl:1392-1398 gd:1 +ttp: b641/782 bl:2.3010 bb:0.9462 rl:2.2536 rb:0.9664 dl:1343-1349 gd:1 +ttp: b633/782 bl:2.2893 bb:0.9471 rl:2.2544 rb:0.9660 dl:1297-1302 gd:1 +ttp: b625/782 bl:2.4169 bb:0.9757 rl:2.2580 rb:0.9662 dl:1255-1260 gd:1 +ttp: b617/782 bl:2.3227 bb:0.9425 rl:2.2593 rb:0.9657 dl:1211-1216 gd:1 +ttp: b609/782 bl:2.2844 bb:0.9423 rl:2.2598 rb:0.9652 dl:1172-1177 gd:1 +ttp: b601/782 bl:2.3433 bb:0.9504 rl:2.2614 rb:0.9649 dl:1137-1141 gd:1 +ttp: b595/782 bl:2.3625 bb:0.9773 rl:2.2632 rb:0.9652 dl:1110-1115 gd:1 +ttp: b587/782 bl:2.4146 bb:0.9938 rl:2.2657 rb:0.9657 dl:1077-1081 gd:1 +ttp: b579/782 bl:2.3558 bb:0.9589 rl:2.2672 rb:0.9655 dl:1044-1048 gd:1 +ttp: b571/782 bl:2.3073 bb:0.9343 rl:2.2678 rb:0.9650 dl:1014-1017 gd:1 +ttp: b560/782 bl:2.2775 bb:0.9331 rl:2.2680 rb:0.9646 dl:975-979 gd:1 +ttp: b552/782 bl:2.2834 bb:0.9380 rl:2.2682 rb:0.9642 dl:949-952 gd:1 +ttp: b545/782 bl:2.3456 bb:0.9530 rl:2.2692 rb:0.9640 dl:927-930 gd:1 +ttp: b537/782 bl:2.3862 bb:0.9894 rl:2.2707 rb:0.9643 dl:902-905 gd:1 +ttp: b533/782 bl:2.3819 bb:0.9863 rl:2.2722 rb:0.9646 dl:890-892 gd:1 +ttp: b524/782 bl:2.3822 bb:0.9849 rl:2.2735 rb:0.9649 dl:863-866 gd:1 +ttp: b512/782 bl:2.3144 bb:0.9755 rl:2.2740 rb:0.9650 dl:829-832 gd:1 +ttp: b504/782 bl:2.3321 bb:0.9549 rl:2.2746 rb:0.9649 dl:807-809 gd:1 +ttp: b496/782 bl:2.4363 bb:0.9793 rl:2.2763 rb:0.9651 dl:785-788 gd:1 +ttp: b488/782 bl:2.3068 bb:0.9354 rl:2.2767 rb:0.9647 dl:766-769 gd:1 +ttp: b483/782 bl:2.2670 bb:0.9496 rl:2.2766 rb:0.9646 dl:754-756 gd:1 +ttp: b474/782 bl:2.3497 bb:0.9865 rl:2.2773 rb:0.9648 dl:733-735 gd:1 +ttp: b466/782 bl:2.3992 bb:0.9590 rl:2.2784 rb:0.9647 dl:714-717 gd:1 +ttp: b461/782 bl:2.3874 bb:0.9628 rl:2.2794 rb:0.9647 dl:703-706 gd:1 +ttp: b453/782 bl:2.3494 bb:0.9745 rl:2.2800 rb:0.9648 dl:687-689 gd:1 +ttp: b444/782 bl:2.3226 bb:0.9775 rl:2.2804 rb:0.9649 dl:668-670 gd:1 +ttp: b438/782 bl:2.3198 bb:0.9679 rl:2.2807 rb:0.9649 dl:655-657 gd:1 +ttp: b430/782 bl:2.3923 bb:0.9668 rl:2.2816 rb:0.9650 dl:640-642 gd:1 +ttp: b421/782 bl:2.3033 bb:0.9292 rl:2.2818 rb:0.9647 dl:622-624 gd:1 +ttp: b416/782 bl:2.3849 bb:0.9650 rl:2.2826 rb:0.9647 dl:613-615 gd:1 +ttp: b408/782 bl:2.3050 bb:0.9757 rl:2.2827 rb:0.9647 dl:597-598 gd:1 +ttp: b400/782 bl:2.3184 bb:0.9534 rl:2.2830 rb:0.9647 dl:582-584 gd:1 +ttp: b391/782 bl:2.3206 bb:0.9735 rl:2.2832 rb:0.9647 dl:566-568 gd:1 +ttp: b382/782 bl:2.3034 bb:0.9890 rl:2.2834 rb:0.9649 dl:550-552 gd:1 +ttp: b374/782 bl:2.3136 bb:0.9565 rl:2.2836 rb:0.9648 dl:537-538 gd:1 +ttp: b366/782 bl:2.3509 bb:0.9832 rl:2.2840 rb:0.9649 dl:524-525 gd:1 +ttp: b359/782 bl:2.2595 bb:0.9411 rl:2.2838 rb:0.9648 dl:512-513 gd:1 +ttp: b352/782 bl:2.4371 bb:1.0105 rl:2.2847 rb:0.9651 dl:499-501 gd:1 +ttp: b344/782 bl:2.3918 bb:0.9752 rl:2.2853 rb:0.9651 dl:488-489 gd:1 +ttp: b336/782 bl:2.4163 bb:0.9987 rl:2.2860 rb:0.9653 dl:476-477 gd:1 +ttp: b328/782 bl:2.2934 bb:0.9346 rl:2.2861 rb:0.9652 dl:463-465 gd:1 +ttp: b321/782 bl:2.3666 bb:0.9879 rl:2.2865 rb:0.9653 dl:453-455 gd:1 +ttp: b314/782 bl:2.2505 bb:0.9634 rl:2.2863 rb:0.9653 dl:442-444 gd:1 +ttp: b306/782 bl:2.4024 bb:0.9834 rl:2.2869 rb:0.9654 dl:430-432 gd:1 +ttp: b298/782 bl:2.4301 bb:1.0121 rl:2.2876 rb:0.9656 dl:418-420 gd:1 +ttp: b290/782 bl:2.3429 bb:0.9772 rl:2.2878 rb:0.9656 dl:406-407 gd:1 +ttp: b281/782 bl:2.3028 bb:0.9902 rl:2.2879 rb:0.9657 dl:394-395 gd:1 +ttp: b273/782 bl:2.3480 bb:0.9829 rl:2.2881 rb:0.9658 dl:383-384 gd:1 +ttp: b265/782 bl:2.3798 bb:1.0070 rl:2.2885 rb:0.9660 dl:372-374 gd:1 +ttp: b257/782 bl:2.4572 bb:1.0203 rl:2.2892 rb:0.9662 dl:362-364 gd:1 +ttp: b249/782 bl:2.4585 bb:1.0162 rl:2.2899 rb:0.9664 dl:352-354 gd:1 +ttp: b242/782 bl:2.3936 bb:1.0126 rl:2.2902 rb:0.9666 dl:344-345 gd:1 +ttp: b235/782 bl:2.3040 bb:1.0034 rl:2.2903 rb:0.9667 dl:335-336 gd:1 +ttp: b228/782 bl:2.3474 bb:0.9933 rl:2.2905 rb:0.9668 dl:327-328 gd:1 +ttp: b220/782 bl:2.4222 bb:1.0443 rl:2.2909 rb:0.9671 dl:317-318 gd:1 +ttp: b211/782 bl:2.4139 bb:1.0082 rl:2.2914 rb:0.9672 dl:307-308 gd:1 +ttp: b203/782 bl:2.4345 bb:1.0175 rl:2.2918 rb:0.9674 dl:299-300 gd:1 +ttp: b191/782 bl:2.4363 bb:1.0140 rl:2.2923 rb:0.9675 dl:285-286 gd:1 +ttp: b183/782 bl:2.3374 bb:0.9795 rl:2.2924 rb:0.9675 dl:277-278 gd:1 +ttp: b173/782 bl:2.4819 bb:1.0387 rl:2.2929 rb:0.9677 dl:267-268 gd:1 +ttp: b164/782 bl:2.4416 bb:1.0454 rl:2.2933 rb:0.9680 dl:259-260 gd:1 +ttp: b157/782 bl:2.3751 bb:1.0284 rl:2.2936 rb:0.9681 dl:252-253 gd:1 +ttp: b149/782 bl:2.3782 bb:1.0397 rl:2.2938 rb:0.9683 dl:244-245 gd:1 +ttp: b140/782 bl:2.4428 bb:1.0329 rl:2.2941 rb:0.9685 dl:235-236 gd:1 +ttp: b126/782 bl:2.4100 bb:1.0367 rl:2.2944 rb:0.9686 dl:222-223 gd:1 +ttp: b118/782 bl:2.4809 bb:1.0372 rl:2.2948 rb:0.9688 dl:215-216 gd:1 +ttp: b109/782 bl:2.5014 bb:1.0797 rl:2.2953 rb:0.9690 dl:207-208 gd:1 +ttp: b103/782 bl:2.4618 bb:1.0714 rl:2.2956 rb:0.9692 dl:202-202 gd:1 +ttp: b94/782 bl:2.5880 bb:1.1108 rl:2.2962 rb:0.9695 dl:193-194 gd:1 +ttp: b80/782 bl:2.4790 bb:1.0452 rl:2.2966 rb:0.9696 dl:181-182 gd:1 +ttp: b70/782 bl:2.5320 bb:1.1107 rl:2.2970 rb:0.9699 dl:172-173 gd:1 +ttp: b62/782 bl:2.4623 bb:1.0707 rl:2.2973 rb:0.9700 dl:165-166 gd:1 +ttp: b54/782 bl:2.4883 bb:1.0980 rl:2.2976 rb:0.9702 dl:157-158 gd:1 +ttp: b46/782 bl:2.5492 bb:1.1041 rl:2.2980 rb:0.9704 dl:149-150 gd:1 +ttp: b32/782 bl:2.6143 bb:1.1109 rl:2.2984 rb:0.9706 dl:135-136 gd:1 +ttp: b23/782 bl:2.6133 bb:1.1166 rl:2.2988 rb:0.9708 dl:126-127 gd:1 +ttp: b15/782 bl:2.6668 bb:1.1308 rl:2.2993 rb:0.9710 dl:115-117 gd:1 +ttp: b7/782 bl:2.7700 bb:1.1355 rl:2.2997 rb:0.9712 dl:101-103 gd:1 +quantized_ttt_phased val_loss:2.33162252 val_bpb:0.97794283 eval_time:507060ms +total_eval_time:507.1s