Non-record: 11L Int6 + XSA + TTT + SmearGate + BigramHash (pending compute)#277

Open

mohosy wants to merge 6 commits into

openai:mainfrom

mohosy:submission/mohosy-xsa-ttt

Commits on Mar 20, 2026

Non-record: 11L Int6 + XSA + TTT + SmearGate + BigramHash (pending compute)

authored and
mo shirmoahmmadi
committed
Non-record: 11L Int6 + XSA + TTT (val_bpb=1.1429, artifact slightly over 16MB)

authored and
mo shirmoahmmadi
committed
Add EMA (decay=0.997) replacing SWA, tune WD=0.042, XSA last 4 layers

authored and
mo shirmoahmmadi
committed

Commits on Mar 23, 2026

Update to frontier stack: SwiGLU + XSA4 + EMA + U-Net + AdamW TTT

authored and
mo shirmoahmmadi
committed

Commits on Apr 8, 2026

Merge branch 'main' into submission/mohosy-xsa-ttt

mo shirmoahmmadi
authored and
mo shirmoahmmadi
committed
Update approach: depth recurrence + SwiGLU + XSA-all + parallel residuals + AR GPTQ

authored and
mo shirmoahmmadi
committed