Skip to content

Fix: move Ternary UNet submission folder from track_10min_16mb to track_non_record_16mb #730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
janwww wants to merge 1 commit into from
Closed

Fix: move Ternary UNet submission folder from track_10min_16mb to track_non_record_16mb #730

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Happy training!
| Int6 MLP3x + SmearGate + BigramHash | 1.1458 | Raahil Shah | 3x MLP + SmearGate + BigramHash + OrthoInit + Muon WD + SWA | 2026-03-20 | [info](records/track_10min_16mb/2026-03-20_Int6_MLP3x_SmearGate_BigramHash_MuonWD_SWA/README.md) |
| 11L MLP3x + Int6 QAT | 1.1502 | aruniyer | 11 layers, 3x MLP, int6 QAT, zstd-22, WD=0.04, sliding eval | 2026-03-20 | [info](records/track_10min_16mb/2026-03-19_MLP3x_QAT_Int6_SlidingWindow/README.md) |
| SmearGate + OrthoInit + Muon WD | 1.1556 | aquariouseworkman | SmearGate + BigramHash + 3x MLP + int6 STE QAT + sliding eval | 2026-03-19 | [info](records/track_10min_16mb/2026-03-19_smeargate_orthoinit_muonwd/README.md) |
| Ternary Quantization | 1.1570 | Ciprian-Florin Ifrim | 73.7M params quantized to 1 0 -1 + misc arch changes | 2026-03-24 | [info](records/track_10min_16mb/2026-03-24_74M_Ternary_UNet_FP8_10L_8192BPE_YaRN_NeoMuon/README.md) |
| Ternary Quantization | 1.1570 | Ciprian-Florin Ifrim | 73.7M params quantized to 1 0 -1 + misc arch changes | 2026-03-24 | [info](records/track_non_record_16mb/2026-03-24_74M_Ternary_UNet_FP8_10L_8192BPE_YaRN_NeoMuon/README.md) |
| 10L Int6 QAT + Zstd MLP2.6x | 1.1586 | yahya010 | 10 layers, int6 QAT + zstd-22, MLP 1344, Muon 0.99, sliding eval | 2026-03-19 | [info](records/track_10min_16mb/2026-03-19_Seq2048_FP16Emb_TunedLR/README.md) |
| Mixed Quant + Sliding Window Eval | 1.1630 | aquariouseworkman | Int6 block weights + int8 embeddings + 3x MLP + sliding eval | 2026-03-19 | [info](records/track_10min_16mb/2026-03-19_MixedQuant_Int6Int8_SlidingWindow/README.md) |
| Muon WD + 10 layer | 1.1748 | notapplica | Includes prev. wins + Spectral embed init + resid mix | 2026-03-19 | [info](records/track_10min_16mb/2026-03-19_SlidingWindow_FP16Emb_10L_MuonWD_OvertoneInit/README.md) |
Expand All @@ -65,7 +65,7 @@ Breakthrough ideas are rarely immediately state-of-the-art, instead, they're dev
We'd love to see weird & creative ideas in the challenge, since you never know what may work in the end. Most likely, these will be a good fit in our unlimited compute leaderboard as non-record submissions. We have some requests for what we'd love to see people implement:

- [x] 1-bit quantization - [implementation](records/track_non_record_16mb/2026-03-24_106M_Binary_Asymmetric_UNet_FP8_15L_8192BPE_YaRN_NeoMuon_Smear/README.md)
- [x] Ternary quantization - [implementation](records/track_10min_16mb/2026-03-24_74M_Ternary_UNet_FP8_10L_8192BPE_YaRN_NeoMuon/README.md)
- [x] Ternary quantization - [implementation](records/track_non_record_16mb/2026-03-24_74M_Ternary_UNet_FP8_10L_8192BPE_YaRN_NeoMuon/README.md)
- [ ] JEPA
- [ ] Text diffusion
- [ ] H-net tokenization
Expand Down
0 ...et_FP8_10L_8192BPE_YaRN_NeoMuon/README.md → ...et_FP8_10L_8192BPE_YaRN_NeoMuon/README.md
View file
Open in desktop
File renamed without changes.
0 ...t_FP8_10L_8192BPE_YaRN_NeoMuon/RESULTS.md → ...t_FP8_10L_8192BPE_YaRN_NeoMuon/RESULTS.md
View file
Open in desktop
File renamed without changes.
0 ...92BPE_YaRN_NeoMuon/fineweb_8192_bpe.model → ...92BPE_YaRN_NeoMuon/fineweb_8192_bpe.model
View file
Open in desktop
File renamed without changes.
0 ...92BPE_YaRN_NeoMuon/fineweb_8192_bpe.vocab → ...92BPE_YaRN_NeoMuon/fineweb_8192_bpe.vocab
View file
Open in desktop
File renamed without changes.
0 ...10L_8192BPE_YaRN_NeoMuon/requirements.txt → ...10L_8192BPE_YaRN_NeoMuon/requirements.txt
View file
Open in desktop
File renamed without changes.
0 ..._8192BPE_YaRN_NeoMuon/run_cuda_ternary.sh → ..._8192BPE_YaRN_NeoMuon/run_cuda_ternary.sh
View file
Open in desktop
File renamed without changes.
0 ...Net_FP8_10L_8192BPE_YaRN_NeoMuon/setup.sh → ...Net_FP8_10L_8192BPE_YaRN_NeoMuon/setup.sh
View file
Open in desktop
File renamed without changes.
0 ...2BPE_YaRN_NeoMuon/submission_ternary.json → ...2BPE_YaRN_NeoMuon/submission_ternary.json
View file
Open in desktop
File renamed without changes.
0 ...8192BPE_YaRN_NeoMuon/ternary_log_1337.txt → ...8192BPE_YaRN_NeoMuon/ternary_log_1337.txt
View file
Open in desktop
File renamed without changes.
0 ...L_8192BPE_YaRN_NeoMuon/ternary_log_42.txt → ...L_8192BPE_YaRN_NeoMuon/ternary_log_42.txt
View file
Open in desktop
File renamed without changes.
0 ...0L_8192BPE_YaRN_NeoMuon/ternary_log_7.txt → ...0L_8192BPE_YaRN_NeoMuon/ternary_log_7.txt
View file
Open in desktop
File renamed without changes.
0 ...PE_YaRN_NeoMuon/train_gpt_cuda_ternary.py → ...PE_YaRN_NeoMuon/train_gpt_cuda_ternary.py
View file
Open in desktop
File renamed without changes.