Ouroboros — 1.13727008 val_bpb (seed 444)

[newjordan]

Ouroboros

9-flat crawler with loop-aware GPTQ, QK gain 4.0, 2-loop cadence, and brotli compression — stacking five research signals on the Bandit Wagon 9F platform.

Results

Seed	val_bpb (sliding window)	Steps	Size
444	1.13727008	5951	15,034,550 B
4	1.13565882	5963	15,042,594 B
300	1.13638653	5948	15,049,936 B
mean	1.13643848		15,049,936 B

Hardware: 8×H100 SXM · 600s wallclock · bytes_code: 121,677

Architecture

9-flat crawler with recurrent refinement: 9 unique flat transformer blocks followed by 1 shared crawler block looping 2× with differentiated RoPE scales (9,1,1). 26.25M parameters, ~100.85ms/step.

Research signals stacked

This submission is the product of a systematic crawler research program beginning with the beloved but ill fated frugendorff - this is where the dream of recursion lives. - It will not die:

1. Loop-aware GPTQ (confirmed −0.00380 BPB in BW10 full run) — 2-phase Hessian calibration that re-collects crawler importance scores after flat-layer quantization, addressing shared-weight quantization hostility
2. Brotli compression (approved via BW20 gate) — quality=11 replaces zstd level=22, ~5-15% smaller artifacts, freeing 16MB headroom for GPTQ overhead
3. QK_GAIN_INIT=4.0 (high-confidence, −0.006 external signal across 45 runs/3 codebases) — sharper initial attention gradients via per-head q_gain scalar
4. 2-loop cadence (directional −0.054 in BW17 DGX-Spark RAPID) — fewer loops = faster steps (100.85 vs 110.19ms) = 505 more training steps in 600s budget + smaller quant gap
5. Optimized warmdown (WARMDOWN_ITERS=2000, confirmed 2000 > 3500 > 5000)

Key finding from our crawler signal analysis: the crawler's advantage is 85% width, 15% implicit regularization at this configuration. This shifted focus from adding crawler complexity toward maximizing flat depth, reducing loop overhead, and improving post-training quantization. Next is working on an inverse kramer resolution with a 6-7 oscillator.

Parent: Bandit Wagon X (BWX 9F)

Metric	BWX 9F	Ouroboros	Delta
int6_sw_bpb	1.13867894	1.13727008	−0.00141
bytes_total	15,239,617	15,034,550	−205,067
step_ms	110.19	100.85	−9.34
steps (600s)	5446	5951	+505

Reproduce

pip install brotli
SEED=444 \
MAX_WALLCLOCK_SECONDS=600 WARMDOWN_ITERS=2000 \
NUM_FLAT_LAYERS=9 NUM_CRAWLER_LAYERS=1 CRAWLER_LOOPS=2 \
USE_CRAWLER=1 COMPILE_FULLGRAPH=1 \
SKIP_GPTQ=0 LOOP_AWARE_GPTQ=1 QK_GAIN_INIT=4.0 \
GPTQ_CAL_SAMPLES=128 GPTQ_CAL_SEQ_LEN=2048 \
CRAWLER_LOOP_ROPE_SCALES=9,1,1 SKIP_EMA=1 \
MODEL_DIM=512 INST_DIM=32 CRAWLER_MLP_MULT=6.0 \
CRAWLER_TAP_DIM=0 ANCHOR_DIM=0 CRAWLER_MLP_CHOKE_DIM=0 \
XSA_LAST_N=11 BIGRAM_VOCAB_SIZE=2048 ROPE_DIMS=16 \
SWA_EVERY=50 MATRIX_LR=0.03 \
MLP_LEAKY_SLOPE=0.5 CRAWLER_MLP_LEAKY_SLOPE=0.5 \
torchrun --standalone --nproc_per_node=8 \
  records/track_10min_16mb/2026-04-03_Ouroboros_8xH100/train_gpt.py

[newjordan]

Closing to resubmit under track_non_record_16mb. This is crawler architecture research, not a leaderboard attempt. Resubmission will include the full research context linking this to our earlier crawler PRs (#579, #990, #1028, #1140, #1208). See the non-record PR for the complete picture.