Record: Alpha=144 LoRA + Warm-start A + WD 1.0 — val_bpb 1.07209 (3-seed mean)

[renqianluo]

Summary

Four composable small-LOC changes to BatchedLinearLoRA on top of @dexhunter's 1.07193 phased-TTT code. Everything outside the LoRA module (VarLen attention, Fused MLP, multi-phase global SGD, trimmed GPTQ, triple depth recurrence) is unchanged.

1. Alpha/rank output scaling — forward(x) * (alpha/rank). Without this, raising rank directly diverges on some seeds.
2. Warm-start A across batches — only B resets between batches, A accumulates feature directions over the ~780 phased-TTT batches.
3. Raised TTT weight decay 0.5 → 1.0 — counteracts the across-batch A overfit enabled by (2).
4. Alpha lifted 96 → 144 — scale=1.125 on rank 128 gives LoRA more adaptation strength; (3) keeps it stable.

Results

Seed	rank-96 baseline	+ alpha 96	+ warm+WD=1	+ alpha 144
1337	1.07423	1.07379	1.07298	1.07189
42	1.07341	1.07320	1.07298	1.07248
314	1.07214	1.07200	1.07203	1.07189
Mean	1.07326	1.07300	1.07266	1.07209

Every seed improves monotonically across every change.

Compliance

All train ≤596s, eval 455.7–456.7s, artifacts ≤15.94MB. Issue #1017 conditions 1–4 verified.

Attribution

• @bigbag (Record: SP8192 + 3-Layer Recurrence + Parallel Residuals + QK-Gain 5.25 + Legal TTT — val_bpb 1.0810 (3-seed mean) #1493), @EthanYangTW (Record: SP8192 + Triple Recurrence + Banking + Fused MLP + Muon 0.97 — val_bpb 1.0778 (3-seed mean) #1523), @samacqua (Record: Varlen attention + fused MLP + doc-independent TTT (1.07336) #1530), @romeerp (Record: VarLenAttn + PhasingTTT - val_bpb 1.0728 (3-seed mean) #1610), @dexhunter (1.07193), @abaybektursun (Record: LeakyReLU² + Legal Score-First TTT + Parallel Muon — val_bpb 1.1194 (3-seed mean) #549)