Summary
The built-in rsvd transform sketches each needed space — col(A), row(A), row(B) — with a single random projection. At a rank budget M below the numerical rank, that single sketch blurs strong and weak singular components together, so on data whose spectrum decays but is not low-rank (the --fill decaying-spectrum regime) rsvd's accuracy collapses below the floor.
This proposes a new registered transform subspace-iter that runs power_iters steps of subspace iteration on each space — for col(A) it forms (A Aᵀ)^q A Ω, re-orthonormalizing between steps — amplifying the dominant singular directions by the spectral gap each pass. That concentrates the M budget on the components carrying the product's energy, so at a fixed M it reconstructs far more accurately than rsvd. The extra streamed GEMMs are counted in full in basis_flops, so the reported FLOP savings stay honest; on a genuinely full-rank spectrum there is nothing to concentrate and rsvd stays the right tool.
GPU Result (decaying-spectrum, N=8192, rank R=256, M=256, GTX 1650, fp32, seed 0)
| metric |
exact |
rsvd |
subspace-iter (this) |
| accuracy |
1.0 |
0.783 → gated, score 0 |
0.873 |
| latency |
1843 ms |
822 ms |
1225 ms |
| VRAM |
1032 MiB |
273 MiB |
273 MiB |
| FLOPs vs exact |
1× |
7.8× fewer |
5.2× fewer |
| verdict |
baseline |
not admitted (gated) |
improvement (score 2.67) |
subspace-iter dominates exact on every axis (accuracy above floor, faster, less VRAM, fewer FLOPs) — where plain rsvd is gated at the same M. On full-rank data it correctly scores 0 like any subspace method (disclosed honestly).
Most of the work is a new Transform subclass in strategy/transforms.py (the designed hook), with tests for exact low-rank recovery and for beating rsvd on a decaying spectrum.
Summary
The built-in
rsvdtransform sketches each needed space — col(A), row(A), row(B) — with a single random projection. At a rank budgetMbelow the numerical rank, that single sketch blurs strong and weak singular components together, so on data whose spectrum decays but is not low-rank (the--fill decaying-spectrumregime) rsvd's accuracy collapses below the floor.This proposes a new registered transform
subspace-iterthat runspower_iterssteps of subspace iteration on each space — for col(A) it forms(A Aᵀ)^q A Ω, re-orthonormalizing between steps — amplifying the dominant singular directions by the spectral gap each pass. That concentrates theMbudget on the components carrying the product's energy, so at a fixedMit reconstructs far more accurately than rsvd. The extra streamed GEMMs are counted in full inbasis_flops, so the reported FLOP savings stay honest; on a genuinely full-rank spectrum there is nothing to concentrate and rsvd stays the right tool.GPU Result (decaying-spectrum, N=8192, rank R=256, M=256, GTX 1650, fp32, seed 0)
subspace-iter dominates exact on every axis (accuracy above floor, faster, less VRAM, fewer FLOPs) — where plain rsvd is gated at the same M. On full-rank data it correctly scores 0 like any subspace method (disclosed honestly).
Most of the work is a new
Transformsubclass instrategy/transforms.py(the designed hook), with tests for exact low-rank recovery and for beating rsvd on a decaying spectrum.