Skip to content

[strategy] subspace-iteration transform: power iterations for decaying (not low-rank) spectra #184

Description

@glorydavid03023

Summary

The built-in rsvd transform sketches each needed space — col(A), row(A), row(B) — with a single random projection. At a rank budget M below the numerical rank, that single sketch blurs strong and weak singular components together, so on data whose spectrum decays but is not low-rank (the --fill decaying-spectrum regime) rsvd's accuracy collapses below the floor.

This proposes a new registered transform subspace-iter that runs power_iters steps of subspace iteration on each space — for col(A) it forms (A Aᵀ)^q A Ω, re-orthonormalizing between steps — amplifying the dominant singular directions by the spectral gap each pass. That concentrates the M budget on the components carrying the product's energy, so at a fixed M it reconstructs far more accurately than rsvd. The extra streamed GEMMs are counted in full in basis_flops, so the reported FLOP savings stay honest; on a genuinely full-rank spectrum there is nothing to concentrate and rsvd stays the right tool.

GPU Result (decaying-spectrum, N=8192, rank R=256, M=256, GTX 1650, fp32, seed 0)

metric exact rsvd subspace-iter (this)
accuracy 1.0 0.783 → gated, score 0 0.873
latency 1843 ms 822 ms 1225 ms
VRAM 1032 MiB 273 MiB 273 MiB
FLOPs vs exact 7.8× fewer 5.2× fewer
verdict baseline not admitted (gated) improvement (score 2.67)

subspace-iter dominates exact on every axis (accuracy above floor, faster, less VRAM, fewer FLOPs) — where plain rsvd is gated at the same M. On full-rank data it correctly scores 0 like any subspace method (disclosed honestly).

Most of the work is a new Transform subclass in strategy/transforms.py (the designed hook), with tests for exact low-rank recovery and for beating rsvd on a decaying spectrum.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:evalProtected evaluator, metrics, artifacts, and policy (eval/, benchmarks/)area:strategySmart strategies / transforms (strategy/)status:triageNewly opened, not yet triagedtype:strategyNew smart strategy / transform proposal

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions