Skip to content

[strategy] dct transform: fixed DCT-II basis for smooth / decaying-spectrum data #191

Description

@andriypolanski

Summary

The only built-in transform today is rsvd, a data-dependent range finder whose basis construction costs O(N²M) sketches. This proposes a new registered transform dct: the first M type-II DCT modes as a data-independent orthonormal basis Q with closed-form O(NM) construction (basis_flops = N·M).

Measured honesty: CCO's --fill decaying-spectrum uses random singular vectors with decaying weights — not frequency-localized structure — so at M ≪ N dct accuracy collapses (~0.0004 at N=2048, M=256) and is gated, while rsvd recovers (~1.0). Where A/B actually live in the low-frequency DCT span, dct is exact (accuracy ≈ 1.0). On full-rank random data it scores 0 like any subspace method. FLOPs are ~2.36× fewer than exact at M=N/8.

Most of the work is a new Transform subclass in strategy/transforms.py (the designed hook), with CPU-safe tests for shape, orthonormality, and projector exactness on the DCT subspace.

Target regime

  • Matrix content: frequency-localized / DCT-smooth (official decaying-spectrum does not currently favor dct)
  • Expected N, M, dtype, device: N=12000 reference on GPU when available; CPU mirror used below

Expected trade-off

Against the exact baseline, on official --fill decaying-spectrum (RTX 5090, N=12000, M=1500, measured):

aspect vs exact (measured)
accuracy fails floor (0.0019) — gated
time complexity lower (2.4× fewer FLOPs)
latency higher (5868 ms vs 5297 ms exact)
VRAM usage lower (713 MiB vs 2208 MiB)

Notes

  • Distinct from rsvd: fixed frequency basis vs data-dependent sketches.
  • Official fill energy is not DCT-localized — do not claim a frontier win on it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:strategySmart strategies / transforms (strategy/)status:triageNewly opened, not yet triagedtype:bugBug report/fix; not an accepted miner PR lanetype:strategyNew smart strategy / transform proposal

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions