[Feature] Make autotune a safe, adopted path

[jhinpan]

Goal

Make FlyDSL autotune a real, adopted path without making normal serving or CI pay for benchmark search.

This supersedes the discussion in #612 and supports the downstream-kernel direction in #749. The implementation should stay small: reuse the direct JIT entry point, keep one owner for tuning axes, and add machinery only for demonstrated correctness or deployment needs.

Design invariants

1. Search is explicit opt-in. Normal calls use a searched winner, matching offline artifact, or heuristic default before considering search. FLYDSL_AUTOTUNE=1 is the deliberate force-search path.
2. Benchmarking preserves program semantics. restore_value and reset_to_zero make every repetition start from valid state and also preserve clean final/cache-hit behavior.
3. The declared key owns tuning axes. Shape, dtype, layout, and mode values that can change the winner belong there. Artifact lookup must not introduce a second key callback or competing schema.
4. Scratch winners and deployment artifacts have different lifetimes. The scratch cache includes toolchain, device, environment, dtype, and stride fingerprints. Offline artifacts use the declared key plus device identity and are reviewed deployment inputs.
5. Artifacts are generated on the intended GPU. Shared CI verifies deterministic emit/load, validation, fallback, and execution behavior; it does not select or commit a winner from noisy shared-runner timing.
6. Accepted artifacts fail loudly at execution. Missing or invalid artifacts may fall back. Compile, launch, or runtime errors from an accepted artifact are not masked by retrying a default.

Current implementation

• [1/5] autotune: harden cache key + add restore_value (#770) #783 — merged: hardened cache identity and added restore_value / reset_to_zero with GPU-free tests.
• [2/5] autotune: add opt-in search and RMSNorm waves-per-EU tuning (#770) #785 — merged: added explicit opt-in search, heuristic defaults, compiler-hint tuning, and the first direct-JIT adopter (rmsnorm).
• [3/5] autotune: add offline config artifacts (#770) #786 — open: adds optional content-addressed offline artifacts, target-device identity, atomic emit/load, validation/fallback contracts, RMSNorm GPU coverage, and the user guide.
• autotune: keep CI search opt-in (#770) #788 — open: rebased to a minimal CI guard. The shared test runner pins FLYDSL_AUTOTUNE=0; dedicated tests opt in to forced search themselves.

The current design has no builder mode, second RMSNorm factory, artifact-specific key callback, or committed config registry.

CI contract

• Broad source and wheel test runs must not benchmark implicitly.
• GPU-free tests cover config serialization, cache/default/forced-search ordering, cache-key axes, pruning, state restoration, and artifact policy.
• The dedicated RMSNorm device test owns its FLYDSL_AUTOTUNE=1 scope, executes every candidate with deterministic synthetic ranking, and checks the selected result plus artifact emit/load against the reference.
• No extra autotune workflow or permanent checked-in winner is required.

Acceptance criteria

• Cache identity is invalidated by relevant compiler, environment, device, dtype, shape, and stride changes.
• In-place and accumulation kernels can be benchmarked without cross-repetition corruption.
• Normal execution has a zero-search heuristic path and one real kernel adopter.
• Offline artifact generation and serving contracts land through [3/5] autotune: add offline config artifacts (#770) #786.
• Broad CI search remains disabled while dedicated search/artifact tests remain explicit through autotune: keep CI search opt-in (#770) #788.
• Documentation states when to tune, how artifacts are invalidated, and why CI does not choose winners.

Deliberately deferred

Parallel precompile / two-phase compilation (#266) is not part of this issue's acceptance criteria. The direct-JIT path removed the builder-mode motivation, and there is not yet a measured first-run latency budget showing that another compilation mechanism is necessary. Open a focused follow-up only with a reproducible workload, target GPU, current compile/search breakdown, and required latency bound.

References

• Prior discussion: Discussion: make autotune a supported and broadly usable tuning path #612
• Downstream direction: [RFC] FlyDSL kernels in downstream libraries #749
• Compiler-hint history: Feat/autotune compile hints #241, feat(jit): two-phase compilation for autotuning compile_hints #266
• Reference implementations studied: Triton autotuner state restoration and cache identity; quack heuristic/search split; SGLang and aiter offline deployment patterns.