native inference: restore authoritative full-value faithfulness gates before FIFO fusion

[sydneyrenee]

Problem

The production graph is native, Candle-free, and capable of real speech generation, but its strongest component-level numerical evidence is disconnected:

• crates/liquid-audio/native/tests/fixtures/{mel,resample,conformer} have no live consumers;
• designs 05 and 06 cite tests/native_frontend_parity.rs and tests/native_conformer_parity.rs, but neither file exists;
• the real-checkpoint E2E compares native runs to deterministic native runs, not to an authoritative independent trace;
• detokenizer_payload_math_is_owned_by_paired_assembly proves symbol hygiene, not numerical parity or register chaining.

Deterministic, intelligible audio can still hide formula, rounding, layout, state, or sampling drift. Register-FIFO fusion must not proceed without executable full-value evidence.

Contract

• Do not restore a Rust/Candle production model, Rust safetensors loader, tensor bridge, or competing inference path.
• Generate/reference traces offline from the authoritative model implementation, then consume immutable fixtures from native-only tests.
• A “tensor” in fixture metadata is only a typed byte/shape/stride description.
• Tests must compare complete values at semantically meaningful boundaries, not only final tokens or PCM.
• Each fixture records checkpoint revision, source implementation revision, dtype, shape, strides/layout, input digest, output digest, tolerances, and precision/reduction contract.
• Faithfulness and bit-exactness are distinct: every boundary declares which is required.

Required traces

Frontend

• resampler output;
• preemphasis/window input;
• STFT/power rows;
• mel rows before and after log;
• per-feature statistics;
• normalized BF16 output and valid frame count.

Conformer and adapter

• subsampling output;
• positional embedding;
• every layer output;
• representative first-layer FFN, attention, and convolution intermediates;
• adapter output and valid-row geometry.

Backbone and recurrence

• embeddings/modality assembly;
• every layer hidden output;
• KV append and ShortConv carry;
• logits;
• cursor/state after prefill and consecutive decode passes.

Sampler and Depthformer

• top-k threshold and tie order;
• lane maxima/sums/prefix intervals;
• draw target and PRNG before/after;
• selected text/audio token;
• every Depthformer layer/codebook output and feedback embedding.

LFM2.5 audio detokenizer

• fused code embedding;
• each nearest-exact/operator/FFN block boundary;
• terminal spectral representation;
• polar complex spectrum;
• overlap-add/envelope state;
• full PCM over consecutive frames.

Work

1. Inventory the provenance of every existing fixture; do not delete unexamined evidence.
2. Reject fixtures whose generator/checkpoint/layout cannot be proven.
3. Add native fixture readers that validate manifests before comparison.
4. Wire mel/resample/Conformer fixtures into executable native tests.
5. Capture missing backbone/sampler/Depthformer/detokenizer traces from the authoritative implementation outside the production graph.
6. Rename the detokenizer symbol-hygiene test to describe only what it proves.
7. Correct design 05/06 status headers and every claim citing nonexistent gates.
8. Run these gates before and after every numerical/FIFO tranche under native inference: remove residual C++ numerical fallbacks and materialized scratch #158.

Acceptance

• Every retained fixture has complete provenance and a validated manifest.
• Existing mel, resample, and Conformer fixtures are consumed or explicitly rejected with rationale.
• Full-value native tests cover frontend, Conformer, backbone, sampler, Depthformer, detokenizer, and retained recurrence.
• Tests fail on intentional layout, rounding, state, and token-selection perturbations.
• No Rust/Candle inference path enters production or the test execution graph.
• The real-checkpoint gate remains green after fixture wiring.
• Designs claim only gates that exist and run.

Related

• Native numerical ownership umbrella: native inference: remove residual C++ numerical fallbacks and materialized scratch #158
• Real-checkpoint product gate: voice: automate the native real-checkpoint truth gate and preserve evidence #147
• Accounting: native voice: enforce zero post-readiness reads, allocations, and weight materialization #157
• Documentation: docs: reconcile native PCM, Sesame, kcoro, and numerical ownership after the Rust cutover #161

[sydneyrenee]

Verified firsthand and consistent with my own sweep — with one inventory note: fixtures/sesame is consumed (sesame_detector.rs:245, 301, 347, 459 against selected_bins_v1.json), so the orphaned set is exactly {conformer, mel, resample}. The provenance-inventory step should mark sesame as the live example of the target state rather than lumping it into the rejection review.

On provenance: the Conformer fixtures' manifest lineage runs through the deleted Rust oracle, which is fine under the contract here (pinned-commit capture, never a live call) — but work item 1 should record, per fixture set, the generator commit, the checkpoint revision, and whether the capture was full-value or truncated, because the rejection rule ("generator/checkpoint/layout cannot be proven") will need those fields to decide keep-vs-recapture rather than keep-vs-delete blindly.

Also worth stating in Acceptance: the spec 05/06 status headers cite tests/native_frontend_parity.rs and tests/native_conformer_parity.rs — neither file exists, so "Designs claim only gates that exist and run" requires either recreating those files as real consumers or amending the headers in the same tranche (overlaps #161).

— K3

[sydneyrenee]

Implementation handoff — read before changing numerical code

This issue is the dependency root for #171–#176. Do not begin by writing a new kernel.

The nonstandard contract

A conventional ML port keeps the reference framework alive and compares outputs online. This repository deliberately does not do that:

• production and test execution may not instantiate Candle/PyTorch/MLX model objects;
• Rust may not load safetensors or own numerical arrays;
• checkpoint bytes remain in the native immutable image;
• frozen reference artifacts are evidence, not a second runtime;
• a “tensor” is only fixture metadata describing bytes, dtype, shape, layout, and strides.

The goal is not merely “close enough final audio.” Faithfulness must survive future changes to reduction order, layout, scratch aliasing, assembly fusion, and AMX seams.

Start here

1. Read the vendored authoritative Python model/processor/detokenizer sources.
2. Inventory every file under native/tests/fixtures; do not delete or regenerate anything until its checkpoint, input, generator revision, dtype, and layout are proven.
3. Confirm which current native tests actually consume each fixture.
4. Define one versioned manifest schema before adding readers.
5. Add deliberately failing perturbation tests so the gate proves it observes layout, rounding, state, and token-selection errors.

Evidence rules

• Bit-exact BF16 storage does not imply bit-exact F32 accumulation.
• A tolerance must derive from the reference precision/reduction contract, never from “what passes.”
• Compare consecutive calls where state exists; a single frame cannot validate KV, ShortConv, OLA, envelope, PRNG, or cursor behavior.
• Preserve NaN, infinity, signed-zero, tie, tail, padding, and valid-length cases.
• Final token equality is insufficient; final PCM similarity is even weaker.
• The real-checkpoint E2E (voice: automate the native real-checkpoint truth gate and preserve evidence #147) proves wiring and product behavior. This issue proves internal numerical semantics. Require both.

Completion discipline

A component gate is complete only when it reaches the shipping native function, consumes independently produced immutable evidence, fails under intentional corruption, and runs without a framework inference path. Do not claim coverage from symbol scanning, fixture presence, deterministic self-comparison, or audible approval alone.