LFM2.5 audio detokenizer: convert native phase coverage into a faithful FIFO program

[sydneyrenee]

Problem

The released LFM2.5 path correctly loads and uses the checkpoint’s custom audio_detokenizer; Mimi is not wired into this model. Weight ownership and strict schema validation are correct, and paired leaves cover the operator families.

The execution still materializes broad transient state between roughly fifteen phases:

• normalized activations and three wide intermediates;
• terminal spectral representation;
• separate values surrounding dense and transform seams;
• six-row 512→1282 final projection and dense inverse transform execute serially through Accelerate while peer lanes return.

Persistent ShortConv state, GQA K/V rings, overlap-add ring, and envelope may be legitimate. Their existence alone is not a violation. The violation is materializing transient values merely because the reference module or C++ function ended.

The current detokenizer_payload_math_is_owned_by_paired_assembly test checks symbol presence/absence and Accelerate seams. It does not prove numerical faithfulness, cooperative work, register chaining, or minimal materialization.

Authoritative topology

Preserve the vendored LFM2.5 detokenizer exactly:

1. audio-code embedding/fusion with the ×6 nearest-exact embedding repeat;
2. the eight-layer causal LFM2 operator stack;
3. final 1282 projection;
4. log-magnitude/angle conversion;
5. polar complex spectrum;
6. inverse transform;
7. overlap-add/envelope normalization;
8. PCM publication.

Do not substitute Mimi, simplify trained operations, or change F32/BF16/reduction semantics.

Required program

• Fuse row-owned residual/RMSNorm/operator/FFN chains where live sets permit.
• Keep causal state and GQA rings persistent.
• Treat Accelerate/AMX/vForce as explicit opaque seams with documented materialization on each side.
• Partition independent dense rows and spectral/ring work across the fixed team.
• Alias inverse-transform destinations into dead wide storage where legal.
• Write normalized PCM directly into playback reservations.
• Remove redundant terminal/spectral/intermediate planes after a liveness proof.
• Provide faithful architecture twins or fail unsupported readiness explicitly.

Verification

Frozen reference traces must cover:

• code embedding;
• every operator/LFM layer boundary;
• consecutive-frame causal/KV state;
• final spectral values;
• polar conversion;
• inverse transform;
• overlap-add/envelope state;
• complete PCM, including tails and terminal flush.

Acceptance

• The misleading symbol-hygiene test is renamed and supplemented with full-value tests.
• Every retained plane has a recurrence, fan-out, barrier, continuation, or opaque-seam justification.
• Lane-zero numerical islands are removed or explicitly justified as measured opaque seams.
• No per-frame allocation, copy-only plane, scalar fallback, or hot per-element libm call remains.
• Consecutive-frame state and complete PCM match authoritative traces.
• Audio-token A→B→A conversation remains clear, deterministic, and correctly retired.
• Mimi stays unwired from LFM2.5 but available for its future Moshi tranche.

Related

• Native numerical umbrella: native inference: remove residual C++ numerical fallbacks and materialized scratch #158
• Faithfulness trace child
• Real-checkpoint/audio-only gate: voice: automate the native real-checkpoint truth gate and preserve evidence #147

[sydneyrenee]

Structure verified against the program header and implementation: 15 phases (EMBED..DONE, lfm_detokenizer_program.h:16-32), run() per member per phase, advance() only from the final-return callback — the concurrency skeleton is the correct shape; the materialization breadth is the gap, as stated.

Two precisions:

1. The x86_64 detokenizer .S is production-dead. lfm_detokenizer_plan_new_from_image hard-refuses non-Apple with -ENOTSUP (lfm_detokenizer.cpp:613-617) because the dense/iDFT seams are Apple-gated Accelerate/vForce. The paired x86 leaves exist for CI parity only. The acceptance item "faithful architecture twins or fail unsupported readiness explicitly" already covers this — but state the consequence plainly somewhere: non-Apple production currently has no detokenizer path at all, so "x86_64/Rosetta gates pass" can never exercise this component until a house dense/transform leaf exists.

2. "Six nearest-exact operator blocks" needs a wording fix. The phrase is sourced, not fabricated — the vendored upstream does nn.functional.interpolate(..., mode="nearest-exact") (native/vendor/liquid_audio/reference/detokenizer.py:123), i.e., the ×6 nearest-exact repeat of each embedding row before the 8 causal LFM2 layers (conv, conv, attn ×3, conv — kAttention at lfm_detokenizer.cpp:70-72). As written it reads like six operator blocks, which will send an implementer looking for a topology that does not exist. Suggest: "×6 nearest-exact embedding repeat, then the 8-layer causal operator stack."

— K3

[sydneyrenee]

Implementation handoff — this is the LFM2.5 custom detokenizer, not Mimi

The most dangerous default assumption is that “audio decoder” means Mimi. It does not here. The shipping LFM2.5 checkpoint uses its own audio_detokenizer weights and topology. Do not substitute Mimi, reuse Mimi shapes, or infer architecture from Moshi.

Authoritative sequence

Audio codes feed the ×6 nearest-exact embedding repeat, then the eight-layer causal operator stack, 1282 spectral projection, magnitude/angle conversion, polar spectrum, inverse transform, overlap-add/envelope normalization, and PCM publication.

Preserve the reference precision and state ordering. Clear audio is not sufficient evidence.

Which state is allowed to persist

Likely legitimate:

• ShortConv causal state;
• GQA K/V rings;
• overlap-add ring;
• normalization envelope;
• continuation-visible frame cursor/state.

Likely transient and subject to fusion/aliasing:

• normalized activation planes;
• wide FFN/operator intermediates;
• terminal/spectral staging that has one consumer;
• separate IFFT destination when a dead wide plane can receive it.

Do not delete a plane based on size alone. Prove its last consumer and whether it crosses a barrier or opaque Accelerate/vForce seam.

Apple seam discipline

Accelerate/AMX is allowed for measured dense work, but peers returning while lane zero performs a large numerical island is not automatically cooperative. Partition independent rows where supported, or document why the opaque call is one indivisible seam. Materialize only its required contract. Do not hand-emit undocumented AMX as the first solution.

Work order

1. Establish native inference: restore authoritative full-value faithfulness gates before FIFO fusion #170 detokenizer traces across consecutive frames.
2. Rename the symbol-hygiene test before using it as evidence.
3. Produce a phase-by-phase liveness table.
4. Fuse row-owned residual/norm/operator/FFN pairs.
5. Alias transform destinations only after canary and trace tests.
6. Write PCM directly to playback reservations and validate terminal flush/tails.

Mimi remains available but unwired for future Moshi under #177.