flashkern backbone/prefill/Depthformer: finish FIFO leaves and remove pass-time fallback

[sydneyrenee]

Problem

Backbone decode has the healthiest Flashkern stage topology, but it still bypasses assembly for byte movement and depends on intrinsics debt. Prefill contains a full C++ numerical FIR stage and pass-time scalar selection. Depthformer has correct cooperative phase advancement but chains nineteen phases through broad planes.

Decode debt

• C++ K/V cache append in ST_AT_PREP;
• C++ hidden-plane zeroing for audio embedding;
• attention, RMSNorm, activation, add, and conversions remain in architecture intrinsics TUs;
• AArch64 has assembly RNE GEMV leaves while x86_64 uses C++ intrinsics despite the assembly TU claiming no twin is needed.

Prefill debt

• prefill_conv_fir_stage implements FIR, gate, carry recurrence, and BF16 rounding in C++;
• prefill_linear/run_gemm can select scalar GEMM per pass when SIMD is unavailable;
• this fallback is linked into the product archive instead of failing readiness.

Depthformer debt

• nineteen generations materialize QKV, K/V, attention, FFN, logits, and projection planes;
• V append uses C++ memcpy;
• projection bands use C++ std::fill;
• phase boundaries follow plane boundaries rather than per-codebook register liveness.

Required work

1. Replace decode K/V publication and plane clearing with assembly destination leaves.
2. Provide paired AArch64/x86_64 implementations for every mounted decode leaf.
3. Move prefill FIR/gate/carry/rounding into a fused row-group assembly leaf preserving causal order.
4. Resolve ISA specialization once at readiness; remove all pass-time scalar fallback.
5. Re-cut Depthformer around per-codebook row-owned chains:
   • norm → projection → mixer → residual;
   • FFN norm → gate/up → activation → down → residual;
   • logits → cooperative sampler → direct feedback embedding.
6. Materialize only persistent KV/recurrence, true collective partials, or fan-out values.
7. Preserve exact KV, ShortConv, cursor, sampler, PRNG, and codebook update order.

Acceptance

• No C++ numerical loop, payload copy/fill, or pass-time ISA branch remains in these paths.
• Unsupported ISA/configuration fails before model readiness.
• AArch64 and x86_64 use paired assembly semantics.
• Prefill and decode produce identical hidden/logit/state traces.
• Depthformer full layer/codebook/code traces pass.
• Consecutive turns preserve KV/ShortConv/PRNG/cursor exactly.
• No post-readiness allocation or weight materialization.
• Real-checkpoint typed, audio, retained-context, and audio-only gates pass.

Related

• Native numerical umbrella: native inference: remove residual C++ numerical fallbacks and materialized scratch #158
• Sampler assembly child
• Faithfulness trace child
• Post-readiness accounting: native voice: enforce zero post-readiness reads, allocations, and weight materialization #157

[sydneyrenee]

Verified. Two scoping precisions:

1. The K/V append bypass is decode-only. The prefill path performs the same append through the lfm_bf16_copy_bytes assembly leaf (flashkern_engine.cpp:2840-2847); only ST_AT_PREP in decode uses std::memcpy (1581-1587). So the fix is "route decode through the existing leaf," not "write a new leaf" — same shape as the sampler issue (flashkern sampler: move the complete top-k ladder into cooperative assembly #171), where the leaf predates the program.

2. Add two undocumented debts to the inventory: fill_listen_readout_head (3008-3060) with std::exp/std::log in the production TU (test-facing, but the release gate forbids libm in the TU), and prefill_conv_fir_stage (2739-2806), which is absent from FLASHKERN.md's documented debt table — the table should be corrected in the same tranche so the ledger stops understating what remains (overlaps docs: reconcile native PCM, Sesame, kcoro, and numerical ownership after the Rust cutover #161).

The x86_64 asymmetry note is exactly right and worth quoting verbatim for whoever picks it up: x86_64/flashkern_math.S:39-42 claims "callers on this target must route those projections elsewhere" while the engine calls lfm_bf16_gemv_rne_{bf16,add_bf16} unconditionally (1487, 1521, 1554, 1620, 1734, 1746) and the x86 TU silently supplies the C++ twins (flashkern_x86.cpp:329, 358). The comment is not just stale, it is wrong about the link graph.

— K3

[sydneyrenee]

Implementation handoff — three related programs, three different dependency laws

Do not flatten decode, prefill, and Depthformer into one generic GEMM pipeline. They share leaves but not state semantics.

Backbone decode

One token is a strict causal recurrence. KV, ShortConv carry, cursor, and PRNG updates are committed state. Existing stage topology is mostly correct; repair leaf ownership and destination writes without creating new phases gratuitously.

Prefill

Rows are known together, but ShortConv/KV commits remain causal by row. M≤4 row grouping is an optimization, not permission to reorder carry recurrence. The C++ FIR/gate/carry body must become a faithful fused leaf with exactly the current rounding and commit order.

Depthformer

Codebooks form a recurrence. Per-codebook chains may fuse within their live set, but sampled embedding feedback and sampler state create real boundaries. Do not parallelize codebooks that depend on earlier sampled codes.

Repository-specific rules

• Unsupported ISA fails during plan construction; never branch to scalar during a pass.
• AArch64/x86 symbols must implement the same contract, not merely share a name.
• KV/V append should be the destination behavior of the owning leaf, not a C++ memcpy and not an assembly wrapper around an avoidable plane.
• Plane zeroing should disappear through full destination coverage where possible; otherwise it is an explicit assembly destination operation.
• No numerical retry exists for stateful work.
• Never restore Rust/Candle fallback or parity execution.

Work order

1. Use native inference: restore authoritative full-value faithfulness gates before FIFO fusion #170 to pin hidden/logit/KV/ShortConv/Depthformer/code traces.
2. Remove the decode byte-movement bypass using the already existing leaf.
3. Port prefill FIR while keeping row commit order visible in tests.
4. Resolve ISA specialization once at readiness.
5. Build a liveness graph for one Depthformer codebook before reducing phases.
6. Validate two consecutive turns and interruption boundaries.

A passing single-token test is insufficient. The acceptance evidence must include state before/after consecutive prefill/decode/codebook operations.