Problem
The immutable weight image follows the byte-view design, but persistent conversation state does not. LfmConversation currently owns a forest of independent std::vector payloads:
- six attention layers each allocate separate K and V buffers;
- ten ShortConv layers each allocate a separate carry buffer;
- RoPE retains BF16 cosine/sine plus complete F32 construction copies;
- hidden, resampled PCM, mel, and adapted rows are additional independent allocations.
At the production geometry (128,000 context rows plus 256 runway rows), recurrence alone is approximately 1,576,091,648 bytes across 22 payload allocations. RoPE retains another 49,250,304 bytes, including approximately 32.8 MiB of F32 construction/refill storage after initial publication.
These allocations happen before readiness, so they do not violate the zero-post-readiness gate. They still weaken ownership proof, complicate hibernation/snapshot geometry, fragment a multi-gigabyte conversation, and retain much more RoPE scratch than rollover requires.
Required architecture
Build a checked persistent-state layout at conversation construction:
- one 64/128-byte-aligned conversation-state allocation;
- exact byte-offset views for every attention K/V plane and ShortConv carry plane;
- checked arithmetic for physical rows, heads, dimensions, layer counts, and dtype bytes;
- immutable layout metadata sealed before the conversation becomes ready;
- no C++ typed dereference of potentially unaligned checkpoint bytes;
- no tensor objects, Rust loader, weight copy, dtype staging, transpose, or repack.
RoPE must not retain complete F32 cosine/sine twins:
- generate directly into the final BF16 destination where faithful;
- otherwise retain only bounded formula-derived refill scratch sized to the maximum rollover/refill operation;
- preserve exact position, theta, rounding, and rollover behavior.
Routes and Flashkern passes borrow generation-checked views into this arena. A physical worker or coroutine frame never owns persistent numerical state.
Verification
- Assert the computed offset map, alignment, total bytes, and non-overlap for every supported production geometry.
- Assert one persistent numerical payload allocation per conversation after plan/model objects are excluded.
- Compare complete KV, ShortConv, RoPE, hidden, logits, sampled codes, and PCM against the current native implementation before deleting the old vectors.
- Exercise 128k rollover, cancellation at every pass boundary, and two conversations sharing one immutable image.
- Run ASan/UBSan/TSan owner-drop and stale-view tests.
- Prove zero allocation/growth after readiness.
- Record before/after resident bytes and allocation count in the real-checkpoint evidence.
- Delete the vector-backed implementation in the same cutover; retain no compatibility path.
Acceptance gates
Related
Problem
The immutable weight image follows the byte-view design, but persistent conversation state does not.
LfmConversationcurrently owns a forest of independentstd::vectorpayloads:At the production geometry (128,000 context rows plus 256 runway rows), recurrence alone is approximately 1,576,091,648 bytes across 22 payload allocations. RoPE retains another 49,250,304 bytes, including approximately 32.8 MiB of F32 construction/refill storage after initial publication.
These allocations happen before readiness, so they do not violate the zero-post-readiness gate. They still weaken ownership proof, complicate hibernation/snapshot geometry, fragment a multi-gigabyte conversation, and retain much more RoPE scratch than rollover requires.
Required architecture
Build a checked persistent-state layout at conversation construction:
RoPE must not retain complete F32 cosine/sine twins:
Routes and Flashkern passes borrow generation-checked views into this arena. A physical worker or coroutine frame never owns persistent numerical state.
Verification
Acceptance gates
std::vectorcan reallocate after publication.Related