Path F format v3: persist per-head INT8 KV scales for INT8 + persistent KV interop

[ai-hpc]

Why

alpha.12 ships INT8 KV as the default (Path I, #62). But Path F's persistent KV cache files (format v2, #50) don't carry the per-head INT8 scales — saving INT8 bytes without the scales would silently produce garbage on the next turn's hydrate.

Workaround in alpha.12 is skip Path F save when kv_int8 is active, with a stderr warning. That means INT8 users lose the multi-turn warm-start TTFT win (Path F #45, ~−48 % TTFT on warm turns). This issue closes that gap.

Scope

Bump file format v2 → v3:

[header 128 B]
[tokens used_tokens × 4 B]
[KV body body_bytes]                   ← packed_used_bytes
[scales body used_tokens × n_kv_heads × 2 × sizeof(float)]   ← NEW in v3

The scales body adds ~9 KB per saved turn at Qwen3-4B (37 tokens × 8 kv_heads × 2 K/V × 4 B). Tiny vs the KV body (~5.5 MB at the same shape).

Phased plan

Phase	Scope	Effort
Fv3.1	Format bump v2 → v3 in src/persistence/kv_cache_file.{cpp,h}. Header version = 3. Read path detects v2 vs v3 and refuses v2 when kv_int8.	1 hr
Fv3.2	KVCachePool gains gather_scales_used_to_host() + scatter_scales_from_host() mirroring the existing gather_used_to_host / scatter_from_host.	2 hr
Fv3.3	Engine::generate() save path: also gather scales, pass to save_kv_to_file (new arg). Hydrate path: also scatter scales after the KV body.	2 hr
Fv3.4	Remove the alpha.12 "skip save when kv_int8" guard. Document INT8+persistent-KV interop.	30 min
Fv3.5	Smoke test: 2-turn conversation with --int8-kv + --conv-id, verify warm turn 2 hydrates correctly and produces coherent output.	30 min

Total ~6 hr calendar. Single PR feasible; could split into "format" and "engine wiring" if reviewability matters.

Acceptance

• test_kv_serialize and test_kv_int8_convert still pass.
• 2-turn test on Jetson with --int8-kv --conv-id chat-int8:
  • Turn 1: save succeeds (writes header v3 + scales region).
  • Turn 2: hydrates the matched prefix, output is coherent vs FP16 reference.

Not in scope

• Per-conversation scale eviction / size cap (Path F5 already handles total *.bin budget; the ~9 KB scales region barely moves the needle).
• v2 → v3 migration. Files saved by alpha.11–alpha.12 in FP16 mode are still v2; they continue to be loadable. v3 is only emitted when INT8 KV is active.