Add native MLX backend and backend benchmarks

[i386]

Summary

Adds the native MLX backend work, backend benchmarking, and the current Qwen2-focused MLX runtime fixes.

This branch currently depends on an mlx-rs integration branch rather than upstream oxideai/mlx-rs directly:

• cache trim support: oxiglade/mlx-rs#336
• native Qwen2 support: oxiglade/mlx-rs#337
• padded-buffer / KV reproduction: oxiglade/mlx-rs#338

mesh-llm is currently pinned to the fork integration commit:

• i386/mlx-rs@22432d4c4e45d171937678e0dfbe7d9c6684c241

What Is Working

• Native MLX backend starts faster than llama.cpp on the tested pairs.
• Llama-family MLX path supports exact-repeat and safe prefix-extension cache reuse.
• Qwen2 now runs through a native mlx_lm::models::qwen2 path instead of the old Llama-compatibility shim.
• Qwen2 correctness is fixed on the native MLX path.
• Qwen2 prompt reuse is using safe checkpointed prefix reuse instead of unsafe trimmed-cache reuse.
• Lower-level KV cache materialization fixes in the fork improved end-to-end Qwen2 latency.

What Still Needs Improvement

• llama.cpp is still ahead overall on TTFT and most sustained generation paths.
• Qwen2 MLX is improved and now benchmark-quality, but it does not consistently beat llama overall.
• Partial shared-prefix reuse for Qwen2 is still limited to safe checkpointed reuse rather than arbitrary trimmed-cache reuse.
• The benchmark environment is noisy enough that claims should stay narrow and tied to the specific focused runs below.

Stable Full Matrix

Latest full phased serial matrix previously added to this PR:

• matrix-20260329-174731

That matrix still shows the overall shape:

• MLX startup is faster.
• llama.cpp still wins most full-matrix TTFT and total-time comparisons.

Focused Qwen2 Progress

Earlier focused Qwen2 baseline after the lower-level copy() fix

Results:

• /tmp/qwen2-focused-kvcopy

Key MLX totals from that focused run:

• long-briefing: 1620.8 ms
• code-context: 851.4 ms
• cache-base: 1049.1 ms
• cache-extend: 1593.5 ms

Latest focused Qwen2 run after switching KV materialization to contiguous(false)

Results:

• /tmp/qwen2-focused-contiguous

Key read from the latest run:

• this is not a clean overall MLX victory over llama
• the run is noisy, so the claim should stay narrow
• the main useful signal is that cache-extend reached rough parity on total time while preserving the corrected Qwen2 output path

Latest focused Qwen2 numbers:

• main
  • long-briefing
    • llama: 95.9 ms TTFT, 2592.4 ms total
    • mlx: 128.9 ms TTFT, 3194.1 ms total
  • code-context
    • llama: 87.9 ms TTFT, 2069.7 ms total
    • mlx: 128.6 ms TTFT, 1644.5 ms total
• cache-sequence
  • cache-base
    • llama: 102.1 ms TTFT, 2391.5 ms total
    • mlx: 194.8 ms TTFT, 2748.6 ms total
  • cache-extend
    • llama: 87.7 ms TTFT, 3028.9 ms total
    • mlx: 118.8 ms TTFT, 3013.5 ms total

Objective Conclusion

This branch now has:

• a working native MLX backend
• native Qwen2 support in the fork
• safe Qwen2 checkpoint reuse
• lower-level KV materialization fixes that improved Qwen2 MLX behavior and latency

It does not yet show that MLX is broadly faster than llama.cpp.

The current best claim is narrower:

• Qwen2 MLX is materially less broken than earlier in this branch
• Qwen2 MLX latency improved through lower-level fork fixes
• on the latest focused run, Qwen2 MLX cache-extend total time reached rough parity with llama

[i386]

at some stage we will need to make this work with #56 - probably after merge