perf(cpu): bring up, benchmark, and profile Qwen3.5-2B on Raspberry Pi 5

.agents/NOW.md

@@ -22,7 +22,8 @@ checkpoint on `upstream/main` at `59674cf1d`.
 | Kimi-Linear-48B (KDA+NoPE-MLA+MoE) | **Full-model GB10 e2e RUNS** (bf16-resident §13): CPU+CUDA 13/13·656, no OOM. **Token gate NEAR-TIE 106/128** (6/8 token-exact) | device GDN/MLA islands + bf16 stream; 1.59 tok/s; default OFF |
 | 35B fresh grid | **BOUND** @`1ea26427`: tput 0.93-1.03x, c16 0.93x. INTAKE + Option A both **RESOLVED NEGATIVE** (H2D-out-of-capture tput WASH) | Real lever left: prefill glue (task #61) |
 | Qwen3.5-4B revalidation | 0.9971x @`59674cf1` (#35); TTFT/PSS pass, TPOT/ITL open | `docs/bench-evidence/` |
-| MXFP4 parity | **TERMINAL (`QUANT-CT-MXFP4-FINAL-STACK`)**: c1 1.020x PASS + mem 2.63x; c2-c8 0.962-0.969 GPU-intrinsic. Both last levers exhausted: num_splits cap `VT_FA2_NSPLITS_CAP` gated-OFF (c1-only, green, 32B strict char-identical); glue folds via `vt::FusedChain`, residual out-of-catalog Inductor GEMM-epilogue fusion (#46). `VT_MARLIN_DENSE` on | Record; branch not merged |
+| RPi5 A76 CPU | **R3 BASELINE GREEN:** QEMU-built artifact, 16/16 x86-token exact, PMU trace | Portable Q8 dot is 20.1% of user cycles; prove C++ SDOT/compiler gap, then A76 assembly A/B |
+| MXFP4 parity | **TERMINAL:** c1 1.020x pass; c2-c8 0.962-0.969 GPU-intrinsic; final cap/glue levers exhausted | Record; branch unmerged |
 | ROW-SERVE-ASYNC-DENSE-MIRROR | **LANDED+dgx-VERIFIED** (`f9c969ae`): #31 async mirror on classic dense Qwen3; gate RED→GREEN, SACRED 184/184 | Residual: sibling scope one-liner |
 
 In-flight branches (default-OFF, not pushed): `laguna-fp4proj-prod` (fp4),

.agents/backend-matrix.md

@@ -223,7 +223,7 @@ on every listed target.
 | `BACKEND-ABI-VT` | Backend registry, device/queue, capture, drop-in adapter ABI | platform contract `vllm/platforms/interface.py:67-229`; stable stream `csrc/libtorch_stable/torch_utils.h:76-82`; raw Marlin boundary `csrc/libtorch_stable/moe/marlin_moe_wna16/ops.cu:344-356` | queue ID [device.h:9](../include/vt/device.h#L9), explicit resources [backend.h:53](../include/vt/backend.h#L53), scalar/layout ABI [ops.h:13](../include/vt/ops.h#L13), CUDA workspace/raw probe [cuda_dropin.cu:104](../src/vt/cuda/cuda_dropin.cu#L104) | [ported ABI tests](../tests/vt/test_dropin_abi.cpp#L86); CPU 94/94 + repair-focused 1/1; GCC13 repair at [test_dropin_abi.cpp:245](../tests/vt/test_dropin_abi.cpp#L245); exact sm_121a all-target build 100%, focused CUDA/ABI CTest 2/2, sanitizer 9/9 + 196/196 with 0 errors/leaks, 35B/27B gates 2/2 (`1141b79`, evidence manifest `4adbe952…601`) | [drop-in ABI](specs/dropin-kernel-abi.md) | `ANCHOR-BACKFILL` - sm_80/sm_90a cross-build and unchanged-trace/model A/B-memory proof remain; common scalar forwarder and legacy backend-shim migration remain; no production family migrated | CLAIM-BACKEND-ABI-W0-GPU-1 |
 | `BACKEND-ATTN-REGISTRY` | Attention-backend registry + platform-driven priority selection (extensibility **item 4** — the second portability seam, now REALIZED). Backends SELF-REGISTER per `(DeviceType, name)`; `Platform::get_attn_backend_priority()` advertises a capability-ordered name list; the selector returns the first REGISTERED name. Selection is DATA (register + priority slot), not an inline code edit. The concrete attention KERNEL stays at the vt:: op-table (`vt::PagedAttention`→`GetOp`), already device-additive | `vllm/v1/attention/backends/registry.py` (self-registration), `vllm/platforms/cuda.py:361-470` (`get_valid_backends`/`get_attn_backend_cls`) + `:84-166` (`_get_backend_priorities`, non-MLA major-10 vs else), `cpu.py:75-87` (CPU_ATTN) @ pin `e24d1b24` | registry+selector [registry.h:44](../include/vllm/v1/attention/registry.h#L44), [registry.cpp:60](../src/vllm/v1/attention/registry.cpp#L60); priority [interface.h:92](../include/vllm/platforms/interface.h#L92), CUDA major-10/else [cuda.cpp:44](../src/vllm/platforms/cuda.cpp#L44), CPU [cpu.cpp:29](../src/vllm/platforms/cpu.cpp#L29); FLASH_ATTN self-reg [backend.cpp:83](../src/vllm/v1/attention/backend.cpp#L83), GDN [gdn_attn.cpp:153](../src/vllm/v1/attention/backends/gdn_attn.cpp#L153) | [registry tests](../tests/vllm/v1/attention/test_attn_backend_registry.cpp#L61) (self-register + Make/throw + CUDA/CPU priority order + first-registered walk + override; 8 cases/25 assertions) + [platform priority](../tests/vllm/platforms/test_platform.cpp#L53); clean CPU `-Werror` + full CPU CTest green; behavior-preserving ⇒ same FA2 attention selected (**DGX-CONFIRMED @ `2c732e7`: 27B 235/235 + 35B 315/315 token-exact, FA2 sm_121a, memcheck 0/315**) | [Platform seam plan](specs/extensibility-platform-seam-2026-07-18.md) (item 4) | `ANCHOR-BACKFILL` — item-4 seam LANDED CPU + DGX-CONFIRMED; FLASHINFER/TRITON_ATTN/FLEX/TURBOQUANT are named in the priority (fidelity) but unimplemented ⇒ walk falls through to FLASH_ATTN (the behavior-preserving outcome); MLA-branch priorities deferred until an MLA model ports | CLAIM-ATTN-REGISTRY-1 |
 | `BACKEND-PLATFORM` | Platform capability/memory-model seam (`is_cuda`/`is_cpu`, `is_unified_memory`, `has_device_capability`, `supported_dtypes`, `residency_policy`, `supports_graph_capture`; `get_attn_backend_priority` REALIZED by `BACKEND-ATTN-REGISTRY` item 4; **+ the S3 capability predicates `supports_fp8`/`cutlass_fp4_supported`/`opaque_attention_op`/`is_integrated_gpu`/`support_static_graph_mode`/`is_device_capability_family`, mirroring `interface.py:914,933,977,1058,441-476` + `nvfp4_utils.py:56`, added by `CLAIM-BACKEND-SEAM-S3-1` and consumed by the fp4/fp8/graph gates in `qwen3_5.cpp`**); self-registered per `DeviceType` via `CurrentPlatform()` | `vllm/platforms/interface.py:134-229` (`class Platform`), `:409-439` (get/has_device_capability), `:181-187` (supported_dtypes) @ pin `e24d1b24` | interface [interface.h:56](../include/vllm/platforms/interface.h#L56), registry/CurrentPlatform [platform.cpp:57](../src/vllm/platforms/platform.cpp#L57), CPU [cpu.cpp:11](../src/vllm/platforms/cpu.cpp#L11), CUDA [cuda.cpp:14](../src/vllm/platforms/cuda.cpp#L14); composes `vt::Backend` [backend.h:22](../include/vt/backend.h#L22) | [platform tests](../tests/vllm/platforms/test_platform.cpp#L28) (registration + CPU capabilities + has_device_capability), clean CPU `-Werror` + full CPU CTest; 7 memory-model/residency sites migrated (`runner.cpp`, `model_registry.cpp`, `qwen3_5.cpp`) — **per-tensor sites key on `GetPlatform(<obj>.device.type).is_cuda()` (the OBJECT's device), NOT the process-global `CurrentPlatform()`, which is accelerator-first and would mis-route a CPU queue/tensor on a GPU box (regression fixed 2026-07-18; red DGX CPU tests `test_platform`/`test_qwen27_dense_forward`); `CurrentPlatform()` is reserved for process-level "which accelerator" questions only**. DGX behavior-preserving model gates 27B 235/235 + 35B 315/315 CONFIRMED @  (CUDA -Werror-clean after a GCC13 dangling-pointer FP fix). **Item-2 residency_policy() CONSUMPTION LANDED (2026-07-19, `CLAIM-BACKEND-PLATFORM-2`):** the host-free / load-stream / DevicePool-cap decisions in `qwen3_5.cpp` now READ `GetPlatform(<obj>.device.type).residency_policy()` (per-device) instead of an inline `device.type`/env gate — host-free `qwen3_5.cpp:BuildMoeMarlinResident` via `ShouldReleaseHostWeights`, load-stream `Qwen3_5Model::PrepareMarlinResident` via `ShouldInterleaveLoadStream` (both `interface.h`), DevicePool soft cap via `residency_policy().device_pool_cap_bytes` (`DBuf`/`DevicePool`). `CudaPlatform::residency_policy().release_host_weights_after_upload` flipped **false→true** (now CONSUMED ⇒ reproduces today's GB10 host-free-after-Marlin-build EXACTLY); `MarlinMoeEnabled()` stays the orthogonal KERNEL-PATH gate; `VT_MOE_HOST_FREE`/`VT_MOE_LOADSTREAM` env stay as overrides. Consumption unit-tested `test_platform.cpp` (7 cases/43 assertions). A new (discrete) GPU sets `residency_policy()` values ⇒ ZERO model edit. **DGX gate PASSED @ `62fc0e0` (production flags CUTLASS sm120a + FA2 sm_121a + Triton AOT, one flock): clean CUDA `-Werror` 0 warn, 27B 235/235 + 35B 315/315 token-exact, 35B VmHWM ≈ 4.0 GiB (load-stream win preserved), memcheck 0 errors.** | [Platform seam plan](specs/extensibility-platform-seam-2026-07-18.md) | `ANCHOR-BACKFILL` — item-1 extraction DONE + DGX-CONFIRMED; item-2 residency consumption LANDED + DGX-PROVEN (`CLAIM-BACKEND-PLATFORM-2`); kernel-shape dispatch branches deliberately left for items 4/5 | CLAIM-BACKEND-PLATFORM-1 / CLAIM-BACKEND-PLATFORM-2 |
-| `BACKEND-CPU` | CPU correctness and production path | `platforms/cpu.py:42-125`, CPU ops rooted at `csrc/cpu/torch_bindings.cpp:123-139`; llama.cpp `ggml-cpu.c:471-610,3024-3390` owns the added native pool reference | [cpu_backend.cpp:11](../src/vt/cpu/cpu_backend.cpp#L11), [threadpool.cpp:78](../src/vt/cpu/cpu_threadpool.cpp#L78), [chunked ops](../src/vt/cpu/cpu_ops.cpp#L88) | [backend tests](../tests/vt/test_backend.cpp#L10), [op parity](../tests/parity/test_op_parity.cpp#L34), [threadpool/upstream-test port](../tests/vt/test_cpu_threadpool.cpp#L63); 1/3/20 full CPU suites + TSAN pass, but B4 speed/RSS and full model-context thread-safety gates remain open. **macOS UNBLOCKED 2026-07-22 (`CLAIM-BACKEND-FANOUT-1`, W0):** the CPU backend was SILENTLY BROKEN on macOS — `CMakeLists.txt` gated the registrar force-link behind `if(UNIX AND NOT APPLE)`, which is FALSE on Apple, so ld64 dropped every static registrar and `test_backend` failed 5/7 with `vt: no backend registered for device type 0`. Fixed to `if(APPLE) -force_load elseif(UNIX) --whole-archive`; `test_backend` is now **7/7 (18/18)** on the M4 and the WHOLE tree builds `-Werror`-clean there. This was a Metal prerequisite but the CPU backend is the bigger beneficiary: macOS is now a usable CPU-tier target (108,952 portable-tier assertions pass on Apple arm64). Two macOS-only test gaps remain, both PRE-EXISTING and unrelated to any backend: `test_serve_low_tools` (Linux-only `os.sched_getaffinity` / `POSIX_FADV_DONTNEED` in the Python bench tooling) and `test_safetensors` (`MappingRssKb` reads `/proc/self/smaps`) — recorded as debt in [environment.md](environment.md), not fixed | [threadpool leaf](specs/gguf-cpu-threadpool.md) | `PARTIAL` | - |
+| `BACKEND-CPU` | CPU correctness and production path | `platforms/cpu.py:42-125`, CPU ops rooted at `csrc/cpu/torch_bindings.cpp:123-139`; llama.cpp `ggml-cpu.c:471-610,3024-3390` owns the added native pool reference | [cpu_backend.cpp:11](../src/vt/cpu/cpu_backend.cpp#L11), [threadpool.cpp:78](../src/vt/cpu/cpu_threadpool.cpp#L78), [chunked ops](../src/vt/cpu/cpu_ops.cpp#L88) | [backend tests](../tests/vt/test_backend.cpp#L10), [op parity](../tests/parity/test_op_parity.cpp#L34), [threadpool/upstream-test port](../tests/vt/test_cpu_threadpool.cpp#L63); 1/3/20 full CPU suites + TSAN pass. The binding 20-core Arm/i8mm Qwen3.5-2B Q8 single-stream gate is at or ahead of same-file llama.cpp on prefill/decode/RSS; server concurrency remains open. **Raspberry Pi 5 / Cortex-A76 lane GATING 2026-08-06 (PR #65):** R0 pins the exact Q8_K_XL hash, four-core A76 DotProd/no-i8mm target, recursive gates and llama.cpp floor. R1 [PMU harness](../examples/cpu_kernel_bench/main.cpp) is CPU-gated. **R2-R3 GREEN:** QEMU-built AArch64 artifacts pass smoke and execute on the Pi; Qwen3.5-2B is exact 16/16 against the x86 token golden, four operation fixtures retain exact checksums, and unthrottled PMU baselines are captured. A zero-loss model trace ranks portable Q8 dot at 20.10% of user cycles (BF16 GEMM 57.76%); the Pi has DotProd but no i8mm, so C++ SDOT and AAPCS64 assembly A/B are the active next gate. No optimized or llama.cpp-parity result is claimed; no 20-core/i8mm result transfers. **macOS UNBLOCKED 2026-07-22 (`CLAIM-BACKEND-FANOUT-1`, W0):** the CPU registrar force-link is fixed; `test_backend` is **7/7 (18/18)** and the whole tree builds `-Werror`-clean on M4. Two unrelated macOS test gaps remain recorded in [environment.md](environment.md) | [threadpool leaf](specs/gguf-cpu-threadpool.md); [RPi5 Cortex-A76 campaign](specs/rpi5-cortex-a76-cpu-optimization.md) | `PARTIAL` | PR #65 |
 | `BACKEND-CPU-ZEN` | AMD Zen CPU with ZenDNN/zentorch dispatch and weight prepack | `platforms/zen_cpu.py:12-32`; detection `platforms/__init__.py:153-192`; `tests/test_zen_cpu_platform_detection.py:8-37` | generic CPU backend only; no Zen-specific dispatch | - | [CUDA inventory](specs/cuda-architecture-inventory.md); leaf spike required | `INVENTORIED` | - |
 | `BACKEND-ROCM` | AMD ROCm/HIP | `platforms/__init__.py:110-128`, `platforms/rocm.py:43-125`, ROCm ops rooted at `csrc/rocm/moe_q_gemm_rdna3.cu:1` | Lands through the realized seams: a `platforms/rocm.cpp` Platform + a `kROCM` attention TU self-registering via the [attn-registry seam](../include/vllm/v1/attention/registry.h#L44) + a `rocm.cpp` `get_attn_backend_priority()` slot (mirror `rocm.py`'s ROCM_ATTN/AITER ordering); ZERO selector/model/runner edit | - | [CUDA inventory](specs/cuda-architecture-inventory.md) | `INVENTORIED` | - |
 | `BACKEND-XPU` | Intel XPU loyal port; gating exploration E4 ([backends.md](backends.md) kernel sourcing: SYCL-vs-Level-Zero call + whether upstream XPU attention contracts translate 1:1; explore when T2 scheduling begins) | `platforms/__init__.py:131-150`, `platforms/xpu.py:103-125` | enum slot only [device.h:11](../include/vt/device.h#L11) | [unavailable-backend test](../tests/vt/test_backend.cpp#L37); **SPIKE-ONLY verdict, MEASURED 2026-07-22: HW-BLOCKED — no Intel GPU on any box we own, and no acquisition path recorded.** Compounding finding: the loyalty target is INCOMPLETE upstream — `vllm/platforms/xpu.py:11-13` imports the EXTERNAL `vllm_xpu_kernels._C`/`._moe_C`/`._xpu_C`, and a search of the pinned tree for `*.sycl`/`*.dp.cpp` returns NOTHING, so there is no in-tree SYCL kernel source to mirror 1:1. What stays doable with zero Intel hardware: the attention-selector/priority POLICY port (pure data, unit-testable), oneAPI DPC++ compile coverage, and SYCL kernels executed on the oneAPI OpenCL CPU device for unit-level numerics. **No e2e or performance gate is proposed, because none can be run.** No XPU backend code exists | [backend fan-out](specs/backend-fanout-metal-vulkan-xpu.md); [CUDA inventory](specs/cuda-architecture-inventory.md) | `SPIKE` | `CLAIM-BACKEND-FANOUT-1` |

.agents/feature-matrix.md

@@ -274,7 +274,7 @@ evidence.
 |---|---|---|---|---|
 | `BACKEND-CUDA-SM121` | GB10/sm121a | `PARTIAL` | gate workload built, traced, token/perf gated; full component-family coverage is open | [backend row](backend-matrix.md#cuda-target-rows) |
 | `BACKEND-CUDA-OTHER` | vLLM sm70/75/80/86/87/89/90/100/101/103/110/120 targets | `ACTIVE` | 9 CUDA arches build-supported (sm80/86/87/89/90a/100a/103a/110/120a, single-arch portable-kernels-only, `-Werror` clean, SASS emitted); sm70/75/101 not build-supported; no non-121a target is runtime-validated | [backend matrix](backend-matrix.md), [CUDA inventory](specs/cuda-architecture-inventory.md) |
-| `BACKEND-CPU` | production CPU | `PARTIAL` | persistent threadpool + chunked GEMM/row dispatch is 1/3/20-thread bit-identical and TSAN-clean; idle-host performance/RSS gate and compute-in-quant remain open | [backend matrix](backend-matrix.md) |
+| `BACKEND-CPU` | production CPU | `PARTIAL` | shared CPU path is correctness-gated and the 20-core Arm/i8mm Qwen3.5-2B single-stream llama.cpp floor is closed; server concurrency is open. Raspberry Pi 5 Cortex-A76 R2-R3 is green: QEMU-built artifact, exact 16/16 x86-token and fixture parity, PMU baseline and a zero-loss model trace. Portable Q8 dot is 20.10% of user cycles; SDOT C++/assembly optimization and the Pi llama.cpp floor remain open | [backend matrix](backend-matrix.md), [RPi5 spike](specs/rpi5-cortex-a76-cpu-optimization.md), [harness](../examples/cpu_kernel_bench/main.cpp) |
 | `BACKEND-ROCM` | ROCm | `INVENTORIED` | source/dispatch spike required; no "one flag" support claim | [backend matrix](backend-matrix.md) |
 | `BACKEND-MLX` | Apple Metal through MLX | `ACTIVE` | Metal/MLX skeleton ACTIVE: two models (OPT-125m, Qwen3-0.6B) run e2e + pass correctness, native-MSL GEMM, batched command buffers 1.50x (compute-bound at 98%+); optional MLX GEMM provider | [backend matrix](backend-matrix.md) |
 | `BACKEND-VULKAN` | Vulkan | `INVENTORIED` | runtime absent | [backend matrix](backend-matrix.md) |