perf(cpu): schedule Cortex-A76 Q8 dot with AAPCS64 SDOT - #79
Open
richiejp wants to merge 7 commits into
Open
Conversation
FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5 [Codex]
Pin the Q8_K_XL vehicle, observed Pi hardware and PMU surface, recursive optimization method, correctness gates, and same-file llama.cpp floor before implementation. Record Pi evidence as pending without changing the binding 20-core Arm result. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5 [Codex]
Add a review-capped vt-op harness with deterministic quant GEMM fixtures, calibrated timing, cache-pressure profiles, JSON output, affinity and system metadata, plus grouped generic and Cortex-A76 perf_event_open counters. Keep unsupported and multiplexed events explicit and preserve the legacy quant benchmark. CPU-gated with GCC warning-as-error, clang-format, CTest CLI/schema checks, deterministic checksums, and direct 1-thread and 4-thread PMU runs. Pi execution and every performance claim remain pending. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5 [Codex]
Build ARM64 artifacts under buildx/QEMU, verify the exported binaries and exact Qwen3.5-2B tokens on Cortex-A76, and record the PMU-ranked portable baseline that selects Q8 SDOT as the first assembly candidate. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5 [Codex]
Inventory the measured Q8_0 DotProd opportunity, bind compiler-SDOT versus scheduled AAPCS64 gates, and claim the independent kernel row before implementation. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5 [Codex]
Add runtime-gated exact-order compiler and AAPCS64 variants, QEMU gates, and bind the physical-Pi compiler-gap result. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5 [Codex]
Record the same-file Raspberry Pi 5 speed, memory, and exact-output comparison, including the reproducible b9892 reconstruction and void intrusive-sampler attempt. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Codex:gpt-5 [Codex]
richiejp
marked this pull request as ready for review
August 7, 2026 12:10
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Row
KERNEL-CPU-A76-Q8-DOT— one row per PR.What changed
Adds the first optimized Q8_0 x Q8_0 provider above the portable C++ fallback:
an exact-order GCC/ACLE SDOT control and a hand-scheduled AAPCS64 Cortex-A76
leaf. Runtime HWCAP and MIDR selection enables assembly by default only on
Cortex-A76 with DotProd; unsupported CPUs retain the portable implementation.
It also adds explicit benchmark variants, direct exactness/edge tests, QEMU
ARM64 gates, and the binding physical-Pi compiler-gap evidence.
Evidence
scripts/agent-preflight.shpasses. The branch-specific record andcheckpoint checks pass; full preflight retains two unrelated failures
already present on
upstream/main: MiniMax-H3 merged-GEMM routing andfour undocumented DFlash/H3 tuning variables.
build-rpi5-cpu/tests/test_ops_quant_dot: 20/20 cases, 150,139assertions.
test_ops_quant_dot: 20/20 cases, 150,258assertions, followed by an explicit
a76-asmsmoke with the expectedchecksum.
indexed in
docs/bench-evidence/rpi5-a76-q8-dot-20260806.md.docs/STATUS.md,docs/BENCHMARKS.md,docs/FEATURES.md, roadmap, backend/feature/kernelmatrices, state and parity ledger.
check-agent-record.py,check-doc-checkpoint.py,check-now-current.py,check-model-checklist.py, andcheck-readme-structure.pypass.scripts/check-pr-size.py --base 16c848326 --head 9044c2a7d --branch row/KERNEL-CPU-A76-Q8-DOT: 387 non-exempt lines across eightfiles, within the 900-line cap for the eventual kernel-only range.
Speed claims
5, with randomized/interleaved arms,
throttled=0x0throughout, exactchecksums/tokens, and binding results recorded in
docs/BENCHMARKS.md.Assembly versus compiler SDOT is 3.66% faster at M=1/T1, 5.08% faster at
M=128/T1, and 3.69% faster at M=128/T4. In the recursive 64-token
Qwen3.5-2B gate it lowers median TTFT 1.55% and E2E 0.13%, with TPOT
neutral and every output byte-identical to the x86 golden.
Honest gaps
M=1/T4 is 2.43% slower than compiler SDOT (and 13.4% slower than portable),
so the row remains
GATING, notDONE. The threadpool partition, dominant BF16GEMM, peak memory, concurrent serving, and same-file Raspberry Pi llama.cpp
floor remain open. This PR makes no whole-engine competitor-parity claim.