opencl: fix q6_K flat mul_mat for Adreno A6x/A7x GPUs with older E031 compilers - #26476
Draft
wanghqc wants to merge 2 commits into
Draft
opencl: fix q6_K flat mul_mat for Adreno A6x/A7x GPUs with older E031 compilers #26476wanghqc wants to merge 2 commits into
wanghqc wants to merge 2 commits into
Conversation
…r SIGSEGV) The Adreno 740 (A7X) compiler E031.41 crashes inside clBuildProgram when building the flash_attn programs whose KV path is mixed-type or dequantized: flash_attn_f32_f16, flash_attn_f32_q8_0, flash_attn_f32_q4_0. It is a driver crash rather than a compile-error return, so build_program_from_source_ex() cannot catch it. The uniform f32 and f16 programs build correctly. Decline the three KV-convert variants on the A7X in supports_op so they never lazy-compile; those attention layers run on the CPU backend instead. Same idiom as the existing Intel DK=512 and X1E carve-outs. test-backend-ops FLASH_ATTN_EXT on the 740: 226 OK / 0 FAIL, previously exit 139. Other parts are unaffected - the gate is dead code there.
kernel_mul_mv_q6_K_f32_flat produces ~10x-wrong output on the older Adreno
E031 compilers while q4_K and q5_K are correct. Four codegen defects, each
confirmed on-device against the CPU reference:
1. 64-bit ulong arithmetic is miscompiled, so every weight and scale read
hit the wrong address - the primary cause, and why q5_K (int offsets)
was unaffected. The block index is computed in int and widened only
inside the pointer expression.
2. The vectorized dequant (int4/float4 bit-ops, convert_*4, dot()) is
miscompiled; the 6-bit weights are reconstructed and the dot done
scalar.
3. vload4 of the f32 activations is miscompiled; replaced by a
scalar-indexed load.
4. The accumulation is miscompiled unless a side effect forces the partial
sums to materialize. A printf under a guard the compiler cannot prove
false acts as a zero-cost optimizer barrier; its placement is
load-bearing.
The defect tracks the compiler, not the GPU generation: it reproduces on
E031.38 (Adreno 642L) and E031.41 (Adreno 740) and is fixed by E031.45
(Adreno 619), so the workarounds are gated on the compiler version. Where
they are not needed they cost real throughput - 42.4 -> 35.1 GFLOPS on an
Adreno 840 q6_K GEMV. The explicit compiler-type check is required, not
redundant: newer_than_or_same() is false for every non-E031 compiler, so
negating it alone would enable the workarounds on E17 and DX.
test-backend-ops MUL_MAT is 919/919 on the Adreno 740, 642L, 619, 840 and
850; the 740 and 642L were 909/919 before. The 642L additionally needs the
A6X per-kernel-program support to reach these tests at all.
github-actions
Bot
added
ggml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR is to workaround the issues seen on Adreno 740/642L GPUs with specific compiler version E031.
mul_matproduces wrong answers on the older Adreno E031 compilersm=16, k=256family), which is exactly theMUL_MAT909/919 seen on an Adreno 740 and anAdreno 642L today.
-D ADRENO_OLD_COMPILER=1.-
test-backend-ops MUL_MAT, all 919/919 OK with this change:Additional information
Over-applied where they are not needed the workarounds cost real throughput
Why the gate is on the compiler and not the GPU generation.
adreno_gen == A7X.-It would keep applying the workarounds to an A7X whose driver had been updated past the fix
Requirements