ggml-cuda: enable GGML_CUDA_GRAPH_OPT by default on RDNA3.5

[roberteg16]

Summary

• The CUDA/HIP graph optimization pass (ggml_backend_cuda_graph_optimize) was previously opt-in, requiring GGML_CUDA_GRAPH_OPT=1.
• Default it to enabled on RDNA3.5 (GGML_CUDA_CC_IS_RDNA3_5) while keeping the env var as an explicit override: GGML_CUDA_GRAPH_OPT=0 disables it, =1 forces it on.
• All other architectures keep the previous behavior (off unless the env var is set to 1).
• The compute capability is only queried when the env var is unset, so the override path avoids the extra lookup.

Benchmarks

gfx1151 (RDNA3.5), llama-bench -ngl 999 -r 40; VLM via llama-mtmd-cli (decode, 128 tokens). OFF = GGML_CUDA_GRAPH_OPT=0, ON = GGML_CUDA_GRAPH_OPT=1 (new default).

Models:

• unsloth/Qwen3.5-0.8B-GGUF (Q4_K_M)
• unsloth/Qwen3.6-35B-A3B-GGUF (UD-Q4_K_M)
• unsloth/Qwen3.6-35B-A3B-GGUF (UD-Q4_K_M + mmproj-BF16, vision)

model	test	OFF t/s	ON t/s	delta
Qwen3.5-0.8B	pp128	7744	7734	-0.1%
Qwen3.5-0.8B	pp2024	9215	9192	-0.2%
Qwen3.5-0.8B	tg128	210.2	211.1	+0.4%
Qwen3.6-35B-A3B	pp128	1085	1093	+0.7%
Qwen3.6-35B-A3B	pp2024	1540	1555	+1.0%
Qwen3.6-35B-A3B	tg128	54.9	58.9	+7.3%
Qwen3.6-35B-A3B (VLM)	decode128	56.6	61.5	+8.7%

[mgehre-amd]

Which graph optimizations or covered by this flag? Which impact does this have on perf?
Is that enabling hipGraphs?

[roberteg16]

Which graph optimizations or covered by this flag? Is that enabling hipGraphs?

It has nothing to do with hipGraphs, the name is a bit unfortunate. It just enables concurrent execution of ops for two graph patterns:

• shared expert (the one we implement)
• k/v computation for attention (already in llama.cpp)

Which impact does this have on perf?

+7% on decode for MoE due to shared expert. I don't recall regressions for dense models

[roberteg16]

Which graph optimizations or covered by this flag? Which impact does this have on perf? Is that enabling hipGraphs?

I updated PR body with a table

[mgehre-amd]

Is this an alternative to #36? How do both relate?

[roberteg16]

Is this an alternative to #36? How do both relate?

• llamacpp already had k/v overlapping support and it was behind GGML_CUDA_GRAPH_OPT
• ggml-cuda: overlap the MoE shared expert on a separate stream #36 implemented overlapping for shared expert (also behind GGML_CUDA_GRAPH_OPT)
• This PR enables GGML_CUDA_GRAPH_OPT by default on RDNA3.5

[roberteg16]

@mgehre-amd friendly ping