Summary
The official Linux x64 Vulkan release b10018-1.3.0 produces corrupted Gemma 4 31B output: control tokens followed by gibberish/repetition. The same GGUF produces correct output with a separate pre-existing upstream llama.cpp binary on the same host/GPU.
This also makes the bundled Gemma 4 MTP path unusable: with the HauhauCS target + provided draft-mtp head, draft acceptance was 0.00026 (4 accepted / 15318 generated).
Environment
- Atomic release:
llama-turboquant-linux-x64-vulkan.tar.gz, b10018-1.3.0 (build 10408, commit 5bc5c248d)
- OS: Ubuntu 22.04.5 LTS x86_64
- GPU: NVIDIA GeForce RTX 5090, driver 580.119.02
- Vulkan device is detected and model fully offloads successfully.
Reproduction
Model:
llmfan46/gemma-4-31B-it-uncensored-heretic-GGUF
File:
gemma-4-31B-it-uncensored-heretic-Q4_K_M.gguf
Atomic command (MTP disabled to isolate the base decode path):
atomic-llama-cli \
-m gemma-4-31B-it-uncensored-heretic-Q4_K_M.gguf \
-ngl 99 -ctk turbo4 -ctv turbo4 -fa on -c 4096 -t 14 \
-n 32 -p '请只回答：测试成功' -st \
--no-display-prompt --no-warmup -co off \
--reasoning off --reasoning-format none
Actual result
<|channel>thought
<channel|>H lC/ or single ly single l orP wife/ deSまだまだ deH ly single l or single lL,C// or
The server API shows the same corruption. It also logs:
W load: control-looking token: 212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
W load: special_eog_ids contains '<|tool_response>', removing '</s>' token from EOG list
I reproduced the same behavior with a second Gemma 4 31B target (HauhauCS/Gemma4-31B-QAT-Uncensored-HauhauCS-Balanced-MTP) both with and without its MTP drafter, and with the embedded template and --chat-template gemma.
Control result
On the same host, GPU and target GGUF, a pre-existing upstream llama-cli build (b1-876a432) returns the expected text:
<|channel>thought
<channel|>测试成功
It runs at about 45 tok/s in this small test. Therefore this does not appear to be a damaged download, a target-model issue, or an MTP-only issue.
Expected result
Atomic's Linux Vulkan release should generate coherent content from the same Gemma 4 GGUF, including when TurboQuant KV cache is enabled. If there is a known Gemma 4 incompatibility in this release/backend, please document it and consider disabling the affected configuration rather than returning silent/corrupted output.
Summary
The official Linux x64 Vulkan release
b10018-1.3.0produces corrupted Gemma 4 31B output: control tokens followed by gibberish/repetition. The same GGUF produces correct output with a separate pre-existing upstreamllama.cppbinary on the same host/GPU.This also makes the bundled Gemma 4 MTP path unusable: with the HauhauCS target + provided
draft-mtphead, draft acceptance was0.00026(4 accepted / 15318 generated).Environment
llama-turboquant-linux-x64-vulkan.tar.gz,b10018-1.3.0(build 10408, commit5bc5c248d)Reproduction
Model:
llmfan46/gemma-4-31B-it-uncensored-heretic-GGUFFile:
gemma-4-31B-it-uncensored-heretic-Q4_K_M.ggufAtomic command (MTP disabled to isolate the base decode path):
atomic-llama-cli \ -m gemma-4-31B-it-uncensored-heretic-Q4_K_M.gguf \ -ngl 99 -ctk turbo4 -ctv turbo4 -fa on -c 4096 -t 14 \ -n 32 -p '请只回答：测试成功' -st \ --no-display-prompt --no-warmup -co off \ --reasoning off --reasoning-format noneActual result
The server API shows the same corruption. It also logs:
I reproduced the same behavior with a second Gemma 4 31B target (
HauhauCS/Gemma4-31B-QAT-Uncensored-HauhauCS-Balanced-MTP) both with and without its MTP drafter, and with the embedded template and--chat-template gemma.Control result
On the same host, GPU and target GGUF, a pre-existing upstream
llama-clibuild (b1-876a432) returns the expected text:It runs at about 45 tok/s in this small test. Therefore this does not appear to be a damaged download, a target-model issue, or an MTP-only issue.
Expected result
Atomic's Linux Vulkan release should generate coherent content from the same Gemma 4 GGUF, including when TurboQuant KV cache is enabled. If there is a known Gemma 4 incompatibility in this release/backend, please document it and consider disabling the affected configuration rather than returning silent/corrupted output.