System
- OS: Windows 11 Home (Build 22610)
- GPU: NVIDIA GeForce RTX 3060 Laptop (6 GB VRAM)
- Driver: 595.79
- CUDA: 13.2
- Compute Capability: 8.6
- CPU: AMD Ryzen 9 5900HS
- Build:
prism-b1-62061f9 (July 14, 2026)
Configuration
llama-server.exe -m Bonsai-27B-Q1_0.gguf --host 0.0.0.0 --port 8080 -c 81920 -np 1 -ngl 99 -fa on -ctk q4_0 -ctv q4_0 --spec-type ngram-mod --spec-ngram-mod-n-match 24 --spec-ngram-mod-n-min 16 --spec-ngram-mod-n-max 32
Model: Bonsai-27B-Q1_0.gguf (3.53 GiB, Q1_0 quantization)
Context Window: 81920 tokens (verified working via llama-bench at d79872+p2048)
The Issue
After a multi-turn agent session at ~50K depth with the server running stable, a CUDA illegal memory access error occurred 9 seconds after a task was cancelled mid-stream:
[Time: 30.28.963.188] W srv stop: cancel task, id_task = 16951
[Time: 39.59.341.755] E CUDA error: an illegal memory access was encountered
[Time: 39.59.341.762] E current device: 0, in function ggml_cuda_mul_mat_q at
D:\a\llama.cpp\llama.cpp\ggml\src\ggml-cuda\mmq.cu:148
[Location: ggml-cuda.cu:103]
The crash is in the Q-matrix multiplication kernel (mmq.cu:148), not in the task cancellation code itself.
Observations
-
Server was stable before cancellation: Task 16951 ran for ~55 seconds, reaching depth ~50K tokens, with successful prompt cache reuse and normal GPU utilization.
-
Crash occurs post-cancellation, not during: The server continued running after the task was cancelled, but crashed 9 seconds later when attempting GPU operations.
-
Illegal memory access, not OOM: This is a pointer dereference error, not an out-of-memory condition. GPU had 77-86 MiB free at the time of the crash (confirmed via separate depth-sweep testing).
-
Context-dependent: The error manifests at 81920 context with deeply-cached KV state. Unrelated llama-bench crashes at d96224 (separate investigation) pointed to VRAM exhaustion, but this crash signature is different.
-
No reproduction with small prompts: Test cancellations at smaller contexts (~10K tokens) and with reduced prompts did not trigger the error, suggesting the crash requires a deep KV cache state to corrupt.
Hypothesis
The task cancellation logic may not be properly cleaning up GPU buffer state when a request is terminated mid-computation at high context depths. This could leave freed pointers or corrupted metadata in GPU memory, which the next matrix-multiply operation then dereferences illegally.
Reproduction (Observed)
- Run llama-server with -c 81920, q4_0/q4_0 KV caching
- Send a request with ~50K tokens (fits in context window)
- Allow it to process until deep in the KV cache (ideally >40K tokens consumed)
- Cancel the request mid-stream
- Server should clean up and remain stable; instead, a CUDA illegal memory access occurs within ~10 seconds
Logs
Full server session logs available on request. The crash signature is stable and reproducible under these conditions (though I was unable to trigger it with synthetic test cancellations, suggesting it may require specific real-world prompt/cache state to occur).
Impact
- Server crash requires manual restart
- Users cannot safely cancel long-running requests at high context depths without risking GPU memory corruption
- Affects agent/interactive use cases (like Hermes Desktop integration) where mid-stream cancellation is common
Workaround
Revert to (716 MiB free margin) if stability at high context is not critical. Do not cancel requests mid-stream when running near the context ceiling.
System
prism-b1-62061f9(July 14, 2026)Configuration
Model: Bonsai-27B-Q1_0.gguf (3.53 GiB, Q1_0 quantization)
Context Window: 81920 tokens (verified working via llama-bench at d79872+p2048)
The Issue
After a multi-turn agent session at ~50K depth with the server running stable, a CUDA illegal memory access error occurred 9 seconds after a task was cancelled mid-stream:
The crash is in the Q-matrix multiplication kernel (
mmq.cu:148), not in the task cancellation code itself.Observations
Server was stable before cancellation: Task 16951 ran for ~55 seconds, reaching depth ~50K tokens, with successful prompt cache reuse and normal GPU utilization.
Crash occurs post-cancellation, not during: The server continued running after the task was cancelled, but crashed 9 seconds later when attempting GPU operations.
Illegal memory access, not OOM: This is a pointer dereference error, not an out-of-memory condition. GPU had 77-86 MiB free at the time of the crash (confirmed via separate depth-sweep testing).
Context-dependent: The error manifests at 81920 context with deeply-cached KV state. Unrelated llama-bench crashes at d96224 (separate investigation) pointed to VRAM exhaustion, but this crash signature is different.
No reproduction with small prompts: Test cancellations at smaller contexts (~10K tokens) and with reduced prompts did not trigger the error, suggesting the crash requires a deep KV cache state to corrupt.
Hypothesis
The task cancellation logic may not be properly cleaning up GPU buffer state when a request is terminated mid-computation at high context depths. This could leave freed pointers or corrupted metadata in GPU memory, which the next matrix-multiply operation then dereferences illegally.
Reproduction (Observed)
Logs
Full server session logs available on request. The crash signature is stable and reproducible under these conditions (though I was unable to trigger it with synthetic test cancellations, suggesting it may require specific real-world prompt/cache state to occur).
Impact
Workaround
Revert to (716 MiB free margin) if stability at high context is not critical. Do not cancel requests mid-stream when running near the context ceiling.