omlx.server - ERROR - [-] - POST /v1/chat/completions → 500 (unhandled): There is no Stream(gpu, 1) in current thread.

[boriken72]

• Regression introduced in: 0.4.0 (also present in 0.4.1)
• Last working version: 0.3.8
• Model: Qwen3-VL-8B-Instruct-MLX-4bit
• Error: There is no Stream(gpu, 1) in current thread on every POST /v1/chat/completions
• SpecPrefill: off
• Likely cause: 0.4.0 VLM loader thread refactor breaks Qwen3-VL's secondary GPU stream

[jundot]

Thanks for the report, @boriken72.

I tested Qwen3-VL-8B-Instruct-MLX-4bit on the current HEAD (d6ab811), and I was able to get a normal response without reproducing the There is no Stream(gpu, 1) in current thread error.

It is hard to identify the exact cause without server logs. Could you set Global Settings > Log level to debug, reproduce the same error, and attach the log file or paste the relevant log section around the failed request?

That should help me check whether this is a loader/thread issue or something specific to the local environment/configuration.

[ajsniderdvm-hue]

Root-caused this on an M1 Max running v0.4.1 + mlx-vlm 6f60ee4 + Qwen3-VL (reproduced 100% on both 4B and 8B 4-bit). Sharing because it is a different layer than the per-engine stream-creation fixes referenced above — which may be why HEAD doesn't repro in some setups while the bug class can persist.

Chain:

1. mlx_vlm/models/qwen3_vl/language.py — Qwen3VLRotaryEmbedding is a plain Python class (not nn.Module). Its __init__ builds inv_freq from lazy ops (mx.arange / power / divide) and stores it unevaluated.
2. Because it isn't an nn.Module, materialize_lazy_state's tree_flatten(model.parameters()) walk (9d5bed8, the fix(engine): per-engine threads to eliminate cross-engine stream contamination #1304/fix(scheduler,cache): materialize bf16 KV on owner thread to end async store-cache SIGABRT #1570 family) never reaches inv_freq. It stays lazy, pinned to the construction thread's default stream.
3. Under the per-engine executor split (Migrate _global_mlx_executor to per-engine threads now that mlx-lm 0.31.3 supports per-instance streams #1248), prefill runs on mlx-engine-<id>_0. The mRoPE applied to keys broadcasts self.inv_freq into the graph; the first mx.eval of the KV state from the engine thread dies with There is no Stream(gpu, 1) in current thread (gpu,1 = the construction thread's thread-local resolution).
4. Diagnostic signature for this variant: after the forward, mx.eval(cache.keys) fails while mx.eval(cache.values) succeeds — values never touch RoPE. The vision tower is unaffected because VisionRotaryEmbedding is an nn.Module and gets materialized.
5. Pre-0.4.0 this was invisible: loader and forward shared one global MLX thread.

Verified fix (one line, mlx-vlm side):

        inv_freq = 1.0 / (
            self.base ** (mx.arange(0, self.dim, 2).astype(mx.float32) / self.dim)
        )
        mx.eval(inv_freq)   # materialize; thread/stream-agnostic thereafter
        self.inv_freq = inv_freq

With only this change, v0.4.1 serves Qwen3-VL image requests correctly (single + concurrent, repeated runs, zero stream errors; previously 100% failure). Running in production here on v0.4.1 since — all probes green.

Suggested durable fixes: extend materialize_lazy_state to also walk non-Module attributes holding mx.arrays (or document that model code must not stash lazy arrays on plain objects); and/or mlx-vlm evals inv_freq at construction / makes the rotary an nn.Module.

@jundot happy to attach debug logs or the minimal repro harness if useful.

[viktike]

I've got the same error with the VLM version (with image prompt) Ministral and Mistral Small models. Text only prompts work.

Version: v0.4.4.rc2
Model: mlx-community/Ministral-3-14B-Instruct-2512-4bit