LocalAI not detecting nvidia gpu - WSL

[zone-zero]

I'm having difficulty getting localAI to pick up on my 4090.

system: Windows 11, WSL 2 (Ubuntu 22.04) docker desktop with WSL 2 integration enabled.

nvidia-container-toolkit installed.

output of nvidia-smi from the wsl system:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.76.04              Driver Version: 580.97         CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        On  |   00000000:2D:00.0  On |                  Off |
|  0%   45C    P8             28W /  450W |    2564MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A              26      G   /Xwayland                             N/A      |
|    0   N/A  N/A              39      G   /Xwayland                             N/A      |
+-----------------------------------------------------------------------------------------+

and from the localai container:

$ docker run --rm --runtime=nvidia --gpus all --entrypoint nvidia-smi localai/localai:master-gpu-nvidia-cuda-12
Fri Dec  5 06:04:14 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.76.04              Driver Version: 580.97         CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        On  |   00000000:2D:00.0  On |                  Off |
|  0%   45C    P8             23W /  450W |    2588MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A              26      G   /Xwayland                             N/A      |
|    0   N/A  N/A              39      G   /Xwayland                             N/A      |
+-----------------------------------------------------------------------------------------+

But localai itself is not detecting the gpu:

$ docker run --rm --runtime=nvidia --gpus all -e DEBUG=true localai/localai:master-gpu-nvidia-cuda-12
CPU info:
    ...
6:05AM DBG Setting logging to debug
6:05AM DBG GPUs gpus=[]
6:05AM DBG GPU vendor gpuVendor=
6:05AM DBG Total available VRAM vram=0
6:05AM INF LocalAI version: 024aa6a (024aa6a55b307ed9843424f398b7b10d685e1f61)
     ...
6:05AM DBG GPU count: 0

Any ideas? I've also tried the cuda-13 image with the same result.