Error 500: named symbol not found

[Nazar78]

Hi,

I'm having the following issue thus it's only using CPU. I do however recall it was working with few previous Windows 11 drivers. If possible I would like to remain using latest Nvidia drivers. Please help to advise. Thank you.

docker run -dit --name=WhisperASR -p 9000:9000 -e ASR_MODEL=tiny -e ASR_ENGINE=fast_whisper --gpus all onerahmet/openai-whisper-asr-webservice:latest-gpu

==========
== CUDA ==
==========

CUDA Version 11.8.0

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

[2024-05-23 11:13:55 +0000] [27] [INFO] Starting gunicorn 21.2.0
[2024-05-23 11:13:55 +0000] [27] [INFO] Listening at: http://0.0.0.0:9000 (27)
[2024-05-23 11:13:55 +0000] [27] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2024-05-23 11:13:55 +0000] [28] [INFO] Booting worker with pid: 28
/app/.venv/lib/python3.10/site-packages/torch/cuda/__init__.py:88: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 500: named symbol not found (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
[2024-05-23 11:13:56 +0000] [28] [INFO] Started server process [28]
[2024-05-23 11:13:56 +0000] [28] [INFO] Waiting for application startup.
[2024-05-23 11:13:56 +0000] [28] [INFO] Application startup complete.

# nvidia-smi
Thu May 23 11:33:17 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 555.42.03              Driver Version: 555.85         CUDA Version: 12.5     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3080        On  |   00000000:01:00.0  On |                  N/A |
|  0%   48C    P8             26W /  320W |     842MiB /  10240MiB |      3%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
#

[jet082]

This is a problem specific to docker and the recent 555.85. Install 552.44 or move the code off of docker and it should work.

[Nazar78]

NVIDIA/nvidia-container-toolkit#520

[Nazar78]

AbdBarho/stable-diffusion-webui-docker#694

Fix with Docker Desktop 4.31.