OpenCL no /etc/OpenCL/vendors/nvidia.icd

[denisstrizhkin]

nvidia-container-toolkit does not expose /etc/OpenCL/vendors/nvidia.icd to containers currently. However, if I just copy this file from the host system into the container, then OpenCL works.

Is this the intended behavior for some particular reason? If not, I think it would be good to expose /etc/OpenCL/vendors/nvidia.icd to containers.

[qhaas]

This might be related to the vulkan icd issue

Update: I cannot get the ICD to automatically mount, but the libnvidia-opencl.so.1 library needed by the OpenCL ICD loader is in there. I suspect this fille name will be relatively stable given it tracks a symbolic link and thus can be placed inside the /etc/OpenCL/vendors/nvidia.icd file in the container image (assuming you stay in the nvidia ecosystem). But I agree, it should likely be auto-mounted.

Consider:

$ cat /etc/redhat-release 
Rocky Linux release 9.4 (Blue Onyx)

$ modinfo nvidia | grep ^version
version:        535.183.06

$ nvidia-container-toolkit --version
NVIDIA Container Runtime Hook version 1.16.1

$ docker version
Client: Docker Engine - Community
 Version:           27.2.1
...
Server: Docker Engine - Community
 Engine:
  Version:          27.2.1
...
 containerd:
  Version:          1.7.21
...
 runc:
  Version:          1.1.13
...
 docker-init:
  Version:          0.19.0
...

$ cat /etc/docker/daemon.json 
{
    "runtimes": {
        "nvidia": {
            "args": [],
            "path": "nvidia-container-runtime"
        }
    }
}

$ cat /etc/OpenCL/vendors/nvidia.icd 
libnvidia-opencl.so.1

$ file /usr/lib64/libnvidia-opencl.so.1
/usr/lib64/libnvidia-opencl.so.1: symbolic link to libnvidia-opencl.so.535.183.06

$ clinfo -l
Platform #0: NVIDIA CUDA
 `-- Device #0: Quadro P4000

$ cat opencl.Dockerfile 
FROM ubuntu:22.04

ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y clinfo && apt-get clean

RUN install -d /etc/OpenCL/vendors/ && \
    echo 'libnvidia-opencl.so.1' >> /etc/OpenCL/vendors/nvidia.icd
ENV NVIDIA_DRIVER_CAPABILITIES=all

$ docker build -t test:opencl -f opencl.Dockerfile .
...

$ docker run --runtime=nvidia --gpus all --rm -it test:opencl clinfo -l
Platform #0: NVIDIA CUDA
 `-- Device #0: Quadro P4000

[elezar]

Hi @denisstrizhkin, yes, I think this may be a similar issue in that we're not injecting the nvidia.icd file from the host or using an alternative mechanism such as the environment variables listed here https://github.com/KhronosGroup/OpenCL-ICD-Loader?tab=readme-ov-file#table-of-debug-environment-variables

One question I have, is the following. In the documentation it states that the full path to a library should be provided but your example uses only the library name. I this also supported?

Requiring the full path becomes an issue if the path in the container and the path on the host are different.

Update: To answer my own question: the loader uses dlopen under the hood, meaning that the library name should be fine assuming the library is in the Ldcache or locatable in the LD_LIBRARY_PATH: https://github.com/KhronosGroup/OpenCL-ICD-Loader/blob/3d27d7ca04d29fabe608a2372ce693601bcc4e81/loader/linux/icd_linux.c#L243-L251

[github-actions]

This issue is stale because it has been open 90 days with no activity. This issue will be closed in 30 days unless new comments are made or the stale label is removed. To skip these checks, apply the "lifecycle/frozen" label.