linux.md

Linux GPU Setup Guide

This guide covers setting up GPU support for DeepSeek R1 Local on Linux systems.

Note: Commands may need to be run with sudo if you are not in the docker group.

Prerequisites

• NVIDIA GPU
• Updated NVIDIA drivers
• Docker and Docker Compose

GPU Setup Steps

1. Install Docker and Docker Compose if not already installed:

make install-docker

2. Install NVIDIA Container Toolkit:

make setup-gpu

If successful, you should see something like the following, if not, see Troubleshooting:

Starting the Service

Start with GPU support:

docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d

Troubleshooting

1. If you see "docker: Error response from daemon: could not select device driver":

   • Ensure NVIDIA Container Toolkit is properly installed
   • Restart the Docker daemon:

     sudo systemctl restart docker

2. If GPU is not detected:

   • Check NVIDIA driver installation:

     nvidia-smi

   • Verify NVIDIA Container Toolkit setup:

     sudo nvidia-ctk runtime configure --runtime=docker
     sudo systemctl restart docker

   • If you have docker installed via a snap, remove it and install it via apt:

   make install-docker