Introduction

Recently, I decided to repurpose my old desktop PC into a home Ubuntu server for running containerized applications with GPU acceleration. The system has a NVIDIA RTX 2060 SUPER, and I was excited to leverage its power using Docker.

However, when I tried to run a Docker container with GPU access using:

docker run --gpus all nvidia/cuda:12.2.0-base nvidia-smi

I was met with this frustrating error:

docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]

The Real Issue

After checking my drivers and confirming that my GPU setup was correct, I discovered the root cause: I hadn’t installed the NVIDIA Container Toolkit. This toolkit is essential for enabling GPU support in Docker on Linux.

The Fix: Installing NVIDIA Container Toolkit

Here’s how you can fix the issue and get your Docker container running with GPU access:

1. Configure the NVIDIA container repository

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && \
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
      

2. Update and install the toolkit

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

3. Configure Docker to use NVIDIA runtime

sudo nvidia-ctk runtime configure --runtime=docker

4. Restart Docker

sudo systemctl restart docker

After completing these steps, test with:

docker run --gpus all nvidia/cuda:12.2.0-base nvidia-smi

This should return a successful `nvidia-smi` output, confirming GPU access within the container.

Conclusion

If you’re setting up a home server with GPU capabilities, remember to install the NVIDIA Container Toolkit — a small but critical step to enable GPU support in Docker. For official documentation, visit: NVIDIA Container Toolkit Installation Guide .

Feel free to share your experience or ask questions in the comments section!