Use GPU in a VM (Passthrough only)
If you’re intentionally passing the GPU to a VM (e.g. Windows or Linux VM with GPU acceleration), then:
You should not install the NVIDIA driver on the Proxmox host.
Instead, install it inside the VM, and keep vfio-pci bound on the host.
Use GPU on the Proxmox Host or in LXC
Start by find the correct NVIDIA driver
https://www.nvidia.com/en-us/drivers
On the proxmox host :
sudo apt update
sudo apt install pve-headers-$(uname -r) build-essential dkms
wget https://us.download.nvidia.com/XFree86/Linux-x86_64/570.144/NVIDIA-Linux-x86_64-570.144.run
chmod +x ./NVIDIA-Linux-x86_64-570.144.run
./NVIDIA-Linux-x86_64-570.144.run
sudo ./NVIDIA-Linux-x86_64-570.144.run -dkms
nvidia-smi
nano /etc/modules-load.d/modules.confnvidia
nvidia_uvmls -al /dev/nvidia*root@pve:~# ls -al /dev/nvidia*
crw-rw-rw- 1 root root 195, 0 Apr 29 08:21 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Apr 29 08:21 /dev/nvidiactl
crw-rw-rw- 1 root root 511, 0 Apr 29 08:21 /dev/nvidia-uvm
crw-rw-rw- 1 root root 511, 1 Apr 29 08:21 /dev/nvidia-uvm-tools
/dev/nvidia-caps:
total 0
drwxr-xr-x 2 root root 80 Apr 29 08:21 .
drwxr-xr-x 20 root root 4800 Apr 29 08:21 ..
cr-------- 1 root root 236, 1 Apr 29 08:21 nvidia-cap1
cr--r--r-- 1 root root 236, 2 Apr 29 08:21 nvidia-cap2nano /etc/pve/lxc/115.conf 1lxc.cgroup2.devices.allow: c 195:* rwm
lxc.cgroup2.devices.allow: c 236:* rwm
lxc.cgroup2.devices.allow: c 511:* rwm
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-caps dev/nvidia-caps none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=filepct push 115 ./NVIDIA-Linux-x86_64-570.144.run /root/NVIDIA-Linux-x86_64-570.144.runOn Lxc :
sh NVIDIA-Linux-x86_64-570.144.run --no-kernel-module
nvidia-smiFor Docker:
# Add Nvidia repository key
apt install -y gpg
curl -fsSL https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/3bf863cc.pub | gpg --dearmor -o /etc/apt/keyrings/nvidia-archive-keyring.gpg
# Add Nvidia repository
echo "deb [signed-by=/etc/apt/keyrings/nvidia-archive-keyring.gpg] https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/ /" | tee /etc/apt/sources.list.d/nvidia-cuda-debian12.list
# Update package lists
apt update
# Install Nvidia container toolkit
apt install nvidia-container-toolkit
nano /etc/docker/daemon.json{
"default-runtime": "nvidia",
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"runtimeArgs": []
}
}
}sudo nvidia-ctk runtime configure --runtime=dockernano /etc/nvidia-container-runtime/config.toml
# Set no-cgroups to true
no-cgroups = trueFor testing
# Run a test Docker container to verify GPU usage
docker run --gpus all nvidia/cuda:12.6.1-base-ubuntu24.04 nvidia-smi
If needed before purge old nvidia driver
sudo apt remove --purge '^nvidia-.*'
sudo apt autoremoveSource :
- Be aware that one time the cgroup2 number as change after a full reboot (510->511) ↩︎