Published on

Install cuda-12.6 on Fedora 40

Authors

Check that we have an Nvidia card

lspci -n -n -k | grep -i nvidia # ...

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1080] [10de:1b80] (rev a1)
	Subsystem: ZOTAC International (MCO) Ltd. Device [19da:1425]
	Kernel driver in use: nouveau
	Kernel modules: nouveau, nvidia_drm, nvidia
01:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
	Subsystem: ZOTAC International (MCO) Ltd. Device [19da:1425]
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel

Installing basic Nvidia drivers + CUDA

dnf install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm \ 
            https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm
dnf install kernel-devel

dnf install akmod-nvidia

dnf install xorg-x11-drv-nvidia-cuda
#wait until modinfo returned the nvidia driver version before rebooting
modinfo -F version nvidia
# this will show something when the module exists

Installing ffmpeg extras

# https://rpmfusion.org/Howto/NVIDIA#Current_GeForce.2FQuadro.2FTesla
dnf install xorg-x11-drv-nvidia-cuda-libs  # For ffmpeg support (?) - already included

Installing CUDA extras

dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/fedora39/x86_64/cuda-fedora39.repo
# This is required, so as not to crunch rpmfusion install
dnf module disable nvidia-driver  

# Now...
dnf install cuda
#dnf install cuda-toolkit-12-6 # included

nvidia-smi # WORKS :-)

cuDNN for Machine Learning

Failed attempt #1

wget https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/linux-x86_64/cudnn-linux-x86_64-9.4.0.58_cuda12-archive.tar.xz
xz -d cudnn-linux-x86_64-9.4.0.58_cuda12-archive.tar.xz

Maybe this isn't needed:

Failed attempt #2

  • Global install via python pip - probably bad idea : NIX THIS
dnf install python3-pip

python3 -m pip install nvidia-cudnn-cu12
# 573Mb : cudnn
# 379Mb : cublas

Successfully installed nvidia-cublas-cu12-12.6.1.4 nvidia-cudnn-cu12-9.4.0.58
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Instead, as a user...

  • Here, let's use uv, since that'll make the venv operations super-quick!
uv venv env312
. env312/bin/activate
uv pip install numpy  # Not 'required' by torch, apparently...
uv pip install torch 
uv pip install torchvision torchaudio

# And now test it:
python
>>> import torch
>>> #... no error displayed
>>> torch.cuda.is_available()
True
>>> quit()

Test out Ollama

As root (after checking out the source to be executed) :

curl -fsSL https://ollama.com/install.sh | sh
ps fax | grep ollama | grep serve  # Check it's running

#ollama run gemma2:2b # This is the same as::
ollama run gemma2:2b-instruct-q4_0 --verbose # Shows token rates
#   62 tok/sec on GTX 1080

# or (larger model)
ollama run gemma2:9b-instruct-q4_0 --verbose
#   28 tok/sec on GTX 1080

All done!

Discuss on Twitter