Compile bug: nvcc fatal : Unsupported gpu architecture 'compute_120'

[jacekpoplawski]

Git commit

commit 3f3769b (HEAD -> master, origin/master, origin/HEAD)

Operating systems

Windows

GGML backends

CUDA

Problem description & steps to reproduce

After replacing 3090 with 5070 I see compilation error:
nvcc fatal : Unsupported gpu architecture 'compute_120'
I found this: ggml-org/whisper.cpp#3030
Adding -DCMAKE_CUDA_ARCHITECTURES="86" solved my llama.cpp compilation problem.
Should this fix be merged also into llama.cpp?

First Bad Commit

No response

Compile command

cmake -DGGML_CUDA=ON -DLLAMA_CURL=OFF .. 
cmake --build . --config Release -j 30

Relevant log output

nvcc fatal : Unsupported gpu architecture 'compute_120'

[JohannesGaessler]

Does deleting and re-creating the CMake build directory fix the issue?

[JohannesGaessler]

Just to make it absolutely clear: when you replace the GPU you also have to recompile the project because the generated GPU code is specific to the GPU compute capability. Also, make sure that your CUDA version is recent enough to support RTX 5000.

[jacekpoplawski]

I always create new build dir, that's the point.
I just tried again with the commands above and I see:
nvcc fatal : Unsupported gpu architecture 'compute_120'

[JohannesGaessler]

Ah sorry, I think I didn't read your error message correctly. The llama.cpp default is to compile for the "native" architecture, meaning to compile for the GPUs connected for the system. With a connected RTX 5000 GPU it tries to build for compute capability 12.0 which is unsupported (I suspect your CUDA version is too old). If you specify compute capability 8.6 the code is built for RTX 3000; because the code is forwards-compatible it also runs on RTX 5000. But this should not be made the llama.cpp default because it would not work for any older GPUs.

[Panchovix]

What CUDA version are you using? nvcc comes from CUDA itself, and you need 12.8 at least for blackwell 2.0.

[jacekpoplawski]

You were right, correctly updating to cuda_12.9.0_576.02_windows.exe resolved the issue.
It was likely caused by nvcc still pointing to version 12.6.