AdvancedEdge / Local 12 min read

AMD GPU + llama.cpp via ROCm (Quick Start)

Run GGUF models on Radeon RX 7900 / 6800 series with llama.cpp HIP backend — what works and what does not.

AMDROCmllama.cppHIPLinux

Supported hardware

ROCm support on consumer AMD GPUs is Linux-only and varies by architecture. RDNA3 (7900 XTX) and RDNA2 (6800 XT) have the best community reports. Windows ROCm for llama.cpp remains experimental.

text

Good: RX 7900 XTX, RX 6800 XT, RX 6900 XT (Linux)
Patchy: RX 6700 XT, older Polaris
Not supported: integrated Radeon graphics

Build llama.cpp with HIP

Install ROCm 6.x, then build with GGML_HIP=ON. Set HSA_OVERRIDE_GFX_VERSION if your GPU is not officially listed.

bash

# Ubuntu 22.04 example
sudo apt install rocm-dev

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_HIP=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
cmake --build build -j$(nproc)

./build/bin/llama-server -m ./model.gguf -ngl 99 -c 4096 --host 0.0.0.0

Deployment guides are educational. Each model is subject to its own license — read the official Hugging Face model card before downloading or deploying.