I'm documenting a reproducible setup for running current llama.cpp on a mixed AMD Vega system with both:

- Radeon RX Vega 64 / Vega10 / gfx900 (8GB HBM2)
- Radeon Instinct MI50 (Vega20) / gfx906 (16GB HBM2) presenting as Radeon VII

Repo + Dockerfile + test: https://github.com/anng-phtk/rocm-vega-llama.cpp

**Why this exists**

Modern ROCm still enumerates these old Vegas via HIP, but prebuilt rocBLAS/Tensile payloads no longer include legacy arches. So:

- rocminfo shows gfx900 and gfx906
- HIP compiles
- llama-server --list-devices works
- first real GEMM fails because rocBLAS kernels are missing

This setup combines:
- ROCm 7.2.1 image patched for gfx906 (mixa3607/rocm-gfx906:7.2.1-complete)
- gfx900 rocBLAS/Tensile payloads copied from ROCm 6.3.4
- llama.cpp built for GPU_TARGETS="gfx900;gfx906"

**Tested hardware**

- Ryzen 5 3600 / 64GB DDR4
- Vega 64 8GB (gfx900)
- MI50 16GB (gfx906)
- Docker with /dev/kfd + /dev/dri

Result: Mixed Vega64 + MI50 runs Gemma 3 27B Q4_K_M under ROCm 7.2.1 once both payloads are present. This is NOT officially supported — experimental, use at own risk.

**1. Build the mixed image**

Create Dockerfile.rocm72-gfx900-gfx906:

FROM rocm/dev-ubuntu-22.04:6.3.4 AS rocm6-libs
RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends rocblas && rm -rf /var/lib/apt/lists/*

FROM mixa3607/rocm-gfx906:7.2.1-complete
COPY --from=rocm6-libs /opt/rocm/lib/rocblas/library/ /tmp/rocm6-rocblas/
RUN set -eux; \
    ROCBLAS_LIB="/opt/rocm/lib/rocblas/library"; \
    find /tmp/rocm6-rocblas -maxdepth 1 -type f -name '*gfx900*' -exec cp -v {} "$ROCBLAS_LIB/" \;; \
    rm -rf /tmp/rocm6-rocblas
WORKDIR /workspace
CMD ["/bin/bash"]

Build:
sudo docker build -t rocm72-gfx900-gfx906 -f Dockerfile.rocm72-gfx900-gfx906 .

Final image: ~128 gfx900 files + 156 gfx906 files (.co, .hsaco, .dat)

**2. Verify enumeration**

sudo docker run --rm -it --device=/dev/kfd --device=/dev/dri --group-add video --group-add render rocm72-gfx900-gfx906 bash
rocminfo | grep -E 'Name:.*gfx(900|906)'

Expected: Name: gfx900, Name: gfx906

**3. Prove rocBLAS actually runs on gfx900**

Device detection is not enough. SGEMM test on device 0 (Vega64):

HIP devices: 2
device 0: AMD Radeon RX Vega arch=gfx900
device 1: AMD Radeon VII arch=gfx906
Calling rocblas_sgemm on device 0...
rocblas_sgemm returned: 0
hipDeviceSynchronize: no error

Test program is test-rocblas.cpp in the repo.

**4. Build llama.cpp for both**

Mount host llama.cpp to keep build persistent:

sudo docker run --rm -it -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add video --group-add render -v /home/$USER/Development/llama.cpp:/workspace/llama.cpp -v /home/$USER/Downloads:/models rocm72-gfx900-gfx906 bash

Inside:
cd /workspace/llama.cpp
cmake -B build-rocm72-dual -DGGML_HIP=ON -DGPU_TARGETS="gfx900;gfx906" -DCMAKE_BUILD_TYPE=Release -DLLAMA_BUILD_SERVER=ON
cmake --build build-rocm72-dual --target llama-server -j$(nproc)
./build-rocm72-dual/bin/llama-server --list-devices

Should show ROCm0: Vega 64 (8176 MiB) + ROCm1: Radeon VII (16368 MiB)

**5. Mixed-GPU run — the important part**

./build-rocm72-dual/bin/llama-server -m /models/gemma-4-26B-A4B-it-UD-Q4_K_M.gguf --device ROCm0,ROCm1 -ngl 999 --split-mode layer --tensor-split 1,4 -c 8192 -np 1 -b 512 -ub 256 --host 0.0.0.0 --port 8080

Why 1,4 not 1,2:

1,2 overloaded the 8GB Vega64 and caused multi-second stalls — browser unresponsive, then inference resumed. 1,4 left headroom and was smooth.

Observed with 1,4:
~57.6 tok/s low context
~56.5 tok/s @ 3.6K
~55.5 tok/s @ 4.4K
~54.3 tok/s @ 6.6K
~53.8 tok/s near 8K
~52.8 tok/s @ 8191/8192
no long stalls



**6. Single MI50 baseline**

Same model on single MI50:
~58-59 tok/s low context
~52-53 tok/s near 8K

Way better than Vulkan path for this model.

**7. Notes**

- Keep custom ROCm image separate from host
- Use bind mounts so docker --rm is safe
- HIP_VISIBLE_DEVICES=1 for single-GPU tests
- Don't set HSA_OVERRIDE_GFX_VERSION — real gfx900/gfx906 enumerate correctly here

Credits: mixa3607/rocm-gfx906:7.2.1-complete, daimonionnn/amd-vega-rocm-vulkan-llm-toolkit, Frigate Vega backport experiments.

Full Dockerfile + test.cpp + commands in repo: https://github.com/anng-phtk/rocm-vega-llama.cpp

Happy to answer questions about mixed Vega setups. I tested on real Vega64 + MI50 hardware and I have an extra Vega64 if anyone in US is looking to replicate — DM me.

ROCm 7.2.1 + llama.cpp on mixed Vega GPUs (gfx900 Vega64 + gfx906 MI50) — got 52-57 tok/s on Gemma 3 27B Q4 with patched rocBLAS payloads [repo + Dockerfile]