AMD GPU configuration for Open Web UI

Apps and Virtualization
1

Trying to configure AMD GPU RX 6700XT to be visible for Open Web UI application.

I understand that it is required to install rocm driver for it (I didn’t know if it should be done in TrueNas Scale or inside application), but still it is not visible and showing the following errors:

2025-05-22 18:28:36.352664+00:00time=2025-05-22T21:28:36.352+03:00 level=WARN source=amd_linux.go:61 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
2025-05-22 18:28:36.353980+00:00time=2025-05-22T21:28:36.353+03:00 level=WARN source=amd_linux.go:443 msg="amdgpu detected, but no compatible rocm library found.  Either install rocm v6, or follow manual install instructions at https://github.com/ollama/ollama/blob/main/docs/linux.md#manual-install"
2025-05-22 18:28:36.353998+00:00time=2025-05-22T21:28:36.353+03:00 level=WARN source=amd_linux.go:348 msg="unable to verify rocm library: no suitable rocm found, falling back to CPU"

How can I add AMD GPU to be visible for Open Web UI?
Passthrough GPUs was configured in TrueNas Scale UI

P.S. I’ve managed to read a bit (not my strongest part) and downloaded ROCM image for ollama instead of the usual one.
Currently I can see that the GPU is visible, but the following error appears:

2025-05-23 07:38:38.445616+00:00time=2025-05-23T07:38:38.445Z level=ERROR source=amd_linux.go:407 msg="amdgpu devices detected but permission problems block access: kfd driver not loaded.  If running in a container, remember to include '--device /dev/kfd --device /dev/dri'"

I have also managed to use HSA_OVERRIDE_GFX_VERSION, because I have 6700xt, and the minimal supported one is 6800.

Currently I need to understand how can I include /dev/kfd and /dev/dri?

2

Managed to make it work only uploading the separate ollama using custom app (even the ollama with ROCm didn’t work)

It was also required to specify the rocm package and HSA_OVERRIDE_GFX_VERSION.

After that, ollama is working pretty quick with GPU support available.

3

Could you provide your solution ? I’ve been fighting against that one too.

From container to VM, it’s always been the same issue. Only once, i was able to make it work with a debian VM by killing the GUI.

Thanks a lot

4

Hey Julien,

I’m replying since I had the same issue as OP and figured it out. I wanted it documented in case anyone else runs into the same issue. I had this error when using the Ollama app, but I’m sure it is applicable to OpenWebUI as well. You can change the Docker image in the YAML and it should work.

I created a custom app using this YAML:

services:
  ollama:
    cap_drop:
      - ALL
    deploy:
      resources:
        limits:
          cpus: '8'
          memory: 8192M
    devices:
      - /dev/dri:/dev/dri
      - /dev/kfd:/dev/kfd
    environment:
      HSA_OVERRIDE_GFX_VERSION: 10.3.0
      NVIDIA_VISIBLE_DEVICES: void
      OLLAMA_HOST: 0.0.0.0:30068
      TZ: Etc/UTC
      UMASK: '002'
      UMASK_SET: '002'
    group_add:
      - 44
      - 107
      - 568
    healthcheck:
      interval: 10s
      retries: 30
      start_period: 10s
      test: timeout 1 bash -c 'cat < /dev/null > /dev/tcp/127.0.0.1/30068'
      timeout: 5s
    image: ollama/ollama:rocm
    platform: linux/amd64
    ports:
      - mode: ingress
        protocol: tcp
        published: 30068
        target: 30068
    privileged: False
    restart: unless-stopped
    security_opt:
      - no-new-privileges=true
    stdin_open: False
    tty: False
    user: '0:0'
    volumes:
      - bind:
          create_host_path: False
          propagation: rprivate
        read_only: False
        source: /mnt/.ix-apps/app_mounts/testollama/data
        target: /root/.ollama
        type: bind
volumes: {}
x-notes: >+
 x-portals: []

Obviously change the cpus, memory, and image fields to whatever applies to you, but these lines are the significant ones that fixed it for me:

**devices:**

** - /dev/dri:/dev/dri**
** - /dev/kfd:/dev/kfd**
** environment:**
** HSA_OVERRIDE_GFX_VERSION: 10.3.0**

The devices shouldn’t change regardless of your GPU model, but the HSA_OVERRIDE_GFX_VERSION will depend on your GPU model:

  • for GCN 5th gen based GPUs and APUs HSA_OVERRIDE_GFX_VERSION=9.0.0
  • for RDNA 1 based GPUs and APUs HSA_OVERRIDE_GFX_VERSION=10.1.0
  • for RDNA 2 based GPUs and APUs HSA_OVERRIDE_GFX_VERSION=10.3.0
  • for RDNA 3 based GPUs and APUs HSA_OVERRIDE_GFX_VERSION=11.0.0