I want to get my arc 750 working with ollama, but it doesnt seem to be using it with the docker container from the truenas catalog.
I found ipex-llm and llama.cpp as a way to get the intel GPU working. Has anyone had success with ollama and Intel GPU?
I wonder if I need to run this as an lxc.
Did you make the Arc available to the Docker container?
Run this from within the container and give the output:
ls -l /dev/dri
Does the process in Docker have permissions to the above?
this is what I get
total 0
crw-rw---- 1 root video 226, 0 Apr 16 14:16 card0
crw-rw---- 1 root 107 226, 128 Apr 16 14:16 renderD128
Yes I checked the box to pass through the “non nvidia” gpu to the container.
I believe it has permissions, its running at apps (568)
From that, it looks like you don’t have permissions.
Verify the userid running the process that needs to access the gpu from the container:
ps aux
Verify that userid is in the render group on the host.
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> root 1 0.2 0.6 3870624 915280 ? Ssl 14:16 0:31 /bin/ollama serve
> root 320147 0.0 0.0 2616 784 pts/0 Ss+ 18:20 0:00 /bin/sh
> root 320320 0.0 0.0 2616 784 pts/1 Ss+ 18:23 0:00 /bin/sh
> root 320543 532 1.8 4693132 2386132 ? Sl 18:27 18:07 /usr/bin/ollama runner --model /root/.ollama/models/blobs/sha256-5ee4f07cdb9beadbbb293e85803c569b01bd37ed059d271
> root 331859 1.0 0.0 2616 796 pts/2 Ss 18:30 0:00 /bin/sh
> root 331865 0.0 0.0 5900 1588 pts/2 R+ 18:30 0:00 ps aux
> #
This is inside the ollama container and I was wrong its running as root.
You could do it outside the container too, but probably easier from inside since the process list will be much shorter… If you know the name of the process, you could ps aux | grep [process name]
How do I do that?
I’ve never used Ollama or any other local AI model, so I’m certainly not an expert.
First, some good news: it’s not a permissions issue.
Now the news you’re probably not hoping for:
How did you install it? I think it’s safe to assume you installed it from the TrueNAS apps catalog.
I used a search engine to look up how to use Intel Arc with Ollama. From that, I learned that Ollama does not include Intel support and many people suggested using ipex-llm produced by Intel with the Ollama portal zip.
Then I converted the Ollama app from “Apps” to a custom app to get a view of the compose. I see that it is just the ollama/ollama image (that does not have Intel support).
You will need to make a custom container, use an instance, or VM to set it up.
Well, this is an interesting problem.
Do you know if ipex-LLM works with Open-Web-UI?
@honeybadger Can this be a feature request for the TrueNAS catalog to update their ollama image to include ipex-llm?
@Rocketplanner83 My only ARC card at the moment is a B-series so I unfortunately can’t poke at this as much as I’d like first-hand, due to xe driver issues.
Submitting an issue to request the feature on the Apps github directly is probably the way to do it, although let me see if I can find anyone internally with an A-series card.
I am trying with a custom app in dockage and first attempt is giving me an error.
> services:
> ipex-llm:
> image: intelanalytics/ipex-llm-inference-cpp-xpu:latest
> container_name: ipex-llm-container
> devices:
> - /dev/dri:/dev/dri # Map GPU devices
> privileged: false
> volumes:
> - /mnt/floki/ollama:/models # Mount your models directory
> environment:
> - no_proxy=localhost,127.0.0.1
> - bench_model=mistral-7b-v0.1.Q4_0.gguf # Example model, adjust as needed
> - DEVICE=Arc # Or iGPU, Flex, Max
> - OLLAMA_HOST=0.0.0.0 # Allow remote connections to Ollama
> ports:
> - 11434:11434 #Expose Ollama port if needed
> shm_size: 16g #Shared memory size
> mem_limit: 32g # Memory limit
> networks: {}
the error in dockge was “ipex-llm-container exited with code 0”
here is the link to the github page
Could you give more information from logs to determine why it exited?
You’re on the right path, though.
I was doing it in dockge where do I find more detailed logs?
Sometimes I feel like the only one with access to documentation, search engines, and LLMs 
Try this?
docker logs ipex-llm-container
lol I should try that more
The problem is in your screenshot…
The port is already allocated to something.
If you have another ollama container, perhaps the one you were trying to use at the start of this thread, stop the container so the port is free…
Logs?
Im not getting any when I run the logs comand
