Electric Eel Nvidia gpu passthrough support

[MG-LSJ]

When installing apps on the nightly build, gpu pass-through shows this.

Is nVidia gpu pass through supported or planned. And are there any workarounds.

[Go_blin]

If your GPU is Nvidia please see my post for solutions

@MG-LSJ

[MG-LSJ]

This one rt?

Truenas Scale 24.10 cannot recognize Tesla P4 graphics card driver

Starting in 24.10, we are no longer including a default Nvidia driver. This is to allow it to be updated separately from TrueNAS releases. There will be a UI selectable option available to install/update nvidia drivers before 24.10.0, but for now you can set it using midclt call -job docker.update '{"nvidia": true}'.

So we need to manually install nVidia drivers

[MG-LSJ]

Were you able to run immich? Or do we need to wait for the maintainers to update it.

[Go_blin]

Yes,I also couldn’t run Immich, and the developer told me I needed to wait.

@MG-LSJ

[Rocketplanner83]

@Go_blin

ouldn’t run Immich, and the developer told me I needed to

My immich wont seem to run with an intel iGPU. It was happy with nvidia.

[MrTea]

I was first running passthrough and had docker running through there for testing. Once i felt ready, I unticked it, but it took 2 restarts for nvidia-smi to start working. I noticed on nightly from last week I had the 560 driver installed, today it’s back to the 550.

Plex sees it and transcodes, Ollama works quick once it loads (think cpu is starting to age out), tried ComfyUI which I got working in the VM, but reaching a size limit since my boot drive is so small. Getting the “var/lib/docker/buildkit/containerdmeta.db no space” error.

**Just realized this was in regards to apps (app store), not in general. I haven’t used apps in a while, jailmaker than started switching over to native docker a few weeks ago here and there. I was referencing isolated GPU passthrough in advanced, for VM’s.

[rambro1stbud]

I struggled getting Electric Eel to recognize my Quadro P400, since it requires legacy drivers. Then, I stumbled upon this post.

After following the directions there, Nvidia passthrough became available as an option when installing Jellyfin, and hardware transcoding is tested and working. Note that you may also have to run this command from the solution linked above in addition to making sure nvidia-smi is up and running:

midclt call -job docker.update ‘{“nvidia”: true}’

I’m pretty new to pretty much all of this — TrueNAS, PC building, etc. — but if anyone has questions I’ll try to help.

[Sawtaytoes]

Is there a newer post available on how to install the NVIDIA drivers? I can’t remember how to enable aptitude in TrueNAS either.

I have an RTX 4060 for Plex transcoding, and I was told in the past NVIDIA passthrough “just works”, but apparently, that’s not the case.

Can I go back to Dragonfish or is it fine to stay on Electric Eel? Do I have to reinstall drivers each time I bump the OS version?

[neofusion]

@Sawtaytoes

Is there a newer post available on how to install the NVIDIA drivers?

Yes:

Nvidia GPU not showing for apps

Take a look at the Upgrade Notes section here for how to enable nvidia drivers in the BETA

@Sawtaytoes

I can’t remember how to enable aptitude in TrueNAS either.

Small correction, apt is short for Advanced Package Tool.
aptitude was a GUI front-end to apt but is, if I remember correctly, not maintained anymore.

[Autopilot2301]

I found that Jellyfin would not work with nvidia gpus, even if the “Install NDIVIA Drivers” option is checked.

[AVHWDeviceContext @ 0x55a7b2ebbec0] Cannot load libcuda.so.1
[AVHWDeviceContext @ 0x55a7b2ebbec0] Could not dynamically load CUDA

To get it working I had to adapt the capabilities:

deploy:

resources: {“reservations”: {“devices”: [{“capabilities”: [“compute”,“utility”,“video”], “device_ids”: [“all”], “driver”: “nvidia”}]}}

With that added to the compose file, transcoding is now working with nvidia gpus.

[duderuud]

Stupid question but where do you edit the yaml file exactly?

[Autopilot2301]

If you mean where is the yaml located, then you can find this within the hidden apps dataset:

/mnt/.ix-apps/app_configs/<>/versions/<>/templates/rendered/docker-compose.yaml

Since any changes are going to be overwitten, I suggest to make a new yaml file and use something like portainer or perhaps directly with the ‘custom app’ function.

[duderuud]

Not an expert here

Tried creating a new stack in Portainer with the default docker-compose from Jellyfin as a base and changed the specific deploy properties.
After shutting down the original Jellyfin docker the custom docker starts normally but when I change the quality settings of a stream to a lower quality I see the CPU usage jump up significantly.
So that’s not working…

One thing to notice, the original Jellyfin docker can run “nvidia-smi” but it never shows a running process.
Same goes for the custom Jellyfin docker.

Will try to create a custom app now…

FYI: My default docker compose file looks like this:
resources: {"limits": {"cpus": "4", "memory": "8192M"}, "reservations": {"devices": [{"capabilities": ["gpu"], "device_ids": ["GPU-<<long ID>>"], "driver": "nvidia"}]}}

[Autopilot2301]

Assuming you have set Nvidia NVENC under transcoding, the best bet to know what Jellyfin is doing would be to look in the transcode logs.
Also, nvidia-smi should be run in the host, I would run the command in the truenas shell to see if the transcode active (the process is jellyfin-ffmpeg).

You may also have to add the Jellyfin user (root) to the video group.

[rambro1stbud]

I mean, the app version of Jellyfin not showing a running process with nvidia-smi might be a problem. One thing you could try with your compose is adding an environment variable:

  - NVIDIA_VISIBLE_DEVICES=all

Not sure you need to get that granular with your deploy command. This is what’s working for me:

deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu

[Chris_Holzer]

I am not sure if this also applies to jellyfin, but I had the exact same issue with the Plex App.

The problem is that the TrueNAS middleware fails to detect the UUID of the nvidia GPU, and so it does not get passed to the container.

The ‘fix’ is to setup Plex (and mostlikly this applies to jellyfin as well) inside portainer, and then pass the UUID to the container.

See this for how to get the UUID: TrueNAS 24.10-RC.2 is Now Available! - #92 by Chris_Holzer

[rodneysing]

I took rambro1stbud’s suggestion, dumped my old Ollama container and created a new one with the following compose file, being carefully intentional to include the appropriate GPU environment variables and deployment resources:

name: ollama-project
services:

ollama:
container_name: ollama
restart: unless-stopped
image: ollama/ollama:latest
runtime: nvidia
environment:
- NVIDIA_VISIBLE_DEVICES=all
volumes:
- “/mnt/storage/windows_share/Apps/Ollama:/root/.ollama”
ports:
- 11434:11434
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
healthcheck:
test: ollama list || exit 1
interval: 10s
timeout: 30s
retries: 5
start_period: 10s
networks:
- ollama_network

ollama-models-pull:
container_name: ollama-models-pull
image: curlimages/curl:latest
command: >
http://ollama:11434/api/pull -d ‘{“name”:“llama3.1”}’
depends_on:
ollama:
condition: service_healthy
networks:
- ollama_network

networks:
ollama_network:
driver: bridge

Sadly, this didn’t work, as the poor response of the LLM model demonstrated the GPU resources are not being used at all.

I was able to get the UUID from the Nvidia Tesla P4 installed, with nvidia-smi -L:
GPU 0: Tesla P4 (UUID: GPU-7d073f23-6ec9-13d5-ea9b-52bcebf1f0a9)

But I don’t know what to manually do with it. Will this problem be fixed soon? I there a work-around? Or, will the fix show up in the next RC kit?

Thanks!

[Chris_Holzer]

@rodneysing

I was able to get the UUID from the Nvidia Tesla P4 installed, with nvidia-smi -L:
GPU 0: Tesla P4 (UUID: GPU-7d073f23-6ec9-13d5-ea9b-52bcebf1f0a9)

But I don’t know what to manually do with it.

Tried this in your compose file?

- NVIDIA_VISIBLE_DEVICES=GPU-7d073f23-6ec9-13d5-ea9b-52bcebf1f0a9
- NVIDIA_DRIVER_CAPABILITIES=all

[rodneysing]

Chris,

Thanks very much for the tip! I have not tried this, but will the first chance I get and will let you and the rest of the thread know.

Thanks again!

-Rodney