TrueNAS 24.10.1 Update - No Nvidia driver

TrueNAS General
,
1

Hello everyone,

[TL;DR] - Cannot get my Nvidia GPU (1660 Super) to work after the update but it works fine on 24.10.0.2.

I’m a happy TrueNAS Scale user for about half a year. I started running docker containers in jails around August and transitioned to running them without jails after EE got released. I basically have custom yaml compose files with all my apps so I can easily version and manage them.

I updated recently from 24.10.0.2 to 24.10.1 and encountered an issue with the Nvidia driver. I discovered this when my Plex and Immich containers failed to start due to missing Nvidia device.

I started debugging by running nvidia-smi and it printed NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running. . I also checked /dev and there’s no dri folder.

What I tried (and has not worked):

  • Uncheck then Check Install NVIDIA Drivers in the Apps → Configure → Settings. Same result.
  • Many reboots.
  • Unchecking driver install and running nvidia-smi resulted in it being an unrecognizable command. Checking it and running nvidia-smi resulted in the error above.
  • Booting back to 24.10.0.2, deleting the 24.10.1 update and updating again.
  • Fresh install of 24.10.1 and restoring config.
  • Fresh install of 24.10.1 without restoring config.
  • Fresh installing 24.10.0.2, restoring config and updating to 24.10.1.
  • Choosing/Unsetting the application pool.

Everything works fine while on 24.10.0.2, so I do not know what to try next.

Something that could interfere with the update process is that both of my pools are encrypted with a passphrase. So they are not automatically decrypted at boot, thus if anything is supposed to run on the apps pool, it might not. But this is just a hunch.

You can find my server info in my signature, but I’m happy to provide anything else if I missed it.

Please let me know what else I can try.

TIA!

2

I am having issues with the upgrade as well… Install Nvidia drivers checkbox has disappeared after I unchecked it. I am not able to get it back. In my case, nvidia-smi works fine.

update: Mine appear to be related to the jlmkr script instead of the updates.

update-2: Working through issues, but documenting for future me…

  1. Need to update the jlmkr.py script to look for “nvidia-uvm” instead of “nvidia-current-uvm”
  2. There is an issue with Truenas not loading all the modules properly. Through a search of the forum, I found this post which mentioned nvidia-persistenced: https://forums.truenas.com/t/dev-nvidia-modeset-missing/7260/10
  3. TODO: Nvidia-container modules are different versions than expected and can’t create container.
3

Recreated the jail with the config changes from @dasunsrule32 and its happy now. Container virtual storage and mounts makes everything so easy.

4

Glad that you got it working! Unfortunately this won’t work for me bacause I’m not using jails anymore.

I’m still very interested in a fix, becasuse for now I have no other option than running 24.10.0.2.

5

The TrueNAS team identified the issue causing this. Aparently I had the Debug Kernel enabled and that caused the issue. More info here.

Thanks TrueNAS Team!

1 Like