SCALE not loading nvidia drivers

TrueNAS General
,
1

Hi folks!

I recently upgraded my SCALE install to Dragonfish from Cobia, and now nvidia-smi returns

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Running lspci |grep NVIDIA does return the card:

01:00.0 VGA compatible controller: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GP107GL High Definition Audio Controller (rev a1)

However, when running systemctl status systemd-modules-load.service , the nvidia service is nowhere to be found, and VFIO fails to load

● systemd-modules-load.service - Load Kernel Modules
     Loaded: loaded (/lib/systemd/system/systemd-modules-load.service; static)
    Drop-In: /etc/systemd/system/systemd-modules-load.service.d
             └─override.conf
     Active: active (exited) since Sun 2024-08-25 22:28:58 BST; 2h 27min ago
       Docs: man:systemd-modules-load.service(8)
             man:modules-load.d(5)
   Main PID: 2044 (code=exited, status=0/SUCCESS)
        CPU: 11ms

Aug 25 22:28:58 CobaltSoul systemd[1]: Starting systemd-modules-load.service - Load Kernel Modules...
Aug 25 22:28:58 CobaltSoul systemd-modules-load[2044]: Failed to find module 'vfio_virqfd'
Aug 25 22:28:58 CobaltSoul systemd-modules-load[2044]: Inserted module 'ioatdma'
Aug 25 22:28:58 CobaltSoul systemd-modules-load[2044]: Inserted module 'ntb_split'
Aug 25 22:28:58 CobaltSoul systemd-modules-load[2044]: Inserted module 'ntb_netdev'
Aug 25 22:28:58 CobaltSoul systemd[1]: Finished systemd-modules-load.service - Load Kernel Modules.

Does anyone know what I can do to fix this? According to Nvidia, the 1050Ti is still supported.

My specs are:

OS Version: TrueNAS-SCALE-24.04.2
CPU: Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz
GPU: Nvidia 1050 Ti (Rev a1)
MoBo: SuperMicro X10SLL-F
RAM: 32GB ECC

Thanks!

2

Unfortunately upgrading means that since the xeon you have you’ll need a second graphics card for you to use anything related to graphics card since truenas now needs a dedicated gpu

3

The supermicro MB he uses, has onboard IPMI with ASPEED AST2400 graphics. So passthrough is not a problem, but he didn’t mention he wants to use passthrough. Truenas will see and use the IPMI GPU.
Also using the single GPU on a system for apps is not a problem.

4

As Vlad said, the MoBo has onboard graphics. and previous to Upgrading, TrueNas showed both the graphic cards. Now It just doesn’t see any Nvidia cards at all.

5

Hi @Pidge,

Did your issue get fixed with latest version 24.1.0.2? In my tests it sees both Nvidia and Intel cards with no issues. For Nvidia cards you have to either tell Truenas Scale to install hte Nvidia drivers from the UI (in apps I think, but might be wrong) or use this command as instructed in the notes for the RC versions: midclt call -job docker.update '{"nvidia": true}'

Hope all is well. Have a nice day :slight_smile: