NVIDIA Kernel Module Change in TrueNAS 25.10 - What This Means for You

My need went well beyond what @zzzhouuu did (thanks it was essential!) - with that invaluable breadcrumb trail and my mate claude i got a solution that:

  1. uses github workflows to build truenas sysext environment (using free runners)
  2. auto checks for new nvdia open drivers and runs workflow
  3. lets me run a workflow for any truenas build or nvida driver version using a UI
  4. injects displaymodeselector in sysext at install time (not included in package as it has license agreement that prevents distribution)
  5. defines MIGs at each boot using systemd service in the sysext (WIP)
  6. assigns one compute MIG UUID to all containers with UUIDs cofnigured (WIP)
  7. auto re-application of sysext package on truenas update (WIP)

and i got this far after 72 hours…..

yes this is a blackwell 6000 pro workstation card with 4 MIGs (2 compute MIGs, 2 GFX migs)

@HoneyBadger please can ix-systems consider includig 580.x in the fall release so i don’t have maintaing the github repo i just had to create, these are the open drivers so should work just fine for my need. :slight_smile: oh an please include all normal user mode tools except displaymodeselector which i know you cant include (and include persistencd daemon)

truenas_admin@truenas1 ~ 17:45:13 $ sudo nvidia-smi
Sat Mar  7 17:45:15 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.18             Driver Version: 580.126.18     CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX PRO 6000 Blac...    Off |   00000000:0F:00.0 Off |                  Off |
| 30%   34C    P1             81W /  600W |    2681MiB /  97887MiB |     N/A      Default |
|                                         |                        |              Enabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| MIG devices:                                                                            |
+------------------+----------------------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |              Shared Memory-Usage |        Vol|        Shared         |
|      ID  ID  Dev |                Shared BAR1-Usage | SM     Unc| CE ENC  DEC  OFA  JPG |
|                  |                                  |        ECC|                       |
|==================+==================================+===========+=======================|
|  0    3   0   0  |              64MiB / 24192MiB    | 46    N/A |  1   1    1    0    1 |
|                  |               0MiB /  8327MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+
|  0    4   0   1  |              64MiB / 24192MiB    | 46    N/A |  1   1    1    0    1 |
|                  |               0MiB /  8327MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+
|  0    5   0   2  |            2489MiB / 24192MiB    | 46    N/A |  1   1    1    0    1 |
|                  |               0MiB /  8327MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+
|  0    6   0   3  |              64MiB / 24192MiB    | 46    N/A |  1   1    1    0    1 |
|                  |               0MiB /  8327MiB    |           |                       |
+------------------+----------------------------------+-----------+-----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0    5    0            33175      C   frigate.detector.onnx                   586MiB |
|    0    5    0            33177      C   frigate.embeddings_manager              470MiB |
|    0    5    0            34614      C   /usr/lib/ffmpeg/7.0/bin/ffmpeg          188MiB |
|    0    5    0            34640      C   /usr/lib/ffmpeg/7.0/bin/ffmpeg          186MiB |
|    0    5    0            34650      C   /usr/lib/ffmpeg/7.0/bin/ffmpeg          186MiB |
|    0    5    0            34654      C   /usr/lib/ffmpeg/7.0/bin/ffmpeg          188MiB |
|    0    5    0            34659      C   /usr/lib/ffmpeg/7.0/bin/ffmpeg          190MiB |
|    0    5    0            34664      C   /usr/lib/ffmpeg/7.0/bin/ffmpeg          190MiB |
|    0    5    0            34669      C   /usr/lib/ffmpeg/7.0/bin/ffmpeg          190MiB |
+-----------------------------------------------------------------------------------------+

truenas_admin@truenas1 ~ 17:23:04 $ nvidia-smi -L
GPU 0: NVIDIA RTX PRO 6000 Blackwell Workstation Edition (UUID: GPU-a1d956ee-5f28-8140-014f-234454549cfa)
  MIG 1g.24gb     Device  0: (UUID: MIG-f54d3804-de1c-5d53-b2df-189722b5c186)
  MIG 1g.24gb     Device  1: (UUID: MIG-dfe0cf87-9354-5326-a6a7-187f0c7ab4d3)
  MIG 1g.24gb     Device  2: (UUID: MIG-a0496dd0-fa2c-5546-953c-f6b5a63a8a1e)
  MIG 1g.24gb     Device  3: (UUID: MIG-87144e2e-17bb-5983-95c6-b84fa3d63613)

truenas_admin@truenas1 ~ 17:45:15 $ sudo nvidia-smi mig -lgi
+---------------------------------------------------------+
| GPU instances:                                          |
| GPU   Name               Profile  Instance   Placement  |
|                            ID       ID       Start:Size |
|=========================================================|
|   0  MIG 1g.24gb           14        5          6:3     |
+---------------------------------------------------------+
|   0  MIG 1g.24gb           14        6          9:3     |
+---------------------------------------------------------+
|   0  MIG 1g.24gb+gfx       47        3          0:3     |
+---------------------------------------------------------+
|   0  MIG 1g.24gb+gfx       47        4          3:3     |
+---------------------------------------------------------+

this method works to assign the MIG to a docker container

1 Like