TrueNAS 26.0.0-BETA.1 NVIDIA 590.44.01 no longer supports Tesla P4 (10de:1bb3)

[BeastNAS]

TrueNAS 26.0.0-BETA.1 NVIDIA 590.44.01 driver does not support Tesla P4 (10de:1bb3) that worked under TrueNAS 25.10.x

Environment

TrueNAS Version:
26.0.0-BETA.1

Hardware:
Dell PowerEdge R630

GPU:
NVIDIA Tesla P4
PCI ID: 10de:1bb3

Previous Version:
TrueNAS 25.10.x

Issue Summary

After upgrading from TrueNAS 25.10.x to TrueNAS 26.0.0-BETA.1, the NVIDIA Tesla P4 is no longer recognized by the NVIDIA driver.

The card functioned correctly prior to the upgrade.

The GPU is visible on the PCI bus and appears healthy, however the NVIDIA 590.44.01 driver refuses to initialize the device.

Expected Behavior

The Tesla P4 should be recognized and initialized by the NVIDIA driver as it was under TrueNAS 25.10.x.

nvidia-smi should display the Tesla P4 and applications should be able to use the GPU.

Actual Behavior

nvidia-smi fails:

NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver.

Docker GPU containers fail with:

nvidia-container-cli: initialization error: nvml error: driver not loaded

Apps → GPU selection only shows the onboard Matrox G200eR2 and does not list the Tesla P4.

Evidence

GPU detected by PCI subsystem:

82:00.0 3D controller: NVIDIA Corporation GP104GL [Tesla P4] (rev a1)

PCI ID:

10de:1bb3

Kernel module version:

version: 590.44.01

Driver rejection from dmesg:

NVRM: The NVIDIA GPU 0000:82:00.0 (PCI ID: 10de:1bb3)
NVRM: nvidia.ko because it does not include the required GPU
NVRM: The NVIDIA probe routine failed for 1 device(s).
NVRM: None of the NVIDIA devices were initialized.

Verification Performed

Verified GPU is visible via lspci

Verified PCIe link is healthy:
Speed 8GT/s
Width x16

Verified no nouveau driver conflict

Verified card is not bound to vfio-pci

Verified NVIDIA kernel modules are installed

Verified Docker GPU containers fail due to NVML initialization failure

Verified Tesla P4 functioned correctly prior to upgrade to TrueNAS 26.0.0-BETA.1

Additional Notes

This appears to be a regression introduced by the NVIDIA 590.44.01 driver package included with TrueNAS 26.0.0-BETA.1.

The same hardware worked correctly on TrueNAS 25.10.x before the upgrade.

Because the storage pool has been upgraded, rollback to the previous release is not practical for validation.

root@beastnas[~]# midclt call system.version

TrueNAS-26.0.0-BETA.1
root@beastnas[~]# lspci -nnk -s 82:00.0

82:00.0 3D controller [0302]: NVIDIA Corporation GP104GL [Tesla P4] [10de:1bb3] (rev a1)
Subsystem: NVIDIA Corporation Device [10de:11d8]
Kernel modules: nouveau, nvidia_drm, nvidia
root@beastnas[~]# lspci -vv -s 82:00.0

82:00.0 3D controller: NVIDIA Corporation GP104GL [Tesla P4] (rev a1)
Subsystem: NVIDIA Corporation Device 11d8
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 95
IOMMU group: 4
Region 0: Memory at c8000000 (32-bit, non-prefetchable) [size=16M]
Region 1: Memory at 3ffe0000000 (64-bit, prefetchable) [size=256M]
Region 3: Memory at 3fff0000000 (64-bit, prefetchable) [size=32M]
Capabilities: [60] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [78] Express (v2) Endpoint, IntMsgNum 0
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 25W TEE-IO-
DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 256 bytes, MaxReadReq 4096 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM not supported
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 8GT/s, Width x16
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range AB, TimeoutDis+ NROPrPrP- LTR+
10BitTagComp- 10BitTagReq- OBFF Via message, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-
AtomicOpsCtl: ReqEn-
IDOReq- IDOCompl- LTR- EmergencyPowerReductionReq-
10BitTagReq- OBFF Disabled, EETLPPrefixBlk-
LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
Status: NegoPending- InProgress-
Capabilities: [250 v1] Latency Tolerance Reporting
Max snoop latency: 0ns
Max no snoop latency: 0ns
Capabilities: [128 v1] Power Budgeting
Capabilities: [900 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Kernel modules: nouveau, nvidia_drm, nvidia

root@beastnas[~]# modinfo nvidia | head -20

filename: /lib/modules/6.18.13-production+truenas/kernel/drivers/video/nvidia.ko
import_ns: DMA_BUF
alias: char-major-195-*
version: 590.44.01
supported: external
license: Dual MIT/GPL
firmware: nvidia/590.44.01/gsp_tu10x.bin
firmware: nvidia/590.44.01/gsp_ga10x.bin
softdep: pre: ecdh_generic,ecdsa_generic
srcversion: 5E16B1FC70F7A43C0111BFE
alias: pci:v000010DEdsvsdbc06sc80i00
alias: pci:v000010DEdsvsdbc03sc02i00
alias: pci:v000010DEdsvsdbc03sc00i00
alias: of:NTCnvidia,tegra264-displayC*
alias: of:NTCnvidia,tegra264-display
alias: of:NTCnvidia,tegra234-displayC*
alias: of:NTCnvidia,tegra234-display
depends: drm
name: nvidia
retpoline: Y
root@beastnas[~]# dmesg | grep -i nvidia

[968964.285179] nvidia-nvlink: Nvlink Core is being initialized, major device number 235
[968964.285192] NVRM: The NVIDIA GPU 0000:82:00.0 (PCI ID: 10de:1bb3)
NVRM: nvidia.ko because it does not include the required GPU
[968964.292245] nvidia 0000:82:00.0: probe with driver nvidia failed with error -1
[968964.293919] NVRM: The NVIDIA probe routine failed for 1 device(s).
[968964.295919] NVRM: None of the NVIDIA devices were initialized.
[968964.328115] nvidia-nvlink: Unregistered Nvlink Core, major device number 235
[968976.824644] nvidia-nvlink: Nvlink Core is being initialized, major device number 235
[968976.824657] NVRM: The NVIDIA GPU 0000:82:00.0 (PCI ID: 10de:1bb3)
NVRM: nvidia.ko because it does not include the required GPU
[968976.833399] nvidia 0000:82:00.0: probe with driver nvidia failed with error -1
[968976.837808] NVRM: The NVIDIA probe routine failed for 1 device(s).
[968976.840066] NVRM: None of the NVIDIA devices were initialized.
[968976.868700] nvidia-nvlink: Unregistered Nvlink Core, major device number 235
[968977.329379] nvidia-nvlink: Nvlink Core is being initialized, major device number 235
[968977.329391] NVRM: The NVIDIA GPU 0000:82:00.0 (PCI ID: 10de:1bb3)
NVRM: nvidia.ko because it does not include the required GPU
[968977.337894] nvidia 0000:82:00.0: probe with driver nvidia failed with error -1
[968977.342781] NVRM: The NVIDIA probe routine failed for 1 device(s).
[968977.345511] NVRM: None of the NVIDIA devices were initialized.
[968977.372908] nvidia-nvlink: Unregistered Nvlink Core, major device number 235
[968977.813782] nvidia-nvlink: Nvlink Core is being initialized, major device number 235
[968977.813794] NVRM: The NVIDIA GPU 0000:82:00.0 (PCI ID: 10de:1bb3)
NVRM: nvidia.ko because it does not include the required GPU
[968977.821036] nvidia 0000:82:00.0: probe with driver nvidia failed with error -1
[968977.821357] NVRM: The NVIDIA probe routine failed for 1 device(s).
[968977.821580] NVRM: None of the NVIDIA devices were initialized.
[968977.825732] nvidia-nvlink: Unregistered Nvlink Core, major device number 235
[968978.247058] nvidia-nvlink: Nvlink Core is being initialized, major device number 235
[968978.247071] NVRM: The NVIDIA GPU 0000:82:00.0 (PCI ID: 10de:1bb3)
NVRM: nvidia.ko because it does not include the required GPU
[968978.254858] nvidia 0000:82:00.0: probe with driver nvidia failed with error -1
[968978.255235] NVRM: The NVIDIA probe routine failed for 1 device(s).
[968978.255488] NVRM: None of the NVIDIA devices were initialized.
[968978.259002] nvidia-nvlink: Unregistered Nvlink Core, major device number 235
[969150.262202] nvidia-nvlink: Nvlink Core is being initialized, major device number 235
[969150.262219] NVRM: The NVIDIA GPU 0000:82:00.0 (PCI ID: 10de:1bb3)
NVRM: nvidia.ko because it does not include the required GPU
[969150.274858] nvidia 0000:82:00.0: probe with driver nvidia failed with error -1
[969150.278854] NVRM: The NVIDIA probe routine failed for 1 device(s).
[969150.281038] NVRM: None of the NVIDIA devices were initialized.
[969150.310837] nvidia-nvlink: Unregistered Nvlink Core, major device number 235
[970143.195740] nvidia-nvlink: Nvlink Core is being initialized, major device number 235
[970143.195758] NVRM: The NVIDIA GPU 0000:82:00.0 (PCI ID: 10de:1bb3)
NVRM: nvidia.ko because it does not include the required GPU
[970143.207490] nvidia 0000:82:00.0: probe with driver nvidia failed with error -1
[970143.208016] NVRM: The NVIDIA probe routine failed for 1 device(s).
[970143.208494] NVRM: None of the NVIDIA devices were initialized.
[970143.239068] nvidia-nvlink: Unregistered Nvlink Core, major device number 235
[971593.481057] nvidia-nvlink: Nvlink Core is being initialized, major device number 235
[971593.481067] NVRM: The NVIDIA GPU 0000:82:00.0 (PCI ID: 10de:1bb3)
NVRM: nvidia.ko because it does not include the required GPU
[971593.488666] nvidia 0000:82:00.0: probe with driver nvidia failed with error -1
[971593.493262] NVRM: The NVIDIA probe routine failed for 1 device(s).
[971593.495509] NVRM: None of the NVIDIA devices were initialized.
[971593.524532] nvidia-nvlink: Unregistered Nvlink Core, major device number 235
root@beastnas[~]# nvidia-smi
NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

[LarsR]

Are you sure the p4 was supported natively under 25.10? Because if my memory serves me right Support should already be dropped with the driver in 25.10

Edit:
yep according to this list it’s no longer supported under 25.10

NVIDIA GPU List for 25.10+

Thought this list would be handy for the community. It’s refined from this source - GitHub - NVIDIA/open-gpu-kernel-modules: NVIDIA Linux open GPU kernel module source I’ve had to refer to this list a number of times, so I thought this would be easier for people to quickly find an Nvidia GPU. Also attached. nvidia_truenas.txt (3.1 KB) Matrox D-Series D2450 Matrox D-Series D2480 NVIDIA A10 NVIDIA A100 80GB PCIe NVIDIA A100-PCIE-40GB NVIDIA A100-PG509-200 NVIDIA A100-SXM-64GB NVIDIA A100-SXM4-…

[BeastNAS]

I can confirm it was working with v25.10. I had it working with Ollama an Frigate. I would go back, but I’ve updated my ZFS pool.

[BeastNAS]

Additional Investigation

The Tesla P4 device ID (10de:1bb3) does not appear to be present in the installed NVIDIA kernel module.

Commands:

modinfo nvidia | grep 1BB3

Result:
(no output)

strings /lib/modules/$(uname -r)/kernel/drivers/video/nvidia.ko | grep -i 1bb3

Result:
(no output)

This appears consistent with the kernel log message:

NVRM: nvidia.ko because it does not include the required GPU

This suggests the NVIDIA 590.44.01 package included with TrueNAS 26.0.0-BETA.1 may not contain support for Tesla P4 devices.

[SmallBarky]

Did you make changes to run the old driver?
The inital post by @HoneyBadger listed the Tesla P4 as not working under 25.10. Later in the thread @zzzhouuu shows how they made changes to run a Tesla P4. (post 28)

NVIDIA Kernel Module Change in TrueNAS 25.10 - What This Means for You

With the upcoming release of the TrueNAS 25.10 “Goldeye” beta, there have been some changes made to the NVIDIA drivers and supporting modules that are compiled and shipped with TrueNAS. TrueNAS 25.10 now uses the NVIDIA open GPU kernel modules with the 570.172.08 driver. This enables TrueNAS to make use of NVIDIA Blackwell GPUs - the RTX 50-series and RTX PRO Blackwell cards - which many users have requested support for. [1] [2] [3] Unfortunately, what NVIDIA giveth, NVIDIA taketh away. The N…

[BeastNAS]

I also checked for another Pascal GPU device ID, GTX 1080 / 10de:1b80, and it is also absent from nvidia.ko:

strings /lib/modules/$(uname -r)/kernel/drivers/video/nvidia.ko | grep -i 1b80

Result:
(no output)

This suggests the TrueNAS 26.0.0-BETA.1 NVIDIA 590.44.01 driver package may exclude Pascal GPU support broadly, not only Tesla P4.

[BeastNAS]

This was my first attempt at installing an nvidia GPU in my server. I may have done this. I’ll try to research my internet history (dangerous task) to see if I’ve read/followed that article. I don’t recall, tbh.

[scyto]

if can wait a couple of weeks (i am about to go on vacation) i can try and make you a sysext for that driver if you want to test it

i already have tested various 595 and 580 driver trains on 2026 Beta1 - what is the last NVidia confirmed driver train that works with your card (irrespective of truenas)?

[BeastNAS]

Thanks for the offer.

Unfortunately I don’t know the exact NVIDIA driver train that was working previously.

What I do know is:

- The GPU is a Tesla P4 (10de:1bb3).

- The card worked correctly under TrueNAS SCALE 25.10.x.

- After upgrading to TrueNAS 26.0.0-BETA.1, the bundled NVIDIA 590.44.01 driver rejects the card with:

NVRM: nvidia.ko because it does not include the required GPU

Additional testing shows the Tesla P4 PCI ID (10de:1bb3) does not appear in the installed nvidia.ko module.

I have not independently tested 570.x or 580.x on another Linux distribution with this card, but I would be happy to test a sysext built against either train.

If there is any additional information or logs you would like from the system, let me know and I can provide them.

Do you know whether NVIDIA removed Pascal support in the 590 train, or is this likely a packaging issue specific to TrueNAS 26?

Thanks again.

[LarsR]

@BeastNAS

Unfortunately I don’t know the exact NVIDIA driver train that was working previously.

The version of truenas 25.04 should work, which is 550.142

[zzzhouuu]

TrueNAS no longer supports Tesla P4 from version 25.10. If you want to use it after upgrading to 26.beta.1, you can use the NVIDIA driver I provided.

GitHub - zzzhouuu/truenas-nvidia-drivers: TrueNAS GPU Legacy Drivers for GTX 700-series,...

[BeastNAS]

I’ll try it tonight. Thanks. I’m hoping to save myself from having to buy a T4 since I just bought this P4 in March.

[HoneyBadger]

@BeastNAS

Do you know whether NVIDIA removed Pascal support in the 590 train

Direct answer: NVIDIA has removed Pascal (and Volta) support in the 590 driver, which is the default for TN26.

[BeastNAS]

Thank you.

Do you know the last NVIDIA driver before they removed Pascal support?

Based on the info on NVIDIA’s website, it should still be supported, albeit in maintenance mode (at least for another month):

GPU	End of Full Support (yyyy-mm)	End of Extended Full Support (yyyy-mm)	End of Maintenance Support (yyyy-mm)	Last Supported Release Branch
NVIDIA A40	2024-12	2027-12	TBD	TBD
Tesla M6	2018-12	2023-07	2026-07	16
Tesla M10	2022-11	2025-11	2028-07	19
Tesla M60	2020-03	2023-07	2026-07	16
Tesla P4	2019-03	2023-07	2026-07	16
Tesla P6	2021-08	2023-07	2026-07	16

[DjP-iX]

There are two different “flavors” of NVIDIA driver, open and proprietary. TrueNAS 25.10 shifted to the open drivers, which are required to support newer NVIDIA cards, but drop support for earlier ones. Take a look at https://www.truenas.com/docs/scale/25.10/gettingstarted/versionnotes/#nvidia-gpu-support.

@BeastNAS

Based on the info on NVIDIA’s website, it should still be supported, albeit in maintenance mode (at least for another month):

It may be that its still supported on the current proprietary driver but not the current open driver.

[BeastNAS]

Thank you. That is discouraging news, and their workaround is very inconvenient. I understand the P4 is older, and not really suited for heavy lifting AI workloads, but for someone just “dabbling” on a home lab environment, it should be good enough. Unfortunately, it seems TrueNAS is following the PC gaming methodology, develop solely for the newest tech and force the users to upgrade hardware to participate. I know, that’s a bit dramatic, but it reminded me of when I was a heavy PC gamer and had to upgrade every year or so.
Thanks again.

[DjP-iX]

@BeastNAS

Unfortunately, it seems TrueNAS is following the PC gaming methodology, develop solely for the newest tech and force the users to upgrade hardware to participate.

NVIDIA are the ones who chose to only support newer cards on the open drivers and drop support for earlier ones with them, not TrueNAS.

[BeastNAS]

Gotcha. I apologize to the TrueNAS team. I’ll redirect my disdain towards NVIDIA.

[Farout]

Pass the card through to a VM of your choice. No need to run it directly on the base OS.

[BeastNAS]

I’m dabbling with Ollama and Frigate via docker containers. I’d rather run that through TrueNAS docker instead of building a VM and doing it in there, personally, since I want to point some of my AI capable containers and homegrown containers to use Ollama. However, it is always an option. Thanks!