Problem/Justification
I’ve recently bought an RTX 5060 Ti to use for various ML tasks, but I’ve stumbled into an issue when trying to use it for those tasks - all the modern apps like vLLM, localai require CUDA 13 framework to utilize some of the newer models, while the 570.172.08 driver that gets installed on 25.10 supports up to CUDA 12.8.
I could have waited for the spring 26.04 release with a newer driver no problem, but now that Truenas is on annual fall release, waiting for a new driver at least a whole year and getting one that is at least 6 months old is frankly a bit ridiculous.
I understand that it’s an appliance OS, but I think there should be compromises made to keep the product actually usable.
Since TrueNAS 25.10 and newer now downloads and installs the nvidia drivers per user’s request at runtime level, instead of just baking the driver into the ISO, I propose taking an inspiration from what Ubuntu does, and providing a UI for the user where they can select the version of their choosing.
By default, an older version that was tested with the TrueNAS release would be picked, but users would be free to choose and install a newer one.
Impact
- No impact for users who choose the default, older driver - the appliance experience will stay the same
- Latest hardware and feature enablement thanks to newer drivers for users who opt-in for one