Not installing the Nvidia CUDA toolkit limits the compute capacity of the TrueNAS appliance to serve up AI LLM container content for Nvidia users. Is this a bug, or must we lobby for inclusion?

[E_B]

I think lobbying is the best approach:

Nvidia container toolkit

Problem/Justification The Nvidia Container Toolkit is “a collection of libraries and utilities enabling users to build and run GPU-accelerated containers”. At the moment, Nvidia based hardware is not fully available for trueNAS users, unlike Intel GPUs whose toolkit has already been enabled in TrueNAS. Impact Any TN user wishing to use docker containers will be disadvantaged without the toolkit if their container needs to use Nvidia GPU hardware. For example, Frigate users are unable to im…

BUT - I may have hit upon a way to install the toolkit via docker commands and hence not needing apt and I have written up as far as I have got on the Frigate github discussions page here:

[Support]: On TrueNAS APP: Restart from the UI is not working ·...

It seems (to my uneducated eye, at least) that the toolkit is now installed and working, but the next part of the puzzle eludes me for now.