Problem/Justification
The Nvidia Container Toolkit is “a collection of libraries and utilities enabling users to build and run GPU-accelerated containers”. At the moment, Nvidia based hardware is not fully available for trueNAS users, unlike Intel GPUs whose toolkit has already been enabled in TrueNAS.
Impact
Any TN user wishing to use docker containers will be disadvantaged without the toolkit if their container needs to use Nvidia GPU hardware. For example, Frigate users are unable to implement more advanced configurations of Frigate because the TN provided app has limitations and it is difficult to know how the app implements Frigate. Furthermore, the Frigate GitHub community provides full support for Docker methods but none for TrueNAS app methods.
User Story
Frigate is a superbly written network video recording application which is able to employ GPU acceleration to provide machine learning and AI functionality.
The TrueNAS app is one way to implement it but the other way, preferred in some cases due to the extra flexibility it would enable, is to use docker compose (perhaps via dockge or portainer) natively within TN to provide the best flexibility when fault finding or developing.
Frigate, via docker, requires the Nvidia container toolkit:
which explains that
Additional configuration is needed for the Docker container to be able to access the NVIDIA GPU. The supported method for this is to install the NVIDIA Container Toolkit and specify the GPU to Docker. How you do this depends on how Docker is being run:
Docker Compose - Nvidia GPU
services:
frigate:
...
image: ghcr.io/blakeblackshear/frigate:stable
deploy: # <------------- Add this section
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['0'] # this is only needed when using multiple GPUs
count: 1 # number of GPUs
capabilities: [gpu]
Without the Nvidia container toolkit, there is no safe supported way to implement the docker approach other than developer mode and enabling apt
which may be to the detriment of the TrueNAS appliance functionality.
edit - I investigated applications other than just Frigate which could be containerised and which might benefit from the toolkit:
- DeepStack
- TensorFlow Serving
- PyTorch
- OpenVINO
- HandBrake
- Blender
- Stable Diffusion