Xinference need cuda 12.9, but TrueNAS use cuda 12.8
With the next major release propably around q3/4 this year. The first beta for the new release started around 2 weeks ago… if you’re adventureous give it a go…
1 Like
Many llm inference frameworks need cuda 3.9 at lease. Wish update process goes faster !
1 Like
At this rate, when TrueNAS supports CUDA 12.9, Xinference will likely require CUDA 13+ support.Really wish update process go faster
If you require the latest drivers, you need to pick a different product to run. TrueNAS is a conservative, enterprise targeted product. You always have the option of a VM and pass through of the video card to the VM.