Hardware passthough of GPU to a Vm - with only one GPU?

Apps and Virtualization
1

Hi

Has this feature been implemented/fixed in truenas scale yet.

I use truenas scale a a NAS, but have always been disappointed with the virtualisation options.

For now, I am using a very old ivy bridge machine as a NAS. It works just fine with the i7-3770k cpu.

It is a NAS only.

So I may be using another machine instead, which I have, MSI B450 tomahawk motherboard & 3600x CPU.

There will be one GPU in there, an RTX 3050 6GB. All slots will be occupied.

I want to pass through to a linux VM, for media playback, as well as hosting jellyfin, navidrome & other applications. I am no fan of the truenas scale applications, and have been trouble in my experience.

So can it be passed through to the linux virtual machine, with ONLY A SINGLE GPU in the system.

Can it be done without messing around in the command line, which could break with an update?

For context, I am already doing this with in proxmox, and working as expected. It is a dell optiplex 3060 micro, passing through the onboard HD630 to a virtual machine.

There are some grub changes and some other files to modify - but does not break when updates are installed.

I am using hardware I own already, and are not buying more. If this is surprising, then check the PC hardware prices!

Useful and informed feedback is appreciated.

Replies with just buy this or that hardware are not so much appreciated.

Thanks

2

Sounds ambitious, a lot of jobs for a single GPU. There’s a different concept of pass through in truenas. The system doesn’t really use it for anything but your applications normally claim it, once you set them to do that. I don’t know how you plan to run these apps outside the truenas ecosystem, but it’s my understanding that not every app can or will share equally, seems like it’s “assign to an app and that’s it”.

So can you use your card without the os hogging it? Probably so. Can all your apps use it instead? I’m not sure but at least one definitely can.

I have a test server. Has an amd chip with onchip video and an Intel GPU. Effortlessly assigned the discrete GPU to plex but the system seemed to want the onchip video for itself. Hard to explain.

3

From the 25.10 Documenation section on Virtual Machines.

Users with multiple GPUs who wish to pass a GPU to a VM must first isolate a GPU for VM use. One GPU is always required by TrueNAS.

1 Like
4

Thank you for the reply.

My whole point of the post there will be a single GPU.

I am aware already it can be done with 2.

Do you know if it be done or not?

There is no technical reason it cannot.

Thanks

5

Nope still not fixed, and probably never will be…

6

I also don’t get it - I don’t think many people do, but yeah as @LarsR said - no way afaik.
You could of course always buy a very cheap VGA if you have a free PCIe slot…

7

Or use a CPU with an igpu…

1 Like
8

Thanks for the reply. I stated I would not be buying more hardware.

9

There will be no available slots, as I stated.

10

This is sad, but good to know.

Virtualisation on Truenas scale is problematic. It seems it is best to use it just as a NAS.

So an option could be to virtualise truenas in proxmox.

That can cause issues though.

I will need to test if the onboard sata controller can be passed through to a truenas VM.

It is far from ideal.

11

To be fair, its called TrueNAS. Apps and virtualisation are add ons.
Also, Truenas expects to be run on
A a system with enough sata ports (no partitioning of drives supported)
B for passthrough to have at least IPMI/BMC with includes a crude VGA adapter
C One IOMMU Group per device (no ACS overrride possible)

If your system fullfils these requirements, Truenas VMs are as performant as on proxmox (albeit with a lot less options exposed to the GUI).
For a while I ran 2 Win11 VMs, with 1 GPU each for remote gaming…

So its definitely possible to use Truenas for VMs…it just expects certain hardware.

1 Like
12

Single-GPU VM passthrough isn’t currently on our radar. It’s something that might be a consideration, but generally speaking it’s more efficient to use host-level GPU paravirtualization (eg: Apps, Containers, Docker) to do this.

Our Apps framework has made some major strides, but if you want full control @ThisTruenasuser you can always deploy a Dockge or Portainer stack and give the GPU to that - then you’ve got the full control without having to deploy a VM.

13

A, B ,C are fullfilled as far as I know, with the single GPU slot directly connected to the CPU.

There are enough sata ports.

I think the ACS override is an option in proxmox to pass through the sata contrller(s).

I just checked the grub file in my proxmox machine. The onboard single GPU is passed through just fine.

It means running proxmox, with the linux virtual machine and truenas virtual machine.

That is very inefficient, but want a passthrough of a single GPU to the linux VM on truneas scale. it seems it is not available.

14

The apps in truenas, using portainer and such are not good enough for my needs.

Firstly the GPU will be playing media, as attached to the TV.

Also jellyfin, with AI based upscaling plugins (still to test).

Also metadata privacy tools.

As far as I am aware need to be installed on the same machine. A linux VM with enough resources is needed.

15

There’s a way to get around the limitation but it’s hardcoded in the TrueNAS middleware so requires modifications to the OS, so comes with caveats and without support, etc. I won’t link it here but you can find it online

16

Why not? Provided it’s got the “Here be Dragons” level of warning, it would be beneficial to those to see it. Make it a Resource, even.

17

Not my guide so I won’t create a resource, but here’s the link. It’s inside the creation guide for Talos VM’s using forgetool from TrueForge (formerly just TrueCharts)

Basically need to install-dev-tools, unset /usr (something like systemd-sysext unmerge) and modify some python, so not for the faint of heart. Insert all the warnings here, but if you mess up, all you gotta do is copy config and restore to a fresh install boot disk

18

LOL…so this limitation is just put there for no reason at all or what ?

This peace of code just checks if there is more than one GPU and throws an error if not… again LOL

1 Like
19

Thanks for the info.

I am aware of that guide.

Also it could be broken through an update. I am only doing it in truenas scale if it is supported though the gui.

Truenas scale is great as a NAS, but lacking a lot when it comes to virtualisation.

That is fair enough, as it is a NAS first.

To virtualise truenas scale and the linux VM in proxmox seems the stable option.

It seems a lot of work

20

I do not know why there is such a limitation in truenas scale.

In proxmox, there is no issue, when configured.

It is truenas scale that is lacking in virtualisation support.