Until recently, I had SR-IOV working for passing the i915 iGPU from my i5-13600K to a VM. It required this dkms module driver and some experimentation to get things to work. Definitely broke the “warranty void if sticker removed” for TrueNAS Scale so not something I’d recommend in a system where stability, uptime, and data integrity are most important.
In my case, I had to jump through the dkim module route as my iGPU couldn’t be isolated despite having two display adapters (other being the BMC) and not using the iGPU for anything within TrueNAS.
It’s been a while so I’m a little fuzzy what exactly I needed to do, but the critical step I believe I had to do, in addition to the dkms driver and adding the grub cmd line parameters to enable the VFs, was to pass the gpu to the VM not as a GPU, but as a generic PCIe pass through device.
More recently with Docker now native, I’ve migrated my setup away from a dedicated HTPC VM that utilized podman pods, and just use the native docker with yaml configs. Hardware transcoding runs fine through jellyfin and while just a brief synthetic test, was able to have four 4k HDR videos transcoding simultaneously with minimal load on the server.
1 Like
out of curiosity i just tested my ryzen 5 pro 5650g iGPU, passed through without single warning or error in logs. Passed GPU without audio.
Running on asrock B550 uATX motherboard.
It’s a £36 price difference between 4650g and 5650g worth considering an upgrade?
Plex happily uses it for transcoding and it’s faster than quadro p400. Nearly instant playback vs 3-10 seconds delay on p400.
2 Likes
To add to the topic - Why is it necessary to have a GPU for the system? I’ve heard many run TrueNAS without a gpu at all… And as far as I know proxmox can pass all the gpus to the VMs?|
I use TrueNAS scale, and have two gpus, (no more lanes for more gpus), but, I am unable to parallely passthru both - which I do need… Is there a plan to make this possbile? Why does TrueNAS even need a gpu?
I think its actually the UEFI and motherboard that forces GPU to be available, otherwise it refuses to POST.
Only server motherboard have the capability to POST without GPU.
And without GPU you only option to install anything and access UEFi would be serial console or IPMI.
I now updated to 25.04.
I get this error while trying to passthrough my GPU:
[EINVAL] device.GPU.gpu_type: Field required
If I add the GPU, Multimedia device and audio device as single passthrough devices, I can start the vm but then it hangs during boot.
I get the following messages during boot and the forced stop afterwards:
Apr 16 17:22:06 truenas1 kernel: [drm] initializing kernel modesetting (RENOIR 0x1002:0x1636 0x1458:0x1000 0xD9).
Apr 16 17:22:06 truenas1 kernel: [drm] register mmio base: 0xFC800000
Apr 16 17:22:06 truenas1 kernel: [drm] register mmio size: 524288
Apr 16 17:22:06 truenas1 kernel: [drm] add ip block number 0 <soc15_common>
Apr 16 17:22:06 truenas1 kernel: [drm] add ip block number 1 <gmc_v9_0>
Apr 16 17:22:06 truenas1 kernel: [drm] add ip block number 2 <vega10_ih>
Apr 16 17:22:06 truenas1 kernel: [drm] add ip block number 3 <psp>
Apr 16 17:22:06 truenas1 kernel: [drm] add ip block number 4 <smu>
Apr 16 17:22:06 truenas1 kernel: [drm] add ip block number 5 <dm>
Apr 16 17:22:06 truenas1 kernel: [drm] add ip block number 6 <gfx_v9_0>
Apr 16 17:22:06 truenas1 kernel: [drm] add ip block number 7 <sdma_v4_0>
Apr 16 17:22:06 truenas1 kernel: [drm] add ip block number 8 <vcn_v2_0>
Apr 16 17:22:06 truenas1 kernel: [drm] add ip block number 9 <jpeg_v2_0>
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: Fetched VBIOS from VFCT
Apr 16 17:22:06 truenas1 kernel: amdgpu: ATOM BIOS: 113-RENOIR-035
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: Trusted Memory Zone (TMZ) feature enabled
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: MODE2 reset
Apr 16 17:22:06 truenas1 kernel: [drm] vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: VRAM: 1024M 0x000000F400000000 - 0x000000F43FFFFFFF (1024M used)
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: GART: 1024M 0x0000000000000000 - 0x000000003FFFFFFF
Apr 16 17:22:06 truenas1 kernel: [drm] Detected VRAM RAM=1024M, BAR=1024M
Apr 16 17:22:06 truenas1 kernel: [drm] RAM width 128bits DDR4
Apr 16 17:22:06 truenas1 kernel: [drm] amdgpu: 1024M of VRAM memory ready
Apr 16 17:22:06 truenas1 kernel: [drm] amdgpu: 47702M of GTT memory ready.
Apr 16 17:22:06 truenas1 kernel: [drm] GART: num cpu pages 262144, num gpu pages 262144
Apr 16 17:22:06 truenas1 kernel: [drm] PCIE GART of 1024M enabled.
Apr 16 17:22:06 truenas1 kernel: [drm] PTB located at 0x000000F43FC00000
Apr 16 17:22:06 truenas1 kernel: [drm] Loading DMUB firmware via PSP: version=0x01010028
Apr 16 17:22:06 truenas1 kernel: [drm] Found VCN firmware Version ENC: 1.22 DEC: 7 VEP: 0 Revision: 1
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: reserve 0x400000 from 0xf43f800000 for PSP TMR
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: RAS: optional ras ta ucode is not available
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: RAP: optional rap ta ucode is not available
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: psp gfx command LOAD_TA(0x1) failed and response status is (0x7)
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: psp gfx command INVOKE_CMD(0x3) failed and response status is (0x4)
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: SMU is initialized successfully!
Apr 16 17:22:06 truenas1 kernel: [drm] Display Core v3.2.301 initialized on DCN 2.1
Apr 16 17:22:06 truenas1 kernel: [drm] DP-HDMI FRL PCON supported
Apr 16 17:22:06 truenas1 kernel: [drm] DMUB hardware initialized: version=0x01010028
Apr 16 17:22:06 truenas1 kernel: [drm] kiq ring mec 2 pipe 1 q 0
Apr 16 17:22:06 truenas1 kernel: kfd kfd: amdgpu: Allocated 3969056 bytes on gart
Apr 16 17:22:06 truenas1 kernel: kfd kfd: amdgpu: Total number of KFD nodes to be created: 1
Apr 16 17:22:06 truenas1 kernel: amdgpu: Virtual CRAT table created for GPU
Apr 16 17:22:06 truenas1 kernel: amdgpu: Topology: Add dGPU node [0x1636:0x1002]
Apr 16 17:22:06 truenas1 kernel: kfd kfd: amdgpu: added device 1002:1636
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: SE 1, SH per SE 1, CU per SH 8, active_cu_number 7
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring gfx uses VM inv eng 0 on hub 0
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring comp_1.0.0 uses VM inv eng 1 on hub 0
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring comp_1.1.0 uses VM inv eng 4 on hub 0
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring comp_1.2.0 uses VM inv eng 5 on hub 0
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring comp_1.3.0 uses VM inv eng 6 on hub 0
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring comp_1.0.1 uses VM inv eng 7 on hub 0
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring comp_1.1.1 uses VM inv eng 8 on hub 0
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring comp_1.2.1 uses VM inv eng 9 on hub 0
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring comp_1.3.1 uses VM inv eng 10 on hub 0
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring kiq_0.2.1.0 uses VM inv eng 11 on hub 0
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring sdma0 uses VM inv eng 0 on hub 8
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring vcn_dec uses VM inv eng 1 on hub 8
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring vcn_enc0 uses VM inv eng 4 on hub 8
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring vcn_enc1 uses VM inv eng 5 on hub 8
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: ring jpeg_dec uses VM inv eng 6 on hub 8
Apr 16 17:22:06 truenas1 kernel: amdgpu 0000:0f:00.0: amdgpu: Runtime PM not available
Apr 16 17:22:06 truenas1 kernel: [drm] Initialized amdgpu 3.60.0 for 0000:0f:00.0 on minor 2
1 Like
