Hi, I have a scale installation on a supermicro mini it’s board with an epyc 3201 CPU. Now, I have a docker vm and would like to pass a GPU for plex. The GPU is a Intel arc a310.
After installing the GPU on the system, I’ve proceeded to set the the GPU isolated (I have 2 GPU: onboard from ipmi and the arc) and rebooted.
One strange thing is that when I have isolated the GPU I had to refresh manually the page because I didn’t see any confirmation message.
Then I’ve added the GPU to the VM but the VM won’t start anymore. The error is:
Failed to start VM Ubuntu_Docker: [EFAULT] internal error: qemu unexpectedly closed the monitor: 2024-09-19T20:22:53.202365Z qemu-system-x86_64: -device {“driver”:“vfio-pci”,“host”:“0000:08:00.0”,“id”:“hostdev0”,“bus”:“pci.0”,“addr”:“0x9”}: vfio 0000:08:00.0: failed to setup container for group 23: Failed to set group container: Invalid argument
Looking better in dmesg when i try to start the VM i get this:
[47573.932318] vfio-pci 0000:08:00.0: Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping. Contact your platform vendor
Others dmesg messages about iommu:
root@truenas[~]# dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
[ 0.026237] AMD-Vi: Unknown option - 'on'
[ 0.026294] DMAR: IOMMU enabled
[ 1.493772] AMD-Vi: Using global IVHD EFR:0xf77ef22294ada, EFR2:0x0
[ 4.015230] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 4.237977] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 4.243567] AMD-Vi: Extended features (0xf77ef22294ada, 0x0): PPR NX GT IA GA PC GA_vAPIC
[ 4.251884] AMD-Vi: Interrupt remapping enabled
[ 4.256670] AMD-Vi: Virtual APIC enabled
[ 4.289731] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[ 4.840106] AMD-Vi: AMD IOMMUv2 loaded and initialized
Searching online i found this thread from proxmox forum where the issue seems to be a more strict enforcement of the iommu rule on new kernels. Unitl 6.5 seems that the 1:1 error was not shown.
Any hint?