Internal error: Non-endpoint PCI devices cannot be assigned to guests

Apps and Virtualization
1

When attempting to start VM I get internal error: Non-endpoint PCI devices cannot be assigned to guests. I’m new to linux/truenas so I have no idea where to start.

25.10.4 with Hexos.

Intel Core i5-4460 @ 3.20GHz

16 Gigs of ram

nvidia 1050Ti

Error Name: EFAULT
Error Code: 14
Reason: [EFAULT] internal error: Non-endpoint PCI devices cannot be assigned to guests
Error Class: CallError
Trace: Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/middlewared/plugins/vm/supervisor/supervisor.py”, line 191, in start
if self.domain.create() < 0:
^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/libvirt.py”, line 1373, in create
raise libvirtError(‘virDomainCreate() failed’)
libvirt.libvirtError: internal error: Non-endpoint PCI devices cannot be assigned to guests

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/middlewared/api/base/server/ws_handler/rpc.py”, line 361, in process_method_call
result = await method.call(app, id_, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/api/base/server/method.py”, line 57, in call
result = await self.middleware.call_with_audit(self.name, self.serviceobj, methodobj, params, app,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 956, in call_with_audit
result = await self._call(method, serviceobj, methodobj, params, app=app,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 773, in _call
return await methodobj(*prepared_call.args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/api/base/decorator.py”, line 108, in wrapped
result = await func(*args)
^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/vm/vm_lifecycle.py”, line 57, in start
await self.middleware.run_in_thread(self._start, vm[‘name’])
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 670, in run_in_thread
return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 667, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3.11/concurrent/futures/thread.py”, line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/vm/vm_supervisor.py”, line 68, in _start
self.vms[vm_name].start(vm_data=self._vm_from_name(vm_name))
File “/usr/lib/python3/dist-packages/middlewared/plugins/vm/supervisor/supervisor.py”, line 201, in start
raise CallError(‘\n’.join(errors))
middlewared.service_exception.CallError: [EFAULT] internal error: Non-endpoint PCI devices cannot be assigned to guests

2

25.10.4 doesn’t support this video card by default. How do you have it set up?

3

Not entirely sure how to answer this as I didnt really do anything for the GPU. it gave me no errors when I installed hexos. The dashboard of hexos/truenas doesn’t report anything about the GPU so maybe that’s an indication of the problem?

4

I am assumimg, you are trying to passthrough the GPU to a VM ?

If so, there are some hardware requirements to do so:

  1. You need 2 GPUs. One for the VM and one for Truenas itself. You cannot pass the only GPU. You have an iGPU and the 1050 —> good
  2. The GPU that you want to passthrough needs to be in its own, discreet IOMMU group, with no other devices in it other than the sound device.

This is often not the case on consumer motherboards.

The error is about a “non endpoint PCI device”. So it could be the case, that there is a PCI switch in the same group as your GPU.

You can list your IOMMU groups with this script:

#!/bin/bash
for d in /sys/kernel/iommu_groups/*/devices/*; do
  n=${d#*/iommu_groups/*}; n=${n%%/*}
  printf 'IOMMU Group %s ' "$n"
  lspci -nns "${d##*/}"
done

Your GPU doesnt need to be supported by Truenas itself to passthrough. It only needs to be recognised. Which it should.

1 Like
5

I cant seem to find a place to run a script, I tried in init/shutdown scripts but I think I would need to restart for that to work?

6

You’d run it at the shell.

Is there a reason you aren’t asking HexOS Support? That’s who you’ve paid, after all.

7

I guess I didn’t think about that at all lmao. I will try to update this if I find a solution.