Virtualized guest with multiple NUMA nodes

Hi all,
My truenas host has 2 sockets, 2 NUMA nodes.
I created a vm with 2 vCPUs, pinned to the cores such that each vCPU gets cores from the corisponding socket.
The guest shows all the cores, /proc/cpuinfo shows their ‘physical id’ correctly but they are all on the same NUMA node in the guest OS, not split as i would expect

How do i configure a vm such that the guest OS has internal NUMA nodes that reflect that of the host?

(if it matters, this is for CPU inference on large LLMs so it’s i need a single instance of ollama to distribute layers across the numa nodes to maximize memory bandwidth)
thanks

You would have to provide custom configuration to QEMU.

This may be possible using the vm API in middlewared

This seems to document the QEMU SMP/NUMA configs

mm yeah I thought the answer would end up being something like that. It would just get overwritten whenever I used the GUI though?

Ideally the numa nodes should be passed through automatically, though i guess there might be some awkward logic around implementing that with some systems

Just updathing this, it appears 25.04 beta now does create VMs with numa nodes reflecting the underlieing hardware!

1 Like