Are there any storage settings that people have found to work really well with XCP-NG and storing the VMs on an NFS share?
I’m trying to find that magic setting that will let me migrate a VM from one storage device to another, and do it faster than 400mbps which is what I’m seeing right now. I can get up to 7gbps during benchmarks, so I know the system is capable of handling faster transfers. Connection is 10gbps ethernet (DAC on host and base-T Truenas)
I’m thinking about the “block” size that is used when you first make a pool. The record size in a dataset has some effect, but I think I need to start with a different “block” size when the pool is created. Yes I may be mixing words to describe what I’m trying to do.
Could ashift change the way the storage is operating enough to make a difference? Currently using 6 spinning drives so a 512 byte sector size might be the correct thing. But would moving to something larger allow for increased speed?
I’ve pretty much tapped out what I can find with XCP-NG, I’m pretty sure we need a newer kernel to support NFS nconnect and this might fix the areas that I have concerns in, the under 64k “block” sizes in the benchmark tool are where the performance really lacks, getting around 20mbps in 4k “blocks”. The real bottleneck seems to be when migrating VMs from one storage device to a second storage device, I’ve managed to get about 50mbps increase by fooling with record size options in both Truenas and XCP-NG NFS mounts, but 420mbps and still way slower than I would like.
Operationally, with the basic stuff I run, the speed isn’t a problem… But I know there is faster to be had, ESXi8 (in my lab) shows me this and it defaults to nconnect=4 for the NFS mounts.
