What is the recommended benchmarking tool to assess SMB vs. NFS file transfer speed performance.
I have 10Gb ethernet on my Windows and TrueNAS server and I’m getting a measly 31 MB/sec instead of 1000 MB/sec theoretical via copy from windows to SMB mounted truenas share. Is that typical?
How can I max out the speed?
I have an SSD pool on the truenas and the throughput is no better.
What are others getting?
I’ve got a Supermicro with 64 GiB of RAM and 48 cores, so the server isn’t the problem
So I’m able to get 565MB/sec transfer over SMB between my two systems now. Is NFS faster over 10Ge?
Here is what limited my speed: I was using the wrong color USB cable and the wrong color USB port.
I have a cable with a white tab from my Ledger and realized it pretty quickly. That limited the speed to 37 MB/sec!
So between SSD on my desktop to the truenas was 565 MB/sec.
Am I limited by SMB or is that as good as it gets?
I would think about your workload first. After that run fio.
Sure? Maybe you use an old CPU with low single thread performance. SMB was single threaded, last time I checked.
Run NFS to compare it against with.
But to answer your question, there are multiple different benchmarks, multiple of drive options, multiple pool options, which results in a million different results. You would need to offer A LOT more information to get an answer if this is a good result or not. We don’t even know if this is read or write
Writing to an external USB drive for speed testing rarely ends well. Too many variables, including the USB bridge crying uncle.
I suggest a benchmarking tool that sends random data. Otherwise, you end up with compression making results looking better than they are. @HoneyBadger has a nice starting point here.
We don’t have an idea of how your system is set up, ditto the datasets.
It’s like someone calling tech support and complaining that an optical mouse won’t work without disclosing what type of surface they’re using it on.
So, how is your pool configured? Ditto the dataset? On the receiving end, how / where is that data being stored? Fwiw, I’m getting 400MB/s for large file writes to a pretty empty pool. Single Z3 VDEV assisted by a sVDEV, etc. For details see signature.
If you’re the only user, I doubt there is a huge difference re: NFS or SMB based on what I’ve read here. But, I have no personal experience with NFS, I have never used it.
SMB is a single-threaded process, so from a NAS POV, the faster the CPU, the better. Besides that, there is the pool config, dataset config, and other factors (such as using a sVDEV, SLOG, etc) that may make a difference depending on your use case.
Great. fio and iperf3 are both pre-installed in TrueNAS so I’ll give it a go. These were the 2 most obvious tools… should have checked before asking, but now everyone else knows.
I actually got better (and more consistent) performance when transferring a massive amount of data to a USB drive when I had a fan constantly blowing air over it.
Welcome to the wonderful world of Windows networking. I could write a novel on the amount of tinkering I’ve done, it’s impressive how badly MS have built their networking stack. Luckily, someone’s already written a novel on it for me.
While a lot of this isn’t relevant, it’s all interesting stuff, posts 3 and 5 in particular:
Give this a look (by the same person), I’ve been deploying it on every 10GiB machine I own and it’s been bumping my 4-5gbps single stream network benchmarks up to 8-9gbps, and I’m seeing the same improvement with SMB.
(And no, I’m not saying to blindly run it, please don’t blame me if your NIC explodes! Though it is nice to note it takes a backup of anything that is modified).
EDIT: I hastily misread 31 MB/sec as 310 MB/s, whoops! Either way, this is still very good reading material!
I do notice that from my PC I can upload at 565MB/sec, but only download at 170MB/sec (the PC and truenas server both have 10Gb adapters). I’ll be testing from my UDM Pro to see who the troublemaker is… I suspect the PC. Then I’ll try that optimization script.
