iSCSI only moderate performance. Can it be improved?

TrueNAS General
,
1

Since long I wondered why the communication from my PC towards TrueNas did not reach maximum speed via my 10G connections. This where that should not be a problem for PC and NAS since both are powerfull and using NVME-SSD’s.

Yesterday I discovered the problem. I have always been testing using iSCSI and not via iPerf or SMB. Reason for that is and was quite simple, that I would like to measure the real performance.

However I did ‘end up’ on a post starting with “iSCSI is a SAN protocol. NFS, CIFS, etc., are NAS protocols.” jgreco

https://www.truenas.com/community/threads/why-iscsi-often-requires-more-resources-for-the-same-result.2817

After reading that post I decided to do a speed test using SMB. And guess what the speed was 10G and not the (unexpected / not ok) 5G!

So where file transfers using SMB towards TrueNas easily flow with 10G, using iSCSI the tansfer speed is a disappointing 5G (given the used hardware and network).

So I just started trying to get NVME-oF working as a potential alternative, I really wonder is it should not be possible to get better performance using iSCSI.

note:

  • I am using jumbo frames which in general improve performance (tested).
2

Protocol is one thing (SMB or NFS vs. iSCSI) the other, IMHO way more important difference is that it is blockstorage (zvol) vs dataset.

Blockstorage has huge potential disadvantages, like pool geometry, padding, compression, rw amplification, ARC, svdev usage, and probably other stuff I don’t even know.

So if you don’t need blockstorage and can use a SMB dataset instead, you should always do that.

PS: you link is broken, here is a working one Resource - Why iSCSI often requires more resources for the same result | TrueNAS Community

2 Likes