TrueNAS scale share. Slow read transfers. high performance network. High performance drives

TrueNAS General
1

Hi All,
I have situation with an existing installation. Truenas scale running a 12vdev volume with metadata vdevs, slog and l2arc. As well as 256GB of ram. 40gbe nic to mikrotik switch. Client is Linux machine with 40gbe nic on the same switch.

I’m getting relatively slow read from the NAS to client. Around 350MB/s 2.5Gbps.

But on iperf we nearly max out the nic. 37+Gbps.

And fio tests on the shared volume show 2500MB/s write and 3300MB/s reads.

So network connection is good. Drive configuration is good. But performance is poor.

Any ideas? NFS and SMB both doing similar numbers.

2

To add. 10Gbe MAC clients to the same server are fully saturating the connection. ~900MB/s read/write.

I would think it was a client nic issue, except iperf shows full bandwidth.

3

UPDATE: Upgrading to Fangtooth seems to have fixed the issue. Unclear what the cause was.