Twice as much written to drives as saved to pool

TrueNAS General
,
1

I’m in the process of replacing my SATA SSD NAS with an NVMe one. Both pools are comprised of mirrors and sharing NFS with sync disabled.

In the process of testing and tuning, I noticed that the receiving pool writes twice as much data to the disks as is read from the sending pool. The sending pool will read around 60-70MBps, I’ll see 500-600Mpbs network transfer, but the receiving pool writes 150MBps.

Is this due to the ZIL? I read it isn’t used for async writes.

2

How did you exactly get these numbers?

3

Both via TN Reporting and ZFS CLI tools.

Ten SATA drives in five mirrored vdevs were reading around 10-15MBps each.
That times 8 lines up with the network traffic of 500-600Mbps.
Four NVMe drives in two mirrored vdevs were writing about 60-70MBps each.

I don’t have exact numbers because I had to replace the mobo and I just got it working again.

1 Like