Is there a specific reason for this speed decreasing?

TrueNAS General
1

Hello community,
TrueNAS Scale Dragonfish 24.04.0 here, running under Proxmox (host CPU i9 14900K 128GB RAM total). TrueNAS VM = 16 Cores, 32GB RAM assigned.
On host I have a 10Gb Intel X710-T4 while on a client I have the Intel x550-T1 10Gb.
Pool are formed by 4x12TB Ironwolf Pro (vdev 2 mirrors).

When I trasfer big files, it starts with about 1 GB/sec:
1

But after a while the speed decrease to 400 MB/sec:
2

This is happening on all big files I trasferred from PC to NAS, while from NAS to PC I can get 500/600 MB/sec constantly.

Is it normal this behaviour? What could be the technical reason?
Thank you
Lucas

2

You’re likely hitting a limit on transaction group timeout (which is 5 seconds by default, as seen in /sys/module/zfs/parameters/zfs_txg_timeout). Once that drops you’ll be writing at the speed the pool can sustain.
As for why you can recv at a much higher rate from the NAS, sustained reads from pools are typically a lot faster than sustained writes, the data you’re accessing may also be in ARC.

2 Likes
3

Hard drives write at about 100-250MB/s max.

2x 200MB/s sounds about right.

For faster sustained speed add more mirrors, or use SSDs.

1 Like
4

Aside from ARC cache etc, when you read from mirrors you can spread you reads over all the disks, so potentially 4x the speed of a single disk.

Again 600 = 4x 150MB/s so well within expected performance band.

5

Thanks both, so this is clearly a physical drives limitation

1 Like