RAIDZ expansion speedup

TrueNAS General
1

I am expanding a RAIDZ2 pool from 4 to 5 disks on TrueNAS 24.10.0.2.

The expansion speed was ~55 M/s but I got it to ~110 M/s by increasing the raidz_expand_max_copy_bytes parameter:

admin@truenas ~ $ cat /sys/module/zfs/parameters/raidz_expand_max_copy_bytes
167772160
admin@truenas ~ $ echo $((10 * 167772160)) | sudo tee /sys/module/zfs/parameters/raidz_expand_max_copy_bytes
1677721600
4 Likes
2

Screenshot_20241112_215108-1
Screenshot_20241112_215108-11920×875 202 KB

1 Like
3 
/*
 * Maximum amount of copy io's outstanding at once.
 */
static unsigned long raidz_expand_max_copy_bytes = 10 * SPA_MAXBLOCKSIZE;

Effectively 10 * 16MiB → 100 * 16MiB.

I’d be interested if you see the same gain just by doubling.

4

It doesn’t seem so… I have briefly tested x2, x4, x8 :

Screenshot_20241112_230243-1
Screenshot_20241112_230243-11920×873 346 KB

5

This is how it should be: Optimize RAIDZ expansion by amotin · Pull Request #16819 · openzfs/zfs · GitHub .

4 Likes
6

@mav do you know if this will be in Fangtooth? Or even earlier?

7

It will probably depend on OpenZFS… we prefer to use a standard version if possible.

1 Like