Astounding effect of recordsize 1M

TLDR: For WORM data, 1M record size is fantastic.

After much reading I changed from 128k to 1M record size for my datasets storing large WORM data. After a copy and delete process, the results were far better than I expected, so much so I feel compelled to share them. In summary, on a 7disk RAIDz2 dataset with LZ4 enabled, async writes, 7200rpm drives, not SMR:
Total space used fell about 6.6%! Even for folders with mean and median file sizes of approx 1M the folders shrank! Metadata dropped from 0.12% to 0.05%.

I also had an impossibly large improvement in file copy speed, but it was just me noticing block cloning. So a little less astounding.

5 Likes

This is very interesting thanks for sharing.

I use 1M as my “default” recordsize.

2 Likes

And scrub time fell from 9.5 hours to 8.5 hours.