5-wide vdev Vs. a 6-wide vdev (in RAIDZ2) - how much worse is it?

TrueNAS General
1

Is a 5-wide vdev much inferior to a 6-wide vdev, in RAIDZ2?

Or do many people do it, and it’s totally fine?

Reason I ask:

Building my first TrueNAS and can easily go with two 5-wide vdevs of 24 TB HDD’s (capacity of 10 or 11 HDDs total, in my Cooler Master HAF 922 case).

Or if I can cram an extra HDD in there somewhere (to bump the capacity to 12 HDDs total), I could run two vdevs of 6 HDDs each.

I was looking at it from an efficiency standpoint, it seems like useable TB’s would increase by a good amount, just by adding the extra HDD.

Just trying to determine the cost / effort / benefit ratio.

Thank you in advance for any suggestions.

2

Since expanding pools is now an option I guess it works out either way. Some minimal space losses & also expect GUI to hate showing the correct space after expanding; unless you rebalance after expanding.

1 Like
3

Thanks Fleshmauler.

Hmm just realized my case can hold up to 14 x 3.5” HDD’s.

Thinking even of running a 7 wide vdev of 24 TB Ultrastar’s.

Would that be a bad idea?

Versus a 6 wide vdev of 24 TB drives?

(All in RAIDZ2, of course)

4

Why? Because it’s an odd number? Not a power of two?
ZFS no longer has such preferences for vdev width, go with what suits you (within reason).

2 Likes
5

I did find this interesting post where they show that the efficiency of both 6, 7, and 8 wide vdevs is 66.6%. Not sure if anything has changed though to make that untrue (or if this is an accurate analysis):

6

A six-wide z2 used to be considered one of the sweet spots re performance, capacity loss to parity, etc. IIRC, ashift also had to do with it? But for a small SOHO system, who cares?

I’d focus on what gets you the performance / capacity you need, ideally with one drive bay left open. That empty bay becomes very useful if you ever have to replace a drive, pre-qualify a new drive (bad blocks, et al), etc.

5 Likes
7

I run a 7 wide z2 array of HDDs (with a metadata only L2ARC SSD). It is fine.

Having read the linked article, I should add I’m using it with 1M record size and lz4 compression for mostly WORM data - But that can include tens of thousands of small files and it performs well reading them. My VMs live on a 3 wide mirror. Larger mixed IO including app storage lives on a 6 wide z2 SSD array.

2 Likes
8

Thanks guys. I decided to go with 6-wide vdev.

Ordered the HC580 Ultrastars (refurbished), just waiting for all parts to come in now.

2 Likes