RAIDZ3 vs 3-WAY Mirror

[MasterCATZ]

I have run the 3way mirror many years and been through a lot of recoveries up to 2x drives in same Vdev failing in multiple vdev’s

however I am now facing free space limitations , it is the pool that I run VM’s from using 16x bays built into the chassis

however now that I think about it I might have chosen 3-way so I could have expanded into a 24 bay latter on if I needed to by adding another 3x Vdev’s

I have other 24bay racks for storage that I only turn on when needed

I was wondering if its worth while swapping over to raidz3 to gain a little more free space and sacrifice some re-silvering speed ?

when I was looking at the calculator
it said my 3way only has 2 drive redundancy while the raidz3 has 3 ?

[sfatula]

Yes, with a 3 way mirror, per raid group (vdev) as it says, you can lose 2 drives “safely”. 5 drives is too small for a Raidz3 group, I would go much larger for each vdev. Maybe 8 drives for example, or 7 if you can’t fit 16 and use the last one as a hot spare.

You’ll also lose some performance, but shouldn’t be much if you have sufficient memory. You’ll gain space and some additional fault tolerance.

[Stux]

Of course, mirrors is optimal for running VMs. And RaidZ is not.

It has to do with the volblocksize typically being smaller than the raidz stripe size.

[sfatula]

True, I usually run them on non rust though as even mirrors are too slow for my VMs. But it can be done.

[Sara]

I am not sure what the question is here

Just a small note about something that is not covered in these online calculators:

While fault tolerance might be smaller in mirror, what these calculators are missing is the bad drive batch problem.

Imagine that you build a pool 5y ago with 8x WD greens and 8x Seagate. Now all of the greens start to die like flies because of the head parking problem (not really true for ZFS, just for the sake of the argument). Maybe they aren’t even a bad batch, and just because they are pretty old at 5y, they start to fail. Let’s assume all the greens go bad at once.

Scenario A:
You built a two RAIDZ2 vdevs.
Each vdev has 4 greens and 8 Seagates.
Since you can only tolerate 2 drives failing, your pool is now gone.

Scenario B:
You built mirrors.
Each mirror has one green and one Seagate.
Since you can tolerate 1 drive failing, your pool is still fine.

[Stux]

This scenario is not as ludicrous as you think…

A similar but totally different thing happened to me, where I knocked over a stack of 5 drives… out of 4 stacks… (18 drives total) when moving a system.

3 of those drives failed when re-installed.

Luckily none of them were in the same mirror.

BUT if I had’ve had only two raidz2 vdevs… which was the configuration before I switched to mirrors, then I probably would’ve lost a vdev.

(happy ending, most of the drives were replaced under warranty…)