Hello,
I am new here and I discovered TrueNas few weeks ago.
Before put TrueNas in prod, I installed a POC to test and see it.
I make a VM with 9 disks for data’s and 1 for OS.
Have you any advice before install/config it?
For information my future TrueNas will be on a physical server like a HPE ProLiant ML110 Gen10
What do you thing regarding this?
Thanks
Opinions incoming, since I don’t know what your use case is I will have to guess and pass on some things I’ve learned along the way.
First, instead of counting disks for the array, grab a pair of cheap 128GB ssd drives. They will become your boot drive mirror. You would install to one of them and later mirror that drive to the other ssd in the interface. Rock solid and not wasting big storage on your boot.
Now, if you had a total of 9 drives plus one for the OS, I’m assuming you have lots of sata ports available, so losing 2 to the boot mirror leaves you with 9 or 10 right? You could make 3 pools, raidz1, total storage of each pool would be n3-1 so 3 20tb drives would yield 40tb with one lost to parity. With 9 drives you have a vast array of choices on how you want them configured vs how much space you need.
Thanks
My POC on VMware is an example with 9 drives for data because I saw on a website that is more perf with 3 or 5 or 9 drives for data. I don’t know if it is right…
My futur physical server will have 16 slots : 2 drives for OS and one for spare =3 + a lot of drive (to be defined) for data (and one spare drive) with a maximum of failover drive possible.
I think to have one pool with all data drive with a raid-z3.
Maybe SSD or SAS for OS and HDD or SAS for data.
What do you think about this ?
There’s no magic number for raidz performance, but with respect to safety, about 10-12 is only as far as most would like to go with raidz2 or raidz3.
Better have two 8-wide raidz2 with 16 drives than a single 16-wide raidz3 (or 14-wide).
Do not waste drive bays for the boot drive. Small and cheap M.2 are meant for that.
Mirrored boot is largely unnecessary for a home server if you always keep a recent copy of the configuration file at hand.
I am sorry but I don’t understand this : « about 10-12 is only as far as most would like to go with raidz2 or raidz3.
Better have two 8-wide raidz2 with 16 drives than a single 16-wide raidz3 (or 14-wide). »
Can you explain me why?
I cannot use M.2 drive. It is only SATA SSD or HDD or SAS SSD or HDD.
Careful : it is not for a home server but for an enterprise server that why I would like to use a HPE ML110 gen10 server.
A mirrored disk for the OS is mandatory.
And added a spare disk for the OS is to be sure the server can be safe for a 24/24/7.
You’d be surprised how many here have enterprise rack servers at home…
Anyway I seriously doubt that the HPE ML110 G10 (or whatever else you may actually use) has no provision for dedicated boot drive(s)—internal M.2, rear 2.5" bay, whatever—and forces to waste some of the main LFF storage bays for boot. Keep storage bays for storage.
The wider a raidz vdev, the longer it takes to resilver, and the higher the risk of further drive failures during resilver. The general advice is not to go wider than 12 for a raidz3; you may ignore it, especially if you have a comprehensive backup strategy, but you’ve been warned.
1 Like
Ok. I’m sticking to my mirrored SSD boot drive recommendation…because they’re cheap and reliable. Don’t need to be big, or fast, just a trusted name.
Back to the raid level for 9 drives. But let’s review what we should know about raidz first and then draw some conclusions.
Raidz is software raid with zfs. Raidz1 is a pool with a parity drive. In a 5 disk pool, raidz1 would deduct a single drive from the pool, leaving you with 4 drives, because it spreads parity data evenly across the drives, in case you ever lose one from the pool, the rest can rebuild that data from parity onto the new replacement drive. Doesn’t matter how big or small the disks are, you always lose the equivalent of one drive to this parity arrangement. Pros: pretty fast. Cons: the resilvering process, which rebuilds a new drive from that parity data, is stressful for all the drives involved. If one is cooked and a second is teetering on the edge, you may find yourself in a risky situation. That’s the trade off for max storage with some protection.
Now let’s look at a raidz calculator. https://www.raidz-calculator.com/ will let you play around with various configurations so you can quickly caclulate redundancy and speed. It’s ideal to have smaller, mirrored vdevs that can resilver quickly for your pools. Each mirrored drive equates to losing that drive’s total space but doubles up read speeds for the vdev and ultimately the pool. Tinker at that site, it simply explains what the pros and cons can be for each combo.