So I have been running TrueNAS for quite a long time. since the FreeNAS 9 days.
Currently on TrueNAS Core 13.3-U1 for my Main NAS.
And the latest TrueNAS Scale on my Backup NAS.
The Main NAS pool configuration consists of:
vdev raidz2 6x 6TB Ironwolf hdds
vdev raidz2 6x 10TB Ironwolf hdds
With all the data backed up, I was thinking of altering the pool configuration of my Main NAS and move over to TrueNAS Scale at the same time.
As you will see in my MainNAS signature, the case Im using is a hot swap 20bay silverstone case.
I was thinking of changing the pool configuration to take advantage of all the 20bays to increase capacity and speed it up a little for the metadata and tiny files.
I was thinking of altering the pool to be:
vdev raidz2 8x 6TB Ironwolf hdds
vdev raidz2 8x 10TB Ironwolf hdds
special vdev raidz2 4x 2TB Samsung 870 EVO ssds
This should give me plenty of capacity for a while without spending too much on the upgrade.
But I got a couple of questions and I can’t seem to find the answers so far:
Can a special vdev be a raidz2 like the regular vdevs or is it mirror only configuration ?
If I use the TrueNAS zfs replication back from the BackupNAS to the MainNAS after the new pool, will it automatically distribute the files and metadata to the special vdev depending on what I set with the small block file size, the BackupNAS will never have a special vdev as its purpose is pure storage ?
Bonus question: what’s your opinion on using the Samsung 870 Evos for special vdev, do you think it will be reliable enough. Note I can’t afford the server grade ssd’s. So would be up for recommendation of other brands that are a equivalent price to the Samsungs if you think the Sammy’s are not worthy of TrueNAS ?
If I were you, I would not purchase 2 additional 6TB HDDs, but 2 x 10TB HDDs. ZFS will ignore the additional space. But, at some future date you could grow your 6TB vDev to 10TB disks. Of course, if their is a big price difference, or if you already have the 6TB HDDs, it is understandable why you would continue to use 6TB HDDs.
Mirrors only as far as I know. And if you are using RAID-Z2 on your main data vDevs, then 3 way Mirror would be best for similar redundancy.
Yes.
Some consumer grade SSDs don’t have as much write longevity as desired. A Metadata & small file vDev can have a lot written to it, as directory entries get updated for any file change.
Plus, loss of a Special Metadata vDev means loss of the entire pool. Now one way to combat the risk, is to use different brands to hopefully scatter failure rates. So 1 x Samsung 870 Evo, then perhaps a Crucial, and then maybe even a 3rd vendor. This should cause any failure to be a single SSD, leaving you with 2 way Mirror until replacement.
Theoretically, you can use 2 different SSD lines from the same vendor for different failure rates. My old NAS used 2 x WD Reds, (CMR), and 2 x WD Red Pros, using RAID-Z2. So worst case, I could loose the cheaper 2 x WD Reds and still have my data.
Also I would suggest leaving a spare slot in the chassis. In the event of a drive issue its always worth having a slot to use to replace a drive in place rather than removing a drive and then adding a new one
AFAIK, it’s mirror-only VDEVS. Though you could likely set up multiple parallel VDEVs inside the sVDEV pool to grow sVDEV capacity?
AFAIK, yes. The sVDEV is a repository for small files and metadata, so such content would be automatically identified and distributed to the sVDEV as appropriate, in both directions (i.e. backup vs. restore)
I have heard uneven things about the EVO 870 series. I would research whatever brand you want to contemplate using. I elected to go with used Intel S3610 Data center drives in a 4-way mirror and they have been 100% reliable. By virtue of buying them used, they are from different build dates, have experienced different use cycles, etc. so that’s another way to reduce the probability of all the holes in the data-loss cheese lining up.
As far as using the new enclosure, there is something to be said for leaving one slot open to allow for burn in, replacement, etc.
Separately, I would ensure that each slot has a similar pressure drop so that the cooling across all drives will be similar. That suggests putting the 2.5" drives elsewhere inside the case. For example, some folk mount them on spare PCIe slot carriers. Unused 3.5" slots should have something hard-drive sized to block air-flow just like a regular hard drive would.
