Im learning here so please be gentle I have read various posts and concluded that adding additional disks to a ZFS dataset is not possible and I planned accordingly by adding all the HDDs I wanted prior to transferring data across.
I have 4 additional disks available to me now, 2 of which I have the ability to physically add to the server, and 2 I was planning on keeping in storage in case of drive failure.
Can I add the additional 2 drives and associate them with with other Vdevs such as “dedup” and “spare” now or is it too late? If it is possible to add Vdevs to the pool after it has been created, what is the best use case for the additional 2 x HDDs based on my current topology? If the answer is no, Im fine with what I have, I just wanted to get the brains-trust involved with my best course of action.
As of now with Truenas Scale Dragonfish, you could
set up a 2nd Pool with 2 Drives as Mirror
add a Mirror vdev to your existing Pool BUT in doing so you will reduce your redundancy because then, if both your drives in the new vdev would fail, your entire pool will be toast - also performance will be as slow as the slowest vdev.
im questioning the usefullness of your “cash” vdev
dont do dedup, its a RAM and CPU hog
IMO spare, log and cash vdevs can be removed safely. Others require migration of the stored data back.
With the next big release of Truenas, Electric Eel you will be able to add the new drives to your existing raidz3 vdev.
Thank you for your reply, appreciated. With regards to your points:
Ah, ok, a second pool as a mirror, that’s a good idea. I thought I had some form of mirroring going on with the RAIDZ3 setup, but clearly this is not the case, so thanks for that suggestion. Does a mirror create redundancy or is it for increased throughput (or both)?
The second point makes sense so I will probably stay away from that.
The cache vdev is on a 500GB nvme drive with 32GB ECC system RAM. I thought that was a recommended course of action. Maybe I misunderstood, but either way I don’t think I made that clear in my OP.
No dedup - roger. I do have loads of CPU headroom (not ram though), with no virtual machines, apps or anything yet, only 2 users, and a Xeon E5-1650V2 (6C/12T). But noted, cheers. As I do believe my files have some significant duplication in them, is there an app that you’d recommend that can scan the files for duplication that would work on the TrueNAS server? Is it safe/practical to scan for duplication from within windows?
Noted that the safe ones are spare (seems like a no brainer to add) and a log vdev.
So based on the above, the fact that I do have ample space on the data vdev for my needs, the cache being nvme; your advice seems to me to be pointing towards adding a spare and wait until Electric Eel and reconsider.
Any comments or suggestions are very welcome, again thanks for your advice and time. Much appreciated.
RAIDZ3 uses parity information to rebuild your data. You can lose 3 drives. But its not mirrored. Mirrors are just copies of each other without any parity to be calculated. Thats why they are faster.
2way mirror = you can lose one drive.
If you have easy access to the server, e.g. in a home lab, i woud not bother with a spare.
Burn the drive in to be sure its ok and then store it.Although that will then have the effect of it running out of warranty and just collecting dust and age.
I see “Spares” as a only temporary replacement drive that will eventually be replaced again with the new drive.
The “cash” drive is in ZFS/Truenas not a “Read Cash” like on a synology. Its Level 2 Cash. Your main Cash is your RAM. So more RAM is always better that a cash drive.
Only if you maxed out in RAM, and/or you see in your ARC reporting, that your hitrate is low, then consider a L2ARC (Cashdrive)
On the subject of dedup “scripts”:
So yes, i would wait until electric Eel, but expanding vdevs is new territory. If you are unsure about this new feature → Destroy and rebuild
In your current hardware state, I would get rid of the L2ARC (Cache) as anything stored on that eats into your RAM and thus your ARC (read cache in RAM).
Thanks all, thanks SmallBarky, yes I read that article and I also read this article as well and did some calculations. Currently I am on a 99.6% ARC hit ratio so as you guys have pointed out I am certainly not needing that L2ARC device. I might instead use it for applications as I learned, through trial and error, that issues arise putting applications on the data vdev. Live and learn.
I have read various posts and concluded that adding additional disks to a ZFS dataset is not possible and I planned accordingly by adding all the HDDs I wanted prior to transferring data across.