I currently have a pool of 10x 6TB raid-z2 that’s half full. I have another pool of 2 vdev mirrored using 4x 18TB, fully empty.
I want to snapshot the first pool and migrate that snapshot to the 2nd pool, then decommission the first pool.
If I added more vdevs with 2x 18TB mirrored drives each time down the road to expand, would data evenly redistribute to the new disks or are they still fully contained in the 4 first disks?
Would snapshots be the best way to migrate data? Would adding additional vdevs with 2 mirrored drives be the best choice for performance, risk reduction, and resilvering speed? I don’t mind the loss in capacity doing mirrored vs raidz.
Yes for performance (IOPS), flexibility and resilvering speed.
No for risk reduction: You can only lose one disk (per vdev), and any hiccup with the remaining disk before resilver is complete then puts data at risk.
For better security, especially with these very large drives, you’d need 3-way mirrors—at an obvious cost—, or raidz2—which you would then expand either by adding further raidz2 vdevs (min. 4 disks at a time) or through the new raidz vdev expansion (with all its caveats and gotchas).
Those rebalancing scripts are mainly done/made/designed from before block-cloning was a thing.
I don’t know if block-cloning is activated again by default. But if it is, the data might just be cloned instead of actually “distributed” over the disks…
The reason I wanted even data “distribution” is if I lose the original first 2 vdevs, I won’t lose everything, whereas if it’s spread evenly out later, I will only lose some data.
Data is broken into chucks (“recordsize” at play here), and the chucks would be distributed across vdevs, so losing one vdev would mean that most large files would NOT be recoverable from the remaining vdevs even using Klennet ZFS Recovery.
So this implies that if were to purge old files and snapshots, new files will still get written to the more empty vdevs. But would this still have the problem of pool fragmentation as seen with old pools filled more than 50%?
I honestly think people spend too much time worrying about this stuff.
Notice my pool is in the process of growing via replacing 4T disks with 8T disks… The most fullest vdevs are the oldest vdevs… and they were the most fragmented… but now they have lots of free space… and are relatively less fragmented…
And notice, that all but the newest vdevs are fairly well balanced (fill wise), even though I’ve been adding vdevs over time.
And yes, I’m using mirrors, because raidz didn’t provide the performance I needed to saturate 10gbe.
If performance was that important, perhaps I’d be using SSDs these days.
