So, i’ve a NAS which has 8 disk in RAID-Z2 and i’m planning to add 8 more disks to the same pool. Can i create another VDEV with the another 8 disks and then add it to the existing pool? Will it work? Is this the recommended way? Are there any downsides like data imbalance? Will i’ve to use any script (i read on this forum somewhere) to re-balance the data between both the new and existing VDEVs so that it does not affect the performance and have same level of data/parity across all the disks in both the VDEVs?
Writes on the pool will favour the emptier vdev until over time parity is restored. In reality I doubt you’d notice a difference depending on workload.
I’m not personally familiar with rebalancing pools so can’t comment but it’s not necessary for the pool to function.
One of the recommended ways. The other way is to simply upgrade your existing 8 disks to larger disks one by one and resilvering each time. It will take significantly longer, but you will have more capacity after you replace the last disk.
The pool will be imbalanced initially for the first method until you’ve done enough read/write for quite some time (unless you rebalamce manually), but should still remain balanced with second method.
There is a script somewhere in the forums for rebalancing the pool. Performance will probably be somewhat subpar since data isn’t spread across both vdev’s equally. Remember, a ZFS pool performance scales with number of vdev’s, not number of disks. So if the data isn’t spread out evenly across all vdev’s, you’re basically only getting performance of 1 vdev.
Parity will remain the same for the vdev level. Again, fault tolerance exists only at the vdev level. Lose one vdev, lose whole pool. This is true even if you started with only 1 vdev in the beginning and expand it to two later.
Is that really the right way to upgrade to larger capacity? Cause, from what i’ve read from the guides, it says to use same size of disks but in a pool, for example, if a 10TB drive fails, one should replace it with the 10TB and not less or more capacity. If i understand correctly, you say that to increase the pool capacity, one method is to replace the disk from 10TB to 12/14/16/18/20TB and keep resilvering until the last disk and the whole pool capacity increases.
Will resilvering the disks after adding another VDEV not work? Is there any better workaround? I heard that TureNAS now has pool expansion feature.
Not less? Of course. You will not even be able to replace the failed drive with a smaller capacity.
Not more? Why not? Sure, it will only treat10 TiB of the drive as “usable” capacity, even if the drive is 20-TiB large. But you will still have a fully functioning pool. It’s not until all drives are replaced with a higher capacity that ZFS’s (auto) pool expansion will trigger, and you’ll immediately have a pool with 20-TiB members (instead of the previous 10-TiB members).
The reason to hold off or avoid replacing drives with higher capacity ones is price and future management. (If you commit to a pool that uses high capacity drives, you will not be able to purchase smaller drives in the future for resilvers. You’ll lock yourself with the higher capacity purchases. Can’t go backwards.)
For me personally the biggest drawback to staggered expansion is the fact you end up with a pool that has 50% 3 year old drives (let’s say) and 50% brand new. Therefore as your old drives need replacing your new ones are just getting started. Having said that you could then explore the option that is being mentioned i.e replace those old drives with newer bigger ones.
I think you misunderstood me. To replace a failed drive; yes, you would want the same size drive.
No, I’ll give you an example. If you want to upgrade a 4x10TB vdev, you’d replace a 10TB drive with say a 14 TB drive, resilver that, wait till it’s complete; then replace a second 10 TB drive, resilver that till it’s complete, replace another 10 TB and resilver, so on and so forth until the last drive has been replaced and resilvered.
Both adding another VDEV and RAIDZ expansion would require rebalancing if you want absolute optimal capacity/performance, no way around it. Think about it, the brand new disks you added are empty unless you somehow can just wave a magic wand to get that filled… in which case, I would want to know where you bought that magic wand also.
The link was posted above here. It will likely take some time for it to do its thing.
So, Simple resilver will not make the data balance? Also, if that’s the case, how come people add JBOD to their existing system? I thought add vdevs and just reilver and you’re done
Yes, @etorix mentioned that. Is that an official script?
What resilver? There is no resilvering if you’re just adding another VDEV to an existing pool. You would just have to use the pool as normal and overtime, it would rebalance itself as writes get prioritized to the new VDEV.
I don’t believe so unfortunately. You would just have to trust the script. You could verify it yourself if you don’t trust it of course, if you understand bash. Seems like it’s less than 300 lines, so you could do it in a day or two.
Resilver is about sanitising data. As much as possible, this is done in place: No data move, no rebalance.
If capacity is all that is needed, add another vdev and don’t bother about imbalance or suboptimal performance.
No, it is a third-party script.
To work, it copies snapshots, deletes the originals and then renames the copies. Make sure you understand the side effects, and associated risks.
Yes, but then what about the parity? I think unless the data is re-balanced, the parity will not be spread across both the VDEVs. Is that true in my case?
Regarding the performance, how much performance will be affected if i don’t use the script to re-balance the data?
OMG. What are the side effects and the associated risks? I thought it would all work fine but now i’m so much scared ;(
Parity is fine ZFS will ensure data is protected (at the vdev level).
The difference is that the majority of new writes will be specifically targeted at the new vdev to bring it in line with the other. This is the performance impact that’s being mentioned. It will perform more like you have one vdev and not two but let’s be honest that’s what you already have so don’t sweat it.
For both the VDEVs? Note that i’ll be adding a second VDEV with same number of disks and the capacity.
Yeah, i see that. So, lets say a few months down the road and then i’ll get the performance of two VDEVs when the second VDEV has the equal number of writes? If so, that can work. But if not, i can see how can i backup the data and then create two permanent VDEV when setting it up.
Yes, wanted to be sure cause i don’t want that if a disk fails from the newly added VDEV to the pool and then the pool dies because the data was not re-balanced or parity issues.
Gotcha
So, in short, adding another VDEV to the pool will add the parity (as per the layout i.e. RAID-X) but the performance will be of one VDEV until the data is re-balanced for optimal performance. Did i understand it correct?
