Title feels a bit longer than I think I’d prefer, but effectively what I’m looking for.
Backstory:
I’ve had this pool for over 4 years and it’s suited me well.
I’m aware that at around 50% usage, you start seeing pool performance degradation, and you don’t really want to go over 80% because of larger degradation. As well as the further you get to 100%, the more likely you are to have problems with your pool due to ZFS being a CoW filesystem.
I figured that I’d upgrade some drives, and found some 18 TB SATA drives used for Chia at a decent price.
Over the past several days I’ve been burn-in testing them with bad-blocks (these Toshiba MG09s seem to be faster at reading than these Seagate Exos drives).
My current hardware, for what’s potentially relative:
NetApp DS4246
Single IOM6 module
HDD: 12x 3 TB Hitachi SAS HDDs
R730xd
CPU: 2x E5-2643 v4
RAM: 256 GiB (ECC of course)
HBA: LSI 9300-8e
Info on my current pool:
2x RAIDz2 vdevs of 6x drives
76% used space
25% fragmentation
Actual question:
I’d like to replace the 3 TB drives with the 18 TB drives, but I’m curious on the preferred way to do so.
Since I have the NetApp shelf, I can (and am) comfortably running both sets of drives, so regardless all drives would be online.
The way I see it, there’s two potential ways I could go about this:
- Online-replace one-by-one (potentially two-by-two due to 2 vdevs) and expand at the end
- Need to deal with a resilver between each addition
- Resilvers aren’t sequential
- Not really complicated
- Need to deal with a resilver between each addition
- Create a new pool on the new drives, sync the data, export the pools, rename, and import
- I’d probably zfs send | zfs recv for this, doing it once for initial data, stopping everything that writes to the pool, re-syncing, and performing the rest
- Not sure if zfs send | zfs recv is sequential, but probably faster
- More complicated
- I’d probably zfs send | zfs recv for this, doing it once for initial data, stopping everything that writes to the pool, re-syncing, and performing the rest
I’m not in a massive need of the free space currently, and can easily delete things to free space before it becomes a concern.
I’m not sure if there’s any other pros or cons. I’d imagine that the send | recv would probably reset my free-space fragmentation, but maybe fragmentation would also drop since there’d be more sequential free-space with the new drives?
Just curious to hear thoughts, I’m probably still several days away, seeing as how the Exos are almost done with the write cycle of the second pass, and the Toshibas are around 20% done with the read cycle of the second pass.