I had 4 TB WD RED SMR drives in a raidz2 pool that i wanted expand by 1 drive. I bought the same drive 4TB WD RED and proceeded to expand my pool. After sometime it gave a write error on 2 of the drives which i cleared after some talks in the truenas discord. After some research i found out about the difference between SMR and CMR drives and that these errors were probably caused by the SMR write timeouts and were not that problematic.
I decided to buy 5 new ironwolf drives that are CMR to replace the old SMR ones. The issue now becomes. I currently took the new drive offline to stop the extension in case the errors were of a greater consequence. It said it would take 5 more days to finish, and that value would probably increase.
I asked on the truenas discord for some advice and some users told me to stop the extension, add the new disks to the system and create a new pool. After that do a zfs send/receive to that new pool. I’m just not exactly sure what the plan is for the new 5th drive since the extension was stopped and how that would effect the replication and if replicated what would happen to the extension.
The pool data does not care about the RAID-Zx expansion. It should be good if your pool is otherwise healthy.
You basically “lost” a disk when you offlined the expansion disk. But, since you have RAID-Z2, you still have at least 1 disk’s worth of redundancy. (Possibly 2 disks worth of parity for some data due to the prior layout…)
You can replace the offlined SMR disk with a CMR disk. That will resume the RAID-Z2 vDev expansion, though slowly because of the other SMR disks. Whence that is done, (days, a week...), that will restore your 2 disk redundancy.
Then you can replace other SMR disks with a CMR disks, 1 at a time. Again it will be slow due to the SMR disks. But, every SMR disk removed it will get better.
Or if you have enough disk slots, you can create another pool and copy the data over.
