I have 3X14TB drives. I expanded my pool with 1X32TB drive so I had 3X14TB and 1X32TB in the pool. I than removed 1X14TB drive and put in another 1X32TB drive. In Truenas I selected the replace. It re-silvered and after that, it said I had a bunch of corruptions and missing data sets. I searched the forums and found that this is a bug and requires two scrubs to clear (which is kind of ridiculous). So now I’m starting the first scrub, but my question is about capacity. My pool is 2X14TB and 2X32TB drives. Truenas only shows as 40.8TB as total capacity in RaidZ1.
Anyone have a fix to get my capacity to the correct level because 40.8TB is less than 50% of the overall capacity for RaidZ1. Thank you for any help.
This will help you understnad what is happening after adding disk.
In addition, after each drive adding to the pool have to do a scrub.
I having simillar issue BTW.
So, the only solution is to move my data and wipe the existing pool and restart it? If this is a beta future and bugged than why is it in the main release, LOL? It did a scrub after adding the first 32TB drive. I’m not sure if it did it after I did the 2nd 32TB drive.
You have two different problems with your setup. Your pool started as a 3x 14TB Raid-Z1? If you are just using RAID-Zx-expansion , those two 32TB drives will be treated as 14TB. The size is determined by the smallest drive in the pool.
If I understand, you ended up with 2x 14TB & 2x 32TB drives in a Raid-Z1 VDEV. If that pool was created new with those drives it would be sized as a 4x 14TB Raid-Z1 VDEV. You would have to replace all drives in your original Raid-Z1 VDEV (14TB to 32TB) to get the space of the 32TB drives. You should consider a new pool / VDEV layout. Linking ZFS primer and a link to pool layout whitepaper below.
The other problem is the GUI not reporting space as accurately as the command line after using RAID-Zx-expansion . That is just an ‘normal’ issue. You can search for that tag and read about it.
There is documentation that with the newest ZFS, you can add different size hard drives and still maintain Raid-Z1 or do I have that wrong? Still new to linux, NAS, and truenas. Thank you.
When I expanded my pool with the first 32TB, the capacity did increase. Just not when I replaced an existing hard drive.
Your pool capacity increased because you went from 3x 14TB Raid-Z1 to 4x 14TB Raid-Z1 layout. The 32TB drives are being treated as 14TB drives. You would have to replace all the 14TB drives with 32TB drives and then do a manual EXPAND to get a 4x 32TB Raid-Z1 VDEV with all the space available.
Raid-Z1 isn’t recommend with those large drives due to how long it takes to resilver and you could lose the pool with no more redundant drives left, due to a single failure. The usual recommendation is 8TB? and under. It can be fine if this is easily replaceable data that is stored elsewhere but I wouldn’t use Raid-Z1 as an only source of data storage.
In case anyone runs into the same problem as me, after adding 32TBX2 drives, I had checksum errors. I ran scrub and it said I still had checksum errors. I rebooted my NAS and had no checksum errors and ran another scrub. It produced no errors. In the end, a simple reboot solved my issue. Still upgrading to all 32TB drives to expand my pool. Reported pool has increased with every 14TB drive I replaced with a 32TB drive.
