Of course, “redundancy vs. capacity” is the name of the game…
The solution would be to aim for drives larger than the current 8 TB, say 12-16 TB, as a starter for the new pool and progressively replace the 8 TB as you. Initial raw capacity is 4*8 = 24 TB for the mixed 6-wide raidz2 pool, and after replacing the last 8 TB you have one button to press to extend to 4*12 (or whatever). No weird space reporting.
This will be your only way to increase capacity once you have reached the maximum number of drives.
The outline is:
- optionally unplug the old pool to prevent mistakes while dealing with the CLI
- Create the degraded pool.
Attach 4 new drives, format them on the CLI to mimick what TrueNAS would do (this is where some research is needed), create 2 sparse files of appropriate capacity and then create your 6-wide raidz2 with the sparse files and the UUIDs of the real drives
zpool create newpool raidz2 /home/admin/sparse1 /home/admin/sparse2 UUID1 UUID2 UUID3 UUID4
(plus probably a few options for compatibility such as-R /mnt)
then offline the sparse file devices (and delete these files for good measure)
Once you can export and reimport the degraded pool from GUI, you’re done dealing with the CLI. - plug back oldpool (you need to attach 7 drives at this point, but it is acceptable that one lies outside of the semi-open case, with data and power cables coming out…)
- Replicate your data from oldpool to new pool.
- Shutdown and detach two drives from oldpool, which you keep as (degraded) offline backup.
- Take a deep breath and reboot. Degraded newpool should come up; replace one of the missing devices (sparse files) with the remaining old 8 TB drive.
(At this point, you have two non-redundant copies of your data. Once resilver is complete, newpool has one degree of redundancy.) - Power down, bring in another old 8 TB drive, reboot and replace the other missing device with the 8 TB drive (which then destroys oldpool beyond recovery).
- Rename newpool as oldpool according to the Resource.
- Wipe the last remaining old 8 TB for use as cold spare.