Hi, So I know I cannot change from 1 to 2 directly. Currently I have 3x16tb hdds running raidz1. Now I have 8 total slots, so 5 spare. I am wanting to change from 1 to 2 as I feel 2 vdevs running raidz1 is slightly better for io but not for resilience. Is the best way to do this to buy 5x16tb hdds create a raidz2 pool and migrate the data from the raidz1 pool to the raidz2 one before wiping and adding these 3 drives to raidz2 via the zfs expansion. My question would be how does this work as if I create the z2 pool first I will have 2 parity drives, and with 1 parity still in the raidz1 will it automatically adjust for this? Does truenas need to do something similar to a rebase for programming where it will move the data around so at the end I have 6 usable drives and 2 parity rather than 5 usable and 3 parity.
Yes: Create a new pool (NOT a new vdev in the same pool!), and migrate the data.
Since these are two ZFS pools, a one-off replication task is ideal.
During this task, ZFS migrates the data, not raw blocks. Data may be reorganised into different records if record size is not the same, may be spread over a different width, and parity will be recalculated in any case. So I’m not sure what the concern is…
Not as much concern, but just never needed to do this and having a lot of data on the drives I would rather not mess it up so explore all details beforehand. I figured it would be easy to just migrate, but I am relatively new to truenas scale so better safe than sorry 
But it is good to hear it automatically does all of this.
Plus, this type of migration also allows changing Compression algorithms or Checksum algorithms.
I still use LZ4 compression, but eventually changed from the default Fletcher4 Checksum, to SHA256. I figured a crypto-graphically secure Checksum would detect bit-rot to the extreme. (So I am paranoid, yet data corruption IS out to get me!) Because I set it after all 4 of my non-TrueNAS computers were built, only new files, (and changes to old files), have this non-default Checksum. Better than before…
Do you have this data backed up elsewhere? It may be better to verifiy all data on a backup source. Destroy current pool and then make pool with a 8 wide Raid-Z2 configuration. You copy your data back from your backup source and you have the advantage of your all your data using a 8 wide Raid-Z2 setup, 6 data and 2 parity. You also get more accurate space reporting over what the Raid-Z Expansion method will provide.
While you have a different (hardware) situation, you may be interested in reading this topic. OP even made a big post in his blog about this migration.
Not really due to the size of the drives, it is 48TB (3x16tb).
How much data is on it actually? Raid-Z1 would top at 32TB theoretical for 3x16TB
You should have the data backed up to another source. We have had people loose the entire pool, even with Raid-Z2 or Z3. Power problems and then the pool not importing. There are multiple threads.
It helps if you list full hardware details so we can offer better suggestions.
Its completely full, 29TB. So I cannot really viably transfer it to another source and create the pool and import the data, I need to do a switcharoo which I believe is possible from etorix’s answer. I wasn’t going to go out and buy that much storage due to transfer it. I have checked, but there was a lot on other things such as direct conversions so thought it easier to ask, maybe the search bar screwed me. As for hardware I have 3 Seagate IronWolf Pro 16 TB (ST16000NT001) and plan to buy 5, create the raidz2 pool and migrate the data before using the drives from the raidz1 to expand the raidz2, I have a UPS which ensures it will not be without power.
Although I am curious with the hdds being so large is it possible to pause the process if there is a power outage?
If you gracefully shut down, the process will begin again where it left off. A sudden loss of power, without the UPS, may cause problems. We have a few posts with pools going offline after power interruptions.
Your method is fine. Search RAID-Zx-expansion and go over a few of the threads so you can get an idea of what to expect for space reporting in the GUI and CLI afterwards.
I recently did something similar, as @swc-phil mentioned.
Couldn’t you do this?
- Buy 3x16 TB disks
- Create a 5x16TB RAIDZ2 pool using the new 3x16 TB disks and 2x16 TB fake disks (sparse files). New pool has 48 TB of capacity.
- Offline the fake disks from the RAIDZ2 pool.
- Migrate data from the old RAIDZ1 pool to the new RAIDZ2 pool.
- Offline a disk from the RAIDZ1 pool and use it to replace a fake disk on the RAIDZ2 pool.
- Destroy the old 3x16 TB RAIDZ1 pool.
- Migrate the remaining two disks from the destroyed RAIDZ1 pool to the RAIDZ2 pool.
It should be safe as long as you don’t have two concurrent disk failures.
Between steps (3) and (5), you’ll be writing a lot of data to the RAIDZ2 pool while it’s in a degraded state with two missing disks, but even if a disk fails, you still have the primary copy of the data on your RAIDZ1 pool with redundancy.
Seagate now sell 30 TB Ironwolf Pro/Exos M, so external backup in a single drive is (just) possible…
STH just shucked a $330 Seagate Expansion external USB drive, and found inside a yet unreleased Barracuda 28 TB, which should be based on the same platform as the above 30 TB drives.
I’m not specifically suggesting you go for any of the above, but think about some kind of external backup.