Recommended way to migrate my main pool from RAID1 (2 disks) to RAIDZ1 (4 disks)

I’m very happy with truenas scale. I started with just 2 10 TB disks in a mirrored config for my only pool,

now I want to migrate that pool to 4 physical 10 TB disks in a RAIDZ1 configuration. I only have 2 TB of data right now and conveniently I have a 4TB SSD pool on the SuperMicro server which I can use to backup the contents my entire main pool.

What is the recommended way to do the data migration via my SSD pool?

Do I use the GUI data replication tasks?
Or should I do zfs-send to the SSD pool?
Or some other way?

And should I save the config BEFORE I do anything and restore the saved config at the end? Or does that just create more issues because the VDEVs are now different?

I’m sure I’m not the first person to have this problem, and it seems like there are many options, so there must be a thread on the RECOMMENDED way to do this somewhere…

Thanks!

1 Like

Whatever floats your boat.

A replication task from the GUI with the appropriate settings is probably what I would do. Tweak the settings accordingly (for example full filesystem replication if you want to keep everything).
Feel free to let us review your settings here before you make the jump.

A config backup does not hurt, however creating a new pool does not require restoring the config. You only need to restore the config if you do a fresh install of Truenas (or if you want to revert to a precious configuration because you feel the need to revert recent changes).
The config backup would also not know about your new pool if you create it after the backup.

1 Like

OK, so when I disconnect the existing pool, there is an option: “Delete saved configurations from TrueNAS?” I think I want to NOT check that?

This is a different configuration, I meant the configuration you can download from the GUI.

If you’re regarding the configuration when destroying the pool, then yes, leave that unchecked. I have no first hand experience but I would imagine if you name you new pool as you named your old pool, your shares etc. should still work afterwards. Ultimately they are referencing /mnt/poolname and that will be the same then.

But before you do that, you need to replicate your data to your SSD pool.

1 Like

Pretty much.

1 Like

this is the plan then:

  1. add the 2 more disks, but don’t put in pool
  2. save TrueNAS config (just to be safe)
  3. stop all my apps
  4. Use the GUI to copy main pool to SSD pool
  5. also do copy via zfs-send for safety to SSD
  6. Use GUI to disconnect main pool (don’t destroy anything including config)
  7. Use GUI to recreate new main pool with all the disks with same name as old pool
  8. copy data over from backup to the main pool

Now the only question is: Do I need to reboot? If so, why?

I do this fairly often.

This is how I do it

  1. add new disks… make new pool.
  2. replicate existing datasets of existing pool onto new pool using once off data replication in the gui. Ensure that the destination is NOT set read-only. Verify the new pool looks like how you expect…
  3. export old pool, don’t erase, don’t zap configs.
  4. rename new pool to match old pool following this guide:
    HOWTO: Rename a ZFS pool

Tada.

Maybe restart.

4 Likes

actually, that’s a great idea. When I copy the main pool to the SSD, I now have a clone. I should be able to swap the names in the pool and check everything works as expected. I can then kill the old pool.

That way the only risk is renaming the pools and I always have an active working backup.

right?

Yes. Really not much risk, and your data is not at risk.

1 Like

awesome! that really lowers the risk.

for those who might be wondering, if you want to expand an existing raidz1 vdev to add more disks to it, you have to go through the same procedure above.

1 Like

I made the mistake of assuming. When you said you have 2x10 TB disks and want to go 4x10TB, are you going to purchase an extra 2 or 4 disks?

We talked about not zapping the config, but I’m stuck at the renaming part. If you reuse 2 disks you do not need to rename the pool, because it will be already destroyed when you create the new one. Or at least I think it’s easier if you verify that you correctly copied everything to your SSD pool, destroy old pool and recreate new pool.

If you get additional four disks, then yes, what @Stux said.

I just Purchased 2 additional drives, so my new pool = 2 old drives + 2 new drives.

so i will copy main to ssd pool, export main, rename ssd to main and verify everything works.

The question is this: if I export/disconnect a pool, and then change my mind, can I use the “Import pool” to reverse the disconnect?

I would honestly skip that step. If it doesn’t work then it will not work after you created your new pool. You will not gain anything here.
You’ll have to reconfigure some services then, not the end of the world. Most stuff should survive after you’ve created your new raidz1 pool with the same name.

Just make sure all your data, snapshots is there as @Stux also pointed out.

Exporting / Importing is like ejecting a flash drive, you can reimport the pool afterwards.

But this will not help you / is not needed.

  1. Replicate your datasets to your SSD pool
  2. Verify that Replication worked as you intended
  3. Destroy your current HDD pool without destroying the config
  4. Create new raidz1 pool and basically repeat 1) but this time with the new pool as your target

I stand to be corrected, but I think @Stux assumed you will have the new and old pool coexisting at the same time which is not the case since you need the old pools drives.

Edit: 2 TB isn’t a lot for internal Replication, but is your SSD pool redundant (mirrored, RAIDZ?).

You want a backup of your data in any case (if it’s important data) so you may as well start with 0) and get an additional backup in place (cloud, other machine, …)

Yes, there’s really no need for the renaming if you replicate to the ssd pool, verify it, and then replicate to the final pool.

The key is that you would get to verify totally each stage with the rename, which is not hard, takes a few minutes.

And export just makes it so that a pool is not claimed by a specific zfs host. Import claims it. Analogous to eject/insert

OK, this is my final checklist of tasks in order with no missing steps. Any final comments on this? (edited to incorporate all comments)

  1. Insert the 2 more disks into the hardware, but don’t create a pool yet. Make sure they are recognized.
  2. save TrueNAS config in System > General. Also, take a few screenshots of the datasets, and SMB shares (just to be safe)
  3. stop all my apps (since they will be unhappy when the pool goes away; is this necessary?)
  4. Use the GUI to copy everything in the main pool to SSD pool
  5. Migrate system dataset from main pool to SSD pool (so I have a system after I disconnect) using System Settings>Advanced>Storage, then select the pool after the prompt for “System Dataset Pool:”
  6. Use GUI to disconnect main pool (destroy the date, but NOT the configs which specify SMB shares, etc.)
  7. Use GUI to create a brand new main pool with all four disks with same name as my original main pool
  8. Use GUI again to copy data over from SSD pool to the new main pool
  9. Migrate system dataset back over to the new main pool using GUI as done before
  10. Done.
  11. Reboot just to make sure there are no issues. There shouldn’t be, but I’d rather find out now than later :wink:
2 Likes

I think you will have trouble using the old pools disks in the new pool.

I just quick erase them first. Then you can.

I haven’t done it in a while, but this problem could perhaps be avoided by erasing the pool on export but you do not want to erase the shares config etc, and I’m not sure if that is an option if you erase the data.

2 Likes

thanks! I’ll be on the lookout for that and update this thread.

but why would i not want to erase the pool’s data? there is a separate option for the system config erase.

Here are the options:

what exactly does “Delete saved configurations from TrueNAS?” do?? That’s confusing.

1 Like

That’d be it. Destroy the data, don’t delete the configurations.

The configurations is anything that references datasets/zvols on the pool, eg smb shares etc

2 Likes

excellent. super helpful. thank you!

Thank you for sharing your migration route and thoughts behind it.

I‘m watching as I want to rebuild my pool soon (once dragonfish is released) and I‘m not sure yet, which route is the best (=safest).