My old TrueNAS CORE system, RAIDZ2 10X 4TB HDD, is failing with two drives down. I have a new TrueNAS SCALE system, RAIDZ mirror 4X 14TB HDD, ready to replace it.
I want to COPY ALL my data and ZFS datasets including permissions from one TrueNAS system to another. Then decommission the old system and use the new one in its place.
I was able to run Replication Task (TrueNAS to TrueNAS, pull from another system, via SSH) successfully but I get a “permission denied” when I try to access it (shell cmd: ls /mnt/tank/Data). See screenshot. How do I make my “replicated” data accessible?
Replication task is the fastest way. Just ensure that the replication is not read-only (read-only is default). The setting is in the replication task menu. Try it out with a small dataset for starters over SSH.
Once you prove that the new pool is working as intended, select all the other datasets also and potentially enable SSH+Netcat in the replication settings to really speed things along if you have the network to support it. My old and new NAS were running SSH+netcat at 400+MB/s when regular SMB maxed out at 250 MB/s.
Your new pool is already filled beyond 80%. You need to figure out better space management.
Other than making sure the destination is not set to read-only, as mentioned by the Vampire Pig @Constantin, you need to have users on the destination system with matching UIDs and GIDs from the source system.
Core starts at 1000. SCALE starts at 3000. This might be why the filesystem permissions/ownerships do not overlap.
While we’re talking “if your network supports it” and speeding things along: enable jumbo frames if the switch(es) between the two servers (and their NICs) support it. This sort of large-scale sustained data transfer is just what jumbo frames are for.
But with that make sure all devices on said network are configured with jumbo frames, as mixing jumbo frames with non on the same vlan/subnet can cause packet loss for other systems not configured for jumbo frames.
All great tips, thank you. I finally got it all working. From a whole new system setup, I encountered a lot of bumps along the way.
Yes, replication worked. Although, I didn’t know I had to create a snapshot on source TrueNAS for it to work. Finished my replication with SSH+Netcat averaging 2-3Gbps or 300MB/s. Briefly tried SSH and didn’t notice much speed difference. First time replication, I did make mistake of read-only. Attempted to get jumbo frame to work but failed (doing iperf3 tests resulted in 0 - no transfer). Had to revert back to default MTU 1500.
I agree 80% filled is not good for ZFS but is temporary. My priority was to get data copied to new system before old system RAIDZ2 with 2 bad drives dies. New system is now 2X mirror vdev. That means I can easily upgrade anytime with a pair of HDDs.
@winnielinnie Exactly “matching UIDs and GIDs”. That was my permissions problem. Old system was TrueNAS CORE had “root” user only and signed into Active Directory. New system is TrueNAS SCALE which uses “admin”. So had to reset all my permissions to include admin and Active Directory user/groups.
