I am still somewhat of a newb to TrueNAS and ZFS and have two questions about replication tasks…
I need to migrate my ZFS storage pool (tank0) to a pair of new disks (RAID1 setup).
(Side-note: My goal is to replicate the old pool (tank0) to new disks, so that I can move the old disks to another NAS and configure that off-site backup. I am hoping that if things go as planned, I would only need to copy the data once to the new disks, because the old disks still contain the replicate that I can sync against after moving them to the backup-NAS.)
Now, when trying to run the replicate for the first time, I found that no data was being copied. I then realized that my snapshot task for tank0 was misconfigured and did not perform recursive snapshots. (Good thing I realized this now, before losing data.)
So after creating a snapshot with the Recursive option checked, it started copying the data to the new pool (tank0_new) during replication.
I have two questions now:
Why is a snapshot needed for the replication? I selected Full Filesystem Replication and it sounded to me as if, in this case a snapshot is not needed. This is the help text on the option:
Help: Full Filesystem Replication
Completely replicate the selected dataset. The target dataset will have all of the source dataset’s properties, child datasets, clones and snapshots that match the specified naming schema. Set Snapshot Name Regular Expression to .* to replicate all snapshots.
Am I doing things correctly such that I will end up with a full copy of the source data-pool? Is it realistic that I can then continue the “old” pool in the backup-NAS?
Only snapshots are replicated. It doesn’t matter if it’s a full (first time) or incremental replication.
A snapshot is an immutable form of the dataset, which includes all its blocks.
The difference between two snapshots is called the “delta”, which can be very small if there is not much of a difference between them. This is why when you incrementally replicate a 2TB snapshot, only 1 MB of data needs to be transferred if there is only a difference of 1 MB between the base snapshot and the one being replicated.
Rather than get technical, I’ll ask you a question that might help you better understand what is happening.
If snapshots were not a mandatory part of replication, what do you expect to be replicated from a dataset? Everything? What if you’re writing new files or deleting stuff just before or during the replication? ZFS should start over again to see if anything was changed and then make the changes on the destination?
What’s the geometry of tank0?
If it is a mirror or a single drive (degenerate mirror of sorts), you can just evolve from your current setup to the new one. (Assuming that the new drives are at least as large as the old one(s) and you mean “mirror” by “RAID1”—vocabulary matters here.)
Thank you for the clarification. That helps a lot (especially the questions…). Do you know of a not-too-technical, but authoritative write-up about how snapshots work? I hope this could help me better grasp the basics…
Yes, I already copied it. But I did not yet do the remaining steps of the previously linked post. But would replace not remove the dataset from the previous disks?
I do want them to keep the data so that I can move them to the backup NAS…
This resource explains why “math” doesn’t make sense with ZFS snapshots.
Box with white sticker = block in the live filesystem
Box with colored sticker = block protected by a specific snapshot
Translucent box = block that still exists in the pool (protected by a snapshot) and still consumes space, but it is no longer part of the live filesystem
I didn’t have time to expand on the resource, but I might in the future. One thing it doesn’t show is how it relates to replication.
My (poor) understanding is that snapshots “freeze” the block pointers & meta data at their current state.
New pointers & meta data are created when any changes are made, so the original data still exists & isn’t freed up or overwritten (until snapshots are deleted). Benefits are that only the changes from snapshot take up additional space; so if not a lot of changes it is very low cost storage wise.
This is as simple as I can explain it, because it is the full level of my understanding of snapshots I couldn’t go into more details or complexity if I tried.
Then you followed the right procedure (but you didn’t explain that you wanted to retain the old pool).
And since you have a ZFS backup you really want to have a periodic snapshot task and periodic replication.
Alternatively, you could have expanded the pool into a 4-way mirror, and then used the command line to split it into two pools of 2*8T and 2*4T.
zfs send works on a fixed, immutable source – which means a snapshot. By not having to worry about anything changing during the processing it makes everything vastly simpler (which means more reliable).
My script to replicate a top-level dataset to another pool says:
DATAPOOL: the name of the data pool I’m backing up one dataset from.
dset: the name of the dataset I’m backing up.
BUPPOOL: the name of the backup pool I’m backup data up into.
The -v on zfs recv is optional, since this is logging to a file I let the verbose information flow so that if anything goes bad I have a record later of where it happened. The other options are needed to get the results I need.
The -d switch, in particular, causes the stream received to be stored under the last name component of the source stream, so it ends up being “$BUPPOOL/$dset”.
This use case is with the backup pool mounted on the local system. Using a pipe through ssh would work just as well to a remote system. (Well; maybe slower. I’m getting only about 50MB/sec writes, so that should go over gigabit ethernet fine.)
I couldn’t go into more details or complexity if I tried.