I set up some replication tasks, to replicate datasets from one TrueNAS (‘source’) to another (‘backup’).
I ran them once, and all files in the source datasets were copied to backup datasets (previously created, on the second TrueNAS), as I expected. No snapshots were involved.
Some time later, I ran one of the replication tasks again, as ‘run once’, but none of the new files in the ‘source’ dataset appeared on the backup TrueNAS. It was, though, reported as having been a success.
I suspect that I’m not understanding something(s), perhaps how replication and snapshots relate/interact; I had assumed / hoped that it would be fairly straightforward but maybe it’s more complicated and nuanced (as seems to be the case with everything TrueNAS). I’ve scanned information here and there, but so far I haven’t understood much.
So, what’s going on? What does replication actually / really do; what am I missing?
This is not possible: Replication is based on snapshots.
No snapshot = no replication.
It’s possible that TrueNAS automatically took a snapshot to complete the task, and you failed to notice. Sending updates will require further snapshots, preferably as a scheduled snapshot task.
Carefully review your replication task, and provide a complete and detailled description of what you have set up, with screenshots.
Replication only replicates snapshots, not files. It works on the ZFS block level. So you need to create a new snapshot (which will contain all files present at the time you do that) and replicate the snapshot.
To work on the file level there’s rsync, but ZFS replication is faster and more reliable.
I’ve just looked, and there is a pre-existing snapshot for that dataset. I assume that, if I looked, I would find pre-existing snapshots for all of the other datasets that I was able to replicate.
I did actually create a (new) snapshot for the (now larger) dataset referred to in my first post, to see if that would make any difference, and re-ran (again) the replication task. But still no change occurred at the ‘backup’ TN.
Also, while I do see folders and files within the dataset at the ‘backup’ TN, there are no snapshots on that TN. If ‘only snapshots’ were replicated from the source TN, then they appear to have become regular files and folders at the backup TN. Is this what I should expect?
I will do as you suggest; review the replication task etc.
There are, but you’re not looking in the right place. You have to either specifically mount the snapshot or browse the hidden snapshot directory.
A snapshot is not a dataset or a folder, it is the state of a dataset at a point in time. Think browsing through TimeMachine: Each stop corresponds to an APFS snapshot. (TrueNAS does not have a fancy Sci-Fi-themed GUI to expose ZFS snapshots.)
I did find some snapshots on the ‘backup’ TN. Not under top-level menu ‘Data Protection’, but under ‘Datasets’/Data Protection/View Snapshots.
I understand the sense / idea of this, but I can’t really conceive of how it actually does it.
Presumably, the snapshot allows full reconstruction of the data, from just that one snapshot? Or are previous snapshots also required? I think I need to read some more.
Each snapshot contains all data present at the time the snapshot was taken.
But if you have more than one snapshot all data that is shared between them is kept only once. Hence the space savings and the “differential” replication etc.
Actually they do make sense if you are experienced with Unix and the concept of hard links.
A hard link is a second (third, fourth, …) directory entry with a name pointing to a certain inode. The inode contains all metadata of a file except the name. You can have 10 hard links pointing to the same inode but that inode and the file data is stored only once. That’s how all traditional filesystems like UFS or Ext4 work.
Once a hard link is created it is not possible to tell which name was the “original” one and which one the “new” one. You simply have N names for the same file. The file data is removed when the last name is removed.
Snapshots work like this, only on the ZFS block level.
Any block can be in N snapshots and will only be deleted when the last snapshot containing it is. The “stickers” in @winnielinnie’s article are the snapshots. The “boxes” are the blocks.
The file/directory/etc. view is an emulation of a traditional Unix file system on top of ZFS. Internally (checksums, snapshots, clones, replication) ZFS only cares about blocks.
So each snapshot contains all data at the time it was created but each different ZFS block is stored only once.
