Snapshots, zfs send/recv, some clarification


I would like to have more insight on understanding the snapshots on TrueNAS.

I configured them on my datasets and I use them for backup:

  • in replication task on a backup server
  • manually twice a year on cold backup
  • roll back when I did some mistakes! :smiley:

And there are some subtleties that elude me… because well, I don’t necessarily need them. :slight_smile:
But I would like to understand it a bit hence my message.

I’m doing a manual backup from a snapshot of a dataset from a server (A) to another server (B, on the LAN).
So I use zfs send/recv through nc from a terminal on each of the servers.

From the source server (A) I have existing snapshots:

root@ServerA[~]# zfs list -r -t snapshot -o name,creation Tank/Document
Tank/Document@auto-20240622.0030-12m     Sat Jun 22  0:30 2024
Tank/Document@auto-20240622.0130-12m     Sat Jun 22  1:30 2024
Tank/Document@auto-20240623.0030-12m     Sun Jun 23  0:30 2024
Tank/Document@auto-20240623.0130-12m     Sun Jun 23  1:30 2024
Tank/Document@auto-20240624.0030-12m     Mon Jun 24  0:30 2024
Tank/Document@auto-20240624.0130-12m     Mon Jun 24  1:30 2024

Note: yeah, don’t ask me about the snapshots at 00:30 and 01:30 on the same day, I just noticed, I have to look into that! :smiley:

So I use the following on server A:
root@ServerA[~]# zfs send Tank/Document@auto-20240622.0130-12m | nc -l 3333

And the following on the target server (server B):
root@ServerB[~]# nc 3333 | zfs recv Backup/Document

And that works perfectly…

It took me several days to finished my backup and I thought, in the meantime I have new snapshots from the Document dataset, let’s send them over…
So I thought I’d sent the latest snapshot:
root@ServerA[~]# zfs send Tank/Document@auto-20240624.0130-12m | nc -l 3333

And I expected then the server B to show two snapshots for this dataset but instead I got an error:

root@ServerB[~]# nc 3333 | zfs recv Backup/Document
cannot receive new filesystem stream: destination 'Backup/Document' exists
must specify -F to overwrite it

So I could use -F indeed but then, as I understand it, it will completely overwrite the existing snapshot (which is fine) but I was wondering, can I send this snapshot incrementally?
I thought I’d use -I in zfs send to send the missing snapshots in between but it didn’t work as expected, I got:

Server A:
root@ServerA[~]# zfs send -I Tank/Document@auto-20240622.0130-12m Tank/Document@auto-20240624.0130-12m | nc -l 3333

Server B:

root@ServerB[~]# nc 3333 | zfs recv Backup/Document
cannot receive incremental stream: destination Backup/Document has been modified
since most recent snapshot

On server B, the data in the Document dataset has not been changed (beside of the zfs recv), so I’m surprised.

Am I missing something?
Maybe a parameter? Or my understanding of how to use the snapshots, or the zfs send/recv?
Or maybe the way I do it is not compatible with sending incremental snapshots (but then, why?)?

Thanks for your insights.

Some updates after some tinkering:

I ended up using the -F flag on zfs recv.
And it didn’t rewrite the complete snapshot as I was afraid it would (note: as mentioned, no problem with doing so, I just wanted to try to avoid it).

I used it in combination with the incremental option:

Server A:
root@ServerA[~]# zfs send -i Tank/Document@auto-20240622.0130-12m Tank/Document@auto-20240629.0130-12m | nc -l 3333

Server B:
root@ServerB[~]# nc 3333 | zfs recv -F Backup/Document

I used the -i instead of -I (which was not intentional! :open_mouth:) and I think I understand now the difference which can be summarize by:
-I @a fs@d is similar to -i @a fs@b; -i @b fs@c; -i @c fs@d

So now I get:

root@ServerB[~]# zfs list -r -t snapshot -o name,creation Backup
NAME                                              CREATION
Backup/Document@auto-20240622.0130-12m        Fri Jun 22  1:30 2024
Backup/Document@auto-20240629.0130-12m        Sat Jun 29  1:30 2024

and since I used -i I don’t have all the snapshots in between, but that’s also fine.
Maybe I’ll try with -I in 2 days, just for the sake of it.

Note: is there a way to set a “solved” flag for the thread or so, just out of curiosity (couldn’t find it)?
Ah now I see that you can promote a post to “solution”… hmmmm ok, not exactly the same I’d say.