Support for ZFS "Bookmarks"

Feature Requests
1

Problem/Justification
Replication tasks depend on the presence of a common snapshot on the source and destination. If replication is interrupted longer than the snapshot retention window on the source, the base snapshot may be deleted… so to restore the backup strem a replication from scratch is needed, with all the negative impact.

ZFS bookmarks instead allow incremental replication to continue even after the original snapshot has been destroyed, with a cost in pool terms space really negligible

Impact

  • reduce the need of a replication from scratch, that can be a nightmare when offsite NAS are involved
  • costs nothing in terms of pool space
  • ideally, people not interested in this capability should be able to not select this feature

User Story
I became aware of this capability purely by chance, thanks to the resource written by @winnielinnie ©

I have experienced the feel to miss the snapshot retention window for various reasons, which forced me to restart replication from scratch. This resulted in significant time loss and, in one case, required physically relocating the destination system onto the same network as the source to complete a full re-replication smoothly.

As a TN user, I would benefit from built-in bookmark support in replication tasks to prevent these situations.

1 Like
ZFS "Bookmarks", and why you don't use them (but should!)
Best way to backup a trueNAS?
2

My “vote” is a thumbs up.

There could be 2 parts to this feature.

  1. Manual bookmarks. When manually creating or viewing snapshots, there can be an option for “Bookmark”. If the box is checked, a bookmark of the same name will be created. If the box is unchecked, the bookmark will be destroyed. It should have a tooltip near the “Bookmark” checkbox to explain what a ZFS bookmark is to the user. (It would be useful to have a batch operation as well if multiple snapshots are selected.)
  2. Automatic for Replication Tasks. A Replication Task can have an option to “Create bookmark of latest source snapshot after successful replication”.

Using the bookmark is more complicated.

For now, it could be the user’s responsibility to manually use a bookmark in the command-line to do an incremental replication that would otherwise require starting all over, after which they can return to normal replications in the GUI. (I have successfully done this myself in a test scenario. Thankfully, so far, I never needed to use a fallback for my real backups.)

It’s possible for this to be automated too. There could be a fallback option to use a bookmark. “If incremental replication is not possible, allow checking for and using a bookmark to attempt an alternative way of doing an incremental replication?” This option could be placed under the “From Scratch” option in the GUI. For safety’s sake, enabling the “Bookmark Fallback” option should disable and grey out the “From Scratch” option.

Syncoid, a command-line tool, allows this with the --create-bookmark option:

AFAIK, this still leaves it up to the user to manually use a bookmark for an attempted fallback incremental replication, if required. (I don’t use Syncoid, so I’m not sure if they’ve improved on this feature in their latest version.)