Currently trying to setup a super simply replication task to replicate dataset to a single drive with USB dock.
The thing is that I have many custom snapshots created (like weekly, monthly and yearly with different name patterns). I found that when I setup replication task it won’t create all snapshot replication. (like it setup by the name pattern and only allow 1, if I add the weekly one then monthly and yearly will not be there).
Are we talking use of zfs send here? Is it the initial base replication, or the incremental replication to update it, that is not sending all snapshots for you?
I’m using this routinely and haven’t had any problems with which snapshots go. Are you doing this, or some other kind of replication?
An incremental update, which I run frequently and hence have handy in a log file, looks like this:
(Each backup starts with creating its own snapshot, that’s the more recent snapshot used there. The argument to -I is the most recent snapshot already on this backup, which is the starting point for the replication.)
Obviously, replace rpool with your source pool name. And then replace newpool with your destination pool name.
This type of copy only works once. Trying to perform incremental ZFS Sends afterwards likely won’t work. Or work irregularly.
Be careful of USB enclosures. I have had poor luck with ones that have no active cooling, (no fan). Meaning my external backup disk got far too hot. A full pool copy will likely cause a LOT of heat! So I have had to pivot and look to re-use older enclosures that have good cooling, (but lack newer USB features).
Can you just confirm for me, that when you do this, that the last snapshot in the series (@copy in this case) actually becomes the file system as well.
The specified snapshot from the source, whether a full or incremental replication, will always be the live filesystem on the destination. The catch is that you cannot use the destination filesystem at all, unless it is read-only. The moment you start using it like a normal filesystem, you will break the ability to do incremental replications from the source dataset.
That method creates a 100% identical clone of the source.
In my case, the zfs_clone script re-creates the pool each and every time. That is intended to allow creation of an identical copy of my existing OS, fully bootable, on alternate media, (USB, SDXC card, etc…), for recovery purposes.
To be clear, in my case, their is more to the zfs_clone script. It modifies the /etc/fstab file for the new file systems. Also copies over /boot and /boot/efi, plus a few other things. The script should probably be named zfs_clone_os. But, the code fragment I listed should do the full clone of a ZFS pool.
I remember using the GUI replication tools to go NAS to NAS, long ago (I think FreeNAS, before TrueNAS Core, before TrueNAS Scale). But I believe that still exists.
Understanding how zfs send / zfs receive work makes it much easier to know what to tell the GUI, to understand what’s really going on. The ability to do incremental replications is for me the main point of the whole exercise – you have to do one full replication to get started, which takes a LONG time; but after that, you can do incremental updates to it, and the result on the recipient system is just like the original. Unlike old-style backups to tape or whatever, you don’t need to do a new full backup every week.
These days, I’m replicating from the main pool on my NAS, onto bare drives plugged into USB drive docks. Works surprisingly well (if not as fast as an actual SATA connect would run). But that waiting time I just spend doing other things, the system isn’t unusable or anything.
