RE 2.: A guide is at HowTo: Using ZFS Autobackup with Scale | TrueNAS Community , but python3 -m venv seems not available in current SCALE versions 24/25? I think, it could make sense to create a python -m zipapp self-executable zfs-autobackup.pyz to pack the app with its dependencies, so there is no need for venv.
@sfatula As you created a couple posts about this tool: Still satisfied with it? (Hopefully it’s OK to tag you here)
Post-note: Built-in TrueNAS periodic snaphot and replication task is not flexible enough for my case. Hence I’d be interested in experiences specifically with zfs-autobackup on a TrueNAS SCALE 24/25 system, which in my view can substitute all TrueNAS tasks best, with no need to be installed.
Very very satisfied, much better than built in replication for a variety of reasons. Sorry, am in Bali so not a lot of time to add more notes. Been using it at least a year and never had to start over, handles any communication errors, seamless picking of datasets to backup, etc.
I have no experience with the following, and cannot vouch for it. On the surface it looks clean and usable. It offers a stub script to defer actual operation inside Docker. (Well, it uses Podman. But that’s easily changed.)
Or you could get fancyfancy and try something like this.
This creates a Python self-executable, single file zfs_autobackup.pyz, which can be copied over to TrueNAS machine. (zfs-autobackup does not have required third party runtime dependencies.)
Then on TrueNAS just execute this file:
# ./zfs_autobackup.pyz
This properly invoked above command on my test system and displayed help message. Now I need to figure out, how and what command parameters to use.
Cool! Does this bundle all relevant dependencies like the shiv build did, or does it only gather its own, releasing any external modules into the user’s home environment?
Above command should contain all zfs-autobackup Python modules in the file zfs_autobackup.pyz.zfs-autobackup itself does not have any required external dependencies: argparse is already included in standard library for half recent Python versions and colorama is only for colored CLI output - if not installed, there is a fallback to normal output.
If you still need this external dependency, you should be able to do something like:
@jct Yeah kinda. As said, you can add dependencies, if needed:
git clone https://github.com/psy0rz/zfs_autobackup
# download deps via venv and place them at top-level of later .pyz archive root dir
# (venv can be deleted after zfs_autobackup_all.pyz creation)
python3 -m venv venv
. venv/bin/activate
python -m pip install -r zfs_autobackup/requirements.txt --target zfs_autobackup/
deactivate
# create self-executable incl. deps
# (zfs_autobackup dir can be deleted after zfs_autobackup_all.pyz creation)
python3 -m zipapp zfs_autobackup/\
-o zfs_autobackup_all.pyz\
-p "/usr/bin/env python3"\
-m 'zfs_autobackup.ZfsAutobackup:cli'
# copy over zfs_autobackup_all.pyz to TrueNAS, then
./zfs_autobackup_all.pyz --help # should work
This should include zfs-autobackup and all its dependencies in a single file zfs_autobackup_all.pyz, which can be executed on TrueNAS system.
Update: I guess questions about --clear-mountpoint are wrong in the first place:
If using an external tool for snapshots and replication, we should strive to be as close as possible to ZFS properties set by TrueNAS, when initiating a replication task and creating target datasets. TrueNAS uses canmount=on, so that’s gonna be the proper setting.
Probably it is best to let TrueNAS create all datasets itself (by GUI replication task and run it once), and then use zfs-autobackup on these existent datasets. Everything is working for me, when doing this.
To give a review, zfs-autobackup is quite awesome.
I switched some of my dataset backups to it for flexibility - no problems so far, tool makes a stable and well-thought-out impression.
If you intend to do replication with zfs-autobackup, I strongly recommend to create the replication dataset via TrueNAS GUI/task, so appliance is aware of this dataset and initiates all expected ZFS properties. Afterwards snapshots can be created, replicated and cleaned up as needed.
Here is an example script template for snapshot creation on main + clean up + replication to tank:
zfs_autobackup --verbose --debug --allow-empty \
--exclude-received --rollback --zfs-compressed \
--keep-source=10,1d1w,1w1m,1m1y --keep-target=10,1d1w,1w1m,1m1y \
--strip-path=1 --snapshot-format={}-%Y-%m-%d_%H-%M --no-progress \
--clear-refreservation --destroy-missing=30d main tank
.