The subject of backing up a TrueNAS server to external disks comes up often enough that I wrote a Resource in the old forum on the subject:
However, today people have such large pools that the need is for multiple external disks. Yet, if those external disks are in a ZFS Striped pool, there is potential to loose the entire backup pool on loss of one disk.
At least 20 years ago I thought a plain file backup scheme using multiple media would be a good idea. At that time I had tape drives and DVD-R/RW burners, so those would have been my media of choice, then. Having each media separate would help preserve some of the files in the backup set in case of a single media loss.
Today, having very large disks and the ability to safely transport them with hard shell cases, (like Seahorse), makes them a better choice. Yet keeping the media in separate ZFS pools makes the backup set more robust and prevents total loss on single media failures.
Having thought about it off and on for decades, here is what I think meets most needs:
- Files must fit on a single backup drive, no fragmentation.
- The path to the files, even crossing Dataset boundries, is preserved in the backup drives.
- The first backup drive, (and probably all), should have the source pool(s) configuration and entire dataset attributes saved. This is so the source pool and datasets can be re-created.
- The backup scheme would sort the source files based on size, and backup the largest first, then continue with the next largest to fit. And so on until the first media is full, (skipping ZFS’ normal 80% suggest limit and going up to 98% full).
Here are the problems I foresee:
- A list of files should probably be on all media, and later media should list what was stored on earlier media.
- Block cloned files, snapshots, bookmarks, clones, DeDupped files and such are simply out of scope. That is where full ZFS Replication shines.
- Some datasets may have different attributes that would be better to preserve in the backup drives. But, that would involve making datasets on the backup drives… potentially have every backup drive need such.
- This needs to be a one disk at a time backup. Most people won’t or can’t have all the backup disks installed at the same time. So whatever scheme is used, it is one backup drive at a time. Then, when full, export that pool and start a new one.
- Incremental backups can be problematic because the changes might scattered across all the backup set’s drives. Thus, this is really only a full backup method.
So, any of you want such a backup method?
Any additional problems?
Features you believe are critical?
Remember, ZFS Replication does not really work across multiple, independent external media. Even if you think it might, it does not as loss of 1 drive would generally cause all later drive’s data to be lost.
So, loss of the 1st drive out of 10, means the backup set is gone. But, lost of the 10th drive out of 10, means only those files on the 10th drive. Too screwy to be useful.