Best way to use storj for a COMPLETE backup?

TrueNAS General
1

I just opened a Storj account and was surprised that the first bucket I created couldn’t be selected in TrueNAS GUI (it was there but greyed out). I created a new bucket and that worked fine. Anyone have any ideas or is this a bug?

The bigger issues is that replicating ix-applications isn’t an option in the backup, so I can’t do a full restore of my system.

The workaround I’ve been using is to have a script that uses zfs send to turn a snapshot of ix-applications into a normal file and I put that into the backup set that gets backed up.

Is that the recommended approach to get a “full backup”? Is there any other piece I’m missing for a full restore (beyond the system config file)??

2

With many apps and esp PVC storage depending on what Scale version you are on, you may find yourself bogging your system down a lot by snapshotting the applications pool. If you do decide to snapshot it, then, I would not suggest keeping very many. There are a lot of posts in the old forums about this issue.

For myself, I don’t do any snapshots of that pool. Nor do I even do a true full backup. I have 4-5 different backups, and, they all are for different purposes, some offsite for example as a home full backup when your house burns = lost backups. In the end, there is a large set of data I never backup, as it’s cache space, recordings, work space, and stuff I would live without if I lost it forever, of minor value like antenna recordings that I watch and get rid of anyway. There are other ways to watch those if I lose them. I organized my pools in this manner. This then allowed me to use different techniques for different data. My main offsite non ZFS backup uses Kopia and it’s just a few minutes a day. I also do replication to a VPS setup with zfs. I also use an external drive to replicate to each night of lesser important stuff and I rotate that to my safe deposit box. And others. All are automated except taking the backup to the safe deposit box so no time spent.

I have not heard many folks of trying to get a full backup, except those who run Scale in a vm.

Back to your point, for me, I simply have a note file for each application as to the parms specified. That’s all I would lose if the app pool was lost somehow. In < 30 minutes, I could type it all in again. Note I have a mirrored app pool, so disk failure wise, both have to die. Fire wise, theft wise, it’s gone of course. So, very unlikely I’d ever need the notes to recreate but possible.

What you do DO want for sure is to download here and there the system config, system settings → General → Download File. Keep this on some machine as that is basically all your configuration, except apps. And a UPS!

This is all going to change in electric eel. I suspect there won’t be a need to backup the dataset as configs will be stored in yaml files. We’ll see.

1 Like
3

I remember setting up NextCloud and it was non-trivial.

If I want a complete backup, the 3 “special” locations are: system, ix-applications, and app data.

The app data I’m pretty sure can be replicated with cloud sync.

I’ve been serializing both system and ix-applications but I suspect that I’m totally wasting my time backing up main/.system… Is that right?

So I think only serializing ix-applications would be useful in shortening the time to restore. Is that right?

4

For system, you simply download the config as I mentioned every so often, or, copy the location where the system takes it’s own config backup somewhere daily. There is no need for system backup per se. If you lose the boot disk, you download same version of Scale, install it on the new boot disk, reload that copied off config, and it’s exactly as before.

I don’t think most people spend much time on the application pool. Someone may. I’m not really sure about restoring it, not sure what would happen and if it would shorten anything as likely things would be running when you backed it up. Not sure I would restore that. You’d have to create datasets, tons of them, as the backup won’t do that except for replication copies, certainly not cloud syncs. Replication copies have their own challenges. My solution was as noted and it’s a one time investment. It’s all fixing to change too.

5

There’s no real reason to back up the applications themselves; they can be reinstalled out of the catalog–though it’d be nice to restore all their settings rather than having to manually redo them. The problem comes with restoring the data for those applications, at least some of which (and all of which, by default, with TrueCharts apps in 23.10 and prior) is also stored in the ix-applications dataset.

There’s a method to backup apps under 23.10 and earlier using Heavyscript–it uses backups of the apps and PVCs, and makes database dumps of anything using CNPG–but it doesn’t send them offsite, and I can’t really see a good way to send them offsite. There really isn’t any way to back them up under Dragonfish. How we’re four major releases into SCALE without iX having developed a consistent way to back up and restore applications is a mystery, but here we are.

But since 24.10 is going to throw the existing apps ecosystem out the window, everything’s likely to change by the end of this year. Whether iX are going to come up with a way to back that up (it shouldn’t be hard if it’s just a collection of docker-compose files) is anybody’s guess.

6

Great. I’ll stick with my zfs send for ix-applications and skip backing up system.

appdata gets backed up to storj.

So that covers all three datasets that would not normally be backed up via the storj rclone method.

Here’s the revised zfs send script I used

#!/bin/bash
# for enabling cloud backups of these three pools
# since the cannot be rcloned. So make them into regular files

# this script is called from the TrueNAS cron jobs
# it takes a while to run because the pools are large

cd /mnt/main/backups/truenas
rm ix-applications-*.gz system-*.gz   # remove old backups
datasets=( "main/ix-applications") # had "main/.system" before and appdata
output_names=("ix-applications")   # removed appdata and system
today=$(date +%Y-%m-%d)  # Get current date in YYYY-MM-DD format

list_length=${#datasets[@]}   # get length
for ((i=0; i<$list_length; i++)); do
  dataset=${datasets[$i]}
  output_name=${output_names[$i]}
  # Create snapshot with current date suffix
  snapshot_name="$dataset@${today}"
  zfs snapshot -r $snapshot_name

  # Send the created snapshot to a file
  zfs send -R "$snapshot_name" | gzip > "${output_name}-${today}.gz"
done