I’m looking for a relative simple backup solution, and “backup” might be a bit of a stretch.
My current setup is 3 pools, Media, Data and Apps. My Media pool has a Pictures folder where all family pictures, for the current year, are uploaded to from our mobiles via Photosync. All I’m looking to do is copy the newly added pictures from a specific folder (e.g. media/pictures/2024) to a corresponding folder in the Data pool (e.g. data/pictures/2024). Ideally I’d like to do this on a daily or weekly basis.
Any suggestions for the best and simplest way to do this?
This is probably what you should look at for simple:
Or you could create a rsync task and schedule via cron. I was lazy and used some AI assistance which should provide some idea on an alternative solution.
Considering the following:
My current setup is 3 pools, Media, Data and Apps. My Media pool has a Pictures folder where all family pictures, for the current year, are uploaded to from our mobiles via Photosync. All I’m looking to do is copy the newly added pictures from a specific folder (e.g. media/pictures/2024) to a corresponding folder in the Data pool (e.g. data/pictures/2024). Ideally I’d like to do this on a daily or weekly basis.
Please assist in creating a rsync one-liner for a cron scheduling. The task should run daily on a truenas core system.
This is NOT validated…
To set up a cron job that uses rsync to copy newly added pictures from your Media pool to your Data pool on a TrueNAS Core system, you can follow these steps:
Open the TrueNAS web interface and navigate to the Shell or use SSH to access the command line.
Test the rsync command to ensure it works as expected. You can run the following command in the shell to copy files from the Media pool to the Data pool:
-a: Archive mode; it preserves permissions, timestamps, symbolic links, etc.
-v: Verbose; it provides detailed output of the transfer process.
--ignore-existing: This option ensures that only new files are copied, skipping files that already exist in the destination.
Create a cron job to automate this process. You can edit the crontab for the user that has the necessary permissions to run rsync. To edit the crontab, run:
crontab -e
Add the following line to schedule the rsync command to run daily at a specific time (e.g., 2 AM):
0 2 * * *: This cron expression means the job will run every day at 2:00 AM.
>> /var/log/rsync_pictures.log 2>&1: This part redirects both standard output and standard error to a log file for monitoring.
Save and exit the crontab editor.
Now, the rsync command will run daily at 2 AM, copying any new pictures from the Media pool to the Data pool. You can check the log file (/var/log/rsync_pictures.log) to see the output and any potential errors. Adjust the time in the cron expression as needed for your schedule.
@ChrisRJ why do you say that? The intent is to have a back up of the pictures in case there’s a catastrophic failure of the Media pool.
One thing I didn’t mention is that quarterly-ish I also do a manual copy of the Data pool to an external USB(s) that is stored off-site. Following the 3-2-1 backup rule.
Sorry, I had overlooked that we are talking about 2 separate pools.
The one thing you should be aware about then is that there various scenarios with risk that both pools die together. Think power surge, disk controller, PSU failure.
@ChrisRJ the data pool is in 3x2TB drives in a mirror and the Media pool is 6X16tb in RaidZ1.
As for the other scenarios, I’ve tried to protect as best I can against those as well. I have redundant PSU’s, UPS and surge protection in the server rack so I think I should be ok for most scenarios.
