One of the dozen or so remaining core users here - TrueNAS-13.0-U6.7
Questions:
- What happens if rclone fails/is interrupted in the middle of uploading a large file to object storage?
- Does it know where it left off? Does it resume from where it left off?
- Does it go “dumb mode” and just re-upload the entire file if there’s a difference detected (size, checksum, timestamp differences between src and dst)?
- Is rclone configurable to do something like the above if it doesn’t by default?
Context:
Over a year ago I took a big backup of all my data (multi-TB) and chucked it in a cloud provider. I figure it’s time to take another big backup and am considering a new provider (AWS instead of Azure).
I don’t have a strong Internet uplink, so doing these irregular bulk backups takes planning and forethought. The last time I did bulk backups of this kind it took months - not exaggerating.
The last time I did this, my backup software split the entire backup into smaller files of about 64MB in size so a complete backup of 1TB was divided nicely so that if rclone failed (or I manually interrupted the task) it would only lose a couple minutes of progress if it had to overwrite one of these files. As I write this paragraph, I’m pretty sure rclone takes the “dumb” approach and just overwrites the destination file.
I’ve since changed how my backups get taken for other reasons and now I won’t have the above file-breaking to mitigate any network interrupts.
I could just do the file split myself using something like 7-zip or split but if I don’t have to - all the better and less steps both for the backup and restore processes, less things to go wrong (KISS).
Pre-empting the choir:
And yes, the plan is I would then do the upgrade to SCALE shortly after I have these backups taken and the peace of mind that afford me. 