I have run settings at all 3 and none change the really long run times. I am currently running in fast storage mode, since that should be the fastest. (I have a very big NAS with very little activity at night and 2GB upload internet). When I first dug into this, I set up Storj as cloud sync and it worked a bit slower than backblaze, but by seconds. I am pretty sure the issue is related to restic and lots of small files. Here is my full job configuration:
Also, I found this is my digging, IIRC packsize is controlled by the transfer setting, but even fast storage does not allow for 60MiB, which should be ideal for Storj. :
Restic can become slow when handling many small files due to the overhead of checking each file’s metadata. Here are some suggestions to improve performance:
- Use --no-scan Option : This option disables the file scanning that restic performs to estimate the progress, which can reduce overhead but also means you won’t see a progress estimate.
- Increase Read Concurrency : You can increase the number of files read in parallel by setting the
RESTIC_READ_CONCURRENCYenvironment variable or using the--read-concurrencyoption in the backup command. This can speed up the backup process when files are stored on fast storage like NVMe disks. - Adjust Pack Size : For very large repositories or fast upload connections, increasing the pack size can reduce the number of files in the repository and improve upload performance. This can be done using the
--pack-sizeoption or theRESTIC_PACK_SIZEenvironment variable.