I started to send some backups from truenas scale to Storj with the integration from truenas itself. All is fine until I noticed that one of the folders I want to copy to storj has files (which are copied successfully) but the subfolders are not.
Please someone knows if there is a way to recursively push all the files and subfolders of the specified path?
@Captain_Morgan Sorry, I have to add my configuration in the signature sooner or later Anyway I am on Truenas scale on EE RC2. I am using Cloud Sync at the moment (even though, I did not think about using Truecloud and it is also confusing on whether to use one or the other?). For the moment I have setup 2 cloud sync tasks and one is doing things correctly while the other is just copying the files in the first level of directory but nothing else (even though it has subfolders and files into these folders too)
By default Cloud Sync is going to traverse directories. However do you have nested datasets in there? It may not be crossing mount points, which is what nested datasets will present as.
Hi @kris , actually the structure looks like this:
MyPool
appdata
vaultwarden
vaultwarden-postgres
where appdata is the parent dataset and vaultwarden and vaultwarden-postgres are children datasets. I basically setup a cloud sync for each one, so vaultwarden has its own task and vaultwarden-postgres has the other one, but while vaultwarden-postgres syncs files and folders correctly (with subfolder etc) vaultwarden directory does not.
I’ve made 2 different cloudsync tasks for each of the datasets, so it means vaultwarden has 1 task and vaultwarden-postgres has another.
Same settings between the 2 were applied (apart of the cronjob definition to avoid uploading all together)
Regarding the rights on these datasets, vaultwarden has root as userId while vaultwarden-postgres has user 70, both group and other have 0 rights (because postgres:alpine uses this user (and I can’t modify this settings).
To be honest I am surprised that truenas doesn’t have any problem with the dataset vaultwarden-postgres (because of the rights look like) while vaultwarden which has user as root and group as apps (both with RWE rights) doesn’t push all the subfolders as expected
Are you continuing to use “root” rather than “admin” or something else?
From our AI bot:
It is recommended to use the admin user as the primary data user in TrueNAS SCALE. The root user password is disabled by default to comply with FIPS security hardening standards, and it is advised to create and use an admin user account for system administration tasks. This approach enhances security by limiting the use of the root account, which should only be activated for limited uses and then disabled again.
But same behaviour as before applies. I delete the folders from Storj, and started both tasks. Vaultwarden-postgres has pushed all the files while vaultwarden sent 1 file.
Now I understood what happens!!! Basically the cloudsync task just pushes folders that contain at least 1 file otherwise these folders are not even pushed! Isn’t this a little odd, or at least confusing? To be honest, I would prefer to be a 1:1 copy of the structure of folder I want to push. There are cases (especially with apps) where maybe the application looks for the presence of a specific directory and just crashes if it doesn’t find it (even though it is empty)
Great - glad you got it sorted. Rclone’s handling of empty directories is a little eccentric due to the fact it works on systems like s3 which can’t have empty directories…
If someone using Storj as object storage can confirm this behaviour I will close this topic