TrueCloud Backup (EE)

TrueNAS General
1

I aborted the task but that left some kind of lock hanging around. Whenever I try to run the task again, I get a “locked” error message but I can’t figure out where I find the “unlock” command it references.

errorFAILED

[EFAULT] repo already locked, waiting up to 0s for the lock
unable to create lock in backend: repository is already locked by PID 11571 on truenas-ee by root (UID 0, GID 0)
lock was created at 2024-09-07 02:59:01 (8h33m53.969079484s ago) storage ID e21b4a69
the unlock command can be used to remove stale locks

2

Does the lock persist after a reboot?
If so, feel free to report a bug.

Information should either be in error message or documentation.

3

It did persist after a reboot but I eventually figured out that the lock was on the Storj side. I’ll file a bug as 1) an abort on TrueNAS side shouldn’t leave the Storj repository locked, and 2) if possible, any error message should distinguish if it is being generated by TrueNAS or Storj.

I found a posting that was somewhat related as it was about Storj, restic, and stale locks which led me to “restic unlock” so I’m guessing that is where the mysterious unlock command lives and that restic is being used instead of rclone (which I think is used by the regular Cloud Backup in Scale).

4

I ended up resintalling EE from scratch as part of the debugging process where I discovered that the resulting install didn’t work as it didn’t create the admin user. I ended up having to install 24.04 and upgrading which worked fine.

Another bug report I guess.

5

Fresh installs of EE create a truenas_admin user by default, instead of admin. This is covered in the release notes

1 Like
6

Thanks. So many changes, so little time :joy:.

It looks like TrueCloud is using restic instead of rclone. Do you know if that’s correct?

Do I need to reproduce an issue in the nightly build before reporting it?

Thanks, Bill

7

Did you get an answer or can you point me at a Jira ticket?
I have the same situation after aborting a test TrueCloud Backup task, now I can’t run backups.

8

Never filed a ticket for various reasons but you can use the Storj GUI to delete the lock and that fixes the problem.

9

Oh!
Ok, I’ll see if I can spot the lock files.
Awesome.

Any chance you can tell me what I’m looking for? If Google tells me first I will let you know.

Thank-you.

10

I don’t remember exactly but I just started browsing the data structure on Storj using the Storj and a file that jumped out at me so “lock” must have been in the name. But I was working with rest data that I was willing to blow-up so I didn’t have production data worries

1 Like
11

I took a quick look and didn’t see it so I suspect it only exists while a backup is running or the lock file wasn’t deleted.

12

I found the file in the lock directory and having deleted it, was able to sync.
Thank-you very much.

Now debating the trade-off between being able to download any file from storj if we backup using old style cloud backups, and the efficiency of syncing deltas.

13

I have large files with small changes so I save a huge amount of time with Storj. Restores seem to work fine using the TrueNAS GUI (but the GUI is infuriatingly small and fidgety) and the Storj GUI is useless as it doesn’t present a view of the file system.

I have to keep reminding myself that this is the last line of defense so I’m likely never to actually restore individual files from it

1 Like
14

For those following this issue, it has recently been addressed :slight_smile:
https://ixsystems.atlassian.net/browse/NAS-132612

2 Likes
15

~Can someone tell me how to manually resolve this until the fix gets released? I’m assuming there’s a way to call restic unlock from the command line and specify the Storj URL?~

Nevermind. I didn’t see the post that you can just delete the lock file from Storj and it resolves the issue.