I’ve recently set up BackupPC on my TrueNAS SCALE server, and it’s a bit strange. It seems to generate SSH keys in /.ssh but SSH itself doesn’t look there, so i kept getting errors trying to do an rsync over ssh backup (specifically because it wasnt able to find the remote fingerprint in known_hosts).
I copied the files in /.ssh to /root/.ssh and things started working. But I don’t think that works across container/pod restarts. I also notice that all the various storages defaulted to PVC, so I can’t readily look at the files directly on TrueNAS.
I mention that because I have a hard drive or two from a former system i ran BackupPC on, and i would like to migrate that over so that it all lives in the new BackupPC setup, both active hosts i will be setting up now, and the archived history of former hosts that have been terminated. I assume that i need to change the storage from PVC to HostPath to do so. I see a warning about changing away from PVC; I’m not sure if that’s actually a problem or if i should just delete the app and set it up again.
Yeah, an application can’t change the ssh keys of the system… it doesn’t have admin privileges for security reasons. Do new ssh keys need to be generated each time it reboots?
So I figured out the actual problem once I came across heavyscript. Trying to troubleshoot from the terminal built into the code server container was actually counterproductive, as it’s not set up the same, and while it can mount the /.ssh directory inside of it, that’s not the right way to go about digging into it.
Anyways, I removed the /.ssh/config file and made sure the permissions on the files in /.ssh were sane, and everything was good. However, after I updated things this morning, it immediately started throwing errors again. Connecting in with heavyscript, I found that everything under /.ssh had permissions set to 0660. Once I corrected them to 0600, everything worked properly. I had just corrected this and run full backups prior to the update, so I’m certain the problem was either related to the update or the container being restarted.
I’m not sure if this is the Docker image itself changing the permissions on those files, or something within the TrueNAS/TrueCharts ecosystem doing so, but those permissions on those files just cause ssh to throw a fit and refuse to load the keys based on their permissions being too open.
I have not, I haven’t had a chance to try doing a custom app yet. I did just migrate some old BackupPC data from a previous system onto TrueNAS yesterday, so I was planning to set up a second instance of it to be able to look at the old data and retrieve contents from it as needed. I’ll give that image a whirl for that setup and see how things go… once it’s up and running, testing one of the hosts I’m backing up in the main instance should be pretty straightforward.
I did confirm that it was the start/stop of the container resetting the permissions rather than the upgrade, btw, as I had to reboot a few times while moving the old disks in and out of the system to move the data over to my primary TrueNAS pool. Each time the container started, it reset the /.ssh permissions back to group read/write. I probably need to discuss that with the TrueCharts folks.
