Any tips on using rsync jobs to distribute my LetsEncrypt ssl certs to my home lab systems after TrueNAS refreshes them? I’m not sure if pulling or pushing is a better strategy. I’m hoping to avoid running acme all over the lab.
Well, first, that I wouldn’t use TrueNAS as the source system for this purpose–the certs are stored in a database from which they can’t readily be extracted. I’d instead get the cert on some separate system (which could be a jail/sandbox on TrueNAS, or a small VM somewhere), and then using the ACME client on that system call a script to deploy them wherever they need to go.
How to do that? Well, depends on the target system. Here’s the script I use with a few examples:
#!/bin/sh
# Install cert to UPS
/root/apc-p15-tool install --keyfile /root/.acme.sh/domain/domain.key --certfile /root/.acme.sh/domain/fullchain.cer --apchost ups2.domain:22 --username certs --password (masked) --fingerprint 7jN807D55BVKdEduE3NMOKaij7v7GgB0c9Nm25thR/M
# Install cert to NAS
/root/deploy_freenas.py
# Deploy cert to PVE
scp /root/.acme.sh/domain/fullchain.cer root@cho-pve1:/tmp/fullchain.pem
scp /root/.acme.sh/domain/domain.key root@cho-pve1:/tmp/privkey.pem
ssh root@cho-pve1 pvenode cert set /tmp/fullchain.pem /tmp/privkey.pem --force --restart
ssh root@cho-pve1 rm -f /tmp/fullchain.pem
ssh root@cho-pve1 rm -f /tmp/privkey.pem
# Deploy cert to IP-KVMs
## pve-kvm
scp /root/.acme.sh/domain/fullchain.cer root@cho-pve1-kvm:/etc/ssl/certs/fullchain.pem
scp /root/.acme.sh/domain/domain.key root@cho-pve1-kvm:/etc/ssl/private/privkey.pem
ssh root@cho-pve1-kvm reboot
## opn-kvm
scp /root/.acme.sh/domain/fullchain.cer root@opn-kvm:/etc/ssl/certs/fullchain.pem
scp /root/.acme.sh/domain/domain.key root@opn-kvm:/etc/ssl/private/privkey.pem
ssh root@opn-kvm reboot
## nas-kvm
scp /root/.acme.sh/domain/fullchain.cer root@nas-kvm:/etc/ssl/certs/fullchain.pem
scp /root/.acme.sh/domain/domain.key root@nas-kvm:/etc/ssl/private/privkey.pem
ssh root@nas-kvm reboot
This is the deploy-freenas script used above:
2 Likes
I wholeheartedly support using the deploy scripts that @dan developed from inside a jail on TrueNAS or FreeNAS.
Whatever goes wrong inside the jail is limited to the jail. Worst case, nuke the jail from orbit and start over. Muck around in the root boot pool and you may be in serious trouble.
If these scripts are no longer needed, etc. then you can simply delete the jail and all associated software packages at the same time.
It’s much cleaner and the only downside of jails is the bit of space they consume.
In close systems like TrueNAS / Opnsense - its difficult to deploy certificates issued on another system so I opted for a hybrid approach.
-
I have my dev/generic server requesting wildcard certificate for my domain and stores it into Hashi Vault.
-
I have ansible scripting that deploys those certs to bunch of random hosts/VMs reading it from Hashi Vault
-
On systems like TrueNAS / opnsense I rely on their UIs and only request certificate for a specific host name : nas.example.com / router.example.com
This works pretty well for me. Systems that have advanced UI/cron built-in (TrueNAS) are perfectly capable to manage renewing themselves.
In Opnsense I use HAProxy quite extensively to act as reverse proxy for internal services - Its nice to be able to just select SSL certs from UI and know that opnsense acme client can handle the house-keeping.
Also, in past I used wild-card certs a lot, but lately I moved away towards per domain certs.
I can’t speak to OPNsense in this regard, but in TrueNAS it isn’t, and I already gave the link to the script that enables it. You’re right that TrueNAS is capable of handling the renewal itself–if you’re using Cloudflare or Route53 for your DNS–but its API is quite capable of handling deploying an external cert.
Thats against TrueNAS philosophy
I also hacked TrueNAS like enabled ksmbd, installed NIX and so on. But they don’t like it here.
Nonsense. Using the API to deploy a cert (which is what the script does) is exactly in line with TrueNAS philosophy. It isn’t “hacking” TrueNAS in any way.
1 Like
I appreciate the spirited discussion. Once I have my morning done with I will read through it. 
The script you recommended is a complete non-sense. Not only its spaghetti python code but also its hiding bugs by doing :requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)
Be that as it may, your claim that it’s “against TrueNAS philosophy” remains demonstrably (and demonstratedly) false. I make no claim as to its being truly Python-esque, but it works, and it’s entirely in line with TrueNAS’ philosophy.
3 Likes
Fair point yes. I forgot that you can upload it via REST API.
Correct. It worked wonders in terms of giving TrueNAS users a level of DNS/SSL provider independence that iXsystems, for whatever reason, denied its user base for a couple of years in TrueNAS GUI-land.
As of 24.10, OVH, Route 53 and Cloudflare are the only choices within the TrueNAS GUI universe. Under 13.3 it was only Route 53, IIRC.
@dan developed a script that worked for TrueNAS. Its popularity likely helped focus iXsystems on making more DNS authenticators available via the GUI. Credit where credit should come due.
If someone doesn’t like the underlying python code, instead of complaining about it, an improvement suggestion could have been filed. Or the code could have been forked. Etc.
I’m grateful that I was able to secure my NAS GUI followed by using his other scripts to also secure my network infrastructure. Thank you, @dan.
1 Like
…or I’ve happily accepted PRs against my code. It’s very hack-y at best, and I’d gladly accept improvements from those who know Python better than I (which would be lots of people).
It’s probably time for a rewrite in any event; I understand the REST API will be going away in favor of a Websockets design. At some point, the script should probably use this instead:
I don’t know if I’m going to be the person to make that change, though.
1 Like
Thanks, I will likely pursue an alternate course. The conversation has been interesting to say the least.
Three “easy” steps
- Install jailmaker,
- Make a jail
- Start experimenting until it works as expected.
Might take some time but will teach you a lot about SSH / SSL / and the vagaries of multiple operating systems and associated GUI services (ngix, lightppd, routerOS, etc.).
In a perfect world I would have a scheduled job that copies the certs from the db to a share when they get refreshed. That way the rest of the lab could have a scheduled rsync that grabs them and restarts nginx. That was my initial thought, because I did it that way on the Synology.
I wrote an interesting script that uses dans script.
It fetches the certs from the gateway (pfsense) that manages them via rsync… figures out new ones… and distributes them to various VMs, servers and dockers… and TrueNAS hosts too. Hupping as nececssary. Will look into posting it…
Sort of what it does, but it’s cleverer about when it needs to restart
If the distribution can happen as part of the acme.sh renewal process then you can automate it fully.
Works perfectly with the scripts I have based on @dan’s routerOS script.
For my piholes (as of version 6) I execute the acme.sh script, then via the reloadcmd option initiate a script to distribute.
The TrueNAS jail script runs the Pem concatenator, transfers the PEM file via SCP to each pihole, then executes a script on each Pihole to install the ssl cert and restart the local HTTP service.
This keeps all the supa-sekrit DNS keys inside the TrueNAS jail and centralized. I’m slowly branching out to secure ALL network equipment. The user on each Pinole that executes the script can only run that one script as Sudo via the sudoers file.
I deployed acme.sh in docker and simply used it to get and renew certs, even wildcard since it works with dns providers. That can then deploy to places as needed.
1 Like
Exactly. It limits the number of places where your API keys can become exposed. By doing it all centrally in a trusted location, there is only one OS, etc. to maintain and be vigilant about vs. having multiple environments with sensitive data in them.
1 Like