403 Forbidden when deploying certificates with deploy_freenas or acme.sh deploy hook

TrueNAS General
1

Hello!

I’m using TrueNAS Scale 24.10.0.2, updated from 24.04 when I started troubleshoot this problem. I’m trying to deploy certificates generated by acme.sh using DNS validation with self-hosted DNS and CA servers. Issuing the certificate was the easier part and I will omit just to be brief: my problem is how to deploy the certificate into TrueNAS. I tried the acme.sh built-in hook and also the deploy-freenas created by the user dan of this forum, and both provided more or less the same error:

With acme.sh truenas deploy hook:

[Thu Nov 28 21:03:53 -03 2024] Testing Connection TrueNAS
[Thu Nov 28 21:03:53 -03 2024] TrueNAS system state: <html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx</center>
</body>
./html>
[Thu Nov 28 21:03:53 -03 2024] Getting TrueNAS version
[Thu Nov 28 21:03:53 -03 2024] Detected TrueNAS system os: unknown
[Thu Nov 28 21:03:53 -03 2024] Detected TrueNAS system version: unknown
[Thu Nov 28 21:03:53 -03 2024] Getting current active certificate from TrueNAS
[Thu Nov 28 21:03:53 -03 2024] Uploading new certificate to TrueNAS
[Thu Nov 28 21:03:53 -03 2024] Fetching list of installed certificates
[Thu Nov 28 21:03:53 -03 2024] Current activate certificate ID:
[Thu Nov 28 21:03:53 -03 2024] Checking if WebDAV certificate is the same as the TrueNAS web UI
[Thu Nov 28 21:03:53 -03 2024] Updating the WebDAV certificate
/mnt/Storage/user/.acme.sh/deploy/truenas.sh: line 147: [: : integer expression expected
[Thu Nov 28 21:03:53 -03 2024] Unable to set WebDAV certificate
[Thu Nov 28 21:03:53 -03 2024] Error deploying for domain: truenas.domain
[Thu Nov 28 21:03:53 -03 2024] Error encountered while deploying.

With deploy-freenas:

[Thu Nov 28 22:28:35 -03 2024] Installing key to: /mnt/Storage/user/.acme.sh/truenas.key
[Thu Nov 28 22:28:35 -03 2024] Installing full chain to: /mnt/Storage/user/.acme.sh/truenas.crt
[Thu Nov 28 22:28:35 -03 2024] Running reload cmd: /mnt/Storage/user/deploy-freenas/deploy_freenas.py
Error importing certificate!
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>

[Thu Nov 28 22:28:36 -03 2024] Reload error for: truenas.domain

I tried to change http and https with both, with password or the API in the deploy-freenas, but the result are always the same. The deploy_config at first had only the API and the certificate path enabled, using the default values for the others and later changed some values too. I erased and generated a new API key. Nothing.

Do I need to enable something else besides the API key? The user is a admin, but I tried to run as root too, no luck.

What am I missing here?

2

I’m not sure, because it’s working fine for me (including on a test 24.10 system). The obvious (to me, at least) thing to check is that you’re using the correct and complete API key (including the number at the beginning of the key), and that you’ve specified it in deploy_config using the api_key field, not the password field (and that the password field is commented out).

3

Indeed, and that’s the first thing I verified. With the acme deploy hook I checked the env variable DEPLOY_TRUENAS_APIKEY and the first line of deploy_config, commenting out the the password. I even deleted the API key and created a new one, testing with both solutions. I followed the example and keep the number and the underline in the start of the API string and put no quotes between it.

As I read some comments on acme.sh github regarding a similar problem, I supposed there was a problem with the API and updated the TrueNAS from 24.04 to 24.10, but I think I misinterpreted and this movement are being made because part of the API will be deprecated just in the next version that actually is in nightly version, so today this is not a problem, please correct me if I’m wrong.

Well, apparently it seems the problem is not the deploy tools themselves, but from the TrueNAS side, but I’m struggling on how to approach this problem. I’m not familiar with the API to make some basic tests and check if I need to enable something else to allow the connection. I’m open to suggestions, as I don’t find many information regarding this on the forum and documentation.

4

Update: Assuming that is a network problem, I reviewed all settings and removed the IP range of part of my local LAN (192.168.200.0/24) in System->Advanced Settings-> Allowed IP Addresses. After the service was restarted, I asked acme.sh to force renew the certificates and, in consequence, to deploy the certificates using your script. Now it works!

To isolate the problem, I reintroduced the IP of my local LAN and force renew again. Problem resurfaced. I removed the IP and introduced 127.0.0.1, assuming that it could allow local connection. Still persisted. Only with the Allowed IP Addresses field cleared I’m able to deploy the certificates.

Ok, er… problem solved, but it doesn’t make sense to me. Someone care to explain, please?