First of all - I’m a complete newbie even at Linux, so please bear with me
I managed to install Plex Media Server as my first app app on Scale 24.04.2 and it seems it’s working fine. To continue with other apps, I’d like to set up Tailscale with Caddy. I’ve set Tailscale up as shown in this video, but I have no clue how to install Caddy. I mean I’ve opened the custom app installation window, but it has insane amount of fields to set up and I’m clueless which ones I need to change to what values Like entrypoints, env vars, networking and ports, storage, etc.
Any pointers? Are there some instructions maybe I could follow?
Thanks
We have a very basic tutorial on installing apps using the Custom app wizard that is undergoing updates. If you want, start with this tutorial but keep watch for the updated article that should be posted sometime in the next few weeks.
Generally, the best place to find application specific dock for using a custom app is hub.docker.com and the source application github. Those will mostly give you the parameters needed to run the app. There isn’t too much for caddy, just 3 mountpoints for the most part and possibly wanting to use a static IP for caddy, which means a network bridge, and following that doc.
Thanks for the links, but they do not make things more clear for me
But I found this old forum post, which gave me hints what I should set up. But when trying to run it, I get:
Back-off restarting failed container ix-chart in pod caddy-ix-chart-977d95cd7-fn4x6_ix-caddy(875c5ab9-a480-43ea-965a-68ab6751e084)
Searched for this, but it seems not much info what it actually means.
Also sometimes while deploying, I’m getting this:
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
I didn’t allocate any GPUs to Caddy, but I have allocated one to Plex. If I disable GPU support in Apps advanced settings, then Plex stops deploying with the same message, and I want Plex to use GPU.
So, when you get that, you have to find the reason. The reason for that is on the apps screen. You would click on caddy on the apps screen, then look on the right for the workload section, and click on the bottom right icon of that box, which has a hint of “View Logs”.
Regarding docker hub, that is going to be true for all custom apps, anything not in the Scale catalog. The things to look for are volume mounts, and in the page I mentioned on docker hub for caddy, you will see those are:
srv is used for their own custom config if you read the text on that page so is not needed. That leaves 3, /etc/caddy/Caddyfile, /data, /config. That is how you read those. Those are the 3 container paths you have to point to somewhere on Truenas via hostpaths in the custom app config. You ignore the left hand side (such as $PWD/Caddyfile, that’s just an example on their system. The right hand side after the : is where you need to mount somewhere via hostpaths.
config, data and etc have the same permissions and these are the 3 mounted paths during the installation
And in the History it’s:
History
Related Kubernetes Events
2024-08-02 10:06:10
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-nv8bd to ix-truenas
2024-08-02 10:06:10
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:06:09
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:06:09
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-g6rkt to ix-truenas
2024-08-02 10:06:09
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-6mvbv to ix-truenas
2024-08-02 10:06:09
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:06:09
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-vhjw5 to ix-truenas
2024-08-02 10:06:09
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
[ Insert here at least a 100x more of Successfully assigned and Allocate failed ]
2024-08-02 10:03:24
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:24
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-c9pk7 to ix-truenas
2024-08-02 10:03:23
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:23
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-8csck to ix-truenas
2024-08-02 10:03:23
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:23
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-wwr5m to ix-truenas
2024-08-02 10:03:23
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:23
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-tdz4k to ix-truenas
2024-08-02 10:03:22
Created pod: caddy-ix-chart-5869779b74-czb7n
2024-08-02 10:03:22
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-czb7n to ix-truenas
2024-08-02 10:03:22
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:22
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-dhrjm to ix-truenas
2024-08-02 10:03:22
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:22
(combined from similar events): Created pod: caddy-ix-chart-5869779b74-wd2vf
2024-08-02 10:03:21
Scaled up replica set caddy-ix-chart-5869779b74 to 1 from 0
2024-08-02 10:03:21
Created pod: caddy-ix-chart-5869779b74-bh2nt
2024-08-02 10:03:21
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-bh2nt to ix-truenas
2024-08-02 10:03:21
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:21
Created pod: caddy-ix-chart-5869779b74-4kp82
2024-08-02 10:03:21
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:21
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-4kp82 to ix-truenas
2024-08-02 10:03:21
Created pod: caddy-ix-chart-5869779b74-m2lrk
2024-08-02 10:03:21
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-m2lrk to ix-truenas
2024-08-02 10:03:21
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:21
Created pod: caddy-ix-chart-5869779b74-m7jgd
2024-08-02 10:03:21
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:21
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-m7jgd to ix-truenas
2024-08-02 10:03:21
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-hb9mx to ix-truenas
2024-08-02 10:03:21
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:21
Created pod: caddy-ix-chart-5869779b74-hb9mx
2024-08-02 10:03:21
Created pod: caddy-ix-chart-5869779b74-xgnj2
2024-08-02 10:03:21
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:21
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-xgnj2 to ix-truenas
2024-08-02 10:03:21
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-786s5 to ix-truenas
2024-08-02 10:03:21
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
2024-08-02 10:03:21
Created pod: caddy-ix-chart-5869779b74-786s5
2024-08-02 10:03:21
Created pod: caddy-ix-chart-5869779b74-5pg7n
2024-08-02 10:03:21
Successfully assigned ix-caddy/caddy-ix-chart-5869779b74-5pg7n to ix-truenas
2024-08-02 10:03:21
Allocate failed due to no healthy devices present; cannot allocate unhealthy devices nvidia.com/gpu, which is unexpected
Tried again just disabling the GPU and now in the logs I see:
2024-08-02 10:23:05.024702+03:00Error: reading config from file: open /etc/caddy/Caddyfile: no such file or directory
But there is such a file:
I’ve created Caddyfile via FileBrowser app. Could there be a permission issue maybe?
Really, it’s best to follow what docker hub says. i.e., the mountpath should be directly to the Caddyfile not a directory, you used /etc/caddy, whereas what I showed you said /etc/caddy/Caddyfile. But that wouldn’t stop it. Not the issue.
I just spun up a generic caddy (I build my own containers so don’t directly use docker hub) and mine worked perfectly. So, yes, it could be permissions. caddy is run as root, so wouldn’t expect it to be. However, I never ever use permissions (ACLs) so not sure how nfsv4 permissions works locally. I just use standard linux permissions.
Not sure why you try and add a gpu to caddy, not necessary and a whole other issue I can’t help with, I don’t use gpu and never will and it doesn’t matter as none should be passed to caddy.
So, my guess is indeed, it must be your permissions but I probably can’t help with that since I have no need for those. If you can sort that out, it will likely work.
How do I mount directly to a file via TNS UI? It allows me to select datasets only
But I didn’t. I added 1 GPU for Plex and left 0 for Caddy
Yeah… I already started looking at nginx (didn’t try pulling yet), but I guess pretty much same issues await.
So far from 5 apps I tried to install (only Caddy as a container), just 2 seemingly work without any issues. I thought TNS would be much more stable with apps. It works perfectly with samba shares and files overall, but strugling with everything else
Thanks for your help in any case. It gave me some hints what to look for
You can pick a file, the picker for hostpath allows you to go to the file level, I use that. Expand that folder as you will see!
I see about the GPU, you had said “Tried again just disabling the GPU and now in the logs I see:”. I guess you mean overall disable it, not from Caddy.
Permissions are permissions and a definite Linux.*BSD thing. I find ACLs are not usually necessary IMHO. Unless you have many users and shares that should be kept separate, etc. but even then standard permissions work well. But your problem has to be that, as I said I spun up a second caddy for testing and it worked first try, but I didn’t use nfsv4 permissions.
Here’s how I set up my permisisons for a share, and I don’t share much as I simply use filezilla or sftp to send files to Truenas, so why make a share. ALmost never do I make any share at all for any app, unless something else needs it which is rare. In this case, Digikam needs access directly on my client so I needed a share. But none of my apps use them.
OK, so I managed to get to the point where I get this now:
Error: adapting config using caddyfile: parsing caddyfile tokens for ‘tls’: getting module named ‘dns.providers.cloudflare’: module not registered: dns.providers.cloudflare, at /etc/caddy/Caddyfile:3 import chain [‘/etc/caddy/Caddyfile:10 (import cloudflare)’]
I’ve searched for the error, and it appears cloudflare is a separate module, for which a custom build is required. Couldn’t find how to do that so I could pull it in TrueNAS
Caddy usually has to be built from source with the cloudflare plugin. There may be some docker images available that are built with that plugin included.
Like entrypoints, env vars, networking and ports, storage, etc.
