New app catalogue request: Caddy (reverse proxy)

Feature Requests
1

Caddy is a reverse proxy that natively integrates with Acme and can use Tailscale certificates.

Docker Hub: https://hub.docker.com/_/caddy/

It is very simple to configure and deploy. It also has a native integration with Docker where it can be configured entirely using Docker labels with no configuration file required and automatically maintains itself with no in-service outage for changes (although the container will need to be reloaded when new app networks are deployed, based on a separate network for each app, and for upgrades). It needs access to /var/run/docker.sock and TrueNAS has made an effort to prevent that (cannot add users to group ‘docker’) so I did not try, but am willing to For Science.

Some apps also don’t work with the ECDSA certs issued by Tailscale, and most won’t support QUIC, which the reverse proxy can do. Jellyfin does not work with Tailscale certs, and has announced plans to remove HTTPS support, so a reverse proxy will be required. It is getting harder to avoid https and harder to avoid certificate automation, with CA/Browser Forum agreeing to reduce max cert lifetime from 1 year now to 47 days in 2029.

There would be an opportunity for a very elegant integration between TrueNAS, the Apps catalogue, Docker, and Tailscale/Acme. For example, TrueNAS App catalogue sets labels on the containers that Caddy uses to publish with the certificate auto-managed from Tailscale, with no config file to manage. TrueNAS could even use it for presenting the Web UI, protected with a real cert.

In the apps catalogue today there is Nginx but it doesn’t have the native integrations with Acme, Tailscale, Docker. Caddy’s features significantly differentiate.

1 Like
2

Caddy is so nice but you should have posted this under feature requests? Maybe? Not sure if new apps should go there.

Caddy runs fine on Eel including docker.sock.

I believe you have to build in labels, but they could do this easily enough. I am using labels and had to build them in, didn’t see an image with labels support.

Another thing to consider is building in other dns providers. That could also be done in an image. I for example use porkbun with caddy.

3

Thanks – I moved to Feature Requests.

/var/run/docker.sock on 25.04 is root:docker 660. I suppose one could run caddy as root but it is not good practice. TrueNAS should allow you to change the membership of group docker.

4

Summary:
The TrueNAS App Catalog currently offers Nginx as a reverse proxy option but lacks native support for modern, container-friendly reverse proxies like Caddy and Traefik. Both of these tools provide significant advantages in ease of use, automatic certificate management, and dynamic configuration, which could improve the overall user experience when deploying applications on TrueNAS.

Caddy for TrueNAS

Caddy is an easy-to-use reverse proxy that natively integrates with ACME for automatic SSL certificate management. It supports Tailscale certificates and QUIC, making it an excellent choice for simple, secure web application deployments. Caddy automatically maintains certificates and updates the reverse proxy without requiring in-service outages or manual configuration changes.

Key Features of Caddy:

  • Automatic HTTPS: No manual certificate management; SSL certs are automatically issued and renewed via ACME and Tailscale.
  • Docker Integration: Native integration with Docker allows for simple container-based deployments using labels to configure services without requiring a configuration file.
  • QUIC Support: Supports the modern QUIC protocol, improving performance for secure HTTP/3 connections.
  • Simplicity: Very easy to configure with minimal setup. The Caddyfile is simple to read and write, making it ideal for users who need quick and easy configuration.
  • Use Case: Ideal for home labs or smaller setups that prioritize simplicity and automated HTTPS.

Challenges:

  • TrueNAS Docker Integration: TrueNAS has restrictions around Docker container management, specifically around /var/run/docker.sock access. This could limit the ability to fully leverage Docker-based features with Caddy on TrueNAS unless further integrations are added to allow this access.

Traefik for TrueNAS

Traefik is a powerful reverse proxy and load balancer that is optimized for containerized environments. It supports dynamic service discovery and automatic configuration updates, making it highly scalable and adaptable for more complex infrastructure. Traefik integrates seamlessly with Docker and Kubernetes, providing advanced routing features and traffic management options.

Key Features of Traefik:

  • Dynamic Service Discovery: Automatically discovers services running in containers and adjusts its routes without needing manual configuration changes.
  • Advanced Routing: Supports advanced routing options, such as path-based routing, load balancing, and middleware configurations.
  • Docker and Kubernetes Integration: Excellent support for Docker and Kubernetes environments, using labels to configure routing and proxy rules.
  • Automatic SSL: Like Caddy, Traefik supports ACME for automatic certificate management.
  • Scalability: Best suited for larger, more complex infrastructures, especially when dealing with multiple services and high traffic.

Challenges:

  • Complexity: Traefik offers more features and flexibility but requires more setup and a deeper understanding of configuration files (JSON or TOML).
  • Integration with TrueNAS: While Traefik could be highly beneficial for containerized environments, integration with TrueNAS would require ensuring proper support for Docker networks and dynamic updates for service discovery.

Request:

We propose adding Caddy and Traefik as reverse proxy options to the TrueNAS App Catalog. Both tools offer unique advantages that could enhance the flexibility, security, and ease of use for users deploying containers and services on TrueNAS.

  • Caddy: A simple, out-of-the-box solution for managing SSL certificates and reverse proxy configuration with minimal setup. Ideal for users who want a lightweight, automated solution for home labs and small-to-medium setups.
  • Traefik: A more advanced, feature-rich reverse proxy that excels in dynamic service discovery, load balancing, and support for complex containerized environments. A great choice for larger, more complex setups that need scalability and customization.

Conclusion:
Adding Caddy and Traefik to the TrueNAS App Catalog would provide users with more flexibility and improve reverse proxy options for diverse use cases, from simple home labs to larger, containerized infrastructures. These tools would complement the existing Nginx option and allow TrueNAS users to more easily secure and manage their web services.

5

Wonderful, thank you!

At the same time the Tailscale app should allow /var/run/tailscale to be stored on a host dataset. This is required for some kinds of certificate automation, so that the Tailscale socket can be shared with other containers who will request certificates over /var/run/tailscale/tailscaled.sock.

Feature request: Tailscale TrueNAS app: make /var/run/tailscale volume configurable to allow other containers to automate certificates

6

All (web) apps also need the ability to add the web server to the network that the proxy is running in so that the proxy can reach the web server. For example:

services:
  caddy:
    networks:
    -  caddy
  ...
...
  jellyfin:
    networks:
    - ix-jellyfin_default
    - caddy
  ...
7

A better idea architecturally would be to instead have a checkbox for a caddy to be built for each web app; that way everything is self-contained. Caddy should have the caddy-tailscale module so each instance can run its own hostname on your tailnet and get its certificate.

8

… and caddy-docker-proxy may be useful for configuring caddy without through labels, without a configuration file.

9

Only way to run caddy imho. Makes everything simpler.

1 Like
10

I need to dig into this a bit, especially under Dockge. Seems likely to be a simpler configuration than Traefik.

1 Like
11

@sfatula my result with caddy-docker-proxy and caddy-tailscale is almost perfect, and good enough for my needs but not to deploy in enterprise. To be perfect we need to be able to define cross-stack dependencies (able to depends_on another container in another stack) and networking (able to use a network in another stack).

Example with 2 containers in different stacks, modelling our Caddy container and a TrueNAS app:

Caddy container has the global caddy config, and a macro for TLS that I can reuse in for the web servers. The syntax took me a bit to figure out, so in case it helps anyone else:

caddy:
  labels:
      - caddy_0.servers.protocols=h1 h2 h3
      - caddy_1=(common)
      - caddy_1.tls.protocols=tls1.2 tls1.3
      - caddy_1.tls.ciphers=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

Web server in a separate stack:

webserver:
  labels:
      - caddy=myweb.tailnet.ts.net:443
      - caddy.bind=tailscale/myweb
      - caddy.reverse_proxy={{upstreams 8080}}
      - caddy.import=common

But the web server and Caddy are not in the same network, and I can’t change a TrueNAS App’s web server to also join the Caddy network. But if I could, the web server can’t verify that Caddy is running before starting, which causes a start failure if the Caddy network doesn’t exist.

The behaviour I would like is that:

  • The server depends_on Caddy running before starting. I have not found a native way to do this.

  • The web server joins the Caddy network (and not Caddy joins the app’s internal network) with this target topology:

Caddy - [Caddy Network] - Web Server - [App-Internal Network] - Database (for example)

So, I used the caddy-docker-proxy labels idea to tag which containers should join the Caddy network when Caddy is available. A script running in a separate container listens for container start/restart events on the docker socket. If a container joins wanting to join an existing Caddy network, it is added via the Docker API. If a Caddy joins, all the containers waiting to join its network are added immediately. It does a rescan every 60s in case something unexpected happened.

Now I have a fully dynamic configuration with no chicken-and-egg issues. Caddy does not need to reload as containers/container networks change, and is fully automated with caddy-docker-proxy for config, and caddy-tailscale for certificates and access to my tailnet. Web apps are not blocked from starting if Caddy is down.

Ideally this should all be done in native docker-compose.

12

As a nice idea for those running Tailscale and worrying about remote Tailscale upgrades… You can run an independent caddy proxy just for TrueNAS GUI, with separate tailnet connectivity from the Tailscale TrueNAS App, with something like:

services:
  caddy:
   environment:
      - CADDY_DOCKER_LABEL_PREFIX=truenas_caddy
    extra_hosts:
      - host.docker.internal:host-gateway
    labels:
      - truenas_caddy_0.servers.protocols=h1 h2 h3
      - truenas_caddy_1=truenas.tail3f4eb5.ts.net:443
      - truenas_caddy_1.bind=tailscale/truenas
      - truenas_caddy_1.reverse_proxy=host.docker.internal:80
13

Docker compose depends_on only works within a stack, so the docker solution is not separate stacks if you need a depends_on. Really though, I am using separate stacks as for my apps, it doesn’t matter what starts first. If mariadb is not started yet but homeassistant or photoprism for example starts first, they don’t connect, but they retry until they do. So, no big deal. One can use dockerize or waitforit if they really needs a cross stack depends_on. Note that what you are asking for only resolves any startup issues, not other issues. So, if you stop the depends_on target, you still have a problem.

There is no issue with network and separate stacks in custom yaml. Use the networks section of docker compose and specify the same network in the 2 different stacks and they can talk to each other.