Switching to Traefik in Dockge on TrueNAS Scale After Nginx Stopped Working

I wanted to share my experience with switching from Nginx to Traefik in my TrueNAS Scale setup after encountering a frustrating bug in Nginx. Nginx had been working fine for routing traffic and handling SSL certificates, but after a recent update, I started facing issues that I couldn’t resolve, including:

  • SSL certificate handling failures.
  • Traffic routing problems.
  • Configuration corruption, making it difficult to maintain the reverse proxy setup.

After spending a considerable amount of time troubleshooting, I decided to switch to Traefik . While I’m glad to have it working, the process was far from easy, and I wouldn’t describe it as seamless. That said, it’s now running and managing my services, which is a relief.

Why I Made the Switch

The decision to switch from Nginx to Traefik was driven entirely by the bug in Nginx. Here’s a summary of the issues I faced with Nginx:

  • SSL certificate renewals were no longer working after a recent update.
  • Failed requests to the backend services that were previously routed fine.
  • Configuration corruption, resulting in additional routing errors and downtime.

Since I was already using Docker containers via Docker on TrueNAS Scale, Traefik was a natural fit. It’s a more modern solution that comes with automatic SSL management, dynamic routing based on Docker labels, and simplified configuration that I didn’t have with Nginx.

Traefik Setup in Dockge on TrueNAS Scale

1. Setting Up Traefik in Dockge

I started by setting up Traefik as a reverse proxy in Dockge. Below is the configuration for the Traefik service in docker-compose.yml:

version: '3.7'
services:
  traefik:
    image: traefik:v2.6
    container_name: traefik
    command:
      - "--api.insecure=true"  # Enables Traefik dashboard on a local network (not recommended for production)
      - "--providers.docker=true"  # Use Docker as the provider
      - "--entryPoints.web.address=:80"  # HTTP entry point
      - "--entryPoints.websecure.address=:443"  # HTTPS entry point
      - "--certificatesresolvers.cloudflare.acme.dnsChallenge=true"  # Use Cloudflare for ACME
      - "--certificatesresolvers.cloudflare.acme.dnsChallenge.provider=cloudflare"  # Set up DNS challenge
    ports:
      - "80:80"  # HTTP port
      - "443:443"  # HTTPS port
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"  # Needed for Docker integration
      - "./acme.json:/letsencrypt/acme.json"  # Mount acme.json for SSL certificates
      - "./cloudflare.env:/etc/traefik/cloudflare.env"  # Mount Cloudflare credentials
    labels:
      - "traefik.enable=true"

2. Creating the Necessary Files

You’ll need to create a few files manually in your Traefik configuration directory. Here’s what you’ll need:

  1. acme.json:
    This file stores the SSL certificates that Traefik generates using the ACME protocol (via Let’s Encrypt or Cloudflare). Make sure to set proper permissions for this file:
touch acme.json
chmod 600 acme.json
  1. cloudflare.env:
    This file contains your Cloudflare API credentials. It allows Traefik to request SSL certificates via the DNS challenge. The file should look like this:
CLOUDFLARE_EMAIL=your-email@example.com
CLOUDFLARE_API_KEY=your-api-key

Create the file and add your Cloudflare API token or Global API Key with DNS management permissions.
3. docker-compose.yml:
This file defines the services Traefik will route traffic to (such as Nextcloud, Vaultwarden, Home Assistant, etc.). It’s where you’ll configure Traefik’s Docker integration.
4. dynamic.yml:
This file contains the dynamic routing configuration for your services, including SSL configuration and how traffic should be routed to each service. Here’s an example for Nextcloud:

http:
  routers:
    nextcloud:
      rule: "Host(`cloud.nextcloud.com`)"
      entryPoints:
        - websecure
      service: nextcloud-svc
      tls:
        certResolver: cloudflare

This file instructs Traefik on how to route traffic to Nextcloud, enforce HTTPS, and utilize Cloudflare for SSL certificates.
5. traefik.yml:
This file contains the static configuration for Traefik, like entry points, certificate resolvers, and logging settings. Here’s a basic configuration:

api:
  dashboard: true
log:
  level: DEBUG
entryPoints:
  web:
    address: ":80"
  websecure:
    address: ":443"
certificatesResolvers:
  cloudflare:
    acme:
      dnsChallenge:
        provider: cloudflare
        delayBeforeCheck: 0

This configures Traefik to listen on HTTP (port 80) and HTTPS (port 443) with automatic SSL certificate management via Cloudflare.

3. Manual Edits in dynamic.yml

Although Traefik allows dynamic routing via labels in the Docker configuration, I still had to manually edit the dynamic.yml file to ensure proper routing and SSL management for each service.

I had to:

  • Add routing rules manually for each service (e.g., Nextcloud, Vaultwarden, Home Assistant).
  • Specify the domain name and TLS settings for each service.
  • Configure Cloudflare for SSL management for every service.

This step isn’t fully automatic in Traefik, but once configured, it’s much easier to manage new services by simply adding them to dynamic.yml.

4. Configuring Services with Traefik Labels

Next, I configured each of my services to be routed through Traefik by adding Traefik labels in the Docker configurations. Here’s an example for Nextcloud:

services:
  nextcloud:
    image: nextcloud
    container_name: nextcloud
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.nextcloud.rule=Host(`cloud.nextcloud.com`)"
      - "traefik.http.routers.nextcloud.entryPoints=websecure"
      - "traefik.http.routers.nextcloud.tls.certResolver=cloudflare"  # Use Cloudflare for SSL
    networks:
      - proxy
    volumes:
      - nextcloud_data:/var/www/html

# Define the shared proxy network used by Traefik and all routed services
networks:
  proxy:
    name: proxy
    driver: bridge


I used Traefik labels to specify routing rules for each container, including the domain name and TLS settings.

5. Configuring Cloudflare for SSL Certificates

I’m using Cloudflare to manage SSL certificates with Traefik’s ACME DNS challenge. Here’s how I set it up in Traefik:

certificatesresolvers:
  cloudflare:
    acme:
      dnsChallenge:
        provider: cloudflare
        delayBeforeCheck: 0
      email: "your-email@example.com"
      storage: /letsencrypt/acme.json

This ensures Traefik automatically handles SSL certificate issuance and renewal via Cloudflare.

6. Restarting Traefik and Services

Once I updated the configuration, I restarted Traefik and my other containers (Nextcloud, Vaultwarden, Home Assistant) for the changes to take effect:

docker-compose down
docker-compose up -d

7. Verifying HTTPS and Access

After restarting everything, I was able to access my services over HTTPS without any login prompts. Traefik handled the SSL certificates automatically, and the traffic was routed correctly to my Docker containers.

Why Traefik Over Nginx?

  • Automatic SSL Handling: Traefik automatically manages SSL certificates through Cloudflare, eliminating the need for manual configuration and certificate renewal.
  • Dynamic Routing: With Docker labels, I can easily route traffic to new services without manually editing configuration files.
  • Simpler Configuration: Traefik’s configuration is cleaner and simpler, especially when using Docker.
  • Scalability: Adding new services is easy—apply the correct labels, and Traefik handles the rest.

Final Thoughts

Switching to Traefik was ultimately the right decision, but I won’t sugarcoat it—it was a huge pain in the ass. While Traefik solved the issues I was facing with Nginx, the setup process involved a significant amount of manual configuration, troubleshooting, and trial and error. The automatic SSL handling, dynamic routing, and simpler configuration are all nice, but they didn’t come without their challenges.

If you’re facing similar issues with Nginx or need a modern and scalable solution for managing your reverse proxy, Traefik could be the way to go. Just be prepared for some headaches along the way.

Feel free to reach out if you have any questions or need help with setting it up in your TrueNAS Scale environment. You’re not alone in this!

1 Like

What exactly is “the bug in NginX”? Could you please link a github issue or similar? Seriously interested.

This, this or this.

The issues started in 2.12.4.
2.12.3 appears to work fine for most people.

These issues are all about the nginx-proxy-manager project. So thanks for the clarification.

There seems to be nothing wrong with NginX, which serves web requests for more than a thousand customer web sites my company hosts.

Kind regards,
Patrick

1 Like

It seems nobody can tell the difference between these two–I don’t know why.

2 Likes

Thanks for asking, I was about to do the same. My nginx instance is working just fine. :wink:

I thought it was implied by the Traefik context, but I can see how it can still be misleading/confusing.

That’s just how it is with Nginx Proxy Manager - you can be lazy and not want to type it all out, but neither nginx or npm are going to be shorthands guaranteed to work every time.

Haha, fair point, @PMH. It does seem like everyone thinks it’s either ‘Nginx’ or ‘NPM’ as the culprit! :sweat_smile: But here’s the real kicker: when Nginx made some breaking changes (I mean, who doesn’t love a little change with no second option), the community was left with ‘fix it yourself’ vibes. :upside_down_face:

Sure, Nginx itself is still solid, but if the proxy layer keeps getting murky (no shorthands that work every time), people are going to look for alternatives like Traefik or Caddy, which, thankfully, actually get it when it comes to dynamic configurations, automatic SSL, and just… working without needing to read an encyclopedia for every fix.

At least with Traefik, I don’t have to start debugging every time something breaks. So, yeah, while Nginx is great for web requests, their proxy manager has been a real wildcard recently. But hey, it’s all part of the fun, right? :laughing:

1 Like

Solid guide, the best part about Traefik is there’s lots of ressources out there, and plugins, like mine has Crowdsec and my Traefik doesn’t use the Docker-Socket so I have it connecting to a Socket-Proxy instead

1 Like

Thanks! Once I stopped yelling at dynamic config and got everything talking, Traefik actually became fun. I pulled in CrowdSec too — using the Traefik bouncer as middleware. Really clean once it’s running, though the docs definitely made me question my life choices a few times.

I’m still using the Docker socket directly for now, even though it feels like handing root access to a toddler hopped up on Pixy Stix and curiosity. Definitely thinking about switching to a socket proxy — just haven’t had the motivation (or a good enough scare) yet.

How are you handling CrowdSec ban propagation? Just HTTP bouncer, or do you have it tied into iptables/nftables too?

Sorry for being out of the loop, but what’s Nginx been doing that affects reverse proxying? I recently got SWAG working on my system, and I’m wondering if it’ll be affected.

The latest version of Nginx Proxy Manager (not Nginx) seems to have some problems. Three threads about it linked in Neo’s post above.

They finally nailed it down in 1.2.6. I installed it today and now it’s back to normal. I’m still gonna try this tutorial though, I think I recognize some of the sources and setting an environment variable for the cloudflare credentials, although Techno Tim uses a file callout that’s more secure and trickier.

NPM development looks like a chaotic mess, so if we can get this in the catalog or manually switch to this/caddy I think the future looks better. OP, thanks for suffering through this and documenting it, I’m gonna bookmark this and attempt it later.

Just HTTP, but I have the additonal AppSec component installed, it’s a must, see Traefik | CrowdSec

I also have Crowdsec installed on my Firewall (OPNsense) so I’m not going to get any crazier than this. The bans worked when I tested them

If there are any tips for that, they are welcome too. I run Opnsense with CrowdSec also. Under Proxmox on the router hardware. Snapshots prior to updates have saved my bacon twice.

Ya, I would prefer Docker Secrets, but that is currently not supported by dockge.

1 Like

Follow the guides up there and then add this section to the compose

crowdsec:
    container_name: crowdsec
    environment:
      BOUNCER_KEY_TRAEFIK: GetkeyfromGUIDES
      COLLECTIONS: >-
        crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules
        crowdsecurity/traefik
      PGID: '1000'
    expose:
      - 8080
      - 6060
      - 7422
    image: crowdsecurity/crowdsec:v1.6.10
    labels:
      - traefik.enable=false
      - wud.tag.include=^v\d+\.\d+\.\d+$$
      - wud.link.template=https://github.com/crowdsecurity/crowdsec/releases/tag/v$${major}.$${minor}.$${patch}
    networks:
      - proxy
      - socket
    restart: always
    volumes:
      - /mnt/Apps/Docker/crowdsec/data:/var/lib/crowdsec/data
      - /mnt/Apps/Docker/crowdsec/etc:/etc/crowdsec
      - /var/log/auth.log:/var/log/auth.log:ro
      - /mnt/Apps/Docker/traefik/log:/var/log/traefik:ro

Then make sure you have Crowdsec plugin inside your dynamic.yml or config.yml, whatever you call it

1 Like

Clarification on the proxy network setup

A few folks reached out asking about the proxy network mentioned in the Docker labels. Good catch—my original post didn’t clearly explain how to define that network, which is required for Traefik to communicate with the services it’s routing.

To fix that, add the following to the bottom of your docker-compose.yml file:

networks:
  proxy:
    name: proxy
    driver: bridge

Each service that needs to be routed (Traefik, Nextcloud, Vaultwarden, etc.) should also include:

networks:
  - proxy

Without this, Docker containers won’t be able to see each other over the shared network, and you’ll get mysterious routing failures.

Appreciate the feedback—hopefully this clears things up for anyone following along!

1 Like

You gave us adequate warning, it’s a hassle until you finally get it to work. Our buddy Techno Tim came along and filled in a blank to make marrying Docker containers and Truenas much better in his recent video. Setup a dataset, pop in the shell, create and own the config folder, then custom config the docker app. As a chef’s kiss he also shows how to install code-server to make future deployments and editing a snap without bothering with the janky web shell.

I still haven’t finished my deployment of traefik, need to forward a port that’s not websecure, i.e. 443, but I get a cert issued and the server tries to answer. I’ll figure it out then dip into the plugins, but Tim was like a mind reader on this one with the troubles alot of us deal with.

I can’t seem to get the acme.json file to pull data into it it just stays empty. I see you are using traefik 2.6 though. Is there a reason you are using the older version?