Electric Eel - How I am using Dockerfile, .env files, compose files

Apps and Virtualization
,
1

So, an earlier discussion noted the limitations of Eel regarding not being able to edit after uploading them via create custom app, inability to use .env files, and, needing to keep Dockerfiles elsewhere via a full path build context. I have solved all that, for the way I want to do it at least. Let’s see if anyone likes.

Here is a caddy reverse proxy setup with custom plugins (porkbun dns and labels) where I illustrate:

Dockerfile:

FROM caddy:2-builder AS builder
RUN xcaddy build \
  --with github.com/caddy-dns/porkbun \
  --with github.com/lucaslorentz/caddy-docker-proxy/v2
FROM caddy:2
EXPOSE 80 443
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
RUN apk add --no-cache tzdata
CMD ["caddy", "docker-proxy"]

Compose file:

services:
  caddy:
    restart: unless-stopped
    init: true
    env_file: .env
    ports:
      - name: Reverse Proxy Ingress
        target: 443
        published: "443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /mnt/tank/Data/Caddy/Config:/config
      - /mnt/tank/Data/Caddy/Data:/data
      - /mnt/tank/Data/Caddy/Html:/usr/share/caddy
    labels:
      caddy_0.email: "{env.EMAIL}"
      caddy_0.acme_dns: porkbun
      caddy_0.acme_dns.api_key: "{env.PORKBUN_API_KEY}"
      caddy_0.acme_dns.api_secret_key: "{env.PORKBUN_API_SECRET_KEY}"
      caddy_1: (encode)
      caddy_1.encode: zstd gzip
      caddy_2: "localhost:80, localhost"
      caddy_2.root: "* /usr/share/caddy"
      caddy_2.file_server:
    build:
      context: .
    healthcheck:
      test: ["CMD-SHELL", "wget --no-verbose -T3 --spider http://localhost:80 || exit 1"]
      interval: 1m
      timeout: 10s
      retries: 3
      start_period: 30s
      start_interval: 5s
    pull_policy: missing
    networks:
      - Caddy
      
networks:
  Caddy:
    name: fatula
    external: true

So, how you say does that work? I use include! So, on the custom app creation screen, app name is caddy. and the compose file is simply:

include:
  - /fullpathtocomposefile

Now, there is no need to edit the compose file in the UI as the compose file is in somedir on the NAS. Since env files and dockerfiles are relative to the compose file, those work fine in the same fullpathtocomposefile dir (somedir). Note that compose specifies env_file: .env so it knows what dir. And the build context is . so same somedir. This also allows me to use the same exact files on another machine to develop them, no paths to change. So, compose.yml, .env, Dockerfile all in one dir somewhere on the NAS.

This gets around not being able to simply use .env files in the UI custom app creator, or to easily do custom builds.

The reason same files on another machine is important to me is I develop them on a non Truenas machine for testing (desktop). I don’t have to want to have to change the files to test it.

16 Likes
2

Yo, using include is actually a genius idea. This also allows to keep an easy backup of the contents of Compose file itself and edit it directly there. If the relative paths work, seems that this has no cons basically. Thanks for sharing.

Btw, so I assume "{env.PORKBUN_API_KEY}" gets expanded to what’s in the .env file. Is this syntax needed, or would something like ${PORKBUN_API_KEY} also work or break? Wondering about this in relation to the issue I mentioned with env in the other thread: whether with include it’s different, or is the {env:...} syntax a workaround that allows using stuff from env_file: inclusion in both cases.

3

Exactly, easy backups, in your filesystem not .ix-apps, etc. Relative paths work as shown. I have many apps hooked in separate compose files to the caddy reverse proxy all automatic from labels.

Yes, it gets expanded from what is in the .env file. I believe it’s needed, not 100% sure. I believe that’s how compose works and your proposed syntax means something different, but am not sure, just the way I’ve always done it. The quotes are needed. I don’t think the include is any different as my desktop uses the same files without any include just fine.

1 Like
4

It’s simple genius! It must be in all FAQs.
You save me an epoch.
:+1:

5

Thanks a ton for this info. I just put all the files from GitHub for the latest version of xwiki into a directory on my server and this worked perfect for me.

1 Like
6

I’m intrigued by this approach. I’ve seen a lot of others recommending Dockge to manage their apps. I also just shelled into Truenas and I can use docker directly in the shell. What is the benefit of using custom apps over just using docker compose directly?

7

I can’t answer that, but, I have no need for Dockge. It’s just another thing to deal with. I greatly prefer the minimum software I need. Using this technique, I still get update notices, click update, and it rebuilds and downloads as needed. Can’t be simpler. I don’t want to do auto updates so that is not an issue either. I suspect it’s personal preference and some guides out there that people read. There’s nothing wrong with using it or portainer, etc. Whatever you prefer.

8

Thanks for this!

9

@sfatula this is awesome. Thank you for posting your workaround.
This is the only way i have found to properly install openproject in truenas.

10

This approach is nice.
I am looking for a way to have a global .env file, to reuse some variables in different compose.yaml files.

Only way i found is

export COMPOSE_ENV_FILES=/mnt/tank/stacks/.env,./.env

Source

It works if i rund docker compose config, but with in the Apps not. I have to export the variable to the user which runs the docker compose in the Apps tab. Is this root, or apps?

11

Does this work?
systemctl edit docker

…then add your environment add to the [Service] block:

[Unit]
StartLimitBurst=1
StartLimitIntervalSec=910

[Service]
ExecStartPost=/bin/sh -c "iptables -P FORWARD ACCEPT && ip6tables -P FORWARD ACCEPT"
TimeoutStartSec=900
TimeoutStartFailureMode=terminate
Restart=on-failure
Environment="COMPOSE_ENV_FILES=/mnt/tank/stacks/.env,./.env"

systemctl daemon-reload
systemctl start docker

Source

Maybe setting EnvironmentFile is another option:

[Service]
EnvironmentFile=/data/env/MYAPP

Source

12

You mean for example:

env_file: 
  - .env
  - .env2
13

No, not yaml. I think it just a basic KEY=VALUE text file.

For example,
systemctl edit incus
has a commented out example of setting EnvironmentFile:

# [Service]
# EnvironmentFile=/etc/default/incus

and that file is on the system and looks like this:

root@TrueNAS01[~]# cat /etc/default/incus 
PATH=/usr/libexec/incus:/usr/sbin:/usr/bin:/sbin:/bin

Likewise per the Source’s guidance on using EnvironmentFile, it says:

When using a file to store the environment variables, use the format “VAR=VALUE” (without quotes)

14

I was responding, as it says, to Benda, who said yaml.

He said:

" I am looking for a way to have a global .env file, to reuse some variables in different compose.yaml files."

Am I missing what he asked?

15 
stacks
.env.global
- pihole
   .env
   compose.yaml
- codeserver
   .env
   compose.yaml

The variables are for the compose interpreter, not for inside the container.

compose.yaml

services:
  app:
    image: node:18-alpine
    ports:
      - 127.0.0.1:3000:3000
    volumes:
      - ${PATHTOVOLUME}/app:/app
    environment:
      TZ: ${TZ}

.env.global

TZ=Europe/Berlin

.env

PATHTOVOLUME=/mnt/tank/docker-data/foo
17

Still confused I guess. env_file IS for compose. Here is my caddy .env file which is a set of key value pairs:

TZ=America/Chicago
PORKBUN_API_KEY=myapikey
PORKBUN_API_SECRET_KEY=mysecret
EMAIL=myemail.com

I can say in compose.yaml for caddy:

env_file: 
  - .env.global
  - .env

And I can move the TZ to .env.global. I can reference any of those in compose.yaml using ${TZ} in the environment section or whatever one you want. Just as you did. I can use path to volume from the env file in the compose.yaml file and not in the environment section, just as you did.

I guess I just don’t get it as my example is exactly the same as your proposed example except I am using the env_file construct within the compose file to allow multiple key value pairs from different files.

Here’s another example where he uses version in the .env file to specify the image version. His example does not use the env_file construct, but that’s how you can do multiple env files which he is not showing.

So, it is for the compose interpreter. But can be for inside the container too if you wish. Just like yours.

I guess I should just say sorry if I still don’t get what you are asking, but your example would work fine using env_file construct and using more than one. You can use ${PATHTOVOLUME} which would be set in .env and you can use TZ: ${TZ} which would come from .env.global

EDIT: Finally get what he is asking for so make a working example later in this discussion. Sorry for not getting it. This example was wrong anyway.

18

Please read this.
I need the variables for the docker compose interpreter, not inside the container.

20

Here is a working example, try it!

.env contents:

TZ=America/Chicago
ENV1=Test

.env2 contents:

SOMEVAR=Steve

compose.yml contents:

services:
  ubuntu:
    image: ubuntu:latest
    restart: unless-stopped
    pull_policy: missing
    command: ["sleep", "infinity"]
    environment:
      TZ: ${TZ}
      ENV2TEST: ${SOMEVAR}

Note, I am interpolating values from both env files.

In the truenas custom yaml box:

include:
  - path: /mnt/tank/Data/Compose/Ubuntu/compose.yml
    env_file:
      - /mnt/tank/Data/Compose/Ubuntu/.env
      - /mnt/tank/Data/Compose/Ubuntu/.env2

Obviously change the paths

Go ahead and enter a shell in that container, both variables were interpolated and exist in the container. Of course, you don’t have to put them in the container if you don’t want, but they are interpolated. Just trying to provide a way to do what you want. If you modify the services via syustemctl, you will lose the changes on restarts or updates. Of course you could do a after bootup script too, but then you’d have to delay your docker startup. WHichever makes more sense for you, use that.

21

Please try this.

compose.yml contents:

services:
  ubuntu:
    image: ubuntu:${UBUNTUVERSION}
    restart: unless-stopped
    pull_policy: missing
    command: ["sleep", "infinity"]
    env_file:
      - ../.env.global
    environment:
      TZ: ${TZ}

…/env.global

TZ=America/Chicago
UBUNTUVERSION=24.04

Folder structure:

stacks
.env.global
- ubuntu
   .env
   compose.yaml

Run docker compose config in the ubuntu folder.
Output is

WARN[0000] The "TZ" variable is not set. Defaulting to a blank string.
WARN[0000] The "UBUNTUVERSION" variable is not set. Defaulting to a blank string.
name: ubuntu
services:
  ubuntu:
    command:
      - sleep
      - infinity
    environment:
      TZ: ""
      UBUNTUVERSION: "24.04"
    image: 'ubuntu:'
    networks:
      default: null
    pull_policy: missing
    restart: unless-stopped
networks:
  default:
    name: ubuntu_default

If i put TZ=America/Chicago and UBUNTUVERSION=24.04 and UBUNTUVERSION in the ubuntu/.env file it will work.

Hope that helps to understand the problem.
If i have multiples compose with the same version tag, i wanna manage it from a central place.
I know this is not TrueNAS specific, but though someone already is doing a similar approach and can help.