QNAP TS-877 Truenas Journal

TrueNAS General
382

Ok encountered my first odd issue deploying docker container using jailmaker

this is for the scrutiny docker container

services:
  scrutiny:
    container_name: scrutiny
    image: ghcr.io/analogj/scrutiny:master-omnibus
    cap_add:
      - SYS_RAWIO
      - SYS_ADMIN
    ports:
      - 6180:8080 # webapp
      - 6186:8086 # influxDB admin
    volumes:
      - /run/udev:/run/udev:ro
      - /mnt/docker/data/scrutiny/config:/opt/scrutiny/config
      - /mnt/docker/data/scrutiny/influxdb:/opt/scrutiny/influxdb
    devices:
      - /dev/sda
      - /dev/sdb
      - /dev/sdc
      - /dev/sdd
      - /dev/sde
      - /dev/sdf
      - /dev/sdg
    ###  this is dom    - "/dev/sdh"
    #      - "/dev/sdc" #If you have 3+ drives, uncomment this line and add more to reflect ALL drives you have
    restart: unless-stopped
networks: {}

[+] Running 10/1
 āœ” scrutiny 9 layers [ā£æā£æā£æā£æā£æā£æā£æā£æā£æ]      0B/0B      Pulled                                            10.8s 
[+] Running 1/1
 āœ” Network scrutiny_default  Created                                                                0.2s 
 ā ‹ Container scrutiny        Creating                                                               0.0s 
Error response from daemon: invalid CapAdd: capability not supported by your kernel or not available in the current environment: "CAP_SYS_RAWIO"

using google i found this

Following up a bit, in case someone else might find this useful. I was able to get docker and portainer up and running in a ā€˜jailā€™ . In order to create some useful containers within portainer, i needed to add the following SYSTEMD_NSPAWN_USERS_ARGS line to the config file for the ā€˜jailā€™ (via jlmkr edit :

ā€“capability=CAP_BLOCK_SUSPEND,CAP_MAC_ADMIN,CAP_IPC_LOCK,CAP_MAC_OVERRIDE,CAP_SYSLOG,CAP_SYS_MODULE,CAP_SYS_PACCT,CAP_SYS_RAWIO,CAP_SYS_TIME,CAP_WAKE_ALARM

I will follow up if I find any additional parameters that are required for creating other containers.

by default, jailmaker passes

--capability=all

Enabling all the capabilities solved it. It DOES NOT seem to be the default behaviour of the docker config script, though.

The docker config template does not include all capabilities by default. Only the minimal amount to be able to run docker inside the jail. It is true that you may have to specifically add additional capabilities for some features of containers or packages installed.

Jailmaker doesnā€™t pass --capability=all by default, only when the docker_compatible=1 config is set. But this contains more capabilities than usually required. Which is why I didnā€™t add this to the docker config template.

In the future I plan to deprecate the docker_compatible config setting and instead introduce a
privileged option to disable seccomp, set DevicePolicy=auto and add all capabilities.

So basically what this means is, if you try deploy docker container and you run into a similar situation, you merely have to edit the docker config to add the necessary item you need. all, or specific items only. and that jailmaker plans to do something in regards to this particular situation sometime in the future.

not really a crucial container for me seeing as truenas is capable of running the scheduled smart tests and notifying about issues. i just merely like the UI. If truenas could be improved, they could maybe add a similar UI to what scrutiny did. That said, even truenas spartan look is not bad either. if you need further detail itā€™s found in the shell or so i heard

383

I used a bridge interface and assigned a static ip via my router/dhcp server.

This seemed the simplest to me and the most like running a separate vm.

It means that host networking in docker will behave as it should.

As it turns out, MAC addresses in systemd-nspawn are hashed from the jail name, so they are stable.

stux i notice you run your networking as such.

atm i have mine as bridged as well.

but i set in truenas admin UI settings to use a 192.168.0.210 for a static ip.

then for the docker jail created by jailmaker, this also is set to a static ip (you have to modify a file within the jailmaker location. refer to jailmaker advanced networking)

this way i can get nginx proxy manager to work without fighting with truenas for ports 80 and 443.

but you said you set your static ips via router dhcp.

then how about containers, can they be set with their own static ip? how do you do that? :thinking:

before i would use portainer to setup a macvlan to setup static ip via the portainer UI.

The other method is adding the networking via docker compose (this i had trouble with, so i didnā€™t resort to this method though iā€™m aware of it). But when switching over to dockge, they do have a basic networking setting, but overall they seem to want you to put the networking stuff within docker compose if you want that (something i didnā€™t have to do in portainer)

so this part i am still figuring out.

But for now i am getting by with the defaults, everything just works with just

networks: {}

maybe itā€™s not ideal/hardened setting, but it just works for a lan only homelab docker setup :smiling_face_with_tear:

384

is this a bug?

i was creating a docker container using

1000:999

for guid/gpid

then later i noticed the owner for the docker was 999 which was netdata.

but when i double checked the compose it was done correctly, but during creation it came out listing netadata who is guid 999 as the owner when it was supposed to be 1000 which is docker user

i went into truenas shell to find out the values for the uids and gids for cross checking.

1245Ɨ558 68.4 KB

why is that? :thinking:

i would be more inclined to believe i messed up somewhere rather than a bug in truenas based on occamā€™s razor

*update

saw the rest of the folders files were correct. it was only the db folder within that container was wrong. no idea xd

385

regardless i got unifi network application container to work.

though i couldnā€™t get it to deploy with only just the compose only. yes i did add the init script which should be correct, but i kept encountering a cannot connect to the mongo database.

so what i did was, recover my unifi docker container from my old nas setup, then paste into the new docker location, and make sure owner is correct (recursive apply), then deploy using the compose that didnā€™t work before, but now it does work.

i was also testing mongo db versions. v7 didnā€™t work for me. in the notes, they said mongo does not like major version changes. so instead i went from mongo v4 to mongo v5 without issue.

tested this as working
image: mongo:5.0.26

386

note: when using images for mongo, itā€™s highly recommended NOT to set to mongo:latest because there could be issues where mongo breaks. Always do manual update for mongodb.

at least this was the requirement for the unifi network application container.

387

was iffed i couldnā€™t figure out how i did the unifi from scratch. if i had to guess, you had to connect via the shell for the first time for authentication.

Once the mongodb container is up and running, I connected to it with:

mongo -u admin -p

I enter my password and Iā€™m in. Here I executed the following command to create a user called unifi, owner of a db called unifi:

db.getSiblingDB("unifi").createUser({user: "unifi", pwd: "KJDNFWI4R3R", roles: [{role: "dbOwner", db: "unifi"}, {role: "dbOwner", db: "unifi_stat"}]});

for myself itā€™s resolved. just leaving clues for others trying to figure this out

388

Managed to deploy/recover immich docker container.

Tested as working and fully recovered

987Ɨ441 50.3 KB

Went through the immich docker changelogs, there are breaking changes. but even after applying the change, i still managed to recover without issue

breaking change
Release v1.102.0 - 30,000 Stars Release šŸŒŸ Ā· immich-app/immich Ā· GitHub

nginxproxymanager duckdns works for immich. but this broke the bottom left version checker. but if you switch over to the regular digit lan ip, it works again. will figure that out later, but is just a minor issue, overall it still functionally works.

389

better understanding of how .env works with dockge.

in portainer, for the docker compose, you have to use a stack.env which is unique to portainer, because it then feeds that to the portainer UI where you can then populate the env values.

Whereas for dockge, you just use the regular .env

then in the dockge UI, just paste your env values there *you can dump it all at once, so no need to do it bit by bit fortunately.

once you deploy, the env gets saved as a .env in the stack folder along with the compose file. Very nice

1 Like
390

No, you donā€™t move it there. Thatā€™s where jailmaker stores the active config for a jail. Ie you create a jail, with a config, jailmaker makes a jail in jails, named what you specify, with a rootfs, and a config, and something else.

If you want to modify the config you use jlmkr edit <jailname>

(Or you could edit the config in the jail)

1 Like
391

I think thatā€™s changed in later versions of jailmaker. Will need to add the necessary capability to the jail config.

1 Like
392

I would suggest that you have two choice for docker networking in a sandbox

  1. port forwarding, which is the normal approach of exposing a specific port on the host (in this case the jail and its ip) and it forwards into the container. Eg 8080 ā†’ 80

  2. host networking, where the container can expose whatever ports it wants on the host (the jail). This is good for certain containers which have special requirements such as scanning DLNA services, but should be avoided. Primarily because there can only be one service per port.

The other way would be to add an additional IP to the interface and you can then bind a compose to that IP.

I would suggest not doing that, and if you need a different IP then setup a different jail with a different ip. And run the container in question with host networking in that additional jail

1 Like
393

working on authentik.

opted to start from scratch.

docker compose works.

after you see it boot up, go to the authentik url. however, you need to to to the initialization page which is

http://yourauthentikurl.com/if/flow/initial-setup/

At this point fill in the password, then use that to login to admin from this point.

By default the user is akadmin. If you hate that, you can later create a new user and make that admin, then delete this one. Donā€™t do this just yet. You need to setup invitations first, so when you create the new user with invitation, they can set their own password. Only after you do that, and turn that user into admin, then you can delete akadmin. Basically complete your authentik setup before worrying about that.

at this point just follow along the guides to set authentik up. i used this

and the official doc

Some tips

If you get locked out, you can go dockge, bash for the authentik server, then type this

ak create_recovery_key 10 akadmin

it spits out a token key. append it to the authentik url, this will then let you log in. It can only do this if you have at least a single user in authentik, so donā€™t delete everyone or this may not work.

authentik feels daunting but if you just follow the guides especially the youtube by cooptonian, itā€™s doable. itā€™s worth it in the long run because you then force authentication to access your docker container web services.

1 Like
394

notes: noticed some minor error about lost connection (but it still works) when using the duckdns.org url. Works without issue using the digit lan ip. Will figure this out later.

395

after deploying your authentik

then going to the initialization page to set your email and password.

login. does it work?

if yes then log out. log in. still work?

if yes then dockge stop docker, then start again. does it still work?

if it does, then authentik is stable enough to continue configuring. If not then you need to check out what is the issue.

396

was getting a connection error reconnectionā€¦

apparently this was why

This error is caused by the web interface not being able to connect via websockets, which are used for notification messages from the backend, see Reverse-proxy | authentik for an example reverse proxy config to ensure websockets are configured correctly

the solution was, for nginx proxy manager, tick enable for the web socket. after that the issue went away.

another thing i did was go to authentik > admin > application > outposts

change the line (third i think) changed that to the duckdns.org url

397

My mistake. the first cooptonian video is slightly outdated. his newer video covers the new changes e.g. setup wizard

because i am using nginx proxy manager, this is what i picked

Forward Auth (Single Application)
For nginxā€™s auth_request or traefikā€™s forwardAuth

or if the container supports it, you can pick oauth instead e.g. Immich and Portainer

both methods are covered in this youtube

398

Noticed that docker integration wasnā€™t setup for me.

no problem.

just go to system > outpost integration

then manually add it

# find / -name "docker.sock"
/run/docker.sock

for me itā€™s /var/run/docker.sock i browsed jailbreaker to reconfirm this as well as shell command to verify.

after adding and waiting a short while the integration for docker has a green sign. initially it was unhealthy, but if you wait a short while it will turn green.

1 Like
399

ran into issue

within authentik admin for outpost 3rd line


authentik_host: https://192.168.0.24:9443

all requests to /outpost.goauthentik.io must be accessible without authentication

location /outpost.goauthentik.io {
proxy_pass https://192.168.0.24:9443/outpost.goauthentik.io;
# ensure the host of this vserver matches your external URL youā€™ve configured
# in authentik
proxy_set_header Host $host;

is this correct?
https://192.168.0.24:9443

i even tried https:// mydomain.duckdns.org but that didnt work.

if i remove the nginx custom script from nginx proxy manager, the regular link worked. but its not fenced behind authentik which was whole point of even installing that x-x;

i sus the issue lies somewhere in that. but iā€™m not fully sure

400

actually after you create user in authentik, you donā€™t need to setup invitation (though you should at some point since itā€™s useful).

From the drop down in user when u select user, there is a hidden part for change password. so after create donā€™t forget to change password.

after that you may want to add the user to the admin group

401

different youtube by christian