Linux Jails (sandboxes / containers) with Jailmaker

[TempleHasFallen]

I’d probably want to start up DNS/DHCP jail first and put a sleep/delay on starting all the rest

[Cellobita]

The next docker version (26.1.1) will fix this pesky bug, check the explanation and fix here: Allow for a read-only "/proc/sys/net". · moby/moby@01ea18f · GitHub

[Jip-Hop]

That may be true. But my advice to move from host networking to macvlan or bridge networking for anyone using docker in a jail still stands. I’m using bridge networking and wasn’t affected by this bug in the first place.

[Jip-Hop]

@Haldi

There is no “empty base image”

There are many empty base images. The default is debian but you can also choose Ubuntu or fedora for example. The templates are for installing something on top of a base image.

Just run jlmkr create emptybase and you have an ‘empty’ debian jail. It is all written in the readme.

[skittlebrau]

@winnielinnie

Can someone explain to me the appeal of using docker / containers within a Linux “jail”, when the jail is already a container itself?

It’s becoming more common for Docker to be the main supported method of installation for some services/applications. I don’t personally like that, but it’s the way of things now unfortunately. Some applications go as far as not offering a native package at all (eg. Tube Archivist).

[mooglestiltzkin]

my experience with jailmaker

QNAP TS-877 Truenas Journal

[image] I had previously experimented with truenas using 2 different qnap models. a TS-253D (scale) and a TS-653A (core). I tested how truenas worked (since it was my first time), and i was checking how that would work on a QNAP. Familiarizing myself with truenas, and also how would setting up truenas on a QNAP to make that possible and test how stable this would be. this was the original project to make that work (credits to elvis and trex) https://forum.qnap.com/viewtopic.php?t=161170 H…

whenever u get that error message, most often the reason is the jailmaker docker jail’s config has something wrong in it.

solution?

undo the last change you did there recently that might be the cause.

OR, if you can’t remember, go back to the default docker config template, try use that. If it works, starting more changes to it, BIT by BIT, until you come across the cannot run warning. Then roll back your change that made it go bad. That’s your culprit.

hope that helps with the troubleshoot

Made this for myself but maybe it will help you, and pretty much the summary of what i just said

why jail docker-compose?

i did jail docker.

my layout in my docker dataset looks like this

/mnt/tank/docker

then within that is

/compose
This is where i have my docker yaml for deploying dockge the first time

/stacks
this is where all my docker yamls and .env go when i deploy through dockge. each container has their own folder inside

/data
this is where all the docker container permanent configs go. each container has their own folder inside

/old-archived
this is from my old nas where i restored from. i use this for reference when deploying my dockers using old stuff. will eventually delete

then you got /mnt/tank2/videos , /mnt/tank2/pictures for media. you also need to bind this dataset in the docker jail config in addition to the docker dataset

anyway why i suggest to name your jail, docker instead of docker-compose, is so whenever you do jlmkr command its jlmkr status docker instead of jlmkr status docker-compose less to type and to the point. no need the extra then again if you are copy pasting that, it wouldn’t matter either way i guess

hope that helps

[mooglestiltzkin]

By the way just wondering, did anyone get authentik to work with nginx proxy manager when using the jailmaker setup?

Yes i am using the bridged networking advanced method. With a static ip setup for the docker. While the truenas also has a static ip. Doing this, you can then safely use nginx proxy manager using port 80 and 443 without conflicting with the truenas which is on a different static ip.

but i ran into issue when trying to setup authentik. I couldn’t get it to work with nginx proxy manager because the url wouldn’t work.

I used this guide to add add duckdng domain for use in local lan to get valid cert in combination with nginx proxy manager.

Quick and Easy Local SSL Certificates for Your Homelab!

they said in npm you could add the docker container names instead of lan ip. That doesn’t work. Instead i had to type out the 192.168… lan ip for the docker host

but if you add the docker container name in uptime kuma, that works

[ALeXX]

Hi, is there a way to install a custom distro with jailmaker?
I’m trying to install Home Assistant OS as a jail.

[pmh]

HAOS as the name suggests is a complete OS so it must be installed in a VM. If running in a jail is more important than having the complete distro, you can pick the Docker based installation method.

[etorix]

@TempleHasFallen

I’d probably want to start up DNS/DHCP jail first and put a sleep/delay on starting all the rest

Such fundamental network services are best run on a dedicated appliance (say, OPNsense firewall) than as an app on a NAS. You do not lose network any time you update the NAS or shut it down to change drives.

[ALeXX]

From my understanding, HAOS is a “docker” on top of Debian.
Is there a difference between Debian that jailmaker is using to create jails and the one used by HAOS?

[pmh]

I don’t know. All I know is that the complete HAOS is distributed as an on-hardware or VM image only. If you install in Docker not all features are supported. It’s a decision by the Home Assistant project, not a technical necessity.

When you run in Docker you are not running HAOS but Home Assistant in Docker, plain and simple.

[TempleHasFallen]

I never said I’m referring to the networks DHCP/DNS…

When running over 50 VMs, Apps, Jails across 6 different networks, I want a separate DHCP/DNS server which starts before everything else.

At the moment that’s just a SCALE VM which is explicitly started manually first at boot but wanting to migrate to a systemd-nspawn jail to reduce overall overhead.

[Stux]

I setup a similar thing for VMs, ie I have a pfsense VM which is my DHCP server and then I want other VMs to start afterwards

So, I use this script as a startup command. The pfsense VM fire up, then when SSH is detected I start other VMs

mrstux/truenas_scripts/blob/main/start_vm_after_host.sh

The same thing could be used with jailmaker. Have the DHCP jail set to start. Then have another script which detects something from the jail (ssh?) that then calls jlmkr start to start other jails, rather than auto starting them

[Stux]

If you set “startVmbyNameTool” in above script to “/path/to/jlmkr.py start” that should convert the script to a “start jail after ssh” script.

[dasunsrule32]

Does dockge have the capability to manage multiple hosts?

I would want a management jail with dockge running in it, then have other jails stood up that dockge could manage and install images into. Is that possible?

This is the first time I’ve seen dockge, so forgive the ignorance.

Some example deployments would be:

• Jail with NPM
• Jail with plex
• Jail with authentik
• Jail with postgres

etc.

[Stux]

@dasunsrule32

Does dockge have the capability to manage multiple hosts?

I would want a management jail with dockge running in it, then have other jails stood up that dockge could manage and install images into. Is that possible?

This is the first time I’ve seen dockge, so forgive the ignorance.

Some example deployments would be:

• Jail with NPM
• Jail with plex
• Jail with authentik
• Jail with postgres

etc

Yes. This is possible.

Although usually you would combine multiple stacks/dockers into a single jail… or a handful.

Dockge can control other dockge agents. You would need to install Dockge in each jail and use a master dockge to control the agents.

How it works?

• Your browser only need to connect to Dockge 1. Dockge 2/3/4 are connected from Dockge 1.
• Dockge 1 works like a proxy, it will forward requests/responses from Dockge 2/3/4 to your browser.

[dasunsrule32]

Thanks for that!

Last question, once dockge is set to autostart, and you install containers using dockge, will it manage starting images on each jail? Also, how would deploying apps be handled, just pick an agent to deploy to? I’m assuming it would use the same datasets and share them across agents.

I have to rewatch your video again. Thanks for that by the way, it was helpful.

[Stux]

Yes.

Technically, it uses docker compose up -d and docker compose down, and then docker remembers the state between reboots.

But in essence, once you click “start” the stack will stay started, until you de-activate or delete it.

With the agents, you should be able to even move a stack between agents and it will change which “node” the docker is running on.

[dasunsrule32]

Ok, awesome!

I have some testing do. I very much appreciate your time and help.