Linux Jails (sandboxes / containers) with Jailmaker

I found docker - Failed to add interface to sandbox - Stack Overflow and docker compose - cannot program address in sandbox interface because it conflicts with existing route - Stack Overflow

From the looks of it it doesn’t seem to be a jailmaker specific issue.

I have found several issues with the latest version of Docker, 26.0.2. Many containers stopped working with networking errors, after upgrading to it.

I have managed to get them working by downgrading Docker to 26.0.1, with the following shell command:

apt-get install docker-ce=5:26.0.1-1~debian.12~bookworm docker-ce-cli=5:26.0.1-1~debian.12~bookworm containerd.io docker-buildx-plugin docker-compose-plugin

YMMV, but for now this has allowed me to start my containers again.

6 Likes

Considering I just did the cross-grade from ubernerd/nerdctl to jailmkr only today, I might have been bitten in the ass by this then.

The containers that were configured with host networking started fine, while the bridged ones didn’t. I’m gonna try downgrading Docker.

–edit: Yay, it fixed it!

5 Likes

Is anyone available to test/review the upcoming release: Deprecate docker_compatible and gpu_passthrough by Jip-Hop · Pull Request #121 · Jip-Hop/jailmaker · GitHub?

It’s providing more secure defaults for running docker containers and deprecating the docker_compatible config option.

I don’t have a spare machine to test this, but please do not break Tailscale - I have only been able to make it work inside a jail with

systemd_nspawn_user_args=–capability=CAP_NET_ADMIN

If there is a documented, safe alternative, I can test it after-hours - I have a few servers directly accessible via SSH, breaking Tailscale wouldn’t be a big problem on them.

1 Like

It’s supposed to be a non-breaking update. And even after removing the docker_compatible option in the future there’s nothing stopping users from manually configuring equivalent options.

1 Like

I’d be happy to help test with my setup. The only host network configured container I have is Home Assistant.

1 Like

Question… I have a container (systemd_nspawn_user_args=--bind='/mnt/Vortex/Media:/mnt/media' --bind=/dev/fuse). I then mount /mnt/media with rar2fs under /mnt/rar2fs using AutoFS. At this point /mnt/rar2fs works as expected.

Now, is it possible to somehow make the /mnt/rar2fs mount visible in the host? Or is there a different way I can use a container with rar2fs where the output filesystem is visible in the host?

Thanks

the contents of /mnt/rar2fs inside the jail should be visible in your jailmaker/jails/<jailname>/rootfs/mnt/rar2fs directory

But what do you then want to do from the host with that? That you could’t do from inside the jail instead?

If you wanted to you could probably bind mount that again somewhere else… into the file system.

While I see all the content inside /mnt/rar2fs in the jail, jailmaker/jails/<jailname>/rootfs/mnt/rar2fs on the host is empty.

I want to be able to share trough SMB the rar2fs output that is visible in the jail under /mnt/rar2fs (preferably trough TrueNAS Sharing page)

I’m sorry, I was going to test what I wrote before posting… but got side-tracked.

You’re right, the above doesn’t work.

I don’t think this is possible, even if you were to somehow take the isolation by the jail out of the equation. SMB on SCALE integrates heavily with ZFS (datasets). Your AutoFS mount is not a ZFS dataset so you can’t share it from the TrueNAS GUI. Your best bet is to share (via SMB or otherwise) from inside the jail. Or you’re going to have to extract the rar file or copy the contents of the mount to a dataset which can be shared via SMB.

Is there something special I missed when trying to do so?

TrueNAS is using x.x.x.124 and the rar2fs container x.x.x.125. I’ve installed samba and configured the share like I always do, added a new smb user (smbpasswd -a <username>) but I can’t seem to be able to mount it from any debian VM or access it from Windows… :worried: (I can ping the container from Windows but when trying to access the share I get an instant error message: Windows cannot access \x.x.x.125)

I’m kind of a noob when it comes to all of this, but just another idea I had that I’m still experimenting since nothing works yet… but…

pre_start_hook=mkdir /tmp/rar2fs

use --bind='/tmp/rar2fs:/mnt/rar2fs'

then symlink the temp folder into a dataset: ln -s /tmp/rar2fs /mnt/Vortex/Media/_rar2fs

Maybe someone with more knowledge knows if this is even possible and maybe how to do it? :sweat_smile:

Sorry it took me a week to do any testing, I got ready last week and was pulled by a work project. Cloned my docker config dataset, pulled the dev jlmkr.py, created a test jail and used the github docker config file switching out macvlan with bridge and also turned off the nvidia passthrough default (is that intentional Jip-Hop?). Homeassistant in host network mode works fine for me. This is not a comprehensive test at all but I’m planning to rebuild my production jail using that methodology once the new version is released - if someone could intervene if I did something wrong.

Here’s a video I made on setting up Sandboxes with Jailmaker

8 Likes

Users who are still using host networking will run into issues when they upgrade docker to v26.0.1 and above. I recommend anyone using docker inside their jail to stop using the docker_compatible config option a.s.a.p. and start using macvlan or bridge networking. See the full announcement on GitHub.

1 Like

Can someone explain to me the appeal of using docker / containers within a Linux “jail”, when the jail is already a container itself?

Is it really that inconvenient for users to occasionally run apt update or pacman -Syyu within a jail’s sandboxed environment, in order to keep their “apps” up-to-date?

1 Like

Wait, you guys manually run it?

1 Like

Here’s how I keep the software in my FreeBSD jails up-to-date with TrueNAS Core.

WARNING. THIS IS A VERY COMPLICATED PROCEDURE:

pkg update -f && pkg upgrade
2 Likes