Can't get networking functional in jailmkr

I’ve been using TrueNAS Core for a while and have a lot of iocage jails setup. It seems that Core will not be around forever prompting a need to switch over to Scale. My plan was to run Scale in an Oracle VM and learn how to get everything setup before switching over my TrueNAS Core server.

I’m running a VM of Scale Dragonfish-24.04.2. The Network is set to Bridge. Following @Stux tutorial on Setting up Sandboxes with Jailmaker. I cloned the jailmaker github.
and followed the directions in the Manual and installed docker with this config

startup=0
gpu_passthrough_intel=0
gpu_passthrough_nvidia=0
# Turning off seccomp filtering improves performance at the expense of security
seccomp=1

# Use bridge networking to provide an isolated network namespace,
# so docker can manage firewall rules
# Alternatively use --network-macvlan=eno1 instead of --network-bridge
# Ensure to change eno1/br1 to the interface name you want to use
# You may want to add additional options here, e.g. bind mounts
systemd_nspawn_user_args=--network-bridge=enp0s3
    --resolv-conf=bind-host
    --system-call-filter='add_key keyctl bpf'

# Script to run on the HOST before starting the jail
# Load kernel module and config kernel settings required for docker
pre_start_hook=#!/usr/bin/bash
    set -euo pipefail
    echo 'PRE_START_HOOK'
    echo 1 > /proc/sys/net/ipv4/ip_forward
    modprobe br_netfilter
    echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
    echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables

# Only used while creating the jail
distro=debian
release=bookworm

# Install docker inside the jail:
# https://docs.docker.com/engine/install/debian/#install-using-the-repository
# Will also install the NVIDIA Container Toolkit if gpu_passthrough_nvidia=1 during initial setup
# https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
initial_setup=#!/usr/bin/bash
    set -euo pipefail

    apt-get update && apt-get -y install ca-certificates curl
    install -m 0755 -d /etc/apt/keyrings
    curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
    chmod a+r /etc/apt/keyrings/docker.asc

    echo \
    "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
    $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
    tee /etc/apt/sources.list.d/docker.list > /dev/null
    
    apt-get update
    apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
    
    # The /usr/bin/nvidia-smi will be present when gpu_passthrough_nvidia=1
    if [ -f /usr/bin/nvidia-smi ]; then
        curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey -o /etc/apt/keyrings/nvidia.asc
        chmod a+r /etc/apt/keyrings/nvidia.asc
        curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
        sed 's#deb https://#deb [signed-by=/etc/apt/keyrings/nvidia.asc] https://#g' | \
        tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

        apt-get update
        apt-get install -y nvidia-container-toolkit

        nvidia-ctk runtime configure --runtime=docker
        systemctl restart docker
    fi

    docker info

# You generally will not need to change the options below
systemd_run_default_args=--property=KillMode=mixed
    --property=Type=notify
    --property=RestartForceExitStatus=133
    --property=SuccessExitStatus=133
    --property=Delegate=yes
    --property=TasksMax=infinity
    --collect
    --setenv=SYSTEMD_NSPAWN_LOCK=0

systemd_nspawn_default_args=--keep-unit
    --quiet
    --boot
    --bind-ro=/sys/module
    --inaccessible=/sys/module/apparmor 

enp0s3 is the network interface that shows up in the networks settings/dashboard.
When I go into the shell for the jail I get :

jlmkr shell docker  
Connected to machine docker. Press ^] three times within 1s to exit session.
root@docker:~# ping www.google.com
ping: www.google.com: Temporary failure in name resolution

What should my networking be set to in the docker config file?

This one is fairly easy, “jlmkr.py install” isn’t a valid command. I recommend you go back to the github and read the instructions.

I found that just after I posted. Jailmaker 2.0 removed the install function used in the video.

When running in a VM you often need to enable “promiscuous node” of snooping or something like that on the VM NIC so that the jail and vm can use customized MAC addresses.

1 Like

That’s not a bridge, you need to create the bridge. Stux’s video shows how to do it quite succinctly.

Edit: I’ll add that you would likely benefit from going over the official jailmaker documentation on networking, understanding the basics will be helpful in the future.

1 Like

Thanks for the link.

1 Like

I followed the video and setup the bridge interface but now I don’t have any connection to the internet.

ifconfig
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.5.43  netmask 255.255.255.0  broadcast 192.168.5.255
        ether d6:de:6a:da:95:87  txqueuelen 1000  (Ethernet)
        RX packets 8994  bytes 1271074 (1.2 MiB)
        RX errors 0  dropped 1391  overruns 0  frame 0
        TX packets 1171  bytes 219455 (214.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::a00:27ff:fec9:7a9a  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:c9:7a:9a  txqueuelen 1000  (Ethernet)
        RX packets 46525  bytes 18696714 (17.8 MiB)
        RX errors 0  dropped 185  overruns 0  frame 0
        TX packets 1875  bytes 262654 (256.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

kube-bridge: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet6 fe80::7027:29ff:fee9:9bad  prefixlen 64  scopeid 0x20<link>
        ether 0a:56:99:19:d9:25  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

kube-dummy-if: flags=195<UP,BROADCAST,RUNNING,NOARP>  mtu 1500
        inet 172.17.0.1  netmask 255.255.255.255  broadcast 0.0.0.0
        inet6 fe80::88de:f5ff:fefe:4344  prefixlen 64  scopeid 0x20<link>
        ether 8a:de:f5:fe:43:44  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 9  bytes 630 (630.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 43799  bytes 27225773 (25.9 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 43799  bytes 27225773 (25.9 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

In the shell when I ping google I get ping: www.google.com: Name or service not known

It seems you’ve failed to set a nameserver. If you don’t otherwise have one, Cloudflare’s at 1.1.1.1 and 1.0.0.1 work well.

My nameserver is the same in my TrueNAS Core server and it works fine. It’s the ip of my PFSense router.

Your nameserver isn’t set at all in the screen shot you posted. You need to set it.

2 Likes

Why have you set up a Static Route? That is not mentioned in the video by Stux.

While it doesn’t necessarily look wrong, it shouldn’t be needed and may complicate things in the future if your network configuration changes.

Definitely add a name server. The IP of your router will, in a typical home environment, work just fine.

1 Like

Thanks I deleted it.