24.10 Electric Eel Apps Networking

Apps and Virtualization
,
1

I am doing some tests in preparation for my 24.10 upgrade. I installed a test instance of 24.10 in a VM and have been playing around with it.

My current implementation has my apps running on one separate network (10.0.70.0/24) and the normal truenas system running on its own (192.168.50.0/24). 24.04 had settings to specify the gateway for apps, so it used the apps network without issues.

I am not seeing a place to specify the network interface for apps. Assuming I am going to have to manually specify it for each app via a custom compose file?

1 Like
2

I would like to know this as well! :+1:

3

The 24.04 and previous releases used an implementation of Kubernetes which simply uses subnets and routing, which requires a default gateway to route the subnets (does not use NAT).

Docker also uses subnets, but with NAT, from the containers to the LAN.
NAT does not need a gateway as it uses the host’s ip-address (which already has a default gateway defined) to open connections to the LAN, so that should work for most apps, except the ones that, like all NAT services, need special arrangements when they need to respond to incoming connections (servers).

That’s when you use network type host (use host’s ip-address, but need to watch for all the other listening ports used by the host), or network type macvlan (add a new bridge interface with dedicated mac address and ip-address).

So that didn’t say much more than test and learn or find a recipe from somebody else.

4

The docker daemon uses truenas default gateway by default…

You could edit it
/etc/docker/daemon.json

I am sure IX would say THAT is unsupported… but it is an option

Option 2 would be use a MACVLAN or IPVLAN (L2 or L3) if you went IPVLAN L3 and put your containers on that you could then just route their IPs however you see fit at your router

So recap

Option 1 edit the docker daemon.json

Option 2 use docker container networking (below is the guide I used to learn it)

****In that video you will learn basically how to make containers network however you want…

2 Likes
5

I guess my question is, how do i specify docker to use a specific network adapter on the host? I want it to run on that 10.0.70.x/24 network and network adapter instead of the normal web ui and SMB adapter that is 192.168.x.x

6

That specifically would probably be in the /etc/docker/daemon.json

Which you probably should not do… and it would be universal to all containers…

But I have found a solution

Create an IPVLAN network, use the video for a full explanation of how that work also this is the option you want… PARENT ALLOW network adapter selection

image
image949Ă—752 58.3 KB

Full reference

That way you would avoid something unsupported and have full functionality… Each Container on that IPVLAN would be on your second network adapters network, make sure you watch the video about how to handle IP address assignment in this type of network

You are certainly using a more advanced option

7

For the simpler solution, use host network.

Docker compose
network_mode: host

A Docker gotcha is the fact that by design, docker containers cannot reach the Docker host/server.

The workaround is to create a macvlan bridge in the Docker host/server and add a route to the Docker IP.

ip link add macvlan-shim link xx type macvlan mode bridge
ip add ip-docker-cidr dev macvlan-shim
ip link set macvlan-shim up
ip route add ip-docker-cidr dev macvlan-shim

That should save you hours/days when you need to be able to check/ping the Docker host/server.

8

Due to the complexity of my network, I would rather not use IPVLAN as it would mean that I would have to maintain routes. I also only want to expose specific ports of certain containers only and not the entire container.

I ended up doing some more googling and research and got ChatGPT involved.

I am still in the process of testing this solution, but so far it appears to be working the way I want it to be.

I created a new bridge network and specified it to use the interface on the 10.0.70.0 network and then I attached my containers to this network.

grep -qxF '200 docker_traffic' /etc/iproute2/rt_tables || echo '200 docker_traffic' | sudo tee -a /etc/iproute2/rt_tables

# Set up routing rule and default route for Docker network
ip rule add from 192.168.101.0/24 lookup docker_traffic
ip route add default via 10.0.70.1 dev <parent_interface> table docker_traffic

# Enable NAT for Docker bridge traffic
iptables -t nat -A POSTROUTING -s 192.168.101.0/24 -o <parent_interface> -j MASQUERADE

The settings are not persistent across reboots, so it will have to be a startup script.

When doing a traceroute with a busybox container attached to this network, I am seeing traffic go out the interface I want. I also see that it is not able to interact with the rest of my network on the other interface or other subnets which is also what I want.

However, I am assuming this is not a supported solution and I have no idea what the long term effects will be yet. Currently just testing in a VM as I haven’t upgraded my production system yet as I need security and segmentation on my containers before I can do that.

9

So far this solution has been working perfectly. I have tested standard containers, VPN based containers, and reverse proxy containers and all works. It may not be a supported solution, but it is simple enough that it is worth the risk to me.