I am doing some tests in preparation for my 24.10 upgrade. I installed a test instance of 24.10 in a VM and have been playing around with it.
My current implementation has my apps running on one separate network (10.0.70.0/24) and the normal truenas system running on its own (192.168.50.0/24). 24.04 had settings to specify the gateway for apps, so it used the apps network without issues.
I am not seeing a place to specify the network interface for apps. Assuming I am going to have to manually specify it for each app via a custom compose file?
The 24.04 and previous releases used an implementation of Kubernetes which simply uses subnets and routing, which requires a default gateway to route the subnets (does not use NAT).
Docker also uses subnets, but with NAT, from the containers to the LAN.
NAT does not need a gateway as it uses the host’s ip-address (which already has a default gateway defined) to open connections to the LAN, so that should work for most apps, except the ones that, like all NAT services, need special arrangements when they need to respond to incoming connections (servers).
That’s when you use network type host (use host’s ip-address, but need to watch for all the other listening ports used by the host), or network type macvlan (add a new bridge interface with dedicated mac address and ip-address).
So that didn’t say much more than test and learn or find a recipe from somebody else.
The docker daemon uses truenas default gateway by default…
You could edit it
/etc/docker/daemon.json
I am sure IX would say THAT is unsupported… but it is an option
Option 2 would be use a MACVLAN or IPVLAN (L2 or L3) if you went IPVLAN L3 and put your containers on that you could then just route their IPs however you see fit at your router
So recap
Option 1 edit the docker daemon.json
Option 2 use docker container networking (below is the guide I used to learn it)
****In that video you will learn basically how to make containers network however you want…
I guess my question is, how do i specify docker to use a specific network adapter on the host? I want it to run on that 10.0.70.x/24 network and network adapter instead of the normal web ui and SMB adapter that is 192.168.x.x
That specifically would probably be in the /etc/docker/daemon.json
Which you probably should not do… and it would be universal to all containers…
But I have found a solution
Create an IPVLAN network, use the video for a full explanation of how that work also this is the option you want… PARENT ALLOW network adapter selection
That way you would avoid something unsupported and have full functionality… Each Container on that IPVLAN would be on your second network adapters network, make sure you watch the video about how to handle IP address assignment in this type of network
A Docker gotcha is the fact that by design, docker containers cannot reach the Docker host/server.
The workaround is to create a macvlan bridge in the Docker host/server and add a route to the Docker IP.
ip link add macvlan-shim link xx type macvlan mode bridge
ip add ip-docker-cidr dev macvlan-shim
ip link set macvlan-shim up
ip route add ip-docker-cidr dev macvlan-shim
That should save you hours/days when you need to be able to check/ping the Docker host/server.