Asymetric routing issues with apps on different vlans

Apps and Virtualization
1

I have my

  • truenas scale (smb/web) on vlan20 over lagg0
  • jellyfin app on vlan21 over lagg0
  • rustdesk app on vlan22 over lagg0

lagg0 is connected to pfsense
A client on vlan23 (192.168.23.54) is trying to connect to:
192.168.22.25:21114
The packet reaches TrueNAS on vlan22:
vlan22 In IP 192.168.23.54 → 192.168.22.25
But then TrueNAS generates:
ICMP host 192.168.22.25 unreachable
using source address:
192.168.20.25

How can you have apps on vlans if they reply via the default route interface?
Doesn’t this make the whole concept of having containers redundant?

2

Simple answer: you can’t. Apps will always use the default route configured in the truenas network settings. There are aldready multiple posts about it and as far as i can remember noone found a solution.

3

grok says to use Portainer so a custom docker-compose.yml can be created to support direct macvlan can be created instead of the default bridge docker NAT. Any ideas if that makes sense?

4

Maybe, i don’t use vlans so i can’t comment on wether portainer and your own compose files would solve the issue.

5

Correct. Many people have an idea, but they simply don’t really know.

It is possible, but you have to get deeper into Docker networking.

I would recommend doing it in the docker compose file and not through the Docker manager.

That way you can change Docker Manager, or not even need one!

(I never liked Portainer. Used Dockge until I found Dockhand).