Core and Kubernetes traffic should be routed to the assigned vlan (and NOT to the default gateway)

My TrueNAS scale system is connected via a trunk towards my network. The trunk does contain multiple vlans.

Traffic arriving from a certain vlan should be answered via the same vlan. And that is NOT what is happening :grimacing:

Traffic related to the TrueNas core and traffic related to Kubernetes are returned via the default route as defined in the network settings. And that is definitively NOT OK in most cases.
(Luckily it does work correctly for VM’s)

As example traffic related to:

  • then management vlan should have its own route
  • the same for e.g. iSCSI or SMB
  • ths same for apps (Kubernetes)
    In this example there are three incoming vlan’s and the traffic should be returned to the incoming / the assigned vlan and definitively NOT to the default gateway.

The fact that you can define static routes does not help at all, since in all cases the destination address could be any thing. Could even be some address on the internet.

So traffic related to a certain vlan should be routed to that vlan (level2) and routed towards the IPV4 and or IPV6 gateway as related to that vlan.

If not there is a routing, security issue asymetric routing issue.

So at it is now:

  • I have to put all TrueNas core functionallity in the same vlan (what I do not like)
  • Using APS is not even an option
  • Luckely VM’s do route the correct way :grinning:

As far as I am aware I can not fix this in the actual TrueNas version. But I hope I am wrong …

Here’s a good summary of what you’re looking for:

Yes that is exactly the problem and how it needs to be solved. More exactly how it should be done by default !!!

I will read the article in more default and try to implement multiple routing tables. I hope I get it up and running.

However I stick to my vision that this is the one and only proper routing scheme and it should be the default way to configure the NAS and of course via the GUI !

Thanks for the link!

I don’t think you’ll get a lot of traction with your “one and only” scheme. There are some good articles out there that explain why stateless routing is necessary and useful, in general.

It’s your specific use case that is at odds with how the internet works.

VLAN’s are there to separate traffic, routing traffic from one VLAN into another VLAN is IMHO against every security principle !

Next to that asymmetric routing, firewall conflicts etc etc

I did some tests based on these sites:

But up to now I was not successful

I tried to route the traffic related to Kubernetes to the intended vlan

  • Kubernetes is supposed to communicate to the outside world via vlan 100.
  • TrueNas core is communicating via vlan 18
  • my VM’s are communicating via other vlans (which does work)

In my actual setup:

  • In the past I did assign vlan18 & br18 to truenas itself. As a result the default IPV4 gateway is 192.168.18.1
  • then I defined Kubernetes with interface br100 and gateway 192.168.100.1 which is vlan100
  • the network screen is showing two default routes, the intended one (192.168.18.1) and the Kubernetes specific one 192.168.100.1 which IMHO is in this regard not related (!)
  • traffic incoming via vlan100 destination Kubernetes is answered via vlan18 / the overall default gateway. This does NOT work and is NOT secure.

So in ^rt_tables^ at the bottom I added ^100 applications^ and defined the application table using
sudo ip route add 192.168.100.0/24 dev br100 table applications
sudo ip route add default via 192.168.100.1 dev br100 table applications
/etc/init.d/networking restart

That did not have effect. Appart from the fact that that setting is gone after a reboot

Trying a second option

create a file ^my_routes^

/etc/network/if-up.d/my_routes

#!/bin/sh

if [ “$IFACE” = “enp4s0d1” ]; then
ip route add 192.168.100.0/24 via 192.168.100.1
fi

chmod 751 my_routes

/etc/init.d/networking restart

Also this option does not seem to work

I also tried to remove the ^192.168.100.1" default route as shown on the network screen, but I still do not know how

For these try’s I did install jellyfin as app in Kubernetes. And tried to access jellyfin via my browser. My firewall (pfSense) shows very clearly that the answers where coming back via vlan18 and NOT via vlan100. These answers where of course blocked by the firewall

So I hope someone knows how to set up the needed routing