I have recently installed TrueNAS Scale 25.10.1
Currently the server has 2 IP addresses. One IP on10.10.10.0/24 and one IP on 10.10.20.0/24.
10.10.10.0/24 is where all client devices are and the TrueNAS’s default gateway is also on this network. 10.10.20.0/24 is used exclusively for Management interfaces of routers, switches etc.
From a device on the 10.10.10.0/24 I can connect to the TrueNAS GUI and all is fine but when I connect to the GUI on the 10.10.20.0/24 IP I can login and things are OK for a few seconds then the GUI shows a message that it has lost connection to the host and soon I am thrown out of the session, I can login back in again but things just repeat.
This looks like TrueNAS is accepting traffic in on the 10.10.20.0/24 network but then sending back via the default route which is on the 10.10.10.0/24 network.
I have found some info that I can add a post init script which configures underlying Linux to use a specific interface (NIC, bridge, etc) for out going traffic which comes in on a specific IP.
Something like this:
ip rule add from 10.10.20.200/32 table 100
ip route add 10.10.20.0/24 dev br2 table 100
ip route add default via 10.10.20.1 dev br2 table 100
In the example above 10.10.20.200 is my TrueNAS management interface and 10.10.20.1 is the gateway.
Hoping someone can confirm this will solve this issue and not break things?
The server has two IP addresses on one network interface, or on two?
If you were able to log in, the routing works in general - this is all TCP and without a bi-directional communication there is no connection. There is something else going on.
One IP address is on a VLAN interface and the other is on a Bridge interface, both of which are separate physical NIC ports.
As I understand it, maybe incorrectly…, accessing the GUI and the login works because TCP is stateless but the GUI session falls apart because websocket/ API calls get replies from the other interface (one with a gateway configured in GUI) that invalidates the session and the heartbeat fails forcing a login again.
There is no difference between a regular webpage and a websocket from the networking point of view; both are (at least here) TCP and in fact a websocket connection starts as an http one, that is then “upgraded” to keep the connection but to switch the protocol from http to ws. That’s why I am more inclined to suspect something transported through one or another protocol, than the IP layer…
Firing up the browser’s debugging tools and looking at the traffic might help.
The following solved it for me so am posting here in case someone else has the same issue.
In shell as root (sudo -i) I used these commands:
ip rule add from 10.1.60.200/32 table 100 || true ip route add 10.1.60.0/24 dev br2 table 100 || true ip route add 192.168.5.0/24 via 10.1.60.1 dev br2 table 100 || true
10.1.60.200/32 is the TrueNAS management interface IP, 10.1.60.0/24 is the management subnet, 192.168.5.0/24 is the subnet where the client device is and 10.1.60.1 is the gateway IP on the management subnet. BR2 is my management interface.
Also, these have been added as post init commands and work after a reboot but have to wait and see if they persist after a TrueNAS upgrade.
Still, something does not add up. If your client is on 192.168.5.0/24 and you are connecting to the TrueNAS’s address on the same subnet, the 10.* and the default route shouldn’t get involved at all. If an interface that is member of the 192.168.5.0/24 does not get a route to that network through itself when being configured through the GUI, that should be IMO reported as a bug. I can also imagine that the Web GUI attempts to open the Websocket to the “wrong” address; that would also be a bug worth reporting.
Of course, if the client is on a different subnet, using some explicit route to reach the TrueNAS without going through a router, i.e. expecting some routing functionality from the TrueNAS, that would be a different story.
