Accessing "shares" from Clients and VMs (with multiple interfaces in different VLANs)

TrueNAS General
, , , ,
1

Hello everyone,

I would like to use some swarm intelligence for my problem because I keep getting the same result. I’ve already searched google and the forum but couldn’t find the right solution. If there is already a suitable thread and I couldn’t find it due to my search skills, I would be very happy about a link to the correct thread.

So that I don’t continue to beat around the bush, now to my actual problem (2.1 Problem explanation:).

In the following I will first try to describe my structure as precisely as possible.

My physical network left to right starting from the gateway:

Infrastructure
Infrastructure1021×616 46.4 KB

|| VLAN 20 = Storage Management = 172.31.20.0/24
|| VLAN 30 = Hypervisor Management = 172.31.30.0/24
|| VLAN 100 = Clients = 172.31.100.0/24
|| VLAN 150 = NFS = 172.31.150.0/24
|| VLAN 160 = CIFS/SMB = 172.31.160.0/24

TrueNAS Scale Intrefaces & Configuration:

enp38s0:
enp39s0:
bond0: {enp38s0, enp39s0}
br0: {bond0}
vlan20: {bond0} [..20.10]
vlan150: {bond0} [..150.10]
vlan160: {bond0} [..160.10]

1. Requirement/Problem:
The TrueNAS Scale interface in the CIFS/SMB network, which provides CIFS/SMB shares, must be made accessible by the Windows client.

1.1 Problem explanation:
In the current configuration, the TrueNAS Scale has already set the default gateway [..20.254] on the storage management interface in order to reach the web interface. That’s why there is currently “asyncrone routing” when connecting the TrueNAS Scale CIFS/SMB shares.

Client connection request:
[..100.100] → [..100.254] → [..160.10]
TruneNAS Scale response:
[..20.10] → [..20.254] → [..100.100]

1.2 Solution:
In order to correct the routing, a gateway must also be set for the TrueNAS Scale interface in the CIFS/SMB network. This can be done using the following commands in the Linux Shell of the TrueNAS Scale.

  • Show current routing table:
    root@truenasscale[~]# route -n

  • Set another route:
    root@truenasscale[~]# route add default gw {Gateway-IP} {Interface}

  • Check routing table:
    root@truenasscale[~]# route -n

2. Requirement/Problem:
The TrueNAS Scale interface on the NFS network that provides NFS shares must be exposed by the various hypervisors (XCP-ng02 & nested XCP-ng03).

2.1 Problem explanation:
The NFS share can be reached and accessed by the XCP-ng02, but not by the XCP-ng03, which runs nested on the TrueNAS scale.
When I run a TCPdump on the TrueNAS Scale, I see incoming packets tagged in VLAN 150 on both the interface “br0” and interface “bond0”, which are sent by the XCP-ng03. However, the packets are not passed on to the “vlan150” interface because they cannot be seen there using TCPdump.

2.2 Solution:
Unfortunately I don’t have it because I’m kind of stuck. Do any of you have an idea or analysis approach to find a solution for this too?
The only approach I would have would be to set the parent interface to “br0.150” for the interface “vlan150”, but unfortunately this is not possible via the GUI or CLI, there I only have the preselection “bond0” or one of those Physical adapter.

*** COPY from OLD-Forum ***

2

It would seem you should have clients, SMB, NFS on the same subnet and VLAN

You get to choose one default gateway, but you can specify static routes. These should on the same subnet as your preferred egress VLAN.

3

Yes but no…

Of course, you could assign an IP to the Truenas in every VLAN/network that is local to a location and thus bypass routing, but as soon as you have more than two networks, the configuration overhead increases enormously. In addition, it is completely against compliance guidelines if you do network segmentation, in which you explicitly chase traffic through a firewall or something similar.

Unfortunately, static routes for each network are not practical in larger environments. In addition, what should the traffic look like if a client accesses the MGMT ip and that of the share from the same source network. In my mind the result would also be asynchronous routing again.

In my opinion, the only correct solution to “1. Requirement/Problem:” is the one I described by setting a second default gateway, through which the traffic is sent out again to the interface through which it was received. (independent of the source network)

4

NFS Issues:

You are virtualizing XCP-ng inside of the TrueNAS? That’s rather an odd deployment strategy, but I guess you can do that. Either way, the problem you have is that you’d need a network bridge. Please see this thread:

SMB Issues:
I think you may be making this more complicated than it has to be. Having L2 adjacencies with a VLAN 100 tag going to the TrueNAS for an additional interface is the “easiest” way, but it’s certainly not a hard requirement.

This is not async routing:

This is expected behavior, TrueNAS SCALE’s IP address is in VLAN 20. It is routing through the routers interface in VLAN 20, as is the client’s but in VLAN 100. Since you are doing traceroutes from the TrueNAS, it will do them over wherever the default gateway is, as will the client.

In other words, the client is configured to talk to the TrueNAS on a different VLAN than the management interface, so of course the traceroute from the TrueNAS to the client will be different.

You can set up a static route from VLAN 160 to VLAN 100 on the TrueNAS if you want to:
Configuring Static Routes | (truenas.com)
But as it stands TrueNAS’s network adapter in VLAN 160 has no idea a router exists on VLAN 160, and as such would only be able to bidirectionally communicate with devices in that broadcast domain. This is common practice for iSCSI networks, where the servers are all L2 adjacent in a datacenter.

In other words, if you want clients to talk to the TrueNAS on VLAN 160 that are not in VLAN 160, you’d need a static route.

Another approach might be that you’d move the default gateway on TrueNAS to VLAN 160, and then only manage the TrueNAS from a device in VLAN 20.