Two NIC's two different subnets

TrueNAS General
1

Hi,

Just to start off, i got to be honest and say that i am not too familiar with Linux/Unix. I have a couple of Linux servers, but that’s about it. So please bear with me. English is also not my primary language.

To get started on my issue it pretty much boils down to me wanting two seperate network interfaces. 1 for management (on my management VLAN 120) and 1 for clients connecting (on my internal VLAN 150).

My plan is for the one on the internal vlan to be aggregated with LACP. I have a 4-port Intel NIC, and a UniFi Dream Machine Pro Max and 24 port UniFi PoE switch. All of this will be connected to my 24 port switch.

The problem i have come accross is that as soon as i set a static IP (or Alias as it is called TrueNAS Scale) on bond0 which is my aggregated link i lose connection to the management interface. I can see that i no longer am able to ping 192.168.120.97 and hence i am not able to access the WebUI.
After much troubleshooting i was thinking that this was a problem with either the 4-port NIC or the setup of the aggregation. However i tried this with the second onboard NIC without any other configuration apart from unchecking DHCP and setting an alias of 192.168.150.140. I still lose connection to the management NIC.

I then set an alias on my management NIC to 192.168.120.97 instead of DHCP, and then set the WebUI to bind to the IP 192.168.120.97. But still i lose the connection to my management NIC.
The settings reverts after 60 seconds if you don’t confirm the setting, so i thought that it may be that i have to wait for like 2-3 minutes. So i tried setting it manually via a keyboard on my computer. But still the same issue.

Since i have become pretty desperate, i have even tested a full reinstall wihtout importing any config. But you guessed it, it still won’t work.

Am i crazy, or just really stupid?

I would also like to note that the standard gateway has been set to 192.168.120.1 and i have also tested with setting up a static route (via the GUI) with destination: 192.168.120.0/24 and gateway: 192.168.120.1.

Anyone know what could be causing this issue, or is it simply just not possible to set it up how i have planned? Thanks in advance!

2

I believe you need to make a bridge. I am assuming you want both NIC’s on the same ip scheme/subnet? What i did on mine is a made a bridge, and assigned it the ip/s i wanted, and removed the ip’s configured to each NIC. If you are doing this from the web gui, you kinda have to do it in one move, so you dont lose access to the gui.

I am still figuring it out since I am new, but I was having an issue with multiple nics, and what I read and got to work is…

#1. add the bridge to the network interfaces. Turn off DHCP. Assign which nic’s you want as part of that bridge. Add ip aliases to the bridge "all the static ips you want to reach the server.

#2. Before you test/save the configuration, you need to remove and ip address assigned to the NIC’s

#3. Test, and if all works, Save.

I have 2 NIC’s which is have hooked to one switch, NOT link agrigation, and I can access the GUI from any of the IP allias’s I have assigned to the BRIDGE.

I think this should work for what I am understanding you want to do. You can have 1 NIC attached to a vlan port of your choosing for management, pick any ip you would like to use, and if you have the other NIC on a seperate port with different vlan, you should be able to access the NAS using any ip or you can assign the GUI access under general settings to a specfic ip.

1 Like
3

OR…
after reading your post again, did you want…

One NIC to be on 1 IP scheme, i.e.

192.168.1.x - management ip on seperate switch port vlan 1
255.255.255.0 - SUB
192.168.1.1 - DG

and One NIC to be on a different IP scheme,

192.168.2.x - NAS access on seperate switch port vlan 2
255.255.255.0 - SUB

4

Hi. Yes, my management interface would be on my management VLAN 120 (192.168.120.97) and for storage/4-port NIC on VLAN 150 (192.168.150.140), but if possible i could have it on the same IP.

Problem is that i got an error message when trying to put two interfaces on the same subnet. I have not tried using bridge, but will test it out.

Thanks!

5

if the ip scheme is different, subnet should not matter. Subnets are older versions of vlan… kindof… a 255.255.255.0 subnet means the ip traffic can only stay on its native ip scheme, i.e. 192.168.1.x traffic can only talk to 192.168.1.x traffic. But if you have a 192.168.2.x it cal also have a 255.255.255.0 subnet and be perfectly fine.

With a 255.255.255.0, your ip scheme is limited to 256 ip address “sort of”, 255.255.254.0 now takes it to 512 “sort of”

Now if you had two NIC’s on different ip schemes, i.e. 192.168.0.x, and 192.168.1.x, you can assign 255.255.254.0 subnet to both nics, and you would be able to access ip on both networks as long as any device on there perspective network has 255.255.254.0 as its subnet. The same runs in reverse as well. Lets say you have a server on a network, and you only want a couple people to access it, you can assign the server a 255.255.255.252 subnet with an ip of 192.168.0.1. only ip’s up to 192.168.1.0-3 would have access to the server.

VLANs are a simpler way to do this, but you now can use software on network hardware to define traffic patterns without having to modify the ip scheme of a specfic device. The only problems are you are limited to 256 ip address’s with out modifying the devices network settings.

So for your problem, forget the bridge at the moment. I would first disconect the secondary NIC, the 150 network, and set your primary interface up for 192.168.120.97, DG 192.168.120.1 “I am assuming that is your router?”, your subnet to 255.255.255.0, and your DNS. Get yourself back in, and assign your management IP to the .97. Once you are back in, before you connect your secondary NIC, put all the settings in, 192.168.150.140, DG 192.168.150.1?? do you have a DHCP server for this ip scheme? If not, you really dont need a DG on this scheme if everything is static, and then set the subnet 255.255.255.0.

This should get you online. Let me know, if it does not work, I can test bench a system with multiple NICs and run you through it.

6

For the secondary NIC, how would i go about setting DG 192.168.150.1? I can only see the global setting for DG and that is set to 192.168.120.1 for my primary NIC.

Unfortunatly not able to upload screenshots or links to screenshots, but I can’t see any setting in a NIC to have a specific DG, only Alias (IP and subnet)

7

Correct, the DG is NOT set per NIC, so just leave it alone. If you have the 2nd ip network with another router running DHCP, then all the other traffic will get the DG to it, the DG is really only neccesary for internet access.

8

So on my management NIC (enp1s0) i have set an alias of 192.168.120.97. As soon as i set alias (192.168.150.140) on the second NIC (eno1) i lose connection to enp1s0. What do you think the solution would be here? Are you able to replicate the same issue?

9

Hmm… Let me fiddle with my other server. I will set it up for 2 seperate networks and see what I can see. it wont me 120 and 150, but that should not matter. Ill get back to you later.

10

Thanks man! Really appreciate you taking time to look at this for me.

1 Like
11

ok… it was bugging me, so I went and messed with it.

I can NOT replicate the issue you are having. I have 3 NICs in this machine and I put them in all different networks with /24 subnets which is 255.255.255.0 and they worked right away. I am wondering if it is a setting with the switch. You might want to try one network maybe the management network on a different switch, and the other switch for the 2nd NIC.

Multi NIC
Multi NIC1570×998 32.5 KB

The only issue I was getting was an error trying to put a /23 or /22 subnet on the NIC. It basiclly said it was already in use which makes since since I have other NIC’s assigned to single subnets.

You COULD use just one NIC, and assign both ip alias’s to the NIC or I would use a bridge and use a wide subnet. That way it will have access to both networks. That NIC will have to be part of BOTH vlans, but it will not cause a conflict if it is set to static. I would also narrow you ip’s, 120 to 150 is WIDE and you would have to use a huge subnet. I would use say 120, 121 if possible or maybe 115,120. You this calulator to figure out which subnet you will need if that the way you want to go.

I set my bridge to do this. I lost connection one time, so I did it step by step i.e., set my computer nic with the new subnet, test, set my ip to my managment network, test, save, change my subnet, test, save, add second alias ip, test, save. I was able to ping from both sides no problem so it seems to be all good!

bridge
bridge1559×838 27.1 KB

Cmd
Cmd816×993 25.1 KB

Do you lose connection if the 2nd NIC is unpluged when you assign settings? OR is it you lose connection when the 2nd NIC is plugged in and then you assign settings?

12

I’m so sorry — this is not directly relevant to the OP — but I wouldn’t be able to sleep. :nerd_face:

192.168.1.1 and 192.168.2.2 are not in the same subnet by mask 255.255.254.0.

192.168.0.0/23 spans from 192.168.0.0 - 192.168.1.255.
192.168.2.0/23 spans from 192.168.2.0 - 192.168.3.255.

I have no experience with LACP, so I can’t be of much use to the actual topic. Good luck! :dizzy:

13

Yes you are correct, it has been a while since I played with subs, which is why I linked the calculator. Also if you look at my screenshots, I am using /22. But good eye.

1 Like
14

Hmm… Very weird that this works for you. May i ask what version of TrueNAS Scale you are using?

I have tried with the second NIC being both connected and disconnected. Either way it disconnects.

BUT. Now i found out if i set the IP “192.168.151.140” (domain VLAN) instead of “192.168.150.140” it does not disconnect the management NIC.
Starting to think this a bug with a conflict for networks inside 192.168.150.0/24…

15

Can you try adding a 150 VLAN on your network for testing, and set your second NIC as 192.168.150.140?

16

I am using the current version 24.10 on my secondary server which is the one i play with.

Question…

I am still a newer transplant to truenas so, I am still working through all of the ways to configure it. How do you have the VLAN’s setup? is it by port on your switch? There is an option under Truenas to use vlan’s, so have you tried to use those?

It seems to me you would have your NIC’s and like a bridge, the vlan option sits on top of the NIC. So instead of assigning the IP’s to the NIC itself, you would setup a VLAN, and assign the alias “IP” to it and then tell it which specfic NIC you want to use it with with the vlan.

Next question, is your management pc, tagged to the 120 vlan before you set the NAS either by the port on the switch or the device itself? if it is just hooked to an access port without a vlan tag, that could be the reason you are losing access.

17
  1. Where do you set a VLAN ID in TrueNAS? I didn’t know you could make the OS VLAN-aware
  2. I have UniFi 24 port switch where the management NIC is currently connected to a tagged port with VLAN 120. However i think this is a bit irrelevant as i am able to set the secondary NIC to whatever VLAN i have on my UDM except for VLAN 150. The secondary NIC isn’t even connected to the switch. It’s not in use, just using it as a test.
18

Update: I got VLAN setup for the management NIC, but still no reply from ping as soon as i change the secondary NIC to IP: 192.168.150.140. All other IP-adresses assigned that is not in VLAN 150 works just fine.

19

Ah — then I think you might be misinterpreting UniFi’s “tagged” VLAN terminology?

As I understand it, a UniFi port works with VLAN clients in two forms:

  • Ordinary devices send regular ethernet packets; UniFi implicitly assigns these to that port’s designated “native VLAN”.
  • VLAN-aware devices might explicitly tag their own packets to a specific VLAN.

So as you configure a UniFi port, make sure to allow your full set of “tagged VLAN” packets (as configured in the TrueNAS interface), in addition to choosing how it handles untagged packets (as set through the UniFi port’s “native VLAN” setting).

20

This is weird though, because as it seems, the NIC which he is using to gain access to the GUI works fine even as set static ip, and subnet, but this issue comes from when he is trying to cinfigure the secondary nic, which is not even hooked to the switch. So some part of the truenas network config is causing all to drop out even if what he is configuring is not what he is connected through.

This is kind of a head scratcher. I am trying to get a good understanding of his topology, but it seems more and more that it is somthing in the server… Just some of the things stated are even stranger, like why would 151.x work but not 150.x on an unpluged interface…

IDK…

1 Like