I’m wondering if someone can help me configure my network interfaces in TrueNAS Scale because I seem to be tying myself in knots and am being disconnected from the UI each time I test a change even though I think I should have access.
This is my current network config post truenas install
A bond for enp6s0 and enp8s0 with IP 172.16.94.x as these two NICs are connected to an air-gapped switch that runs my proxmox cluster traffic.
A bond for enp9s0 and enp7s0 with IP 192.168.86.x1 for the connection to my main lan for storage shares
Assign enp13s0 with IP 192.168.86.x2 for management and maybe some other tasks, I think this is my onboard one so it’ll also be used for WOL.
When I go to make a change to one of the network interfaces, I need to disable DHCP for that nic (which makes sense) and then it asks me to provide a new default gateway (which I provide as the existing gateway that the system already knows( for some reason it seems to delete the existing one if it’s not provided) but I’m unclear why that apparently disables DHCP for the other NICs (really I think that should be in the Global settings if that’s intended behaviour) and becuase of that I then lose connection to TrueNAS UI.
Another thing that appears to happen is if I create a new bond or bridge with multiple NICs assigned it appears to reset any prior configuration I’ve made, i.e. I set a static ip 192.168.86.6 for enp13s0 and I could ping and access the UI from that ip but as soon as I tried to make the bond for enp6s0 and enp8s0 with the chosen IP it wiped my prior config on enp13s0.
I must be misunderstanding something in the Scale LAGG setup docs but for the life of me I can’t figure it out. Any help would be greatly appreciated as I’ve been struggling with this issue for almost a month now.
Okay thanks for letting me know. I gave that article a read pretty much as soon as I woke up for not sure how much I absorbed, I’ll give it another read latter. My takeaway from that though is:
my post-install TrueNAS is weird as all of my NICS have been given IPs that are on the same subnet
I’d be better making a bond for all 3 of those NICS that I want to be on this subnet ( enp9s0, enp7s0 and enp13s0), I can’t use a LAG group because the switch they are connected to doesn’t support it.
If your switch doesn’t support LACP you cannot configure a bond interface. Well, you can, but that will lead to all sorts of problems. TCP does not like reordered packets. Also the number of ports in a LAG/bond should be a power of 2.
In the screenshot it looks like you put all 5 NICs in the same subnet, which is what arwen pointed out one should not do.
Also, make a new post if you make changes and want to post an update.
Directly editing the original post makes the whole thread very difficult to follow.
Oh I didn’t do that, That is how TrueNAS itself configured the networking post the installation of the OS. I’ve been trying to wrap my head around how to configure the networking so I’ve just not made any permanent changes yet.
I didn’t think it justified a new post because I wasn’t changing the core part of my post in any way and the only reason I hadn’t provided that information previously was due to my trust (or more like untrusted) level. My only edits were to clarify information in the post by the way of:
a screenshot
adding a link to the specific part of the documentation I had been referring to
That said, I can see how changing some details after the posts creation can be hard to follow.
Oh, okay. My understanding was always that for me to setup a LAG bond, where I get two pipes for my traffic, my switch needs to support that protocol but that for me to configure a bond where the two interfaces work sort of in a failover setup then my switch doesn’t need to sup[port anything special. Maybe I’m thinking of bridge instead of a bond?
Okay that’s good to know. That would mean I could only really use 4/5 of my NICs then? 2 of that air-gapped network and two for my main Lan.
If I could I will try to not over simplify it.
Each device/computer/server can have only One physical Nic per subnet. (reason for this is you’ll generate an Arp loop and crash you network)
If you are using mulitple nics to load balance/LACP/LAGG you will need a managed switch, then to apply them by powers of 2 ie 2,4,8 etc…
The Managed switch will allow you to group the ports you are bonding.
The other way to use multiple nics on a server is to put them on different subnets and bridge them. this can make routing complicated. where you will have to make routing paths to each subnet and metrics for failover. Believe me you do not want to d o it this way.
If you need the bandwidth with failover then I suggest getting a managed switch and aggregate the ports.
The point is you need to be clear in what you want to achieve.
I use subnetted vlans on a 2x 10ge LACP / bond. I have my reasons for separating subnets for smarthome devices and Wi-Fi connections with a single DHCP service to manage them all.
