Best Practices - Network Interfaces

TrueNAS General
,
1

Hi all,
As I’m not a Linux expert, but more intermediate, I recently learned the subtilities of the bridge interfaces and I did not configured my TrueNAS server using them.
I curently have this network hardware setup:
3x 1Gb interfaces
2x 10Gb Interfaces

I created a bond for my 3 Gb interfaces
I also created a bond for my 2x Gb interfaces (will probably separate them soon)
And all my IPs are assigned to my bond interfaces.

Now my question, what are the best practices ? Sould I still keep this configuration ? or add bridges interfaces (for my 1GB bond) and my 2x “futures” 10Gb interfaces ?

My usage is:
Using 1Gb interfaces for “normal” traffic
Using 1x 10Gb interface dedicated to communication with my Proxmox cluster (SMB/NFS storage)

Or should I add bridges ?
1 br on my 1Gb interfaces for normal traffic
1 br on my 1Gb interfaces for my apps traffic
and sould I use (or not) on my 10Gb dedicated to proxmox or still pass the IP directly to the interface?
and very important, why ? (as I want to learn and not only apply config)

Thanks a lot

2

I think it depends on what you are trying to do…

The simplest approach is have one primary IP address. Until June 1st Apps are on th same IP as the NAS.

3

Thanks for your reply, even if I do not have more clarification :frowning:
but my apps already run on a different IP than my NAS, my interface have 2 IP and one is dedicated to apps.

4

Unless you explain what you want to achieve and describe how your network is set up (VLANs, sbnets, switch, routing…), you cannot get useful advice. :roll_eyes:

5

My goal is to have a cluster of 3 nodes with CEPH storage. And idealy, each node should be able to communicate with mu NAS storage for … storage
And also beeing able to use the storage for the apps on the system, like Plex, and also use the storage as a normal NAS.

6

Ceph needs bandwidth - 1Gb would be painful, you definitely want to utilise those 10Gb ports for that. What is your deployment - do you plan to run Ceph on a 3 node Proxmox cluster? Where does TrueNAS fit into this - virtualised under Promox or on a separate node? Not quite clear still what you are trying to do.

7

You can achieve this easily with the default, and recommended, setting of having one single cable in one of the 10G interfaces, going into the (hopefully) 10G switch.

More complex than that, you really need to explain your use case and design goal.

My cristal ball suggests that you’re going to be the umpteenth user to find out that multiple interfaces do not work as you’d want.

8

OK, to be more simple.
What are best practices on TrueNAS (for now and for future)
Assign IP on the physical interfaces (or bond if it’s used) ?
OR
Create bridges interfaces and assing IP on theses brige interfaces ? and create multiples bridges for multiples needs (ex: direct storage on my 1Gb interfaces and applications using these same 1Gb interfaces)

9

Hold on, apps are changing again? Just when I got 'em set up and working just fine and dandy.

What’s the news on this?

10

On June 1 there is a change coming that allows all apps to use an IP alias defined on the host and, because of this, automatic migration from 24.04>24.10 will no longer be possible. If you’re already on 25.04 with working apps, this just gives you a new networking option to take advantage of (or not).

11

Bridges work like a virtual network switch, forwarding packets between interfaces that are connected to it, physical or virtual. So typically you connect a physical nic to the bridge, and then a bunch of virtual ones for VMs etc. In that scenario, if you want TrueNAS in itself to participate on the bridge with its own IP (to access the GUI, access shares etc), then set that IP on the bridge.

If you do not plan to run virtualisation, then skip the bridge altogether and set the IP directly on the bond or the nic instead.

It is also possible to start with the latter and migrate to the former at a later stage.

12

Optional, I like that. I need to rebuild my apps due to a failing SSD anyway (losing a percentage of health every 3 days despite minimal use) as replacement is an enterprise disk with 20GB less storage, you can be damn sure I’m setting them up exactly the same way, no newfangledness.