Configuring Multiple RJ45 Ports with Unique IPs on TrueNAS Scale

TrueNAS General
,
21

IP alias on a single interface, case closed.

Yes. This exists. It’s called policy routing and a standard function of routers and firewalls today.

But is it not the standard routing mechanism of any IP stack be it BSD based (BSD, Windows, …) or independent (Linux). Routing in IP without policy routing implemented as an extra function is based on destination address and destination address alone. TrueNAS is a host, not a router. It’s running all services on a single IP stack. Like that or don’t, but that’s what it is.

And for a standard host IP stack two interfaces in one subnet is an invalid configuration. I am arguing nothing more, but also nothing less. On a router with policy routing you can implement whatever you like but TrueNAS is not a router.

Read “TCP/IP Illustrated” or “The Design and Implementation of the FreeBSD Operating System” or any textbook on the matter.

1 Like
22

@pmh

Thank you for the suggestion of using IP aliasing on a single interface. However, there are two physical NICs in use for a reason:

  1. Bandwidth Limitations: One NIC (2.5 Gbps) is already handling significant traffic and is on its capacity, while the second NIC (1 Gbps) manages less critical traffic, such as DNS, but still serves an essential role.
  2. Separation of Services: Using two interfaces provides a logical and efficient separation of high-priority services.

With that in mind, could you clarify how IP aliasing on a single interface would work without port forwarding, especially since both services need to use port 80? (Because they are attached to a Domain)

For example:

  • Pi-hole on 10.0.0.19:80 (for the GUI) and :53 (for DNS).
  • Jellyfin container on 10.0.0.20:80 (for web traffic).

How would TrueNAS manage traffic directed to these IPs on the same interface without causing conflicts on port 80? The entire point of assigning separate IPs to different NICs is to avoid these conflicts and simplify access, which port forwarding does not address effectively in this scenario.

Looking forward to your insights.

Best regards,
Ricardo

23

The services would need to bind to a specific IP address. They need to do that in the case of separate interfaces, too.

The rest is left as an exercise to the reader :wink: I was just arguing that you do not need two interfaces for two IP addresses.

I prefer to run services in VMs or FreeBSD jails on TrueNAS CORE where you can realise arbitrary network topologies because the underlying NICs are just layer 2 and every jail or VM runs its own IP stack.

I only use Docker on SCALE when I can’t avoid it. E.g. because of a lack of FreeBSD support by some specific app.

24

@pmh

Thank you for your input. While I understand your preference for using VMs or FreeBSD jails in TrueNAS CORE to achieve arbitrary network topologies, my goal here is to minimize complexity and leverage the apps system available in TrueNAS SCALE.

As I’ve already explained, the use of two interfaces isn’t arbitrary—it’s driven by practical considerations like bandwidth management (one NIC is near capacity, while the other handles less critical DNS traffic) and service separation for simplicity.

Networking is a domain where you either go all-in with advanced features (like Ubiquiti, Cisco, etc.) or keep it simple and intuitive for users, as platforms like Synology, Unraid, and OpenMediaVault do. Trying to strike a middle ground, as TrueNAS does, often leads to a concept that feels incomplete and overly restrictive.

The apps system in TrueNAS SCALE is supposed to make things easier, but without support for straightforward configurations like mine, it creates unnecessary hurdles. If TrueNAS SCALE aims to compete in the broader market, it should focus on simplifying valid use cases like this, rather than expecting users to configure VMs or work around its limitations.

Best regards,
Ricardo

25

TrueNAS leverages an operating system (we’re talking about Linux here and it’s a bit different for UNIX, but similar enough to pay attention to the concerns already raised) in order to do its networking.

If you assign 2 IP addresses on the same LAN subnet to different physical NICs of the TrueNAS host, outgoing routing will only use one of those NICs to send all packets destined for that subnet… you won’t see the expected outcome of things you “elect” to have using IP 2 sending data out from the NIC that “owns” IP 2.

That’s an OS thing, nothing to do with TrueNAS and you can’t really get out of that limitation (regardless of TrueNAS and its interface settings).

It sounds to me more like you’re keen to have some apps or functions talking only out one NIC and others out the other…

You don’t need TrueNAS to have IP addresses defined on both NICs for that at all.

Run a VM and assign its NIC to the second Physical NIC (or officially, to a bridge with that NIC as a member)… don’t assign any IP address to that second NIC at all in TrueNAS.

Run your IP stack in the VM and have that host whatever address you want.

Do anything else (including with TrueNAS if it does eventually modify the network settings to allow what you requested) and you’ll be disappointed.

For bonus points, you could run a few different VMs and run a k3s cluster on them, then use MetalLB to have an address per app (again, with the option to tie each VM to a different NIC/Bridge) and have traffic do what you expect.

2 Likes
26

While i might get “killed” here :grinning:

From a network perspective @ricardo.gomes is correct.
But from an application and firewall perspective you are opening a “HUGE Can of Worms”.

To put it simple :
You have no (easy) way of controlling what interface the “answer would come out of”.
Ie. If a device send a Pi-Hole request to IF1 , and the network stack decides to send the answer out of IF2. The device would probably reject the answer as “Bad”, and ANY firewall it would pass would REJECT the package as “Out of state” - Well it’s UDP (stateless) , but the firewall would still block the package , originating from IF2.

I order to fix (policy route) that, you would have to do some iptables tagging of the package, and policy route based on those tags.

If you have ever tried that, you would already know why an “appliance” as TrueNas would not support (allow) such a configuration.
It would to put it nicely: Be very complicated for any supporter to debug and understand.

If you want a box capable of doing such trickery, you will have to look elsewhere.

I’m not buying your “2.5Gb” IF bandwidth as a solid argument for not using aliases. You could just switch to 10Gb.

But since you have 15yr of Network experience, it strikes me as strange that you dont just create another Vlan , and put your IF2 in there. Problem solved

Hint: Don’t just put an additional def-gw on IF2 … Then you’d open up another can of worms, that quite possible would affect TN operations too.
But with your experience you should know what to do. Provided TN even allows for an advanced (tagging/policy) routing setup.
Edit1: Seems like : ip rule is present on my TN

Edit2:
IMHO - TN isn’t VMware or Proxmox, and with the current restrictions would never even aspire be. I see it as a Solid NAS system “with a bit of extras”.
And I would prob. never use it for any mission critical appliances, like DNS/DNS-Filtering or the likes.

2 Likes
27

Wasn’t SAS Multipathing dropped because it was too easy to shoot yourself in the foot?
Not allowing this is a no-brainer out of a usability perspective and not spending limited resources building and supporting this error-prone configuration seems very sensible.

You as the end user, whom appear both willing and with the prerequisite knowhow, can set this up yourself.

28

Btw, synology handles this perfectly out of the box on Linux and also gives users SMB multichannel that behaves like windows (aka fast).

1 Like
29

Unfortunately i cannot due to limitations on space, thermals an many other factors. Im using a ASUS TUF GAMING B650M-E WIFI on a Inter-Tech 2U 2404L S-ATA 19" Rack, and even limiting the AMD Ryzen 5 8600G (AM5, 4.30 GHz, 6 -Core) to 35W.

30

That’s a fact, some are ignoring or don’t want to believe…

31

They won’t budge.

For a decade or so I’ve watched folks wade-in and fight this battle with your level of passion, expertise, and eloquence. None of them got anywhere. Nothing was gained; in every case the ix folks made zero concessions.

Personally I don’t have an opinion either way. Only here to add one can circumvent quite a few TrueNAS middleware restrictions with some strategic echo blah > /proc/sys/net configured to run as a post-init script.

32

I was only ever arguing that this is not a “bug”. What TrueNAS does is how every standard host based IP stack behaves. My participation in these discussions was only about the “bug” claim.

Now if you have policy based routing or multiple FIBs or whatever baked into the platform of cause all of this can be done. But it’s an entirely separate feature set not covered by something as essential as the Hosts Requirements RFC.

The larges obstacle in TrueNAS is that all services and the management UI run on the same single stack.

For jails and VMs you can already separate today even with good old CORE because with VNET you only need layer 2 connections without the NAS participating in any of the separate networks.

33

Well guess my system is bugged as can be then. On 24.10.2 I was able to assign static IP´s in the same subnet to each individual interface, this worked perfectly well for my Home assistant VM i had running on there sadly updating to fangtooth broke that setup.

34

A VM does not need an IP address assigned to the NAS interface used. You assign the IP address in the guest operating system. The connection itself is strictly layer 2.