Fangtooth Apps and Multiple Nics

[NexGen3D]

I have tried the following

eno2np1:
Alias One: 192.168.0.21/24
Alias Two: 192.168.0.142/24

and

eno2np1:
Nothing

br0:
Bridge to eno2np1
Alias One: 192.168.0.21/24
Alias Two: 192.168.0.142/24

Both work, both setups I can access the GUI via either IP address, but in my switch, 192.168.0.142 keeps vanishing, only 192.168.0.21 is persistant, so I dont know what or how TrueNAS is handling this, and this is why I was trying to use 2 nics on the same subnet, I dont want TrueNAS itself to use the second NIC at all, I want it isolated from TrueNAS, I only want Apps to use it, this is possible with VM’s like a hardware pass through or something.

TrueNAS seems to be still missing this feature, what ever trickery that TrueNAS is doing with multiple Alias addresses under one interface, is not working or playing nice with my Ubiquity gear at all.

[NexGen3D]

So now I have the traffic being routed through 192.168.0.142 but its invisible to my switch, there is magic data being used, but not being reported by any device, and when having a second alias added, Plex does not function to any of the devices in the house, even though I went in and assigned Plex to use the default 192.168.0.21 as this is what it has been using.

I have now had to revert everything back to one IP, so for me I’m chasing my own tail here, I’m just going to setup a separate machine for the apps I want on there own IP address.

The issue I see here, is the the GUI and all Apps should NOT be accessible from both IP addresses, they should ONLY be accessible through the first address assigned end of story, the second, third or however many there is, should be dead unless assigned to an App in the apps configuration, then that address should be active for that App only, very much like when using VM’s each VM is given its own IP on the network by the Hypervisor via your Gateways DHCP server, this is what I’m used to, and I just don’t understand why these Docker containers cant do the same.

To be blunt, the there should be a check box in the App config, that says to create its own IP and isolate the App’s networking, then it can use DHCP and have your gateway assign it an IP, I would prefer this, I would be happy for most apps to have their own unique IP address, it allows me to work with them properly, by having them use the GUI’s IP but with a different port number can be problematic, and the port numbers in some cases are different to the Dockers expected port numbers, and some apps just wont work at all, Plex for some reason will not work if I unselect “Host Network” all the devices in the house, around 6 ATV’s alone, will not connect to the server, they say unavailable, select “Host Network” and they all connect and work as expected.

I may end up moving TrueNAS back on top of Proxmox so I can run two separate instances and gain back control over the networking, as it seems the feature I’m looking for is just not doable in TrueNAS directly.

[NexGen3D]

Right I’ve done it.

I now have both of my “real” nics connected.

I have the second nic connected to an isolated port that is dedicated to one of my VLANS, I have that VLAN routed through my VPN, I also selected “Host Network” in the app, and assigned the GUI port to use the primary NIC, and the download port assigned to the secondary NIC, and once the app (qBittorrent) I had to also in the advanced settings, assign it to use the second NIC as it was still trying to use the primary NIC, I now see it in my switches console, and I can see all internet traffic being routed through my VPN.

Also, I found, I had to lock down the GUI to the IP of the primary NIC, this I believe is quite important and doesn’t seem to be documented anywhere yet.

So to summarise:

App in question = qBittorrent

eno2np1 (Primary NIC): 192.168.0.21/24
This one is on my main network and is locked down for GUI access in the General settings

eno1np0 (Second NIC): 10.1.95.5/24
I have this connected to a fully isolated port that is assigned to only allow access to one vlan (10.1.96.0/24)

In the app config, set GUI access to 192.168.0.21 and set BT Port to 10.1.95.5

In the apps GUI, go to settings → advanced and I set the following:

Then I created a rule in my Switch to divert all traffic from 10.1.95.5 to my VPN.

Job done.

If anyone is interested on how I setup the port on my Ubiquity Switch:

[geri91]

Here is a little example with Portainer/docker compose on how to hook up a container to a MACVLAN interface using a specific NIC of your choice:

Understanding Recent Changes to TrueNAS Scale Apps & Viability of MACVLANs

Recently I tried a little something regarding MACVLAN, could be interesting for you or someone, so I post it here. Most of the time I use Portainer and write my docker compose files myself to have full control over everything. And with that one can get around the TN GUI limitations/bugs. Create a macvlan interface: docker network create -d macvlan \ --subnet=192.168.1.0/24 \ --gateway=192.168.1.1 \ -o parent=enp8s0 \ macvlan_enp8s0 docker compose file example: services: pihole: …

[NexGen3D]

Thanks for the reply, I did try Portainer before all this, and I very quickly, uninstalled it, Docker is witch craft to me, I struggle to grasp the concept of it, and Portainer just has too many things to understand and probably a lot to break, I just don’t have time to drill into it all, maybe when I’m on leave from work I can spend some time on a test system and get a better understanding of it all, I’m just so used to VM’s at this point, and time poor, any change to my network that breaks “Plex” will result in Wife + Kids screaming at me and I also work from home so I need my network to be solid and reliable.

I have accomplished what I wanted in the end, 2 Nics 1 Machine and forcing a particular app to use a specific nic, I understand this is not what a lot of others want or are trying to do, but this is what I wanted in the first place and I got there…eventually.

Hopefully my travels will help others that are trying to do something similar.

[NexGen3D]

That’s if the system they’re on treat them both as the same thing, but not so if they are isolated from each other, this just not the case with TrueNAS.

The network needs to see these nics as two completely separate devices, if one could be isolated and used on select apps only or say all of Docker containers, then having 2 nics on one system should not be an issue, well it has never been an issue for me in the past under Windows or Linux, but I assume TrueNAS is just not designed to work this way, and I want to stay in the bounds of what the devs of TrueNAS allow via the GUI.

I’m no expert, but this is the way I think it needs to work:

Single Nick receives a DHCP IP address, then creates 2 more bridge interfaces with different DHCP IP addresses, 1 for the GUI and services like SMB, the other for Apps, and TrueNAS does the switching for these additional interfaces, so your switch will see the first IP address as nothing more than a Layer 2 style switch, with 2 devices connected to it, I’m quite sure this is how a Hypervisor does it, you can choose to have interfaces visible to one another, or invisible.

All up, standard install should show up as 3 IP addresses in total

Main NIC (call it Switch): 192.168.0.20
br0 (call it GUI): 192.168.0.21
br1 (call it Apps): 192.168.0.22

Accessing 192.168.0.20 should give you nothing
Accessing 192.168.0.21 will give you the TrueNAS management interface, SMB and other service access
Accessing 192.168.0.22:3**** will give you access to the App GUI/Serive/API/Whatever

[NexGen3D]

So I have had to revert everything and just go back to routing my entire machine through my VPN.

No matter what I do, the app still tries and mostly succeeds in access the internet through the main nic that its NOT bound too and uses the global gateway to access the web, if I bind it inside the apps gui settings, it eventually loses internet access on that bound nic which is isolated on a vlan and routed through my VPN.

At first I though it was my VPN or something in my routing rules, but I tested the port on my switch with a different device plugged into it, and no, connection works perfect.

From what I can see, it does not matter about binding it to an interface, as it will always use the global gateway, and it can still see the other interfaces, the apps need a big fat check box that says isolate or sandbox to the selected interface, and you need the ability to allow DHCP or be able to define a gateway and DNS address to that interface separately from the main GUI interface.

Now without the ability to fully isolate to a nic and serve it new separate addressing, I see no point in this feature, someone needs to explain to me what we can even do with it?

Giving an app a new address via an alias only confuses the networking, can create looping/spanning tree issues, it does not isolate the app to that address, it will still use the global gateway, even if you vlan it to an alternate subnet, so I’m lost? What does this really do, and what is it useful for? If I can’t use it to route an app onto its own address, subnet or isolated vlan, then its kind of redundant or broken? Or am I just not understanding this correctly?

All I would like to do is route a select app/apps to use one of my vlan’s that is isolated with its own internet connection, I don’t care if its over a separate hardware nic, or a virtual nic, it can be done with Proxmox and VM/LXC’s and can easily be done using Virtualbox with VM’s, I can even do it Windows.

One would think, if I bind it to an interface, and that interface is an actually hardware nic, and I check the bind to host network, it would be locked on to that nic, and if that nic was connected to an isolated “air” gapped network, it would not see anything at all, but this is not the case, the app can still see everything, main nic, internet the lot.

[Stux]

All the IP binding does is set where the app is listening for connections, ie where its inbound ports are bound.

To control which networks/interface an app uses you’d need to be able to specify a network and/or a macvlan interface

This can be done with a custom compose app.

[coth]

Still a problem. Some routers assign forwarding to mac addresses. And as all IP addresses are assigned to one bridge, it’s impossible to do port forwarding. So how to make unique mac address for each?

[NexGen3D]

I’m quite sure what your trying to do is just not possible with the current implementation, I have tried vlans and bridges on the one primary NIC, on both same and different subnets, along with using a second NIC on a separate subnet.

No matter what you do, the app will always use the default gateway and DNS defined in the Global settings.

Also defining alias addresses under the same NIC will not play well with complex networks, its seriously problematic and plays havic with my Ubiquity switch.

If you do assign new virtual alias interfaces, make sure under general settings you lock down the TrueNAS GUI to the address of your main NIC, also I would recommend changing the http and https ports to something slightly different, but at the end of the day this feature only allows you to access an app on a different IP address than the TrueNAS GUI, and that’s all, it does not allow you to route the app through a different/separate network or vlan, in order to do this, we will need at the very least, some way to pass through a hardware NIC or allow the app to utilise DHCP on a virtual NIC, this way you can either define or feed the app different network details different to the TrueNAS GUI.

[Stux]

@coth

So how to make unique mac address for each?

You need to use docker macvlan networking.

This is not implemented (yet) for the apps, but can be done for custom compose apps

You can vote for this feature:

Allow apps to have their own IP

Problem/Justification Add ability to use separate IPs on apps in Electric Eel. With GUI manageability. Impact It will be similar to a CORE jail. User Story I don’t think this needs much of a story. It’s very practical to have apps using their own network stack.

And this feature too, for macvlan routing

Add support for creating MACVLAN interfaces with "Add Interface"

TrueNAS should support creating and configuring a MACVLAN interface when using “Add Interface”. Ie, MACVLAN should be an option in this menu. [Screenshot 2024-11-27 at 3.38.13 PM] This would prevent complicated CLI scripting in order to create macvlan interfaces and configuring routing tables to allow docker macvlan<->host communication. Macvlan is supported by the Linux kernel and is a well known Linux network type It behaves a bit like a cross between a VLAN and a Bridge. The underlying…

(And i explain how to setup macvlan routing too in that feature request)