Add support for creating MACVLAN interfaces with "Add Interface"

Stux · November 27, 2024, 5:28am

TrueNAS should support creating and configuring a MACVLAN interface when using “Add Interface”.

Ie, MACVLAN should be an option in this menu.

This would prevent complicated CLI scripting in order to create macvlan interfaces and configuring routing tables to allow docker macvlan<->host communication.

Macvlan is supported by the Linux kernel and is a well known Linux network type

It behaves a bit like a cross between a VLAN and a Bridge.

The underlying command line to create the macvlan interface on the host is

ip link add macvlan0 link ens3 type macvlan mode bridge

Where macvlan0 is the name of the new interface, and ens3 is the parent interface. bridge is the macvlan mode, the most permissive, should be the default, but would ideally be configurable

The Linux implementation is lightweight and it eliminates the need for using a Linux bridge for sharing host access with jails and virtual machines, and importantly is supported by Docker.

The interface would allow you to pick the parent interface, possibly the macvlan mode, as well as an optional VLAN tag.

If MACVLAN interfaces could be created and configured, then you would be able to assign an IP address to a macvlan interface, much like how a Bridge network can be assigned at the moment.

For example:

This would mean that Docker macvlan containers would be able to communicate with the host, the same as virtual machines and jails can currently when using a bridge network, as documented in the below links.

YouTube - Setting up a Static IP and Network Bridge
TrueNAS Docs - Accessing NAS from a VM or Apps

For example, the following container which is on the same macvlan network as above

services:
  netshoot:
    tty: true
    image: nicolaka/netshoot
    networks:
      - macvlan0
networks:
  macvlan0:
    external: true

Can easily communicate with the TrueNAS host.

Without performing any custom routing, or command line shenanigans.

This also means that a Jailmaker jail is able to communicate with the TrueNAS host without using a bridge (note: --network-macvlan rather than --network-bridge) in the config.

Meaning that the Jail can successfully communciate with the host.

Again, not usually possible without using a bridge.

And this also means that a VM guest is able to communicate with the host without using a Bridge.

The VM is configured to use ens3 rather than a Bridge. This translates to a MACVLAN tap.

NOTE: In all examples, the Jail, Container or VM was able to communicate with the LAN, other MACVLAN clients, and the Host without using a Bridge, in a simple fashion, and network performance was also improved by bypassing the Bridge.

DHCP lease acquisition delay is also removed due to there being no need for STP on the MACVLAN bridge-like device.

NickF1227 · November 27, 2024, 6:23am

I can see the value in your overall feature request, but I think you are selling it short. A macvlan is also a little more complicated than you give it credit for.

Side note, its also the “Linux version” of a Cisco VIC which does basically the same thing in dedicated hardware (for good reason, in software things can get messy from a performance standpoint).

A MACVLAN is really a more advanced version of a BRIDGE. They both perform most of the same functions, but what seperates MACVLAN apart is that each “entity” (vm, docker container) get’s their own physical port and MAC address, e.g. eth0.0, eth0.1.

      +------------+                         +------------------------+
      |  Host NIC  |-------------------------|      Physical Network  |
      |   (eth0)   |                         |                        |
      +-----+------+                         +------------------------+
            |
      +-----+------+
      |   MACVLAN  | 
      +-----+------+
            |
      +-----+-----+-----+  
      | eth0.1 | eth0.2 | eth0.3 |   +-------------------+
      +--------+--------+--------+   | VMs / Containers  |
                                     | (Same network segment) |
                                     +-------------------+

Without getting too much more into the ramifications of what that actually means at a scale greater than homelab, I’d have concerns with just adding this as a drop down item. At least not without other considerations and UI work.

Stux · November 27, 2024, 6:28am

Oh, I understand that.

The issue is simply that TrueNAS should support it, and currently doesn’t

I hope I’ve sufficiently sold one of the use cases to justify the feature. I have also looked at how it could be implemented etc, (ie looking at the interface driver python code in middleware)

And Fangtooth’s development window is probably closing soon, and I hope this feature will be included.

NickF1227 · November 27, 2024, 6:41am

The scope of this is probably bigger than a last minute shoe in. This request is really “Lets add more managed switching functionality to TrueNAS”,

Historically, TrueNAS has never been a switch. With other forum feature requests for “XYZ VPN support” being in front of mind, this can get complicated.

etorix · November 27, 2024, 7:38am

Based on T3 #5, the window for Halfmoon feature set is closing within a month.

Captain_Morgan · January 5, 2025, 5:44pm

I believe this requirement is addressed by this pull request,

@Stux can you verify? May be the UI work is not yet done?

Stux · January 5, 2025, 6:02pm

Which PR?

Captain_Morgan · January 5, 2025, 6:20pm

Answered above… my mistake.

etorix · January 5, 2025, 8:30pm

Great news. As usual, may I request that this positive answer be promptly followed by its logical conseqeunce of flagging the thread as “accepted” and releasing the votes?

Stux · January 5, 2025, 11:23pm

I believe that PR adds macvlan support to the incus integration.

It’s similar to how the legacy virtualization (and jailmaker) have macvlan support.

What does not have macvlan support is TrueNAS’s own interface/network settings (lagg, bridge, vlan and physical)

This means when you have an incus, or other system on a macvlan it can’t communicate with the TrueNAS host (or more technically the host can’t communicate back)

IMHO, the simplest way to setup that communication is simply to also put the TrueNAS host on a macvlan interface, similar to what is done with bridges for VM access… but without needing a bridge, and also working with docker.

Captain_Morgan · January 7, 2025, 9:48pm

You are correct…

We agree its useful, but not sure which release it will make. We need enough time to test as well.

hihi.greensleeves · January 15, 2025, 4:05pm

If you update your TrueNAS. The macvlan created by shell will be delete.

alexanderniebuhr · March 6, 2025, 4:05pm

@Stux How have you got this working with docker. I created the macvlan interface using the command you posted, but docker doesn’t know about the network?

Stux · March 12, 2025, 12:36am

You need to create a docker network, with something like this

docker network create -d macvlan -o parent=ens3 --subnet 192.168.0.0/24 --gateway 192.168.0.254 --ip-range 192.168.0.40/29 macvlan0

In that network create, I reserve the subnet 192.168.0.40/29 for docker. Docker will then hand addresses out from that subnet to containers.

Then if you want the docker containers and host to be able to communicate with each other you need to route between the two. This has to be done using a post-init script, something like this:

# creagte a mvlan0 interface
ip link add mvlan0 link ens3 type macvlan mode bridge
ip link set mvlan0 up

# add a route to the macvlan subnet via the mvlan0 interface
ip route add 192.168.0.40/29 dev mvlan0

That will route the subnet 192.168.0.40/29 into the macvlan that mvlan0 is a member of.

If you want to specify a specific IP address to route, use /32 to define the “subnet”

The simple alternative would be to create the MACVLAN interface in the GUI… hence this feature request

alexanderniebuhr · March 13, 2025, 10:06pm

Thanks a lot. I didn’t get that I need to setup a docker network too.
However that still means it does not allow me to use DHCP from my router, I was hoping to achieve this, so I don’t need to keep a IP range for docker, I would like my router to manage all IP assignments, and was hoping that it would work when using the linux macvlan adapter.

Stux · March 13, 2025, 10:21pm

Docker does not support dhcp.

You can specify a fixed ip for each docker macvlan container. That could match your dhcp servers config.

But at the end of the day, docker containers don’t run dhcp, and instead docker sets up the networking when starting the container, with an ip it chooses from the pool, or through configuration