mDNS nightmare ...per-App IP addressing doesn't help

Apps and Virtualization
1

Since I setup TrueNAS last fall, I have been trying in vain to get mDNS to play nice on my network. The networking in TrueNAS is a nightmare. No amount of loopback adapters, bridges, subnetting, etc has helped. I was able to get multichannel SMB to work for a bit with 2 NICs on 2 subnets, thus giving me 5gbps bandwidth with 2x 2.5gbe. But that would sometimes fall apart and I decided to go back to a simple configuration. Can someone who is a TrueNAS and networking guru please help me. This is all I want to do:

  • TrueNAS NIC1 running on 192.168.100.10/24
  • TrueNAS NIC2 is disabled
  • Time Machine backups using mDNS (port 53)
  • Home Assistant and Scrypted running HomeKit Bridge needs mDNS (port 53) but it’s never discoverable on the network via APPS, even with the new per-App IP and creating an alias on 192.168.100.10 for 192.168.100.9

My only workaround so far has been to using a VM (instances now). It runs Home Assistant and the embedded version of Scrypted on 192.168.100.2, but since the new updates, I still can’t figure out how to backup/replicate this. I just want to move away from VMs which use more resources and make everything APPS running on Docker. Any suggestions on how to go about doing this?

2 Likes
2

See:

3 Likes
3

You are a god send, not sure how I missed that. So I guess it’s more waiting for me until I get into app nirvana. It’s workable now with instances so I can wait. Thank you!

2 Likes
5

Let me know when you reach it, because I’ve been trying to get there as well. I’ve been running Home Assistant app and trying to get a specific Homekit device to be found on there and I’m not sure what else I can try. I was hoping the new app IP settings would help, but no such luck.

6

I’m in the same boat, Home Assistant app and mDNS not working. We can wait and see if that bugfix for the ports on per app IPs does anything. If it doesn’t then we probably need MACVLANs for this to work. You can vote for that feature request here.

7

Until someone test it, we don’t know whether MDNS works or not.

It will work with muiltiple interfaces

It needs to be validated with IP alias

8

Can you elaborate on that? I actually have multiple interfaces in my TrueNAS machine where I need mDNS to work, but I still can’t seem to get the Home Assistant app to recognize a Homekit appliance. What steps should I take use an alternative NIC with a TrueNAS app and somehow get mDNS working?

9

I was thinking about mDNS used to find the TrueNAS. I’m not sure what is needed to do outbound mDNS discovery.

Someone will need to compare the LXC version vs the the App version and see what is broken. Does Home Assitant think it should work from a docker container?

10

Its seems to be a well known problm with Docker and mDNS:

" Let’s say that you have a service set up on a host in your private network but now you need to access that service from an app you are building which will run in a docker container. Several issues arise with this scenario:

  • First of all you will be limited because the docker container will run in a separate network created by docker.
  • Next, even if you configure the container to run with the host network attached, avahi (the mDNS daemon) requires access to your system’s dbus and the avahi daemon socket.
  • Finally, most of the base images that you can use while creating your app’s image won’t have the necessary tools installed needed for mDNS name resolution.

Now, let’s solve each of these problems one by one by creating and configuring the app’s docker image. For this we’ll need to create a Dockerfile and a docker-compose file. We’ll look at the process in a bottom-up approach."

11

Searching around I found this user post about getting it working in a container a few months ago. Initially they had to disable mDNS in network global config, and set the container to host network. On an older version of scale I was able to get device discovery working in the app by disabling truenas mDNS and checking the host network box in the app. Disabling truenas mDNS didn’t seem like the ideal solution, and host network isn’t available for the Home Assitant app now anyway.

What is more hopeful from the post is the later update stating
" Update: I was able to bypass the need for running home assistant in host networking mode by using the macvlan network driver.
This allowed me to re-enable mDNS discovery in TrueNAS and still have auto-discovery in Home Assistant."

They include the relevant config code in that post as well. So it seems like MACVLAN might be the solution, but this is out of my wheelhouse. Home Assistant documentation does state it uses zeroconf which shows services discovered using mDNS.

12

Good to know, but I assume you had to special things to make this work. Can you write it up in a new thread and post the link here.

13

Just one general comment here for this thread, in a previous life during/ just after COVID… I managed several “Apple Schools” in K12 (among many other adventures there).

mDNS can absolutely be a networking problem. VLANs and PIM vs IGMP vs Querier vs avahi… you can go down a very deep rabbit hole trying to make Apple stuff do Apple stuff in a safe way for a school environment. It’s literally the same problem here, but a larger scale.

My only point is, mDNS isn’t specifically a TrueNAS issue RE Apps. I can say for sure, HAOS as an Instance in TrueNAS works great for Home Assistant. I whole heartedly believe that to be the better deployment model even for home users. Having a full phat VM on TrueNAS managing it’s own seperate network stack prevents weird issues and should allow more portability between systems or even operating systems.

However,

@OneHungryPoboy I’m actually interested too, as @Captain_Morgan mentioned it would be nice to see what you did to help others.

Note: Not suggesting I have any reason to believe that is OP’s problem specifically, but worth mentioning for those who find this thread later on.

14

Yeah, some great point and thanks everyone for the inputs. The TrueNAS team released a podcast last night where they talked about this (well port publishing on Apps), they deployed a fix that didn’t require a system update, it was something in the app catalog configuration and sure enough I saw the App updates and new config items for publishing ports. But it still fails, you can’t publish port 5353/UDP if TrueNAS is using it for Time Machine backups, even if you setup an alias IP address to use just for that App. The one mistake they are making is binding 5353/UDP to ALL ALIASES. Why can’t we just specify, use it only for the main IP address, and free up the ports on other alias IPs to be used for other ports on the Apps? What is weird is they kinda did this, but only with port 80 and 443 so you can access App web interfaces on different IP addresses. So they know how to do it partially, but they didn’t fully bake the cake in a way that would allow us to do this.

I tried to disable mDNS on TrueNAS and try to have Scrypted squat on that port until I could get it up, but that would require turning off Time Machine, and I don’t know what that would do to my backups.

So @NickF1227 might be right. The best solution (and the one I’ve been stuck on) is just using a VM/Instance for Home Assistant and Scrypted, so it’s not stepping on any of the weird networking they have setup with Apps on TrueNAS. I hope the @TrueNAS_forum_owner sees this thread and they finally fix this once and for all.

15

That’s a reasonable point… it would be valid to report this as a bug. Please indicate the Nas-ticket ID.

So the per-App IP addressing is working, but there are so globally assigned ports . TimeMachine being the example here. Are there any others?

16

Not that I’m aware of (yet). Ticket opened. Thanks @Captain_Morgan Jira

17

Here’s a sample of processes and the ports used taken from a running homelab system with all specific IPs filtered out leaving 0.0.0.0, loopback and multicast:

% sudo netstat -tulpn                                                         
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:445           0.0.0.0:*               LISTEN      8947/smbd           
tcp        0      0 127.0.0.1:139           0.0.0.0:*               LISTEN      8947/smbd           
tcp        0      0 127.0.0.1:6999          0.0.0.0:*               LISTEN      7485/netdata        
tcp        0      0 127.0.0.1:6000          0.0.0.0:*               LISTEN      3701/middlewared    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      6179/sshd: /usr/sbi 
tcp6       0      0 :::80                   :::*                    LISTEN      6556/nginx: master  
tcp6       0      0 :::22                   :::*                    LISTEN      6179/sshd: /usr/sbi 
tcp6       0      0 :::443                  :::*                    LISTEN      6556/nginx: master  
udp        0      0 0.0.0.0:40707           0.0.0.0:*                           8642/avahi-daemon:  
udp        0      0 0.0.0.0:55431           0.0.0.0:*                           8675/python3        
udp        0      0 0.0.0.0:123             0.0.0.0:*                           6281/chronyd        
udp        0      0 127.0.0.1:323           0.0.0.0:*                           6281/chronyd        
udp        0      0 239.255.255.250:3702    0.0.0.0:*                           8675/python3        
udp        0      0 0.0.0.0:5353            0.0.0.0:*                           8642/avahi-daemon:  
udp6       0      0 :::123                  :::*                                6281/chronyd        
udp6       0      0 ::1:323                 :::*                                6281/chronyd

Some of these can be set to bind to an interface in the GUI and I just haven’t (ipv6 stuff that I am not using for example). Others have no clear way of doing that.

I am also not sure how to best handle loopback and multicast stuff.

18

@Captain_Morgan and @NickF1227 Apologies for that previous post, I realize I worded that confusingly and more importantly forgot the link!

I got HA mDNS discovery working once, about 18 months ago in Cobia, by disabling Truenas mDNS in global config. That seems like not an ideal solution. I ran it for an hour and turned mDNS back on for Truenas (causing HA to no longer work). I have never had home assistant device discovery working outside of that brief test, trying to reproduce it now I cannot get it to work at all. I think I may have been using the TrueCharts app.

The second part was a quote from another user who currently has it working. @tannisroot in this thread:

Was able to get it working recently using this network code section.

services:
  homeassistant:
...
    networks:
      homeassistant:
        ipv4_address: 192.168.0.242
...
networks:
  homeassistant:
    driver: macvlan
    driver_opts:
      parent: br0
    ipam:
      config:
        - subnet: 192.168.0.0/24
          ip_range: 192.168.0.240/29
          gateway: 192.168.0.1
          aux_addresses:
            host: 192.168.0.241

So unfortunately I am unable to help others outside of scouring forums and gathering information. I would like to test the custom container YAML at some point this summer, but I will be bumbling through that as I have not done anything related to network bridges previously (and I’m not entirely sure if the bridge is needed?). Maybe Tannisroot can answer additional questions as needed, maybe that code section provides enough detail.

19

I’m down to answer any questions!

20

Don’t take my word for it, at least not so much as the Home Assistant devs.

  • Home Assistant Operating System: An embedded, minimalistic operating system designed to run the Home Assistant ecosystem on single board computers (like the Home Assistant Green or a Raspberry Pi) or Virtual Machines. It is the most convenient option in terms of installation and maintenance and it supports add-ons. Home Assistant Operating System is the recommended installation method for most users.
1 Like
21

See, it’s not worth logging bugs with the TrueNAS team. You spend time logging it, trying to explain a possible edge case and they never thoroughly read the ticket and just close it out of laziness: Jira