Help with DNS issues in Kubernetes Cluster on TrueNAS Scale

Apps and Virtualization
, ,
1

Hello everyone,

I am facing an issue in my TrueNAS Scale environment related to DNS resolution in my Kubernetes cluster. After several attempts to resolve the problem, I would like to reach out to the community for support, especially to find a solution that does not require a complete reinitialization of the cluster.

Setup and Context:

  • System: TrueNAS Scale
  • Version: Dragonfish-24.04.2
  • CPU: AMD Athlon 3000G with Radeon Vega Graphics (2 cores, 4 threads)
  • Memory: 32 GiB
  • Network: Ethernet 1000Mb/s
  • Docker Containers: Home Assistant and Pi-hole are running as Docker containers within this cluster.

Previous Issues:

DNS Problems with Pi-hole:

  • Initially, I had issues with Pi-hole not being able to perform DNS requests properly, particularly with its own Gravity updates.
  • The issue was resolved by manually setting the DNS servers in the resolv.conf to Cloudflare (1.1.1.1 and 1.0.0.1).

Home Assistant Cloud Connection Problems:

  • While using Home Assistant, I encountered connection issues with the Home Assistant Cloud. The connection could not be established, even though the DNS servers were configured correctly.
  • Even after adding all relevant domains to the Pi-hole whitelist and disabling DNS blocking, the problem persisted.
  • It seems the same issue exists where Kubernetes does not correctly forward DNS requests.

Attempted Solutions:

Changing the resolv.conf in the Container:

  • I attempted to modify the resolv.conf file in Home Assistant to directly point to an external DNS server (Cloudflare 1.1.1.1).
  • This attempt failed due to permission issues within the container, as no root privileges were available.

Considering a Change to Cluster DNS:

  • I considered setting the cluster DNS entry directly to the Pi-hole DNS. However, there is a risk of an infinite loop if Pi-hole forwards DNS requests back to the cluster, which then uses Pi-hole as the DNS server.

Changing Cluster DNS in TrueNAS:

  • When trying to adjust the DNS entry for the cluster in TrueNAS, I encountered a warning stating that this would require a complete reinitialization of the cluster. This would result in the loss of all installed apps and data, which I am trying to avoid.

My Question to the Community:

Is there a way to adjust the DNS settings in the Kubernetes cluster such that:

  • External DNS requests are forwarded correctly?
  • Pi-hole can function as the DNS server without causing infinite loops?
  • A complete reinitialization of the cluster is avoided?

I am looking for a solution that does not restrict functionality and ideally does not require reinstalling the entire environment. Perhaps there is a way to modify the DNS settings at a lower level or use an alternative method to manage DNS forwarding?

I would be grateful for any advice or suggestions!

Thank you in advance for your support.

Best regards,
Zorunel