Workaround for Apps not started due to Unable to determine default interface

TrueNAS General
1

Hello everyone,

I’ve got a realtek 8127 10g network card whose drivers are not in TrueNAS (yet, they’ll be there in TrueNAS 26). I followed this guide to get the drivers working on 25.10.1 and everything works… most of the times.

Sometimes at boot Apps wont start with this message:

Y1smCTr
Y1smCTr444×200 12.7 KB

Application(s) have failed to start: [EFAULT] Unable to determine default interface

Even if all the network interfaces are correctly up and running.

It eventually gets fixed by removing IPv6 from the network interface,

PcdAvQB
PcdAvQB426×114 3.4 KB

rebooting, re-enabling IPV6 and rebooting again (yeah, I know, it’s that weird).

Funnily enough when doing that I get a message that the default gateway is not reachable, even if it obviously is.

UM6aSZt
UM6aSZt1534×1234 81 KB

The full trace message, if anybody is curious, is

Error Name: EFAULT
Error Code: 14
Reason: [EFAULT] ‘172.27.70.254’ is not reachable from any interface on the system.
Error Class: CallError
Trace: Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/middlewared/api/base/server/ws_handler/rpc.py”, line 360, in process_method_call
result = await method.call(app, id_, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/api/base/server/method.py”, line 57, in call
result = await self.middleware.call_with_audit(self.name, self.serviceobj, methodobj, params, app,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 954, in call_with_audit
result = await self._call(method, serviceobj, methodobj, params, app=app,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 771, in _call
return await methodobj(*prepared_call.args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/api/base/decorator.py”, line 108, in wrapped
result = await func(*args)
^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/network.py”, line 482, in save_network_config
raise CallError(f’{str(gw)!r} is not reachable from any interface on the system.')
middlewared.service_exception.CallError: [EFAULT] ‘172.27.70.254’ is not reachable from any interface on the system.

It’s not a FreeNAS bug per se, as I am sure that loading the drivers with a pre-int script (or post-int, it does not make a difference) screws something up. Even if it was, it’s probably not worth fixing it as it will probably work with version 26 soon.

What I am asking is if anybody knows a way to restore the Apps functionality without this very cumbersome double-reboot process. Maybe some CLI commands that I can automate?

I’ve tried restarting middlewared, containerd and docker to no avail.

I appreciate any help with this and while I fully understand that it’s not a TrueNAS issue, I’d love to find a workaround for a few months, until I can jump on version 26 train.

Thanks!!

2

I also get a weird thing with ipv6 being present on the interface that is used for my bridge (only happens after boot)… I also have some other weirdness other than apps not loading, anyway you could likely do a postinit script for:

ifconfig [interface] inet6 del [ipv6address/subnet]

*Edit, you might also need a quick bounce with ip link set down and ip link set up to hopefully avoiding needing to reboot

You could get the ipv6 address while it is in a non-working state with a quick ifconfig

Hopefully it helps you too.

…my full script is longer due to more random crap, and until mine runs I can’t even access the web GUI (no LAN connection at all) - after it runs I still have to make & revert any random change on the Network tab before WAN works again.

Edit 2:

My full script in case if it is of use to you:
ip link set br0 down
ip link set enp37s0 down
ethtool -K enp37s0 gro off
ethtool -K enp37s0 lro off
ip link set enp37s0 up
ip link set br0 up
ifconfig enp37s0 inet6 del ###/64

3

Thanks for your help! Weirdly enough - in my case at least - IPv6 does not seems to make much difference. I think it’s the double reboot and going back to whatever was working gets the job done.

At the moment IPv6 is working perfectly fine:

iAqmn8h
iAqmn8h1904×184 27.9 KB

Unfortunately fixed IP addresses do not seems to work on the interface (v4 or v6), I believe because the drivers gets loaded after FreeNAS tries to set the network up.

you might also need a quick bounce with ip link set down and ip link set up to

Thanks, I’ll try that next time it happens.

The network IS working reliably when the problem happens tough. I mean, everything is setup correctly: IPs, routes, default gateway, dns, everything is fine. It actually works. There are no real problems, it’s just that probably it takes a second or two longer to startup than TrueNAS expect it and doesn’t gets checked again afterwards.

That’s why I was hoping for a service restart, because if it was to perform its checks again it would (probably, I guess?) pass it and everything should be fine.

4

You using bridges or just straight up interfaces with nothing special?

5

I have the integrated 1gb nic with a fixed IP and no default gateway on subnet A and the 10gb nic with DHCP on subnet B. Subnet B has default gateway and everything “normal”.

This is just to ensure ssh even when the 10gb nic acts up, but I do not really need the 1gb nic

6

I reply to my own post with an update: I managed to reduce the number of reboot to ONE by saving the network gateway without actually changing anything

5zfJkI
5zfJkI774×852 26.1 KB

It still makes no sense but it simplifies things.

Any way to perform the same action from CLI?

EDIT: nope, it’s not reliable