I recently “side-graded” my NAS from TrueNAS CORE to SCALE.
About 24-hours after the upgrade I started getting errors when attempting to “discover” the Apps catalogue or update etc. It appears that connectivity to the internet became disabled. Although I could still access internally.
I also noted that one of my network adapters (eno2) was “down” and speculating the two issues may be linked.
My network settings currently consist of two interfaces joined in a load balance arrangement:
eno1 - working
eno2 - down
bond1 - 192.168.0.251/24
Given the short duration between upgrade and the link going ‘down’ if there is a software setting or “issue” with TrueNAS SCALE that could be diagnosed and resolved before relegating to a hardware failure.
I am noting in the green text at the bottom of the dashboard, repeating:
TrueNAS kernel: br-327224a2ccbd: port 1 (veth251c0d3) entered blocking state
TrueNAS kernel: br-327224a2ccbd: port 1 (veth251c063) entered forwarding state
TrueNAS kernel: br-327224a2ccbd: port 1 (veth251c063) entered disabled state.
TrueNAS kernel: veth251c063: entered multicast state
TrueNAS kernel: veth251c063: entered promiscuous state
The text in parentheses changes each time.
I have tried to “split” the aggregated link, however, this resulted in lost connectivity to the server and a reboot to regain connection again.
In addition:
Connectivity to shares has disappeared. The server has disappeared from “This PC” however I can log in to the web interface. I have tried restarting SMB and restarting my Windows machine.
Thanks for any advice in troubleshooting any setup or system issues that may be causing this.
Try pulling the cable from eno2 to see if things stabilize. I’m always skeptical of the value of bond interfaces without specific switch-side support, as you’ll often end up with inadvertent spanning-tree loops if you’re not careful.
I’ve pulled the cable from eno2 and internet connectivity has not been reinstated. Using Apps as an example I continue to get the following error when I refresh the catalogue:
[EFAULT] Failed to clone ‘ GitHub - truenas/apps · GitHub ’ repository at ‘/mnt/.ix-apps/truenas_catalog’ destination: [EFAULT] Failed to clone ‘ GitHub - truenas/apps · GitHub ’ repository at ‘/mnt/.ix-apps/truenas_catalog’ destination: Cloning into ‘/mnt/.ix-apps/truenas_catalog’…
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/middlewared/plugins/catalog/git_utils.py”, line 34, in pull_clone_repository
clone_repository(repository_uri, destination, branch, depth)
File “/usr/lib/python3/dist-packages/middlewared/utils/git.py”, line 25, in clone_repository
raise CallError(
middlewared.service_exception.CallError: [EFAULT] Failed to clone ‘ GitHub - truenas/apps · GitHub ’ repository at ‘/mnt/.ix-apps/truenas_catalog’ destination: Cloning into ‘/mnt/.ix-apps/truenas_catalog’…
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/middlewared/job.py”, line 515, in run
await self.future
File “/usr/lib/python3/dist-packages/middlewared/job.py”, line 560, in __run_body
rv = await self.method(*args)
^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/api/base/decorator.py”, line 93, in wrapped
result = await func(*args)
^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/catalog/sync.py”, line 27, in sync
await self.middleware.call(
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 1005, in call
return await self._call(
^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 731, in _call
return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 624, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3.11/concurrent/futures/thread.py”, line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/catalog/sync.py”, line 55, in update_git_repository
return pull_clone_repository(repository, location, branch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/catalog/git_utils.py”, line 36, in pull_clone_repository
raise CallError(f’Failed to clone {repository_uri!r} repository at {destination!r} destination: {e}')
middlewared.service_exception.CallError: [EFAULT] Failed to clone ‘ GitHub - truenas/apps · GitHub ’ repository at ‘/mnt/.ix-apps/truenas_catalog’ destination: [EFAULT] Failed to clone ‘ GitHub - truenas/apps · GitHub ’ repository at ‘/mnt/.ix-apps/truenas_catalog’ destination: Cloning into ‘/mnt/.ix-apps/truenas_catalog’…
The green text continues to roll.
I get internet access from other applications in my network; I don’t suspect it being a router issue.
You’ve got an IP conflict between your Incus bridge interface and your gateway.
You’ll need to move your bond interface under a new bridge interface, move your bond interface IP to the new bridge interface, and configure Incus to use the system bridge interface instead of using its default bridge.
Thanks for that. I tried clearing the “192.168.0.1” from Containers → Configuration → Global Settings and letting the system choose a new IP. This worked; I now have access to Apps etc. As a side note, I know have internet connection with my phone; This setting was messing around with other devices!
A few more questions:
[*] Should I still create a new “bridge”? I’m new to bridges and, although I have an awareness/understanding, not show how they are implemented in TrueNAS Scale. Or is it a case of “ain’t’ broke don’t fix it”? FYI: When I experimented creating a new bridge, I could only choose from eno2 and/or bond1. Why not eno1?
[*] Should/could this have resolved eno2 not working? Or is it a hardware issue that failed rather coincidentally at time of switch-over from CORE to SCALE?
I recommend you still create a bridge to prevent this scenario in the future.
Because eno1 was already spoken for as a member of bond1.
It’s possible eno2 failed coincidentally. Try connecting eno2 again. If it properly joins the bond1 interface, it will disappear from the list of candidate interfaces for a bridge.
