System: TrueNAS SCALE 25.04
Issue Started: After enabling a VM “instance” using Incus on 25.04 RC
Summary:
I attempted to create a VM instance via the new virtualization UI and unfortunately I dont remember exactly what happened but it was apparent something went wrong and the system hung up causing me to reboot. From that point on, rebooting the system will cause severe networking issues:
- SSH access and web UI would fail after reboot
Server would appear on the network briefly, then become unreachable
Physical access with monitor/keyboard showed high packet loss and massive interface bloat —ip addrlisted 20–40+ interfaces like:br-xxxx,veth-xxxx,incusbr0,docker0, etc.
Even after I stop Incus, these interfaces return on each reboot
This happened after attempting to enabe a VM, and persisted even after disabling/deleting the VM.
Troubleshooting steps:
- SSH / Web UI Unreachable
- Could not ping known IP reliably
- Found the hostname resolving to a different IP
- Eventually connected via physical monitor & keyboard
- Inspected Interfaces
ip addrshowed dozens ofveth,br-, anddocker0interfaces- Many created by Incus even with no VM running (
incus listwas empty)
3.Disabled Incus
systemctl stop incus incus.socket
systemctl disable incus incus.socket
systemctl mask incus incus.socket
- Tried to Delete incusbr0
- Initially I was blocked because it was “in use”
- Used
ip link show master incusbr0to identify attachedveth-* - Deleted all interfaces with: ip link delete
- Removed teh bridge with: ip link delete incusbr0
- Also Stopped Docker (temporarily)
Docker was running apps I need, but for isolation/testing, I stopped and disabled it
After this the system returned to normal, I can SSH and use the Web UI. Only enp1s0, lo, and docker0 interfaces are present.
Unfortunately the problem returns after reboot.
- Huge list of interfaces recreated
- System unresponsive via network until logging in locally
- Had to stop both Incus and Docker manually again to regain control
I need to prevent Incus and/or Docker from recreating these interfaces at boot. Disabling and masking services appears to not persist cleanly across upgrades or reboots.
- Is there a way to fully remove Incus if I’m not using VM instances?
- Can Docker be limited so it doesn’t create
veth-*interfaces unless needed? - Why is Incus re-injecting networking configs even when nothing is running?
Would love help cleaning this up for good — I want to keep Docker for apps, but avoid this massive interface mess on every boot and I would rather not start with a fresh install unless I could keep everything in my datasets.
In case youre curious, here is the Validation Error I get when going into the instances tab:
Validation Error:
Cannot connect to unix socket /var/lib/incus/unix.socket ssl:default [Connection refused]Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/aiohttp/connector.py”, line 1545, in _create_connection
_, proto = await self._loop.create_unix_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3.11/asyncio/unix_events.py”, line 259, in create_unix_connection
await self.sock_connect(sock, path)
File “/usr/lib/python3.11/asyncio/selector_events.py”, line 638, in sock_connect
return await fut
^^^^^^^^^
File “/usr/lib/python3.11/asyncio/selector_events.py”, line 646, in _sock_connect
sock.connect(address)
ConnectionRefusedError: [Errno 111] Connection refusedThe above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/middlewared/api/base/server/ws_handler/rpc.py”, line 323, in process_method_call
result = await method.call(app, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/api/base/server/method.py”, line 40, in call
result = await self.middleware.call_with_audit(self.name, self.serviceobj, methodobj, params, app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 883, in call_with_audit
result = await self._call(method, serviceobj, methodobj, params, app=app,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 692, in _call
return await methodobj(*prepared_call.args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/api/base/decorator.py”, line 88, in wrapped
result = await func(*args)
^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/virt/instance.py”, line 54, in query
results = (await incus_call(‘1.0/instances?filter=&recursion=2’, ‘get’))[‘metadata’]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/virt/utils.py”, line 105, in incus_call
r = await methodobj(f’{HTTP_URI}/{path}', **(request_kwargs or {}))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/aiohttp/client.py”, line 663, in _request
conn = await self._connector.connect(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/aiohttp/connector.py”, line 563, in connect
proto = await self._create_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/aiohttp/connector.py”, line 1551, in _create_connection
raise UnixClientConnectorError(self.path, req.connection_key, exc) from exc
aiohttp.client_exceptions.UnixClientConnectorError: Cannot connect to unix socket /var/lib/incus/unix.socket ssl:default [Connection refused]