I’ve been dealing with this problem since TrueNAS core, I through that it was because I never upgraded to scale some weird cypher mismatch happens, but it has been happening after I upgraded to Scale too, so it has been a pain.
This is the python traceback:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/aiohttp/connector.py", line 1098, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs, sock=sock)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/base_events.py", line 1113, in create_connection
transport, protocol = await self._create_connection_transport(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/base_events.py", line 1146, in _create_connection_transport
await waiter
File "/usr/lib/python3.11/asyncio/selector_events.py", line 974, in _read_ready__get_buffer
nbytes = self._sock.recv_into(buf)
^^^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/api/base/server/ws_handler/rpc.py", line 323, in process_method_call
result = await method.call(app, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/api/base/server/method.py", line 52, in call
result = await self.middleware.call_with_audit(self.name, self.serviceobj, methodobj, params, app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 911, in call_with_audit
result = await self._call(method, serviceobj, methodobj, params, app=app,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 731, in _call
return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 624, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 178, in nf
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/plugins/update.py", line 143, in set_train
return self.__set_train(train)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/plugins/update.py", line 150, in __set_train
trains = self.get_trains()
^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 178, in nf
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/plugins/update.py", line 105, in get_trains
trains_data = self.middleware.call_sync('update.get_trains_data')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1030, in call_sync
return self.run_coroutine(methodobj(*prepared_call.args))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1070, in run_coroutine
return fut.result()
^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/usr/lib/python3/dist-packages/middlewared/plugins/update_/trains.py", line 63, in get_trains_data
**(await self.fetch(f"{self.update_srv}/trains.json"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/plugins/update_/trains.py", line 25, in fetch
async with client.get(url) as resp:
File "/usr/lib/python3/dist-packages/aiohttp/client.py", line 1359, in __aenter__
self._resp: _RetType = await self._coro
^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/aiohttp/client.py", line 663, in _request
conn = await self._connector.connect(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/aiohttp/connector.py", line 563, in connect
proto = await self._create_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/aiohttp/connector.py", line 1032, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/aiohttp/connector.py", line 1366, in _create_direct_connection
raise last_exc
File "/usr/lib/python3/dist-packages/aiohttp/connector.py", line 1335, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/aiohttp/connector.py", line 1106, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host update.ixsystems.com:443 ssl:default [Connection reset by peer]
From the host, it seems that it has no problem connecting to the ixsystems servers:
truenas01% echo | openssl s_client -showcerts -servername update.ixsystems.com -connect update.ixsystems.com:443 2>/dev/null | openssl x509 -inform pem -noout -text
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
54:29:03:7a:7e:5f:a3:f2:0e:a4:aa:d3:db:b7:60:f0
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, O = Google Trust Services, CN = WR1
Validity
Not Before: Sep 15 06:43:28 2025 GMT
Not After : Dec 14 06:43:27 2025 GMT
Subject: CN = update.ixsystems.com
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
pub:
04:95:97:32:4b:db:f7:5a:1a:1b:50:24:e8:04:8e:
65:27:96:54:7a:58:3f:64:b0:a4:54:6b:7c:cc:3b:
59:f3:e9:d3:30:90:d4:e0:bd:3e:27:8f:32:49:65:
e8:d7:18:db:8e:0b:cd:a1:0c:c3:69:ab:f5:fc:49:
4c:b0:05:16:74
ASN1 OID: prime256v1
NIST CURVE: P-256
X509v3 extensions:
X509v3 Key Usage: critical
Digital Signature
X509v3 Extended Key Usage:
TLS Web Server Authentication
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Subject Key Identifier:
06:B6:13:66:EB:10:E7:52:7B:F3:CC:FB:F9:1E:C1:7B:80:BE:56:BF
X509v3 Authority Key Identifier:
66:69:49:D4:DE:2A:9C:91:03:CF:89:0E:24:B8:0E:30:03:6E:88:2E
Authority Information Access:
OCSP - URI:http://o.pki.goog/s/wr1/VCk
CA Issuers - URI:http://i.pki.goog/wr1.crt
X509v3 Subject Alternative Name:
DNS:update.ixsystems.com
X509v3 Certificate Policies:
Policy: 2.23.140.1.2.1
X509v3 CRL Distribution Points:
Full Name:
URI:http://c.pki.goog/wr1/S3hvOSet3rM.crl
CT Precertificate SCTs:
Signed Certificate Timestamp:
Version : v1 (0x0)
Log ID : 1A:04:FF:49:D0:54:1D:40:AF:F6:A0:C3:BF:F1:D8:C4:
67:2F:4E:EC:EE:23:40:68:98:6B:17:40:2E:DC:89:7D
Timestamp : Sep 15 07:43:28.987 2025 GMT
Extensions: none
Signature : ecdsa-with-SHA256
30:45:02:21:00:E0:24:23:64:92:01:EA:D3:FE:FC:24:
BA:79:83:11:1C:99:10:D4:31:03:C8:B0:EE:63:D6:EB:
62:CA:95:00:C5:02:20:7E:0E:8B:56:09:EC:48:3F:35:
BB:BF:8D:37:9D:4E:E5:FB:7B:8E:8B:3F:E6:57:92:12:
57:22:D9:7E:27:71:13
Signed Certificate Timestamp:
Version : v1 (0x0)
Log ID : DD:DC:CA:34:95:D7:E1:16:05:E7:95:32:FA:C7:9F:F8:
3D:1C:50:DF:DB:00:3A:14:12:76:0A:2C:AC:BB:C8:2A
Timestamp : Sep 15 07:43:28.974 2025 GMT
Extensions: none
Signature : ecdsa-with-SHA256
30:46:02:21:00:9F:9E:2A:3B:DF:37:AC:99:85:5B:E4:
89:DC:C2:A2:AE:29:77:1B:A8:3E:2E:06:6F:9E:1F:4B:
69:F8:5B:DD:0A:02:21:00:EB:35:D0:FB:7D:60:C4:13:
74:7A:F3:52:74:BC:EA:9F:43:EC:2C:F9:7E:65:13:11:
F5:5F:BB:D1:6B:9A:26:BB
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
ab:3f:71:f9:25:b1:7a:a5:12:23:23:84:7d:b4:b6:f2:1f:21:
24:fc:f7:a1:14:1f:10:32:fe:d8:c9:a2:79:b2:d2:30:3d:61:
dc:6f:a0:94:ec:2e:4c:86:3e:41:6b:24:ac:1c:d9:90:f2:eb:
33:0c:4a:19:ac:17:0d:27:07:38:f3:a1:08:4b:5f:93:25:cf:
89:28:de:b9:cb:4c:0a:e0:5a:fe:93:35:4b:3c:c9:17:2b:49:
6f:db:f2:32:db:fc:a4:ec:1c:cf:e7:af:23:91:fa:4a:52:73:
63:5f:f2:7b:61:76:87:4a:92:55:40:18:f4:59:08:3c:23:42:
87:19:ed:a1:10:f1:40:11:f2:16:43:67:4c:f9:1c:92:a7:d2:
78:91:ff:d2:d6:28:0c:f2:2b:bd:18:d7:6d:0b:49:ec:b3:14:
38:ae:db:59:c0:5c:f6:d1:bc:d9:36:7f:94:35:bd:8a:5e:5c:
b1:35:8c:ea:f1:73:b2:65:f2:82:7e:a3:f5:72:81:e5:da:dc:
5d:a6:bd:5e:01:33:95:a6:29:48:aa:a2:94:ab:b1:6f:6b:0f:
80:81:d6:f0:4b:a5:32:4a:9d:9b:c3:a1:c5:cf:f3:fe:60:a0:
fb:63:cd:eb:0a:b6:1f:ef:63:2e:89:76:8a:84:8e:2c:11:17:
26:51:36:27
I tried to capture the exchange with tcpdump, but everything is hidden behind the TLS layer:
