[EFAULT] [SSL: TLSV1_UNRECOGNIZED_NAME] error on establish SSH connection

oxyde · November 4, 2024, 1:53pm

UPDATE INFO: 26/11

Hope someone can point me in the right direction, because im obviously missing something crucial to get a replication task working between 2 system.
The final goal is to put the target system at my parents home, setting up 2 PULL replication task, and let both machine replicate each other on specific datasets… but actually im neither capable to perform the task in the same LAN.
Source system is a previous Core updated to EEL, target system is a fresh EEL install.
Same simpthone occur with another system, Core based, that previously was using flawlessy for a weekly pull backup.

Following the migration path for the source system, i have created a new user, group administrator, set an home directory, generated/assigned an Authorized Keys, SSH password login enabled, Allow all sudo commands, Allow all sudo commands with no password
same check on the admin target system
the public key of source admin user has been saved into backup credentials → ssh keypars of the target (and viceversa)
SSH is enabled, can say it work with putty/remote desktop manager without issue (nor ip and hostname)
both system use the same wildcard certificate and CA’s
TLS 1.2 and 1.3 enabled as default

I was pointing my attention to Nginx Proxy manager, because i have changed the default ports of the GUI, and let NPM manage the ports 80/443… but on the target system i have reverted that and literally nothing changed…

Actually i can only PUSH from the source, but every attempt to establish an SSH connection from the targets sytem fail with

truenas [EFAULT] [SSL: TLSV1_UNRECOGNIZED_NAME] tlsv1 unrecognized name

oxyde · November 4, 2024, 8:18pm

Somehow i managed to connect from source system to the target system with a succesfully push replication, using the IP.
As test i bind the target system again for using diiferents ports for GUI, and no problem, so i think can exclude NPM as well.

But im still unable to connect from target to source, the SSH connection fail with an error on TLS (both system have TLS 1.2 e 1.3)… I really think Is some missconfig on the source inherited importing config file from Core, or related to the user admin created, but i can’t really understand what… Still Hope in a good tip

truenas [EFAULT] [SSL: TLSV1_UNRECOGNIZED_NAME] tlsv1 unrecognized name

oxyde · November 9, 2024, 7:05am

Tryng again… No one has a tip!?
I would really avoid a fresh install without config upload (last last Plan)

oxyde · November 19, 2024, 1:10pm

Last try, hoping to receive assistance

oxyde · November 26, 2024, 9:47am

i’m still there, opened a ticket too but still no assistance

bacon · November 26, 2024, 10:07am

SSH does not use TLS.

You mention you have “wildcard certificate and CAs” - are these certificates trusted by a public CA or did you create your own CA?

I am assuming correctly that you’re using the “Semi-automatic” method? Are you using a https URL in the TrueNAS URL setting and if so, have you tried using a http URL (for testing purposes)?

oxyde · November 26, 2024, 10:27am

Good point, didn’t know. This confuse me a bit more about the error

wildcard is associated to a domain that we use at work, used by 100+site, trusted by public CA. Im just pointing my router dns for use same domain locally

yes, semi automatic.
Tested every combination, nor hostname and ip, nor both schema, without success. Despite, don’t remember if the manual connection work (if i remember well, last attempt produce seems going good but couldn’t effectly read from source the datasets, i will turn on the system and retry)

bacon · November 26, 2024, 11:59am

I don’t think there should be a TLS related error if you use a http:// url. Can you double check that you’re getting the same error?

The replication uses the TrueNAS API in order to configure the remote system. Can you execute the following command on the local TrueNAS instance in a shell and check if you’re getting the same error:

(Replace HOSTNAME with the hostname of the remote TrueNAS instance. Use hostname:port if not listening on default port 443.)

midclt -u wss://HOSTNAME/websocket -U truenas_admin -P password call system.info

In order to check if your certificate is correct please try the following:
(Replace both HOSTNAME instances with the remote TrueNAS hostname)

echo "Q" | openssl s_client -verify_return_error -verify 10 -connect HOSTNAME:443 -verify_hostname HOSTNAME 2> /dev/null | grep "Verify return code"

You chould get Verify return code: 0 (ok) if everything is OK.

oxyde · November 26, 2024, 1:08pm

I run those commands directly in the shell of the main system, meanwhile i can go power on the other nas.

result of the first command:

{“version”: “TrueNAS-SCALE-24.10.0.2”, “buildtime”: {“$date”: 1731090342000}, “hostname”: “truenas”, “physmem”: 33509007360, “model”: “Intel(R) Xeon(R) CPU E3-1260L v5 @ 2.90GHz”, “cores”: 8, “physical_cores”: 4, “loadavg”: [0.09912109375, 0.10205078125, 0.09033203125], “uptime”: “13 days, 2:50:26.531781”, “uptime_seconds”: 1133426.531780862, “system_serial”: null, “system_product”: null, “system_product_version”: null, “license”: null, “boottime”: {“$date”: 1731492466000}, “datetime”: {“$date”: 1732625893000}, “timezone”: “Europe/Rome”, “system_manufacturer”: “FUJITSU”, “ecc_memory”: true}

result of the second command:

Verify return code: 0 (ok)

i will update soon with the other info

bacon · November 26, 2024, 1:18pm

Please execute the commands on the main system, but use the hostname of the backup system.

oxyde · November 26, 2024, 1:52pm

didnt understand what you ask me to do, my english sometimes sux.
Meanwhile i updated to the last Scale version on the bacup system, just to be sure.
Can run smoothly both commands from backup system to the main (not what u ask me).
Instead, first command from main system to backup system produce this error:

Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/websocket/_http.py”, line 155, in _get_addrinfo_list
addrinfo_list = socket.getaddrinfo(
^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3.11/socket.py”, line 962, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/usr/bin/midclt”, line 33, in
sys.exit(load_entry_point(‘truenas-api-client==0.0.0’, ‘console_scripts’, ‘midclt’)())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/truenas_api_client/init.py”, line 614, in main
with Client(uri=args.uri) as c:
^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/truenas_api_client/init.py”, line 293, in init
self._ws.connect()
File “/usr/lib/python3/dist-packages/truenas_api_client/init.py”, line 75, in connect
self.socket = connect(self.url, sockopt, proxy_info(), None)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/websocket/_http.py”, line 122, in connect
addrinfo_list, need_tunnel, auth = _get_addrinfo_list(
^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/websocket/_http.py”, line 167, in _get_addrinfo_list
raise WebSocketAddressException(e)
websocket._exceptions.WebSocketAddressException: [Errno -2] Name or service not known

despite the second command dont respond anything

Trying the first command with the internal IP i got different error, depending what port i use:
with 880 (80)

Traceback (most recent call last):
File “/usr/bin/midclt”, line 33, in
sys.exit(load_entry_point(‘truenas-api-client==0.0.0’, ‘console_scripts’, ‘midclt’)())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/truenas_api_client/init.py”, line 614, in main
with Client(uri=args.uri) as c:
^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/truenas_api_client/init.py”, line 293, in init
self._ws.connect()
File “/usr/lib/python3/dist-packages/truenas_api_client/init.py”, line 75, in connect
self.socket = connect(self.url, sockopt, proxy_info(), None)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/websocket/_http.py”, line 136, in connect
sock = _ssl_socket(sock, options.sslopt, hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/websocket/_http.py”, line 271, in _ssl_socket
sock = _wrap_sni_socket(sock, sslopt, hostname, check_hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/websocket/_http.py”, line 247, in _wrap_sni_socket
return context.wrap_socket(
^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3.11/ssl.py”, line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3.11/ssl.py”, line 1108, in _create
self.do_handshake()
File “/usr/lib/python3.11/ssl.py”, line 1379, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:992)

with 444 (443)

Traceback (most recent call last):
File “/usr/bin/midclt”, line 33, in
sys.exit(load_entry_point(‘truenas-api-client==0.0.0’, ‘console_scripts’, ‘midclt’)())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/truenas_api_client/init.py”, line 614, in main
with Client(uri=args.uri) as c:
^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/truenas_api_client/init.py”, line 293, in init
self._ws.connect()
File “/usr/lib/python3/dist-packages/truenas_api_client/init.py”, line 75, in connect
self.socket = connect(self.url, sockopt, proxy_info(), None)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/websocket/_http.py”, line 136, in connect
sock = _ssl_socket(sock, options.sslopt, hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/websocket/_http.py”, line 271, in _ssl_socket
sock = _wrap_sni_socket(sock, sslopt, hostname, check_hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/websocket/_http.py”, line 247, in _wrap_sni_socket
return context.wrap_socket(
^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3.11/ssl.py”, line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3.11/ssl.py”, line 1108, in _create
self.do_handshake()
File “/usr/lib/python3.11/ssl.py”, line 1379, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for ‘192.168.1.144’. (_ssl.c:992)

EDIT: more info:

manage to create the ssh connection on the backup system, but i end with this trying to create the replication task

bacon · November 26, 2024, 2:10pm

You usually can’t use IP Addresses with SSL/TLS. Certificate Authorities certify hostnames, they typically don’t certify IP Addresses.

The first error you got looks like a resolution error. Make sure that your hostname is 1) fully qualified (for example use nas.domain.tld instead of just `nas) and 2) included in the SAN of the certificate and 3) resolves to the correct ip.

Second error is you trying to connect via HTTPS to a HTTP port. If you use https:// or wss:// you need to use the HTTPS port. If you use insecure protocols such as http:// or ws:// use the HTTP port.

Third error is you trying to use an IP address, but the certificate doesn’t certify that IP address.

I think last error is when you connect with an ssh account that doesn’t have permission to run zfs commands. I think the manual explains how to setup manual replication? What user do you use to connect to the remote machine?

oxyde · November 26, 2024, 3:12pm

got it. Honestly, just tried everyting to see if something changes

Honestly i really didn’t expect that error.
After that command fail, i try a basilar ping on the hostname and my system can’t resolve it.
Hostname is right, DNS is right (every device in the lan can reach it, except the nas), the certificate is a wildcard, so every subdomain is allowed.
For sure something i missed previously, but it can impact if i try to establish an SSH connections from backup to main? The backup system resolve the main, but can’t connect

The user i create following the core migration path guide. For be 100 clear, is not the truenas_admin on the main system, it has been deleted automatically importing my core config

I have another system that is impacted the same (but previously of the update of the main system worked), i will check if there is a dns problem there too

oxyde · November 26, 2024, 5:02pm

Small update: odd (at least for me) but both EEL host can’t resolve any DNS hostname that are bind directly in my router.

Screenshot 2024-11-26 175017

DNS are the 8.8.8.8/ 8.8.4.4 on both. Having free access to the domain, i just pointed the 2 hostname to my local ip addresses (don’t wanna expose my ip) and now they can at least reach the hostname with the ping command.

So i repeat the test above on both, and the result of previous command now is what i think we should expect (first command with the system info, and the second with Verify return code: 0 (ok) message).

I tried again to setup a new SSH connection on the backup system, but the access denied error above is still there. I checked again for find any difference from the root user (inherited from Core config) and the new user i have created before disable root password but i don’t find anything relevant (except i have allow sudo without pass for praticity).

Tomorrow i will power on the Core system and wanna see if there i have the same DNS resolution problem (as i say, before migrate to Scale tha main system, from Core to Core pull replica worked flawlessy)

Did you see anything that i’m missing? thanks a lot for the assistance!

bacon · November 26, 2024, 5:04pm

Can you check cat /etc/resolv.conf on TrueNAS? You need to use the IP of your router as the DNS Server in your TrueNAS installations.

oxyde · November 26, 2024, 5:43pm

truenas% cat /etc/resolv.conf
domain **.cloud
nameserver 8.8.8.8
nameserver 8.8.4.4

everything else seems working fine. If point the main hostname on the net can be enough i will not wanna risk other problem, but im on you.

Just powerup the Core system: the command from Core to Scale fail both.
First command:

Traceback (most recent call last):
File “/usr/local/bin/midclt”, line 10, in
sys.exit(main())
File “/usr/local/lib/python3.9/site-packages/middlewared/client/client.py”, line 558, in main
with Client(uri=args.uri) as c:
File “/usr/local/lib/python3.9/site-packages/middlewared/client/client.py”, line 286, in init
self._ws.connect()
File “/usr/local/lib/python3.9/site-packages/middlewared/client/client.py”, line 124, in connect
rv = super(WSClient, self).connect()
File “/usr/local/lib/python3.9/site-packages/ws4py/client/init.py”, line 216, in connect
self.sock.connect(self.bind_addr)
File “/usr/local/lib/python3.9/ssl.py”, line 1343, in connect
self._real_connect(addr, False)
File “/usr/local/lib/python3.9/ssl.py”, line 1334, in _real_connect
self.do_handshake()
File “/usr/local/lib/python3.9/ssl.py”, line 1310, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_UNRECOGNIZED_NAME] tlsv1 unrecognized name (_ssl.c:1134)

second command:

Verify return code: 20 (unable to get local issuer certificate)

Trying to setup the SSH connection i got the same error of the first command via the GUI

From Scale to Core, despite, both command works

bacon · November 26, 2024, 6:10pm

You may want to set Nameserver 1 to 192.168.1.1 and leave Nameserver 2 empty. That should fix your name resolution error.

I don’t have a TrueNAS Core installation. I do wonder if the SSL trust store is outdated on TrueNAS Core. An alternative is that the configured certificate is missing some elements of the chain. Is it possible that you recently renewed the certificate and that started the problems?

oxyde · November 26, 2024, 6:37pm

I will try for sure.

The wildcard has been renewed some months ago (April if i well remember) and before upgrade the main nas to Scale, the backup system pulled without issue
Got all files from the vendor, already tried to reinstalling It, only something i could have done wrong can explain something break in the chain… But how (Is just a copy paste operation), and on 3 different systems? I will check again just for be 100% sure.
The push from the main nor to the Scale and core works… But no way in the opposite way

bacon · November 26, 2024, 6:54pm

I can only really assist with SSL or networking related issues. If you do manual setup you don’t need to bother with SSL or Certificates. But I can’t assist on setting up the replication itself because I’ve not used it.

oxyde · November 26, 2024, 7:31pm

Really appreciate your help.

To do list for tomorrow