Hi,
We have two NAS boxes with replication enabled between them. We recently updated to TrueNAS-SCALE-23.10.2 from TrueNAS-SCALE-23.10.1.3 and all the replication tasks are failing.
Following is the error we get (by the way DEBUG logs don’t show right away after selecting them in the replication window and you still see DEFAULT logs (which was the previous log configuration) on the subsequent runs for some time):
[2024/04/22 17:00:01] INFO [Thread-992] [zettarepl.paramiko.replication_task__task_3] Connected (version 2.0, client OpenSSH_9.2p1)
[2024/04/22 17:00:01] INFO [Thread-992] [zettarepl.paramiko.replication_task__task_3] Authentication (publickey) successful!
[2024/04/22 17:00:02] INFO [replication_task__task_3] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots:
[2024/04/22 17:00:02] INFO [replication_task__task_3] [zettarepl.replication.run] For replication task ‘task_3’: doing push from ‘/path/to/dataset’ to ‘/path/to/dataset’ of snapshot=‘auto-2024-04-21_18-00’ incremental_base=‘auto-2024-04-21_17-00’ include_intermediate=False receive_resume_token=None encryption=False
[2024/04/22 17:00:02] INFO [replication_task__task_3] [zettarepl.paramiko.replication_task__task_3.sftp] [chan 5] Opened sftp connection (server version 3)
[2024/04/22 17:00:02] INFO [replication_task__task_3] [zettarepl.transport.ssh_netcat] Automatically chose connect address ‘server1.example.com’
[2024/04/22 17:00:10] WARNING [replication_task__task_3.process] [zettarepl.transport.base_ssh] [ssh:root@server1.example.com] [replication_process:task_3] Listen side has not terminated within 5 seconds after connect side error
[2024/04/22 17:00:10] ERROR [replication_task__task_3] [zettarepl.replication.run] For task ‘task_3’ unhandled replication error SshNetcatExecException(ExecException(1, ‘Traceback (most recent call last):\n File “/tmp/zettarepl–transport–ssh_netcat_helper.py–564cc2c2c2638b13411dcdd001299f3c”, line 75, in \n s.connect((args.connect, args.connect_port))\nOSError: [Errno 113] No route to host\n’), None)
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/zettarepl/replication/run.py”, line 181, in run_replication_tasks
… 15 more lines …
File “/usr/lib/python3/dist-packages/zettarepl/replication/process_runner.py”, line 33, in run
raise self.process_exception
File “/usr/lib/python3/dist-packages/zettarepl/replication/process_runner.py”, line 37, in _wait_process
self.replication_process.wait()
File “/usr/lib/python3/dist-packages/zettarepl/transport/ssh_netcat.py”, line 210, in wait
raise SshNetcatExecException(connect_exec_error, self.listen_exec_error) from None
zettarepl.transport.ssh_netcat.SshNetcatExecException: Passive side: Traceback (most recent call last):
File “/tmp/zettarepl–transport–ssh_netcat_helper.py–564cc2c2c2638b13411dcdd001299f3c”, line 75, in
s.connect((args.connect, args.connect_port))
OSError: [Errno 113] No route to host
Here’s the replication configuration:
I have checked the connection and the network and it is working fine so don’t think that is the issue and it appears to be a bug with the later version.
Really appreciate any help regarding the issue.
Thanks,