Replication task from CORE to SCALE, permission error

I am trying to Replicate a dataset from a TrueNAS CORE system to a TrueNAS SCALE system.

On the remote system I create the user replica with these settings:

  • password disabled
  • wheel as primary group
  • allow all sudo commands with no password enabled
  • a home directory
  • add authorized public key from source

On the source system, I setup a SSH connection that uses the user replica.

Unfortunately I get the error
cannot unmount '/mnt/pool2/Destination': permission denied

Any ideas what I am missing?

Here is the full log of the source:

[2025/07/16 10:27:53] INFO     [Thread-4250] [zettarepl.paramiko.replication_task__task_2] Connected (version 2.0, client OpenSSH_9.2p1)
[2025/07/16 10:27:53] INFO     [Thread-4250] [zettarepl.paramiko.replication_task__task_2] Authentication (publickey) successful!
[2025/07/16 10:27:54] INFO     [replication_task__task_2] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2025/07/16 10:27:54] INFO     [replication_task__task_2] [zettarepl.paramiko.replication_task__task_2.sftp] [chan 5] Opened sftp connection (server version 3)
[2025/07/16 10:27:54] INFO     [replication_task__task_2] [zettarepl.replication.run] For replication task 'task_2': doing push from 'pool/Nextcloud' to 'pool2/Destination' of snapshot='NextcloudDaily-2025-07-16_00-00' incremental_base=None receive_resume_token=None encryption=False
[2025/07/16 10:27:55] WARNING  [replication_task__task_2] [zettarepl.replication.run] For task 'task_2' at attempt 1 recoverable replication error RecoverableReplicationError("cannot unmount '/mnt/pool2/Destination': permission denied\nwarning: cannot send 'pool/Nextcloud@NextcloudDaily-2025-07-16_00-00': signal received")
[2025/07/16 10:27:55] INFO     [replication_task__task_2] [zettarepl.replication.run] After recoverable error sleeping for 1 seconds
[2025/07/16 10:27:56] INFO     [replication_task__task_2] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2025/07/16 10:27:57] INFO     [replication_task__task_2] [zettarepl.replication.run] For replication task 'task_2': doing push from 'pool/Nextcloud' to 'pool2/Destination' of snapshot='NextcloudDaily-2025-07-16_00-00' incremental_base=None receive_resume_token=None encryption=False
[2025/07/16 10:27:57] WARNING  [replication_task__task_2] [zettarepl.replication.run] For task 'task_2' at attempt 2 recoverable replication error RecoverableReplicationError("cannot unmount '/mnt/pool2/Destination': permission denied\nwarning: cannot send 'pool/Nextcloud@NextcloudDaily-2025-07-16_00-00': signal received")
[2025/07/16 10:27:57] INFO     [replication_task__task_2] [zettarepl.replication.run] After recoverable error sleeping for 2 seconds
[2025/07/16 10:28:00] INFO     [replication_task__task_2] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2025/07/16 10:28:00] INFO     [replication_task__task_2] [zettarepl.replication.run] For replication task 'task_2': doing push from 'pool/Nextcloud' to 'pool2/Destination' of snapshot='NextcloudDaily-2025-07-16_00-00' incremental_base=None receive_resume_token=None encryption=False
[2025/07/16 10:28:01] WARNING  [replication_task__task_2] [zettarepl.replication.run] For task 'task_2' at attempt 3 recoverable replication error RecoverableReplicationError("cannot unmount '/mnt/pool2/Destination': permission denied\nwarning: cannot send 'pool/Nextcloud@NextcloudDaily-2025-07-16_00-00': signal received")
[2025/07/16 10:28:01] INFO     [replication_task__task_2] [zettarepl.replication.run] After recoverable error sleeping for 4 seconds
[2025/07/16 10:28:05] INFO     [replication_task__task_2] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2025/07/16 10:28:05] INFO     [replication_task__task_2] [zettarepl.replication.run] For replication task 'task_2': doing push from 'pool/Nextcloud' to 'pool2/Destination' of snapshot='NextcloudDaily-2025-07-16_00-00' incremental_base=None receive_resume_token=None encryption=False
[2025/07/16 10:28:06] WARNING  [replication_task__task_2] [zettarepl.replication.run] For task 'task_2' at attempt 4 recoverable replication error RecoverableReplicationError("cannot unmount '/mnt/pool2/Destination': permission denied\nwarning: cannot send 'pool/Nextcloud@NextcloudDaily-2025-07-16_00-00': signal received")
[2025/07/16 10:28:06] INFO     [replication_task__task_2] [zettarepl.replication.run] After recoverable error sleeping for 8 seconds
[2025/07/16 10:28:15] INFO     [replication_task__task_2] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2025/07/16 10:28:15] INFO     [replication_task__task_2] [zettarepl.replication.run] For replication task 'task_2': doing push from 'pool/Nextcloud' to 'pool2/Destination' of snapshot='NextcloudDaily-2025-07-16_00-00' incremental_base=None receive_resume_token=None encryption=False
[2025/07/16 10:28:15] WARNING  [replication_task__task_2] [zettarepl.replication.run] For task 'task_2' at attempt 5 recoverable replication error RecoverableReplicationError("cannot unmount '/mnt/pool2/Destination': permission denied\nwarning: cannot send 'pool/Nextcloud@NextcloudDaily-2025-07-16_00-00': signal received")
[2025/07/16 10:28:15] ERROR    [replication_task__task_2] [zettarepl.replication.run] Failed replication task 'task_2' after 5 retries

Source is TrueNAS-13.0-U6.7
Destination is ElectricEel-24.10.2.3

Try manually entering the new dataset via the replication task. For example if your backup system looks like this tank/Backup point it there but enter the text ‘Stuff’ like this tank/Backup/Stuff. This will then automatically create the Stuff dataset and you shouldn’t get your unmount error.

For info if you want to lock this down then I’d remove this user from the wheel group. Disable ‘allow sudo commands…’ and simply zfs allow replica send pool/dataset. That way this user will only have rights to zfs send and nothing else if it ever got compromised.

PS: this is assuming your are using PULL replication.

Interesting, doing that still failed but the error is different now.

[2025/07/16 16:01:56] INFO     [Thread-4282] [zettarepl.paramiko.replication_task__task_2] Connected (version 2.0, client OpenSSH_9.2p1)
[2025/07/16 16:01:56] INFO     [Thread-4282] [zettarepl.paramiko.replication_task__task_2] Authentication (publickey) successful!
[2025/07/16 16:01:57] INFO     [replication_task__task_2] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2025/07/16 16:01:57] INFO     [replication_task__task_2] [zettarepl.replication.run] For replication task 'task_2': doing push from 'pool/Nextcloud' to 'pool2/Backup/test' of snapshot='NextcloudDaily-2025-07-16_00-00' incremental_base=None receive_resume_token=None encryption=False
[2025/07/16 16:01:57] WARNING  [replication_task__task_2] [zettarepl.replication.run] For task 'task_2' at attempt 1 recoverable replication error RecoverableReplicationError("cannot receive new filesystem stream: permission denied\nwarning: cannot send 'pool/Nextcloud@NextcloudDaily-2025-07-16_00-00': signal received")
[2025/07/16 16:01:57] INFO     [replication_task__task_2] [zettarepl.replication.run] After recoverable error sleeping for 1 seconds
[2025/07/16 16:01:59] INFO     [replication_task__task_2] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2025/07/16 16:01:59] INFO     [replication_task__task_2] [zettarepl.replication.run] For replication task 'task_2': doing push from 'pool/Nextcloud' to 'pool2/Backup/test' of snapshot='NextcloudDaily-2025-07-16_00-00' incremental_base=None receive_resume_token=None encryption=False
[2025/07/16 16:02:00] WARNING  [replication_task__task_2] [zettarepl.replication.run] For task 'task_2' at attempt 2 recoverable replication error RecoverableReplicationError("cannot receive new filesystem stream: permission denied\nwarning: cannot send 'pool/Nextcloud@NextcloudDaily-2025-07-16_00-00': signal received")
[2025/07/16 16:02:00] INFO     [replication_task__task_2] [zettarepl.replication.run] After recoverable error sleeping for 2 seconds
[2025/07/16 16:02:02] INFO     [replication_task__task_2] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2025/07/16 16:02:03] INFO     [replication_task__task_2] [zettarepl.replication.run] For replication task 'task_2': doing push from 'pool/Nextcloud' to 'pool2/Backup/test' of snapshot='NextcloudDaily-2025-07-16_00-00' incremental_base=None receive_resume_token=None encryption=False
[2025/07/16 16:02:03] WARNING  [replication_task__task_2] [zettarepl.replication.run] For task 'task_2' at attempt 3 recoverable replication error RecoverableReplicationError("cannot receive new filesystem stream: permission denied\nwarning: cannot send 'pool/Nextcloud@NextcloudDaily-2025-07-16_00-00': signal received")
[2025/07/16 16:02:03] INFO     [replication_task__task_2] [zettarepl.replication.run] After recoverable error sleeping for 4 seconds
[2025/07/16 16:02:08] INFO     [replication_task__task_2] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2025/07/16 16:02:08] INFO     [replication_task__task_2] [zettarepl.replication.run] For replication task 'task_2': doing push from 'pool/Nextcloud' to 'pool2/Backup/test' of snapshot='NextcloudDaily-2025-07-16_00-00' incremental_base=None receive_resume_token=None encryption=False
[2025/07/16 16:02:09] WARNING  [replication_task__task_2] [zettarepl.replication.run] For task 'task_2' at attempt 4 recoverable replication error RecoverableReplicationError("cannot receive new filesystem stream: permission denied\nwarning: cannot send 'pool/Nextcloud@NextcloudDaily-2025-07-16_00-00': signal received")
[2025/07/16 16:02:09] INFO     [replication_task__task_2] [zettarepl.replication.run] After recoverable error sleeping for 8 seconds
[2025/07/16 16:02:17] INFO     [replication_task__task_2] [zettarepl.replication.pre_retention] Pre-retention destroying snapshots: []
[2025/07/16 16:02:18] INFO     [replication_task__task_2] [zettarepl.replication.run] For replication task 'task_2': doing push from 'pool/Nextcloud' to 'pool2/Backup/test' of snapshot='NextcloudDaily-2025-07-16_00-00' incremental_base=None receive_resume_token=None encryption=False
[2025/07/16 16:02:18] WARNING  [replication_task__task_2] [zettarepl.replication.run] For task 'task_2' at attempt 5 recoverable replication error RecoverableReplicationError("cannot receive new filesystem stream: permission denied\nwarning: cannot send 'pool/Nextcloud@NextcloudDaily-2025-07-16_00-00': signal received")
[2025/07/16 16:02:18] ERROR    [replication_task__task_2] [zettarepl.replication.run] Failed replication task 'task_2' after 5 retries

Ok. I did set “allow sudo command…” because that is what the official doc tells you to do.

I am fine by locking down permission further later on, but for now I would be happy if it even worked :slight_smile:

Push.

I know that this is different from what we discussed in another thread. I came to the conclusion that since I only use root with a random good long password and 2fa, plus a root backup user with public key, and since there is only one IPv6 allowed to even reach TrueNAS destination, this is a risk I am willing to take.

If nothing else works, this might be a sign for me to finally upgrade from CORE to SCALE :sweat_smile:

I havent tried PUSH from CORE to SCALE but I know PULL works fine. Could you try that and reverse your SSH keys and Connection etc?

Honestly I’ve got loads of systems replicating happily from CORE to SCALE (PULL) so I wouldn’t worry we will fix it.

Just remembered to come back here and give you an update :grin:

Upgrading to SCALE did fix the issue.

During the creation of the replication task, I was asked about something about Linux not having the permissions to unmount and either allow sudo or some ZFS unmount option.

Unfortunately I did not take any screenshot.