Replication failed: cannot send snapshot not an earlier snapshot from the same fs

TrueNAS General
,
1

Hi,

I am trying the replicate some ZFS snapshots but I am faced with this error:

Replication "local -> remote2" failed: warning: cannot send 'tank1/backup-ext/myuser/mymachine/home@zfs-auto-snap_hourly-2024-07-17-14h44U': not an earlier snapshot from the same fs
Replication cannot continue because existing snapshot zfs-auto-snap_daily-2024-07-17-14h44U is newer than tank1/backup-ext/myuser/mymachine/home@zfs-auto-snap_hourly-2024-07-17-14h44U, but has an older date in the snapshot name. To resolve the error, rename tank1/backup-ext/myuser/mymachine/home@zfs-auto-snap_hourly-2024-07-17-14h44U with a date that is older than zfs-auto-snap_daily-2024-07-17-14h44U or delete snapshot tank1/backup-ext/myuser/mymachine/home@zfs-auto-snap_hourly-2024-07-17-14h44U from both the source and destination. cannot receive: failed to read from stream..

Context:

My NixOS laptop (referred to as mymachine) uses ZFS and takes snapshots (with NixOS module services.zfs.autoSnapshot). These snapshots are replicated on my TrueNAS Core (referred to as mytruenas, running version TrueNAS-13.0-U6.2) using syncoid (with NixOS module services.syncoid). Specifically, the snapshots are replicated on the dataset tank1/backup-ext/myuser/mymachine/home of my TrueNAS. This works fine.

I want to replicate everything from the pool tank1 of my TrueNAS into another pool remote2 which resides on a single disk that I rotate at regular intervals and store offline. That is were the error is happening.

The snapshots mentioned in the error have the exact same timestamps:

root@mytruenas[~]# zfs get -p creation tank1/backup-ext/myuser/mymachine/home@zfs-auto-snap_hourly-2024-07-17-14h44U
NAME                                                                           PROPERTY  VALUE                  SOURCE
tank1/backup-ext/myuser/mymachine/home@zfs-auto-snap_hourly-2024-07-17-14h44U  creation  1721227477             -
root@mytruenas[~]# zfs get -p creation tank1/backup-ext/myuser/mymachine/home@zfs-auto-snap_daily-2024-07-17-14h44U 
NAME                                                                          PROPERTY  VALUE                  SOURCE
tank1/backup-ext/myuser/mymachine/home@zfs-auto-snap_daily-2024-07-17-14h44U  creation  1721227477             -

They were taken as soon as I started my laptop that day.

  • Why is TrueNAS stating that one snapshot is newer than another since they have the same timestamp? A daily snapshot will be taken at the same time as 1 hourly snapshot, an hourly snapshot will be taken at the same time as 1 frequent snapshot. So there will always be cases with snapshots with different granularity that are taken at the same time.
  • Why would it be a problem anyway? One will likely be taken a few milliseconds before the other, how is that a problem for TrueNAS?
  • How can this be fixed permanently? I can fix it for 1 instance but the next day the same problem happens with different snapshots.

I have looked at the similar issue PULL Replication Fails on Creation Time.. bug? | TrueNAS Community but it does not provide answers. Is this a TrueNAS bug?

2

The following pretty much answers it:

source: What are the requirements for naming a ZFS snapshot? - #2 by winnielinnie

Somehow, zettarepl has determined that one snapshot was older than this other.