Sneakernet replication possible?

When it comes to replication, is replication via sneakernet from one NAS to the backup NAS possible?

Ie

export snapshots from primary NAS to a DAS
Transport DAS to remote NAS
Import snapshots at remote NAS from DAS

Especially if the primary and remote NAS are trying but failing to replicate their initial dataset snapshots due to the size of said snapshots, network interruptions, etc.

I’ve done that using an external USB drive in the past :slight_smile:

Basically the process was:

  1. Connect drive
  2. Create pool on drive.
  3. Setup local replication job from Pool 1 → USB Pool.
  4. Run replication
  5. Export pool
  6. Sneaker-net to new system
  7. Import Pool
  8. Setup local replication job from USB pool back to Remote Local Pool
  9. Run job

Downside of that vs a DAS is that its a single drive, so no redundancy for the sneakernet portion if something goes wrong in transit. But ZFS will let you know if anything checksum-wise looks off.

3 Likes

And afterwards, the primary NAS can replicate to remote NAS just the incremental snapshots?

Correct, if they share the same set of snapshots as the “origin” you can then continue on with replication directly from system A → B.

1 Like

To keep things simplified, and more likely to “just work”, use the same options for steps #3 and #8, whether you use the GUI or manual command-line.

1 Like

The output of ZFS Send is a data stream, which can be saved as a file. This file can be copied via many different methods to the remote site. Including via a ZFS pool on a USB disk. Slightly different from @kris’ solution.

I have personally used a full ZFS Send output file to restore a pool remotely. (The ZFS Send file was copied via SCP.)

One weird / neat thing about incremental ZFS sends to remote locations, via manual methods, is that you can collect a few on the transfer media. Meaning you might do incremental ZFS Sends once a week, but only update the remote pool once a month.

You can collect 4 incremental ZFS send files on the USB drive. Then, as long as you ZFS Receive the files in the right order, (oldest first), you should be good.

That is useful if you have a remote place like a cabin, or relative that you don’t visit as regular as your ZFS snapshot schedule is performed.

3 Likes

Wait. @Constantin, just to make sure about something:

In your opening post, were you implying that when you “travel back to the physical location” of the “remote NAS”, you are going to remove the drives from the DAS and insert them into the remote NAS server to permanently reside here? (In other words, only two pools are involved.)

Or did you mean there are three “pools” involved? Main NAS pool + DAS pool + Remote NAS pool?

Only two pools - one primary, one remote. The initial replication keeps getting interrupted by network disconnects and ZFS send does not handle that well - restarting from scratch every time.

Since my entire primary pool will fit on one 12TB drive, I’ll simply follow @kris instructions to get the initial replication done. Subsequent incrementals are tiny by comparison and should execute without issues.

Btw, latest scale versions will successfully finish a remote replication even in the presence of disconnects.

It automatically uses bookmarking.

Bookmarks or resume tokens?

Probably the latter :slight_smile:

Just to close this out, the sneakernet replication was 100% successful despite breaking many rules such as using a cheap external enclosure, USB, port multiplication, etc.

Multiple TB made their way into the remote pool, while throwing up SATA errors likely associated with the less-than-optimal enclosure / protocol for said transfer. But the data is there and the remaining bits and bobs are now uploading remotely.

Thank you, @kris, @stux, @arwen, and @winnielinnie !!!

3 Likes

Excellent news.

We were just discussing IP over Avian carrier, as well as variants like IP over flying Reptile carrier, (but that is so Jurassic!). However, we decided that IP over Flying Mammal was problematic due to potential virus infection.

Your variant of ZFS via sneakernet avoids the inconsistencies of non-human animal involvement. (I assume you wore the sneakers, not your dog or other carrier animal…)

1 Like