Data migration with hard links

TrueNAS General
,
1

Hello,

I am migrating data from one pool to another.

I have some data hard linked like this:
Dataset

  • Movies
    • Movie folder
      • Movie file.mkv
  • Downloads
    • Movie file.mkv

The two files are hard linked to save space.
How can I move those two folders while preserving the hard links between them? Thanks in advance

2

Both ZFS Send / Receive and RSync support copying hard links.

I routinely use the following with RSync to copy directory trees with all features:
rsync -aAHSXxv --stats /SOURCE/ /DESTINATION/
And if this is an incremental copy, then I add the option below to make the destination look like the source:
--delete

Now for ZFS replication, (aka Send & Receive), it’s a bit more complicated as the options vary depending on:

  • Source is the top level Dataset or child Dataset
  • If the destination pool is empty
  • Or you want to install the source Dataset in a destination child Dataset

I use the following to clone an entire pool to a different, empty pool:

export MY_SNAP="`date +%H%M%S`"
zfs snapshot -r ${MY_SRC_POOL}@${MY_SNAP}
zfs send  -Rpv  ${MY_SRC_POOL}@${MY_SNAP} | \
  zfs receive -dFu ${MY_DEST_POOL}

zfs destroy -rv ${MY_SRC_POOL}@${MY_SNAP}
zfs destroy -rv ${MY_DEST_POOL}@${MY_SNAP}

Replace MY_SRC_POOL and MY_DEST_POOL with the pool names.

I’ve been told that one of the send or receive options is redundant. But, I use it in a script that works, so I have little incentive to change the script.

If you want to copy just a child Dataset, (and it’s further children), the options vary and I don’t have experience in doing that.

2 Likes
3

Thanks I used the rsync and it seems to have worked perfectly.

I had to move the dirs into a new dir first and copy them at once instead of one after the other.

In a previous thread it was mentioned that after raidz expansion I would lose space due to old data using the original parity ratio.
Is there a script or something to rewrite each data so as to recover the lost space?

Thanks

4

There are multiple scripts floating around the Internet, just search zfs rebalance script.
In one of the previous t3 podcast episodes they said that iX is developing its own solution, I believe called zfs rewrite. But afaik there’s no eta when the feature will land in truenas

1 Like