Rsync failing with "error allocating core memory buffers"

TrueNAS General
1

I’m trying to replicate a 150TiB dataset using rsync.

After a long time, it fails repeatedly with this error:

[sender] exceeded --max-alloc=1073741824 setting (file=sender.c, line=96)
rsync error: error allocating core memory buffers (code 22) at util2.c(80) [sender=3.2.7]

I’ve got another dataset on same server replicating to the same destination without issue (although it’s only 64TiB)

Both source and destination servers are running Dragonfish-24.04.2, with 1024GB RAM and sitting at about 5% CPU load.

The remote server is across a WAN and I’ve got most of the data there already, so switching to zfs replication isn’t a great option.

I haven’t tried running rsync outside the gui yet.

Any pointers?

Thanks
dave

2

You can add auxiliary parms to your rsync with --max-alloc, look it up for syntax.

1 Like
3

Trying --max-alloc=2g right now (1g is the default?), is there any science to determining the appropriate size or just trial and error?

thanks

4

Not that I know of. It’s really a rsync thing, not a truenas thing.

5

With –max-alloc=2g rsync completed successfully (took 7 days)

Thanks for the help here,
Dave

6

According to the man page for current versions of rsync, 1gb is the default.