Support samba block cloning on ZFS (Veeam B&R slow merge)

We use the TrueNAS Scale 24.04.2.2 system
X11SSH-F + E3-1220 v6 + 32Gb ECC
12x 6Tb HDD in pool RAIDZ2, 2x SSD 240Gb OS
as a backup storage for latest Veeam Backup&Replication over NFSv4.1 (4.2 not supported by Veeam), and we are facing the problem of very long merges of backups.

While Windows repositories with ReFS backend and SMB merge copies in 10 minutes, our installation merges copies in 10+ hours (!). The size of one incremental copy is about 1TB.
In Windows/ReFS/SMB installations, Veeam uses the faststone mechanism to merge copies and create a synthetic full copy, this works very quickly, but this does not happen in ours: the merge takes a very long time, and the synthetic copy will take more than a few days

I have reproduced a similar configuration in a virtual environment: Veeam, TrueNAS SCALE 24.10-RC.2 with 6x vdrives of 100 GB each. I made sure that the same problem was present, then switched the connection to SMB. The problem remains.

As a result of searching for the reason, I found out that the FSCTL_DUPLICATE_EXTENTS_TO_FILE instruction in SMB, as well as the flag of its support in the SMB2 protocol [MS-FSCC (*1)]:
FSINFO/FileFsAttributeInformation - 0x08000000 FILE_SUPPORTS_BLOCK_REFCOUNTING

This flag is present in Windows/ReFS/SMB and is missing in TrueNAS.
According to the Samba documentation, this instruction is officially supported only in Btrfs, the functionality that is implemented through Samba VFS.

ZFS supports this feature since 2.2.0, which is called reflink or block cloning.

I’ve found mentions of slow Veeam+TrueNAS merges (*2)

In the current Release Candidate version of TrueNAS, we do not see any improvements in this regard. Now there is a question of expanding the space and we will not be able to afford such low performance due to the lack of support for functions that work out of the box in Windows for several years now

Are there any developments to implement this functionality in TrueNAS?

The limitations of the forum do not allow to leave more than two links in a post, so the remaining 2 links that I would like to leave here:
(*1): [MS-FSCC]: FileFsAttributeInformation | Microsoft Learn
(*2): Veeam Backup and Replication - Backup Copy, merge very slow | TrueNAS Community

I just wrote / merged support for the required SMB IOCTLs for VEEAM fast copy over SMB to our samba branch for fangtooth. At present the feature is targeted for our enterprise user base.

and

If you’re morbidly curious.

So great news for enterprise users.

2 Likes

You’ve done a very good job. I am glad to hear this news :grinning:
When can I expect this in the TrueNAS Community Edition?

We have various features (like fibrechannel support and RBAC for AD / LDAP) that are for our enterprise userbase and are not on the roadmap for the community edition. At present, that is the case for this feature.

So do I need to compile samba from the repository to get support for these functions on my installation?

I’m not going to provide guidance on circumventing our licensing structure, but if you are interested in licensing details you can contact our sales department or someone higher up at the company than me.

Unfortunately, it is highly likely that even if we want to purchase TrueNAS Enterprise, it will be difficult, we are from Russia. :slightly_smiling_face: I am considering various options.

Thanks for the information provided, it will be useful to other users who use TrueNAS in similar configurations.

Instead of SMB, aren’t you able to create a Linux repo in Veeam? Block cloning works with that.

OpenZFS block cloning is an experimental feature in Veeam,
I am also working on it.

Do you have successful experience in this configuration?

UPD: in the context of truenas, I do not like this configuration, as it involves enabling developer mode to install the Veeam agent and it will be deleted after updates
I will also try the night build on 25.04, hopefully there will be a new samba package with new functionality.

I will try to configure Veeam Agent in the lab first, enable block cloning, then update truenas to 25.04, reconfiguring it to SMB and try to run it there.
The main goal I am pursuing is the minimum amount of overwriting of existing copies, since there are 40TB of them in the working environment, it will take forever to overwrite them

I do – it works fine as far as I can tell. Although that’s not on TrueNAS, but Proxmox.

Sorry, what? Why would anything be overwriting current backups? Block cloning is used only for synthetic fulls.

I want to use them to perform merging of old incremental copies into a full copy