[Accepted] Enable Mac SMB (Samba) Server-Side Copy Support by Default or Provide a Toggle in SMB Service Advanced Options

Feature Requests
, , ,
1

Reference Discussion Thread: SMB (Samba) Server-Side Copy Support: Enabled for Mac OS? - #3 by SinisterPisces

Hello. I’m using TrueNAS on a network where I’ve got 3 production Macs accessing various SMB shares.

I want to make sure that these Macs are using SAMBA’s Server Side Copy feature. See: https://wiki.samba.org/index.php/Server-Side_Copy

Samba 4.1.0 was the first release to ship with support for server-side copy operations via the SMB2 FSCTL_SRV_COPYCHUNK request. Clients making use of server-side copy support, such as Windows Server 2012 and Windows 8, can experience considerable performance improvements for file copy operations, as file data need not traverse the network. This feature is enabled by default on the smbd file server.

Note - not enabled for OS X (Macs) unless server Samba includes vfs_fruit module and fruit:copyfile = yes in smb.conf.

Samba 4.7.0 introduced support for FSCTL_DUPLICATE_EXTENTS_TO_FILE, which similarly allows for offloaded clone operations.

Currently, per @swc-phil in the referenced thread above, it is possible to specify the needed SMB parameter via the ‘Additional Parameters String’ under the advanced options of the smb share."

Currently, the fruit:copyfile = yes is not present in 24.10 Electric Eel’s smb4.conf file, and can only be set by adding an optional flag for each share.

Problem/Justification
(What is the problem you are trying to solve with this feature/improvement or why should it be considered?)

As noted above, the fruit:copyfile = yes is not present in 24.10 Dragonfish’s smb4.conf file, and can only be set by adding an optional flag for each share. (I’m not sure of its status in 25.04.0.)

Having to manage this setting per-share to enable server-side copy on Mac clients is doable, but adds an extra step to share creation. It’s not at all obvious that this needs to be done for Mac clients when adding shares, and since it has to be done for every share, it is potentially error-prone (e.g., it’s easier to forget to turn on for each share than it would be to enable a global option once).

Additionally, right now, this feature is not discoverable. I didn’t even realize it was an option I could adjust to improve performance when working with files the Mac I use to adminster the server until I randomly stumbled on a thread here that made me realize this feature existed. A quick trip to the SAMBA wiki confirmed my suspicion that it wasn’t enabled by default, because when it comes to Mac OS’s native implementation, everything about SMB has to be just a little bit more complicated. :stuck_out_tongue:

I’m not sure how difficult this would be to implement, or whether it has any negative implications. So, I propose two ways to implement it.

  1. Change Default SMB4.conf File. Assuming no negative performance or stability implications or other issues that make it a bad idea, enabling this by default will ensure that–at least on this point–Windows and Mac users will each get full advantage of this feature out of the box.
  2. Provide a checkbox to enable this at the SMB service-level in the advanced options. The Advanced Options already contain a couple of Mac-specific options. This could be added alongside them. See the attached screenshot for reference.
    image
    image936×1556 63.7 KB

Impact
(How is this feature going to impact all TrueNAS users? What are the benefits and advantages? Are there disadvantages?)

Adjusting the default or even adding a more visible Advanced Setting will improve the SMB experience for every Mac user using SMB to manage data across more than one dataset/SMB share.

When moving files around between two datasets with shares around, it is much easier for users (especially less technical users or those who don’t have permissions sufficient to access the TrueNAS CLI), to just copy the files from one share to another in the Mac or Windows file manager.

This kind of workflow is quite frequent, and should be equally performant on Windows and Mac machines, without requiring an SMB flag be set manually for every share.

User Story
(Please give a short description on how you envision some user taking advantage of this feature, what are the steps a user will follow to accomplish it)

Assuming the default SMB4.conf has been changed, or the user enables Server-Side Copy in the advanced options for the SMB service (depending on how this feature is implemented), users managing TrueNAS data across datasets via mounted SMB shares will immediately see a benefit during file transfers.

SMB is already known to be potentially slow, especially when copying/moving a lot of small files. This is a common worflow for users who aren’t working with large media, for example. Enabling this feature by default–or making it easy to enable–will in many cases provide better performance (and better perceived performance) during a workflow they’re already using.

4 Likes
2

Please test with Fangtooth… 25.04. Fast copy for SMB got some extra love.

The question is whether MacOS is doing something very different. You might also need to test with the their latest version.

1 Like
3

Thanks! I don’t have Fangtooth installed yet (planning on next week after I hopefully finish some admin work on my datasets), but I’ll test this as soon as I’m able.

4

Just a quick correction, the Samba documentation says this is a global option, so currently instead of adding it to each share, it must be added to the “smb_options” config in the service instead. See the following post for more information:

Personally, I’ve been using this option in a home server for a couple of weeks and haven’t had any issue so far. It would be appreciated if it was an (advanced, disabled by default) option in the SMB service GUI and officially supported.

5

I am on 25.04.1 with Apple extensions enabled, and do not see fruit:copyfile = yes through testparm -v.

6

Any evidence that this is using ZFS block cloning or is just offloading the networking for file copy?

7

Yeah. Here’s how I’ve tried:

  1. Ran this:
zpool list -o name,size,cap,alloc,free,bcloneratio,bcloneused,bclonesaved
  1. Copied a file (using Finder in macOS 15.5 connected via SMB).
  2. Ran it again.
  3. BCLONE_USED and BCLONE_SAVED columns grow by exactly the size of the file.
2 Likes
8

Thanks… useful to know it is working.
@kris

1 Like
9

Thanks for testing this, @gabriel . :slight_smile:

Late last month, I posted over on the Samba mailing list looking to get some clarification on the documentation–particularly why this feature isn’t enabled by default and what the danger of the “lock” is. You can see the dev’s response and my original message here: https://lists.samba.org/archive/samba/2025-May/251551.html

Quoting the response here for reference:

There are two styles of SSC:

  • the “normal” protocol style called copy-chunk, where the copy is
    requested in IO ranges by the client and performed server-side
  • the Apple way enabled by fruit:copyfile where the client requests the
    whole file to be copied in one request to be performed by the server

The problem with the latter is that for large file the copy takes some
time and meanwhile the client is blocked waiting for IO to complete. If
the copy takes longer then the SMB request timeout time (iirc default
30s) the requests times out and the client will disconnect the connection.

My recommendation is to stay away from fruit:copyfile for these reasons.

My understanding after reading that is that because of the way Mac OS works, enabling this feature will work great with Macs until it fails in such a way that possibly leaves data in an unknown inconsistent state, and there’s not much the user can do to ameliorate that as-is.

So, I’d adjust my original feature request to make this a global option switch with a big warning on it, similar to the red EXPERIMENTAL notice associated with Incus in Fangtooth. Ideally, there’d be a way to override/extend the timeout, as well. But I’m guessing that’s very much a non-trivial change to the backend and UI.

@HoneyBadger I think this explains a bit more why Samba’s default settings are the way they are–they definitely work more consistently across more systems this way. Does this explanation from the Samba team change how iX might approach enabling this feature?

10

You’re welcome :slight_smile:

I understand that would only be problematic if block-cloning doesn’t happen, because otherwise the copy is pretty much instant.

I don’t see how this is different from any other file copy failure (i.e. disconnection). The only difference is that retrying will most likely not work.

I understand that the Samba folks are designing software that makes no assumption about the underlying file system, nor how the shares are configured, so they may not be comfortable recommending this option. But TrueNAS can, so let’s try to reproduce the problem in practice and see how it affects TrueNAS systems:

Test 1

  • fruit:copyfile is enabled.
  • pool_a/one dataset shared as one.
  • pool_a/two dataset shared as two.
  • Copying a large file from share one to itself performs the instant SSC with block cloning as expected.
  • Copying a large file from share two to itself performs the instant SSC with block cloning as expected.
  • Copying a large file from share one to two did not perform a SSC. It was a slow copy consistent with the data being round-tripped to the client. Progress was reported normally on the copy. It took 17 minutes, a lot more than the timeout.

This did not go as expected. I expected it to attempt a SSC and hang, like the Samba documentation describes. This means that either:

  • The blocking behavior does not exist at all in recent versions of macOS
  • macOS will not attempt a SSC copy between different shares.

As we’ll see, it’s the second one.

Test 2

Here I tried to create a situation in which macOS will attempt a SSC copy, but block cloning doesn’t happen, so we can see it hang:

  • fruit:copyfile is enabled.
  • tank/one dataset encrypted.
  • tank/one shared as one.
  • tank/one/two dataset encrypted with a different key.
  • tank/one/three dataset inherits all settings.
  • Copying a file from one to itself performs SSC and block cloning as expected.
  • Copying a file from one to the one/two directory performs the blocking behavior described by the Samba team:
    • The progress dialog shows 0 KiB.
    • After a few seconds, the client disconnects, the share is unmounted and the progress dialog is replaced with the error message “The operation can’t be completed because an unexpected error occurred (error code 100057).”
  • Copying a file from one to one/three exhibits the same behavior.
  • Curiously, the copies have continued in the background in the server. They eventually complete and match the hash and size of the original file.
  • The server remains responsive to other clients. You can instantly reconnect after the error.

Conclusions

With fruit:copyfile on, macOS will attempt SSC copies within each share, but not across shares. If you can ensure block cloning is performed within each share, you won’t run into the blocking problem. As far as I can tell, this means you’re fine if you’re sharing the leaves of your dataset hierarchy as separate shares, but not if you’re sharing the parents.

If you can’t ensure that: users may be disconnected trying to copy files that can’t be block cloned. When they do, failure is clearly communicated. Copies continue in the background. Server remains responsive.

In my opinion:

  • The “bad” scenario isn’t nearly as bad as we thought, no data is lost and the copy eventually succeeds. Still needs more tests.
  • It’s relatively straightforward to ensure you can safely enable it and never run into the blocking timeouts.

So I still think this should be an advanced setting.

Edit: Now for a slightly spicier opinion: Apple’s SMB CopyFile implementation is not broken, it’s behavior makes it seem like it was implemented not for offloading traffic during copies, but instead, precisely for CoW filesystems that can instantly duplicate files. It’s the only reasonable explanation to justify the existence of a copy feature that doesn’t work across shares, that disconnects if it takes more than 30s, and doesn’t report progress at all. It sounds perfectly reasonable to me that to enable it, we must make sure we only expose shares that can make instant copies.

11

A Samba developer saying “stay away from fruit:copyfile” more likely reinforces the stance from our internal developers not to enable this.

Digging a bit more into what is posted of Apple’s open-source code shows that newer releases seem like they’ll support the native SMB offload.

https://github.com/apple-oss-distributions/SMBClient/blob/c709f70bc2b0572aa7169935d2aa74fc197db053/kernel/smbfs/smbfs_smb_2.c#L7262

/*
 * This routine is used for both Mac-to-Mac and Mac-to-Windows copyfile
 * operations.  For Mac-to-Mac, the FSCTL_SRV_COPYCHUNK ioctl is sent with
 * a chunk count of zero, because the server uses copyfile(3) and doesn't need
 * a list of chunks from the client.  To specify Mac-to-Mac semantics, the
 * mac_to_mac parameter should be set to TRUE.
 */

Opinion below. This is not Engineering’s official response.

It’s better not to masquerade the MacOS copyfile(3) functionality and hope that it works in all cases, as shown by @gabriel 's testing where bclone is unavailable and we get the unusual behavior of throws error code but copy continues which is troublesome as we can’t guarantee the client response. Picture an app that is programmatically doing a copy, receives the error code, and then its internal logic says copy failed, retry it or just raises an error indicating the operation failed, but the copy is continuing in the background with no ability to track/be reliably notified.

I’m not sure exactly what version of MacOS is needed to implement this newer code path to leverage FSCTL_SRV_COPYCHUNK - @gabriel you mentioned you’ve tested with 15.5 - does it still engage bclone and SSC without fruit:copyfile?

1 Like
12

It does not engage bclone without fruit:copyfile.

Since the timeout closes the SMB connection, I don’t expect clients to retry (and if they did retry, maybe the file is locked and it’s all fine? needs testing). The code for fruit:copyfile is specifically concerned with cases where FSCTL_SRV_COPYCHUNK attempts to copy the entire file in one chunk, passing zero chunks, it’s targeted enough it’s safe to assume that only Macs will trigger this behavior, and so we can make reasonable assumptions about the client’s retry policy (by observing Macs).

If a different client does implement FSCTL_SRV_COPYCHUNK with zero chunks, it’s reasonable to assume that it understands Mac OS’s semantics and its caveats (and has a similar retry policy)

13

But maybe, looking at the code @HoneyBadger showed, we could somehow make Macs not do the “Mac-to-Mac” branch and still leverage block cloning. I’m not sure how this flag is populated but I want to investigate further.

14

This post was heavily edited as I found more relevant information.

Looking further into the problem, it does seem like Apple should be sending FSCTL_SRV_COPYFILE chunks for copy offloading. But it only seems to do so if vfs:copyfile is enabled.

vfs:copyfile sends the kAAPL_SUPPORTS_OSX_COPYFILE server capability, which enables the SMBV_HAS_COPYCHUNK flag in Apple’s SMB implementation. It also enables the “Mac-to-Mac” semantics that @HoneyBadger mentioned.

What we would really want is to enable the SMBV_HAS_COPYCHUNK without “Mac-to-Mac” semantics (the 0-chunk request that hangs the client if the copy is not instant). This is possible. Apple’s SMB implementation also enables it here if the share passes the smb2fs_smb_cmpd_check_copyfile check. This seems to be a test that the server supports “Compound Request Chains”, and presumably (since we don’t see Macs using FSCTL_SRV_COPYFILE without vfs:copyfruit enabled), Samba’s current implementation does not pass this test.

I believe that if Samba could pass this test, SSC would work, and indirectly, block cloning. It does not seem to be a limitation in Apple’s SMB implementation (kAAPL_SUPPORTS_OSX_COPYFILE is meant for Mac-to-Mac transfers), but instead a limitation in Samba.

I’m not that well-versed into Samba’s codebase to understand why the test fails, and whether Samba needs to implement “Compound Request Chains” or it already supports this but does not respond positively to this test.

1 Like
15

@gabriel thanks for the detailed information and some pointers into the Apple code for where to start digging. Can’t promise any timelines, but I will see if I can hand out some shovels. :slight_smile:

1 Like
16

I looked at the Apple SMB client source. They detect copy-chunk support via a compounded SMB3 request to open root dir of share / issue ioctl to get offload token / close when the share is first mounted. This is a pretty bad test since typically SMB servers treat ioctl requests as requiring async offload and send an interim response to the client. The SMB protocol requires that compounded requests like this fail if they require going async. Therefore, it always fails against Samba, and possibly also against Windows. I think it’s probably OK to work around the bad test by not doing this for the specific ioctl to get an offload token.

This is a google drive link to a patched samba debian file for TrueNAS 25.04.1 (if you install it on an different TrueNAS version you will break your install). This requires install-dev-tools and then dpkg -i <package> then restart the SMB service. Don’t try this if you don’t know what you’re doing, and of course don’t do this on a production system at work.

I’ll try to get it into 25.04.2, but will have to also upstream the fix.

NOTE: this doesn’t use fruit:copyfile and doesn’t require fruit.

2 Likes
18

If Apple SMB is not doing something correct per SMB specification maybe the Samba team could reach out to them? There is a beta underway for the new macOS version which is a good time to get small problems like that fixed.

19

Thanks for taking the time to push this fix, @awalkerix . This is awesome; I’m really glad to see a solution that doesn’t require VFS_FRUIT, too; working with a more default TrueNAS/Samba setup is always better. :slight_smile:

I’m excited to see this coming to 25.04.2: Jira

Since this patch obviates this feature request since it doesn’t require any invocation of VFS_FRUIT, is there something I need to do as the request initiator to close it?

Also, I assume that, for now, this is a TrueNAS-specific fix for a TrueNAS-specific fork of Samba, is that right? Oddly enough, my inquiry to the Samba mailing list asking why fruit:copyfile wasn’t enabled by default got someone over there interested in working on the problem from their end as of this week.

They’re not anywhere close to the point of implementing their own fix, but someone is poking at it. I suspect they’d be open to your patch being upstreamed.

See:
https://lists.samba.org/archive/samba/2025-June/251726.html

APFS is a copy-on-write filesystem, according to Wikipedia. Assuming it
supports clones / reflinks, a 30s client timeout wouldn’t appear
unreasonable.
Similarly for Linux, we could choose to only enable copyfile if the
underlying FS supports FICLONE (i.e. Btrfs, XFS v5 and perhaps ZFS).
Do Apple clients fallback to read/write on copyfile failure?

https://lists.samba.org/archive/samba/2025-June/251727.html

[dedup] Use APFS clone (CoW) on macOS · Issue #219 · pkolaczk/fclones · GitHub indicates a clonefile
syscall, so mapping to FICLONE on Linux would seem reasonable IIUC.