I have hundreds of ripped CDs that I’m moving from a “sorting” folder to a Music folder entirely within my NAS. Moving things around takes forever.
both folders live in the same library SMB share.
I’m not sending things across the network. (edit: I might be, see reply)
This is entirely in-the-box.
It’s so bad that I wonder whether there isn’t something else going on. Like a setting I haven’t made correctly. I would expect that moving folder locations would be instantaneous. Isn’t this just a pointer operation?
In a previous post, I complained about getting “out of space” errors when trying to move files between SMB shares via MacOS (Sonoma, 14.4.1).
Quick takeaways from that post:
I had plenty of free space on my client machine (~100 GB) and GOBS of free space on the NAS (50+TB).
No quotas set
My NAS was running on some ancient consumer hardware.
The best suggestion put forward in the prior thread was:
The only thing I can think is that it’s maybe a consequence of the 1.7Ghz CPU. I know SMB performance scales with clockspeed, but I wouldn’t expect that to come into play here since I’m just moving files between folders on the server.
See pic attached for what I get to stare at for ~15-45 seconds any time I try to move a single folder over. Transfers are sized anywhere from 500MB - 3GB depending on discography size, and consist of a mix of MP3 & FLAC files with assorted album art + log files, and other metadata.
Googling around, slow SMB performance seems to be a common gripe with mac users. This unraid post talks about macs copying data locally before moving things across shares.
Squid’s reply, assuming it is correct, validates my prior understanding
Moving from one share to another share means that there are two mount points involved. The system is always going to do a copy delete process due to this. Only if the mount point (ie: share) is the same between the source and destination will the move be instantaneous.
Not sure how much of that is valid for TrueNAS, but this is how I imagine things should work.
That thread recommended running smbutils to get details about shares. Here it is in case that’s helpful:
Part of the problem is that you’re moving the files from the NAS to your Mac and back to the NAS. Basically, it has to be read, buffered, sent to your Mac, who then send its back where it has to be re-written locally, which in turn means that it’s interfering with the read operations of the next block.
Instead, I would counsel you to SSH into the NAS and use the “mv” command (at least on CORE) to move files/directories locally instead. That should be significantly faster by virtue of cutting out the network to and from transfer to your Mac out of the equation. Or, since you’re using SCALE, use whatever the Linux eqv. to mv is to move directories locally via the allegedly-usable shell in the GUI of SCALE instead.
I use CORE and achieve write speeds of about 400MB/s if the 10GbE is operational and the files are large (but my NAS also benefits from a sVDEV). One of the issues with Mac is that even if you have a 10GbE interface attached and allegedly in charge (i.e. with the interface listed at the top of your network interface options), sometimes MacOS decides to ignore that and still do all its communication via WiFi, for example. Thus, I turn off all other network interfaces whenever I need to be sure I’m actually getting 10GbE.
AFAIK, you definitely want the Apple Character Encoding to be ON. Otherwise, there will be all sorts of pain re: your file and directory names undergoing unwanted renames.
As an aside though, SMBv3 is smart enough to know that if the source and destination are actually on the same server (regardless of the differing mount points), then the move doesn’t actually traverse the network (it only appears that it does). IE, very very easy to exceed line speed of your network when moving between two differing mount points that are both set to use the cache properly.
Bolding mine. I’ll be the first to admit I don’t know anything about SMB but given my library share is seemingly using SMB v3.1.1 then either Squid is wrong or there’s something else going on.
I’m on SCALE, but your advice is still valid - it most certainly would be faster. the only wrinkle is that I’m selectively merging these two folders. Some artists are new, so it’s a straightforward mv operation. Other artists already exist in the Music folder and only need new albums added. Then there’s the annoying case where certain albums need replacement because prior rips were MP3 and I now have FLAC versions.
It’s not quite so many files that I’d consider writing a script. There’s also no time pressure for doing this so I’m chipping away little bits at a time. Still it is hundreds of folders.
Interesting. Since I am using CORE 13.0u6.1, it’s presumably on a much older version of SMB than the one running in SCALE. The fact that MacOS is getting in the way suggests to me that something non-SMB 3.1.1 is going on. It could very well be a Mac issue.
Since the new files (i.e. FLAC) are the new master, there doesn’t seem to be a case where a conflict is actually a problem. Whether a file exists or not, the file being moved is the one that ought to be there once the operation is complete. So you don’t really need a script as long as mv is smart enough to merge folders appropriately? Not sure if it can, have never used it that way.
I forgot to turn the Mac encoding on, if I go in after I get this scary box. How scared should I actually be? Is it really going to stop me accessing some files on both Macs and Windows?
I don’t know the answer… @anodos on the old forum knew everything there was to know about Macs and SMB playing nice. On this forum his user name may be quite different. Based on the user logo, the old @anodos may have become @awalkerix. But I am not sure.
Are the source and the destination different datasets? Because if so, they’re different filesystems, so the recent server-side copy improvements may not apply.
I have a _sort folder right beneath library (e.g. library\_sort).
@dan is it the case that contents in _sort would belong to the library dataset? If so, would moving that content into the music dataset count as moving them between datasets? Even if it’s a child dataset?
In many cases Finder does not clearly relay failure cause from filesystem clients back up to user. In this case the MacOS kernel client op fails with EXDEX (filesystems are on different mountpoints). I’ve mentioned on several occasions that some SMB clients do not handle nested datasets / filesystems well and that datasets should not be used in lieu of directories (although they’re easier to create in the GUI).
IIRC I filed a bug report to apple about this finder error handling a few years ago.
