MacOS slow SMB for large “file”

TrueNAS General
, ,
1

Hi, I’ve just got a new Mac Mini M4. My old one was over a decade old and now very very slow cough

The new Mac had a lot smaller hard drive as now I’ve got the TrueNas server (ElectricEel-24.10.1 ) I don’t need as much local storage. I thought.

My problem is trying to move some old Apple Aperture files for us to the server - they’re about 60-70GB. If I try to copy the Aperture file as is, it’s saying it’s going to take 20 days to move them, and at the speed it’s going I can believe it.

If I zip up the same files, which end up still being the same size, I can move them to the server in a couple of minutes.

I suspect it’s something to do with the fact that these Mac files are really a folder with loads of sub folders and files inside in reality, but the staggering difference in times to copy doesn’t seem right.

2

IMO likely to be caused by synchronous writes. Make sure the dataset you are writing to has sync=Disabled in advanced properties.

3

I have already tried that and it made no difference.

4

Lots of very small files will always have a very high price penalty when it comes to performance (although 20 days does seem excessive). It took me about 4 hours to download a 7.4GB folder from my corporate Nextcloud instance the other day as it contained 5004 small files however as a test I zipped it uploaded it and downloaded again and it took about 5mins.

PS: My guess is that it wouldn’t really have taken 20 days.

2 Likes
5

And that’s the thing - zipped up, it zipped along in a couple of minutes. But zipped up I can’t do anything with it! I certainly can’t access it to get to photos contained in it. I don’t really want to have to faff around with a USB drive at the side of the Mac Mini.

6

So is this a common situation where you need to move 60-70GB of Aperture files from your local computer to your TrueNAS?

Im not very familiar with Aperture but could you not have your library point to your TrueNAS and then when you want to add photos you just simply open Aperture and import directly to the NAS?

7

Perhaps having something like SyncThing or Nextcloud would work better for you where you could save your data to a local directory (or external USB) and in the background SyncThing or Nextcloud will sync this data to your TrueNAS?

That way you get all the performance benefits of having the data local but also the data is synced to the safer longterm device i.e. TrueNAS.

8

That. SMB is really inefficient for picking the metadat for all small files inside the bundle. It would however speed up when moving the bigger data files in there.

9

Looks like I’ll just have to wait it out, or just put the zip version there. They’re being archived, as Aperture is now long dead, but I want to be able to access them when necessary. I figure there’s no easy way of copying the zip version there and then unzipping it?

10

I would use rsync to sort this. Here is an example script that I use to sync my video files from a local drive on my Mac to the TrueNAS system which is running rsycnd. I am using the “rsyncd” App on the TrueNAS system. If you want to do this without installing the rsyncd services, then mount the SMB share locally on the Mac and replace the “DST=bla” with the correct path to the mount (do not include the trailing /).

You should run this the 1st time with like this “./script test” which just does a dry-run and tells you want the rsync command will try to do without doing anything.

#/bin/bash
#PUT

SRC=/Volumes/data1/Media/Videos/
DST=rsync://barrel:30026/videos
EXCLUDE_FILE=rsync-exclude

if [ "$1" == "test" ]; then
    RSYNC_START="/opt/homebrew/bin/rsync --dry-run"
else
    RSYNC_START="/opt/homebrew/bin/rsync"
fi

if [ ! -d $SRC ]; then
    echo "Local directory is not there. Please fix and try again"
else
    $RSYNC_START \
    --iconv=utf-8-mac,utf-8 \
    --force \
    --inplace \
    --size-only \
    --no-perms \
    --no-owner \
    --no-group \
    --omit-dir-times \
    --delete \
    --progress \
    --stats \
    --recursive \
    --exclude-from=$EXCLUDE_FILE \
    $SRC \
    $DST
fi