I have a all NVME TrueNAS, with dual 25 gig NICs bonded to two swiches. I can get 10+ gig out of it when transferring a few large files without issues. it is domain joined.
The issue i have is a robocopy issue, I have a folder that i sync up. It takes 20+ minutes to JUST start copying. Once it starts, it goes quickly enough, but just to start it takes forever. I have transferred any new or updated data to boxes with a 1gig connections before this even starts updating.
The data is roughly 1.6 million files, accross 3100+ folders, giving a total of 4.54 TB of data. Most of the other systems that I transfer to (this is a community dataset to it has to be transferred to several systems, are VERY VERY quick but they are windows boxes.
I would think the issue is in something to do with the scans from robocopy being slow or having some kind of issue, not hardware related. something with the dataset config etc…
Well, if we do 1333 x 2.84M (avg. file size) x 8 ~ 30Gbit. Which is suspiciously close to 25Gbit link you mentioned. Can it be that robocopy scans and compares entire files before copying (instead of just comparing modification dates)?
In this case the data transferred is 13gig of updates. It should be done within 10 min if that, but still going started at 7:42 it is now 8:19, no file transfer movement yet. Still scanning i guess. So that’s almost 40 min…
It’s clearly taking a long time to enumerate the directory structure. Are you copying ACLs every time? If so could you disable that part and see if that improves things?
I did googling a bit. Seems like robocopy doesn’t scan entire file, so it’s not the case. However, I found people suggested to split the job to the smaller ones. And that it should improve overall time.
I suspect that it can be related to Queue Depth equal to 1, when there is only 1 job running. I’ve checked QD1 IOPS for a single nvme, it is about few thousands. Which is pretty close to the 1300 IOPS mentioned above.
I thought robocopy was slow under Scale also. I ended up adding in a 128GB L2ARC on the computer under my Details section. I sync about 13TB of data over SMB. What use to take over an hour and 20 minutes is down to ten to 15 minutes.
I don’t know of something is off on the usage of ARC, SMB and Robocopy but Core was quicker. Scale behaved like it had to scan the entire set of data instead of just the changed files needing updates.
