So I have a batch file that executes a number of robocopys, each one pulls form a Windows SMB share on a 25gig network. I have 16 servers (all dedicated boxes) each with their own storage (normally just a 2 or 4 TB NVME drive) that gets the data.
Today we had 4.209 Gig update of the dataset. Between windows boxes ,windows 11 and 2022 server, we look at 1718 directories and a bit over 744k files equaling around 1.7TB of data. This takes only say, 1-2 minutes on the NVME Windows boxes.
I have three TRUENAS Scale 24.04.2 boxes, one with 4 NVME 4TB disks in ZRAID5, another with 24 1tb SATA disks on two seperate pools, ZRAID6 in each, and the last with 24 1tb SATA disks each with 12 mirrors setup). In all TRUENAS instances, this process takes 50-60 minutes, if not more. Same file source as the rest, just SUPER SLOW. It’s like it takes longer to read what files are on the disk than anything.
Any thoughts on how to speed this up? Make it much faster ?