Ok you inspired me @Captain_Morgan and after watching your kids play football what else is there really left to do on a Saturday
(other than watch TrueNAS Tech Talk).
I setup a CORE system with a fresh dataset offered out via SMB and the same with SCALE (both running the latest versions) and datasets configured the same way. I created two folders on each share one with with 10K empty .txt files and the other with 50k. Edit added 100K also.
I used a Windows client and along with PowerShell ran the following command: Measure-Command { Get-ChildItem "\\server\share\10K” }
The results surprised me.
CORE: 10k = 6sec
CORE: 50k = 30sec
Edit: CORE: 100k = 58sec
SCALE: 10k = 1sec
SCALE: 50k = 6sec
I then applied the ‘fix’ i.e. “hide unreadable = yes”
SCALE: 10k = 2secs
SCALE: 50k = 10sec
Edit: SCALE: 100k = 19sec
I ran each test about 10 times one after the other and used the rough average and didn’t notice any wild differences between the tests.
So this suggests that SCALE off the bat is much faster than CORE and that even when you apply the ‘fix’ you are still much faster.
Both systems I tested on are recently retired Supermicro servers (same spec) more than capable and the pools were constructed in exactly the same fashion using the same type of HDDs to keep things fair.
I now feel more confident than I did about applying this fix to SCALE but I do still think some consideration should be put into making this default behaviour or at least provide an easy switch for other users.