I need some justification on using ISCSI vs SMB/ NFS and general advises specifically for Windows Client. Just migrated my 80TB NTFS data from windows client to TrueNas ZFS, with 12 HDD Z2 config, 1TB Mem. Synethic benchmark looks great but real world usage with apps are terrible.
I have issue with SMB /NFS having very slow file traversal speed, and strange behavior on windows mostly application side, for instance:
File traversal is slower than single spinning rust can(usually takes 3~4 times amount of time to complete)
perhaps related to #1, anaconda is super slow to open, takes about 15~20 s compare to 2 second on HDD
Some application like equalizerAPO is unable to find some config file located on SMB/ NFS share, w error: "The file is not readable for audio service. change file permission or copy the file to the config directory.
Some game won’t run, like EFT and battleeye(not 100% sure it’s smb / nfs issue but very likely)
overall slow app response time
ISCSI solves all potential problem listed above, but also with great cost specially all the goodies ZFS offers:
File version history
no more dynamic space allocation, can only increase space and not reduce zvol
files no longer managed by truenas, im not sure how would corruption / file checking in this case function since iscsi is block level
just overall complicated to use.
Lastly, If i am going ISCSI, i am thinking mind as well just put my boot drive on network also, should I just netboot xyz my boot drive as well and making my windows client diskless? pros and cons? Really appreciate any input.
Are these large directories? If you can keep metadata in the ARC, it will notably make browsing SMB / NFS folders much snappier. (If metadata is being pruned from the ARC too aggressively.)
As far as “cannot open file”, you’ll have to share your setup and configuration. (And what types of names these files are.)
As far as games go, don’t some just outright refuse to run from a network share?
@winnielinnie Yes, very large directories, and seemingly very slow on traversing folders(if there are many subfolders specially, not 100% sure). I actually not sure about metadata that much, im really new to TrueNas, only hands on like couple days. so you will have to enlighten me on that, but that sounded like saving grace to my issue and im very interested.
Blockquote
As far as “cannot open file”, you’ll have to share your setup and configuration. (And what types of names these files are.)
I have EqualizerAPO installed in C(physical SSD), and it’s trying to access E:\EQs\EQconfig.txt , it sees it under GUI trying to browse the config file, but shows error mentioned on my first post once selected. I assume (service, administrator elevated programs, and some programs does not see network attached drives, and, making things worse, I am connected to my TrueNas via Ipv6, some apps like iscsi initiator also hates it.)
Blockquote
As far as games go, don’t some just outright refuse to run from a network share?
yes… only finding out right now, seems like anti cheat specifically doens’t play very well probably again, due to elevated user which doesn’t see the network drive.
It depends on the (TrueNAS) user you’re authenticating as when you connect to the share, and what types of permissions they have granted via the ACLs in the dataset (“filesystem”) on the server.
Metadata are things like filenames, directories, datestamps, filesizes, etc. All these need to be read and populated whenever you navigate or crawl directories. Ideally, metadata stays in the ARC, so any requests are immediately read from RAM and sent over the network. If your metadata keeps getting pruned from the ARC, then it means the server needs to retrieve all this metadata from the spinning drives or SSDs.
You can gauge how much metadata is currently in your ARC:
arc_summary | grep Meta
Also of note is an adjustable parameter (zfs_arc_meta_balance) that you can dial down or up, depending on how much you prioritize metadata staying in the ARC:
@winnielinnie Very interesting; This makes senses. I don’t expect metadata are pruned from ARC since I have 1TB RAM, well ,right now for testing purpose only 450G, but still. Even after I ran anaconda a couple times, the speed doesn’t seem to change. I suspect from Windows Client, metadata is simply not available and windows can’t leverage TrueNas’s metadata. The way i understood, metadata helps TrueNas to find files and folders in ZFS, but i doubt it helps windows.
I ran arc_summary | grep Meta but shows nothing, i think it’s case sensitive and arc_summary | grep meta shows
Blockquote
Also of note is an adjustable parameter (zfs_arc_meta_balance) that you can dial down or up, depending on how much you prioritize metadata staying in the ARC cat /sys/module/zfs/parameters/zfs_arc_meta_balance
The default is 500.You can read more about it [here(ZFS "ARC" is getting smarter with version 2.2+ :smiley_face_emoji: | TrueNAS Community).
very interesting. This is very useful info, even though might not be what’s causing my issue right now.
