Hum. Okay let me see if I can keep it simple. Hash is a mathematical formula used to distribute something. In the case of what I am talking about a TCP flow between the host to the server when the path has a LAG. The switch needs to know which link to put that traffic on so it looks at some salt (name for input variable of the hash). In the case above it looks at some version of what I linked to (the switch vendor will have documentation on what they do but there is only a limited number of ways to do this.
The formula for unfragmented TCP and UDP packets is:
hash = source port, destination port (as in the header) hash = hash XOR source IP XOR destination IP hash = hash XOR (hash RSHIFT 16) hash = hash XOR (hash RSHIFT 8) hash = hash RSHIFT 1 And then hash is reduced modulo slave count.
So it takes this output and says “hey this means this flow should go over link X of the LAG”.
A new flow will have a new hash and might pick the same link or might pick a different link. In normal network operations things will generally evenly balance out.
As i’ve experimented the SMB Multichannel, its quite clear to me but seems like i’ll also have to experiment with the LAGG as well and then decide.
Just to let you know, there are three clients that will be connected to the NAS, and all are flash (the NAS and the clients) but once i test everything, depending on that, i’ll install the NICs as i’ve a few options for that. However, it will be usually two clients accessing the NAS simultaneously, but for a majority of the time, it will be a single client only.
As far as i’m aware, the switch also needs to support the LAGG feature right?
Also, is LAGG and LACP bond two different things?
In addition, do you guys have any idea when there will be support for SMB Multichannel on TrueNAS Core or any way to enable that? I’ve been running core but i experimented the SMB Multichannel thingy on TrueNAS Scale on a separate NAS and client. Seems like i’m going to use SMB Multichannel for my use case but then, do i really need to switch to SCALE? Are there any other benefits in terms of performance or reliability when comparing Core and SCALE?
LAG = Link Aggregation Group
LACP = Link Aggregation Control Protocol
A LAG is the name for a grouping of physical paths. You can do a static LAG - ie, the device just does what every hashing it wants on its side and the other side can do something different. LACP is like routing protocol. It allows the devices to speak to each other and agree on how they would like to talk and exchange information about the status of the LAG.
The SMB is a single threaded connection and generally, its recommended to use the highest core clock for a CPU. But does this also applies to SMB Multichannel as well?
Secondly, how many cores does the SMB Multichannel can use for a given setup? Does the number of cores being used depends on the number of ports on the NIC?
Lastly, do you have any idea when there will be support for SMB Multichannel on TrueNAS Core or any way to enable that? I’ve been running core but i experimented the SMB Multichannel thingy on TrueNAS Scale on a separate NAS and client. Seems like i’m going to use SMB Multichannel for my use case but then, do i really need to switch to SCALE? Are there any other benefits in terms of performance or reliability when comparing Core and SCALE?
SMB Multichannel is supported by Samba not by TrueNAS Scale or Core.
Latest version of Core comes with Samba 4.15 that supports SMB multichannel perfectly fine. You can enable it in UI or via Auxiliary parameters. Often its good to specify interfaces capabilities explicitly, so that Samba understands that interface supports RSS.
So, does that mean if a NIC supports RSS, then despite being a single port card, the SMB Multichannel can work?
So, if i’m not wrong, for the above results you posted, it seems like that this Mellanox ConnectX-4X can utilize up to 16 threads simultaneously if configured using different clients (SMB) and with a single stream but with SMB Multichannel configured?
And is there any way to increase these channels for more parallel stream for a single connection?
Wow. Are you sure that the version you mentioned has SMB Multichannel support for the CORE?
So I checked my Windows 11 box and my MacBook Pros. The MacBook Pros support RSS and the Intel 225-V 2.5G does not. It did at one point but Intel decided to remove support. There are a few threads on this and how to try to hack it back on but none work.
It’s just generic RSS capability of Mellanox driver. You can tune number of CPU used via a registry parameter. It applies to a nic port as a whole.
I didn’t check but 4 channels is more than enough to saturate 10gbps that is the speed of connection in that example. In theory Samba and Windows look at 2 parameters : RSS and speed to determine how many channels they will negotiate. That’s my understanding.
Yes, Core 13 U6 comes with Samba 4.15 that supports SMB multichannel just fine
