LACP SMB NAS Optimization

After months of testing and researching on my own to maximize the bandwidth with 2x 2.5gbe adaptors on my TrueNAS server to my 10gbe Mac Studio. I had it kinda working with subnetting and SMB3 multichannel, but I was using 2 NICs on my Mac Studio, 1x 2.5gbe via USB-C adaptor and the built in 10gbe. I asked ChatGPT with “deep research” to find other options, and the results are fantastic. I didn’t know I could assign 2 IP addresses to my Mac’s built in 10gbe NIC. So I got it working by removing that extra NIC, and I’m getting 600MB/sec read/write which is fully saturating those 2x 2.5gbe NICs. Took ChatGPT 14 minutes to research these options. Might trying the bonding option next:

Maximizing Throughput with 2×2.5GbE on TrueNAS Scale and a 10GbE Mac Studio

Overview: Combining two 2.5 GbE links into a single 5 GbE logical connection for one client is tricky. Standard 802.3ad link aggregation (LACP) won’t double the speed for a single file transfer because it sends each single connection down one NIC only. The solution is to use SMB3 Multi-Channel (which opens multiple network streams) or a specialized bonding mode. Below are the best approaches, with step-by-step guidance for TrueNAS SCALE, macOS, and your switch.

1. Use SMB3 Multi-Channel (Preferred Solution)

SMB3 Multi-Channel is designed to aggregate bandwidth from multiple NICs for one SMB client/server pair. TrueNAS SCALE does support SMB Multi-Channel, and macOS (since Big Sur 11.3) enables it by default . The key is configuring each NIC on a different IP network (subnet) so the SMB client can establish parallel streams over each NIC. This avoids LACP entirely and is the most reliable way to get ~5 Gb/s throughput from 2×2.5 GbE.

Steps on TrueNAS SCALE (Server):

  1. Disable LACP: In the TrueNAS web UI, go to Network > Interfaces and delete or disable any existing LACP link aggregation. We want the two NICs (ix0 and ix1, for example) to function as separate interfaces.

  2. Assign Separate IPs: Configure each NAS NIC with a static IP address on a different subnet:

• Assign NIC1 (e.g. ix0) an IP like 192.168.1.10/24 (assuming 192.168.1.0/24 is your main LAN) with your normal gateway.

• Assign NIC2 (e.g. ix1) a static IP on a new subnet, e.g. 192.168.2.10/24. Important: Do not set a gateway on this second interface – this keeps all its traffic local to that subnet.

• (The specific subnets can be anything private/unoccupied – for example, 10.0.0.1/24 for NIC2 is fine if 192.168.2.0 is in use elsewhere. The key is that NIC1 and NIC2 are on different networks.)

  1. Enable SMB Multi-Channel on TrueNAS: Go to Services > SMB > (…) Edit. Under Advanced, enable “Multichannel” and save. (If using TrueNAS SCALE 22.12 or newer, SMB Multichannel may already be enabled by default, but verify this setting.) Restart the SMB service if prompted.

  2. (Optional) Tune SMB: TrueNAS’s Samba will automatically advertise multiple interfaces to clients. No further SMB config is usually needed. However, ensure both NICs are active (link up) and on the same switch/VLAN so the client can reach both.

Steps on the Mac Studio (Client):

  1. Ensure macOS support: Verify you’re running macOS 11.3+ (Big Sur or later) so that SMB multichannel is supported . It is on by default; no need to enable anything unless it was manually disabled.

  2. Add a Second IP to Mac’s NIC: Open System Settings > Network, select your 10GbE interface (e.g. “Ethernet”), and add a second manual IPv4 address:

• Keep your primary IP (e.g. 192.168.1.20/24) as is (this is likely via DHCP or manually set, with your normal gateway).

• Add an Additional IPv4 (sometimes called an “alias”) on the new subnet used by the NAS’s second NIC (e.g. 192.168.2.20/24). In macOS Ventura/Monterey, you can do this under Details > TCP/IP > Configure IPv4: Manually (Add Address). Do not set a router (gateway) for this secondary IP. This ensures Mac uses it only for that direct subnet.

• After adding, the Mac’s single 10GbE port will have two IPs (one in 192.168.1.0/24 and one in 192.168.2.0/24 in our example).

  1. Connect to the TrueNAS SMB share: Use the NAS’s primary IP or hostname to mount the SMB share (e.g. smb://192.168.1.10/sharename). macOS will connect on the primary interface, then discover the server’s additional interface and open a second channel automatically. (You can verify this by running smbutil multichannel in Terminal – it should list two channels, one for each NIC/IP.)

  2. Test Throughput: Copy a large file or use a tool like dd or Blackmagic Disk Speed Test to measure throughput. You should see speeds approaching the sum of both links (around ~300–600 MB/s or more, depending on overhead). In one real example, a TrueNAS user achieved 570 MB/s (~4.5 Gb/s) using two 2.5 GbE links with SMB Multichannel.

Why this works: SMB3 Multichannel opens multiple TCP connections – one over each network path – and stripes SMB file transfers across them. By using two subnets, the OSes will route one connection via NIC1 (192.168.1.x) and the other via NIC2 (192.168.2.x), effectively doubling throughput. There’s no complicated routing or VLAN needed – the networks can coexist on the same switch. As long as no gateway is set on the secondary network, all traffic stays local and won’t confuse your router. This method is tested and recommended by iXsystems for TrueNAS SCALE.

Troubleshooting tips for Multi-Channel:

• In TrueNAS, do not assign both NICs to the same subnet (TrueNAS SCALE will complain or behave unpredictably). Each NIC needs its own IP/network.

• If you experienced previous “network issues” with dual subnets, double-check that the secondary interfaces (on both NAS and Mac) had no gateway and that you didn’t accidentally create an IP conflict. Also ensure both NICs were connected to the same layer2 network (same switch/VLAN).

• If using IPv6, consider disabling it on the secondary NICs to simplify things. Some have found sticking to IPv4 on the isolated multi-channel network more reliable.

• Make sure Multichannel is enabled on both ends (TrueNAS server setting, and macOS hasn’t been disabled via an /etc/nsmb.conf tweak). On macOS, the Apple SMB client is multichannel-capable by default – no need to adjust unless you previously turned it off.

• You do not need to create static routes manually – by using distinct subnets, the OSes will “just know” to use the correct NIC for each subnet (no gateway = direct link). This automates load balancing across interfaces.

• No special switch configuration is required for SMB Multichannel. The switch just sees two separate IP interfaces communicating. Ensure the NAS NIC ports and Mac port are in the same VLAN and not aggregated.

2. Consider Alternative Bonding Modes (If Multichannel Is Not Desired)

If, for some reason, you cannot use SMB Multichannel, TrueNAS SCALE’s Linux base does support other bonding modes that can combine throughput for all traffic. The most relevant is round-robin bonding (balance-rr), which sends packets in sequence alternately over both NICs. In theory this can merge two 2.5 Gb links into ~5 Gb for a single stream without needing multiple IPs. However, this approach is advanced and comes with caveats:

• TrueNAS SCALE’s web UI may not expose balance-rr mode (it mainly supports failover, LACP, etc.). You might need to configure it via the CLI or a custom script. This isn’t officially documented and could be overwritten by updates.

Switch Setup: Your managed switch must treat the two ports as a static “trunk” since both will carry the same MAC/IP. Configure a manual link aggregation (static LAG without LACP) on the two NAS ports. This ensures the switch doesn’t get confused by one machine using two ports.

On many switches, this is simply setting the LAG type to “on” instead of LACP. All 2.5Gb ports in that LAG should be in the same VLAN as the Mac’s port.

TrueNAS Setup: Bond the NICs in balance-rr (mode 0) or possibly balance-xor (mode 2) :

• For balance-rr: This will split packets evenly across NICs. It can deliver full aggregate bandwidth, but out-of-order packets might occur, which can lower real throughput due to TCP reordering overhead. Some users report ~75–80% efficiency of the combined links in practice.

• For balance-xor: You can set a hash policy (e.g. XOR of IP or MAC). However, with one client MAC/IP, XOR will still send all traffic down one NIC unless the hash algorithm includes port numbers and you generate multiple flows.

• After bonding, assign a single IP to the bonded interface (e.g., bond0) as your NAS’s address. Connect the Mac as usual (no extra IPs needed in this method).

Pros: No need for multiple subnets or SMB-specific configuration – bonding works at the network level for all protocols.

Cons: Single-stream performance gain is not guaranteed to be 100%. Round-robin can overload switch buffers or reorder packets, and some switches don’t distribute incoming traffic evenly, so one NIC could still bottleneck inbound traffic. This method is also not officially supported via the TrueNAS GUI, so it requires comfort with manual network config. Only attempt this if you’re experienced or willing to experiment, and ensure you have console access in case you lose network connectivity during setup.

3. Optimizing Standard LACP (for Multi-Client or Multi-Stream)

If you stick with traditional LACP aggregation for the NAS’s two NICs (802.3ad), be aware it will not speed up a single SMB file transfer on its own. LACP is great for overall throughput when multiple clients or multiple connections are involved, but any single TCP connection (like a typical file copy) will be pinned to one NIC by design. However, you can still use LACP in combination with SMB Multichannel or multiple streams:

Switch LACP Config: Ensure LACP is correctly set up on the two 2.5 GbE ports to the NAS and that the switch’s hashing algorithm is optimal. Many switches default to hashing by source/dest MAC or IP. Changing it to include Layer4 (TCP/UDP port) can allow different TCP streams to use different links. For example, if SMB Multichannel opens two TCP connections, a layer-4 hashing policy might send one connection over NIC1 and one over NIC2, achieving load balancing. Check your switch manual for settings like “LACP hashing” or “load balance algorithm” and choose one that uses IP and TCP port fields.

TrueNAS LACP Config: In Network > Interfaces, create a Link Aggregation (LAGG) of type LACP (802.3ad) with the two NICs. Assign it a single IP (your normal LAN IP). TrueNAS SCALE’s LACP should work, but note that early versions had a bug where combined bandwidth didn’t improve because both NICs shared a virtual MAC. Ensure you’re running a recent TrueNAS SCALE release (updates have addressed many networking issues).

SMB Multichannel with LACP: It’s generally not needed to bond and multi-channel (and it’s not officially recommended to use LAGG for SMB MC). If you do try it, you could assign multiple IP aliases to the LACP interface (each alias on a different subnet) and then use SMB Multichannel as in option 1. This is complex and usually unnecessary – a simpler approach is to just use separate NICs for SMB MC. LACP alone is best for aggregating multiple independent clients rather than boosting one client’s speed.

In summary, for maximum one-to-one throughput with minimal complexity, SMB Multichannel is the way to go. It’s a built-in solution specifically for using multiple NICs between the same two machines. TrueNAS SCALE expects you to use separate subnets for each NIC, but that “subnetting” is simply a pair of static IPs – no complex routing or VLANs needed. Once configured, it’s essentially “set and forget.”

By following the steps above, you should achieve close to a 5 GbE link between your NAS and Mac. For most users, the SMB Multi-Channel method (Option 1) will be the fastest and simplest path to saturating the Mac Studio’s 10GbE port with the TrueNAS’s dual 2.5GbE NICs.

References:

• TrueNAS community discussion on LACP vs. round-robin bandwidth combining

• Confirmation that TrueNAS SCALE requires multiple subnets for SMB Multi-Channel

• Example of 2×2.5GbE achieving ~570 MB/s with SMB Multichannel

• Apple documentation confirming macOS SMB multichannel support (10.16+/11.3+)

• TrueNAS forum notes on LACP limitations in SCALE

2 Likes

My words exactly since the great war. Thanks for this awesome writeup.