Intel X557-AT2 single port speed?

TrueNAS General
,
1

I swapped motherboards in my server, and the integrated Intel X557-AT2 is only getting about 600Mb/s instead of the 10Gb/s my AQC107 NIC card did.

Is this a limitation of the Intel controller dividing resources among two ports?

2

How did you test speed?

3

I sent data from a 4th-gen PCIe drive in a networked PC to another in the server and watched the transfer rate in Windows.

Both drives are in PCIe 3rd-gen slots.

4

Ok. To benchmark the network speed, you should better use something like iperf3 (not sure how to do it from the win machine, though). Thus, you will be able to mitigate potential drive-related and smb-related bottlenecks.

Also, what speed is shown by sudo ethtool <your-nic-name> in the truenas shell?

5

This is the closest thing I found to Speedtest.net for a LAN.

image
image528Ă—348 9.96 KB

Here’s the output of Ethtool:

image
image408Ă—402 16.1 KB

6

Still looks like an SMB test (at a first glance at least). Nevertheless, 1900/5000 Mbps looks much better than 600 Mbps.

Also, windows explorer shows speed in MB/s (with a capital B). So, there is a chance that “600Mb/s” was actually 600MB/s → x8 → 4800Mb/s. Which is not that bad.

IMO, your pool can potentially saturate 10Gbps with (linear) write. But I’m not sure on how to troubleshoot this issue further… I mean, if you think the issue is related to the new NIC, it’s better to run iperf (perhaps you can boot with some live cd linux distro on your win pc).

Negotiated speed looks ok.

1 Like
7

SMB is the share method I’m using, seemed a safe one to use in the test.

Also, I believe you’re correct on my speed typo. Sorry for the confusion.

However, my primary activity on this server is to store weekly “system images” of my PCs, which are over a terabyte each.

The sooner the transfers complete, the sooner I can resume productivity.

8

Yeah. My point was that if you think the issue is with the NIC, you should test the NIC (and not the overall performance).

Could it be that you are using windows built-in backup&restore as your backup solution?

9

You’re not familiar with OpenSpeedTest or Librespeed? OpenSpeedTest is available as an app on TrueNAS CE iirc. Iperf would probably be the most accurate/lowest overhead, though.

1 Like
10

No: I make a backup to the local PC’s secondary NVMe drive first, then send a copy to TrueNAS for safekeeping.

11

I Agree.

Would running the command from a live USB3 drive be a potential bottleneck?

12

Download iperf3 and execute it from the windows terminal

13

Looks like I just might be getting 9.22Gb/s, but only a few seconds at a time might not be filling the buffer?

image
image1115Ă—628 44.7 KB

14

well this is from my mellanox connectx-3 only set jumbo frames to 9014 no other setting changed

image
image1112Ă—632 47 KB

15

Welp, it seems there is a network issue after all. Too bad that there are no retr and cwnd fields in the result.

Anyway, how is your PC connected to the truenas? Is it a direct connection or via switch/router?

16

Through an unmanaged 10Gb switch:

I also upgraded my server’s network cable from a 6-foot cat7 to cat8, should I try putting the cat7 back in and see what happens?

17

Try to connect them directly. You would need to set static IPs (from the same subnet) on both truenas and the win pc. Then perform the test(s) again.

In my opinion, all these categories are kinda bullshit as long as your cables are below 10 metres and your environment doesn’t have massive EMP. But I’m not a network expert, so I could be terribly wrong. You can try to put cat7 back just in case you think cat8 could be damaged somehow.

18

I swapped cables and plugged it into the other RJ-45 port.

No change.

19

By “directly” I meant “without a switch”. Just to exclude some possible reasons for network instability.

To make that, you would need:

  1. Set a static ip on truenas port. You can use the same ip you have now – 10.0.0.3/24. Not sure whether truenas obtained it from dhcp (your router) or whether it is already static (and not sure about the subnet mask really being 24). This guide can help a bit. Btw, it is generally recommended to have a static ip on the NAS. Set the default gateway to your router ip (perhaps it is 10.0.0.1).
  2. Make sure you still have a connection to truenas.
  3. Set a static IP (for example 10.0.0.10 with subnet /24 aka 255.255.255.0, or just use your current address) on your windows pc port. Essential Network Settings and Tasks in Windows - Microsoft Support – see the section “To specify IPv4 settings manually”. Set the default gateway to your router ip.
  4. If you don’t have other clients with specified ip (10.0.0.10) in your LAN, all should work as before.
  5. Disconnect truenas and win pc from the switch.
  6. Connect their NICs directly with the cable.
  7. You won’t be able to connect to the internet now.
  8. Make sure there is no other connections/cables on truenas and pc.
  9. Perform the test(s).
  10. Reconnect pc and nas to the switch (as it was before).
  11. Optionally: revert to the dhcp setting on pc and/or truenas.
20

I have a crossover cable, but it’s not long enough.

I have a M.2 10Gb NIC with the same AQC107 chip that worked in the old setup coming tomorrow.

If that doesn’t fix the speeds, I’ll try what you suggested.