Hi there,
I’ve just upgraded my newly built TN scale server with a used 10G Mellanox Connect3X card.
The main purpose was to saturate two 2.5G hosts.
Sending and receiving over smb works fine and I get about 280 Mib/s.
When using a single datastream (-P 1) with iperf or rsync (via cifs mount) the server unfortunately only sends at 100-140 mib/s.
Sending to the server with iperf3 gives also the full bandwidth possible.
Any idea how to start debugging this or any general advice?
Thanks in advance!
My Hardware:
SuperMicro X11SSL-F with a XeonCPU E3-1230 v6 @ 3.50GHz (8C)
$ lspci |grep -i mellanox
01:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3]
iperf3 results:
TN sending:
iperf3 -c 192.168.0.13 -f M -P 1 -R
Connecting to host 192.168.0.13, port 5201
Reverse mode, remote host 192.168.0.13 is sending
[ 6] local 172.22.86.169 port 37958 connected to 192.168.0.13 port 5201
[ ID] Interval Transfer Bitrate
[ 6] 0.00-1.00 sec 123 MBytes 123 MBytes/sec
[ 6] 1.00-2.00 sec 139 MBytes 139 MBytes/sec
[ 6] 2.00-3.00 sec 123 MBytes 123 MBytes/sec
[ 6] 3.00-4.00 sec 126 MBytes 126 MBytes/sec
[ 6] 4.00-5.00 sec 138 MBytes 138 MBytes/sec
[ 6] 5.00-6.00 sec 102 MBytes 102 MBytes/sec
[ 6] 6.00-7.00 sec 110 MBytes 109 MBytes/sec
[ 6] 7.00-8.00 sec 109 MBytes 109 MBytes/sec
[ 6] 8.00-9.00 sec 126 MBytes 126 MBytes/sec
[ 6] 9.00-10.00 sec 107 MBytes 107 MBytes/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 6] 0.00-10.00 sec 1.18 GBytes 121 MBytes/sec 1590 sender
[ 6] 0.00-10.00 sec 1.17 GBytes 120 MBytes/sec receiver
iperf Done.
Client sending:
iperf3 -c 192.168.0.13 -f M -P 1
Connecting to host 192.168.0.13, port 5201
[ 6] local 172.22.86.169 port 52914 connected to 192.168.0.13 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 6] 0.00-1.00 sec 284 MBytes 284 MBytes/sec 0 3.37 MBytes
[ 6] 1.00-2.00 sec 280 MBytes 281 MBytes/sec 0 3.37 MBytes
[ 6] 2.00-3.00 sec 280 MBytes 280 MBytes/sec 0 3.37 MBytes
[ 6] 3.00-4.00 sec 281 MBytes 281 MBytes/sec 0 3.37 MBytes
[ 6] 4.00-5.00 sec 280 MBytes 280 MBytes/sec 0 3.37 MBytes
[ 6] 5.00-6.00 sec 281 MBytes 281 MBytes/sec 0 3.81 MBytes
[ 6] 6.00-7.00 sec 281 MBytes 281 MBytes/sec 0 3.81 MBytes
[ 6] 7.00-8.00 sec 281 MBytes 281 MBytes/sec 0 3.81 MBytes
[ 6] 8.00-9.00 sec 280 MBytes 280 MBytes/sec 0 3.81 MBytes
[ 6] 9.00-10.00 sec 280 MBytes 280 MBytes/sec 0 3.81 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 6] 0.00-10.00 sec 2.74 GBytes 281 MBytes/sec 0 sender
[ 6] 0.00-10.01 sec 2.74 GBytes 281 MBytes/sec receiver
iperf Done.