10 GB connection between PC and NAS

TrueNAS General
1

I already have a TrueNAS and several PCs connected to a switch and router using 1GB connections.

I would like to add 10GB cards to one of the PCs and the NAS to enable this one PC to have a high speed link to the NAS. Both the NAS and my PC are Intel boxes with the NAS running TrueNAS Scale 24.04 and the PC running Windows 11.

The first step is to install 10Gb NICs in the PC and NAS.

One solution is to add a 10GB switch to the network and connect the PC and NAS to it and make a 1Gb connection to the existing switch. The NAS should communicate to my PC at 10Gb and to the rest of the devices and the internet at 1GB. This is simple but expensive (for now).

Alternately, I have heard that I could connect my PC and the NAS with a direct cable between the 10Gb ports and also leave them connected to the 1G switch to connect to the other devices and internet.

I’d like to try option 2 if I can find some guidance for the project. Any help would be appreciated.

Thanks,

Mike

2

10G switches are not necessarily expensive. See Mikrotik CRS-305 for a managed switch, or the many reviews of cheap unmanaged 10G+2.5G switches at ServeTheHome.

For option 2, the only requirement is that the direct link uses a different subnet. So if your home network is 192.168.x.x you could use, say, 172.16.0.1 and .2.

1 Like
3

Simple yes, expensive no. And this really is the better solution.

2 Likes
4

172.16.0.0/16 is used for the apps system, at least through Dragonfish (who knows what will happen with EE), so a different subnet would be better.

2 Likes
5

My advice for a direct connection (with no DHCP - so fixed IP addresses) is to define a 169.254.. subnet as this subnet is specifically designed to be used for non-routable, non-DHCP addressing (though I would recommend you use fixed IP addresses rather than rely on the (pseudo-)random automatically assigned IP address.

1 Like
6

I have a similar situation and have already set up a 10G switch along with 10G network cards in both the server and my desktop. The server is configured with a static IP, and the router has a DHCP reservation for both my desktop and the server. Unfortunately, the DHCP server and the wireless access point (AP) are only 1G connections to the switch. I initially thought that since my desktop, server, and switch all support 10G, I would achieve 10G speeds directly between the server and my desktop. However, I am only getting 1G speeds when transferring data between them. Will I need a 10G DHCP server to resolve this issue?

7

Your AP have no influence , since you are running on cable (wired).
Your DHCP Server (Router) does not influence the speed between the 10G machines, provided they reside in the same subnet (L2 lan/vlan segment)

1:
Do you have any active 1G Cards in the 10G machines ?

2:
Are the 10G machines on the same subnet ?
What are the ip addresses of the machines 10G interfaces.

8

It’s wrong to state that you must have different subnets.

The only true requirement is that you use a different IP address than your current ones.

The OS would ARP on both interfaces and find the other side without issues.

Most IT people know Networking, but very few know it well.

9

Oh boy…

10

Sigh …
1:
We are NOT talking about multiple interfaces on the same subnet.
We are talking about performance.

2:
What would happen if they weren’t on the same subnet (L2 lan/vlan segment) ?
Note the OP stated above that his router was 1Gb.
Do you think that would ressemble the OP’s situation (Hint … Yes)

3:
I used the term “subnet” because the OP did’t seem to be that much in to IP networking , but i specifically added (L2 lan/vlan segment) as a qualifier.

4:
Could you please explain why the IP Stack would ARP for an ip address, that is not on the same subnet ?

ARP might “be lucky to get an answer” if the router was running proxy arp, but that was discouraged many years ago.

5:

I might … Click on my avatar

4 Likes
11

Both machines are connected through their 10G ports, each equipped with Intel 10G dual-port cards. However, when I change the MTU to 9000, the connection drops, and the network interface card (NIC) loses connectivity. To restore the connection, I have to log in directly and reset the networking settings.

I can confirm that both cards are on the same subnet and /16 subnet mask.
My windows PC reports the connection as 10G and the card shows in lspci on the server but I just can’t get a 10G connection.

12

I’m mostly just tinkering would like to set this up as my primary nas if I can get it stable. I’m very much a noob at this

13

You would have to set the switch/ports to 9K MTU also
But even w. a 1500B MTU you should exceed 1Gb by far.

I assume in same VLAN too.
And assume you don’t have any 1Gb netcards active on any of the machines.

Try this on the server , to verify the speed.
https://www.cyberciti.biz/faq/howto-determine-ethernet-connection-speed/

I assume you are running TrueNAS directly on the hardware … AKA Non Virtualized.

14

Windows PC
This is the one connected the other port is not

Intel(R) Ethernet Converged Network Adapter X540-T2 #2
Link Speed (Receive/Transmin) 10/10 (Gbps)

In lspci The NIC shows as X540-AT2 (REV 01) and both ports are shown but only one is connected

In my current configuration this is the iperf3 output

D:\Users\lathe\Desktop\iperf3.18_64\iperf3.18_64>iperf3 -c 172.161.8.1
Connecting to host 172.161.8.1, port 5201
[ 5] local 172.161.1.250 port 56797 connected to 172.161.8.1 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 108 MBytes 896 Mbits/sec
[ 5] 1.01-2.01 sec 113 MBytes 945 Mbits/sec
[ 5] 2.01-3.01 sec 112 MBytes 943 Mbits/sec
[ 5] 3.01-4.00 sec 109 MBytes 921 Mbits/sec
[ 5] 4.00-5.01 sec 113 MBytes 944 Mbits/sec
[ 5] 5.01-6.01 sec 113 MBytes 943 Mbits/sec
[ 5] 6.01-7.01 sec 113 MBytes 944 Mbits/sec
[ 5] 7.01-8.01 sec 110 MBytes 924 Mbits/sec
[ 5] 8.01-9.01 sec 113 MBytes 943 Mbits/sec
[ 5] 9.01-10.01 sec 112 MBytes 943 Mbits/sec

[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 1.09 GBytes 935 Mbits/sec sender
[ 5] 0.00-10.01 sec 1.09 GBytes 934 Mbits/sec receiver

15

Yes, it’s running on bare metal.
I have no virtualization running; it is a basic setup currently with one share.
I don’t have any VLANs setup it’s just my home network with a basic Asus router nothing fancy going on

16

An hub with 2sfp 10gb 4 2.5gb costs 35eur on aliexpress. Just remeber to add at least 2 transceiver r45 at 24eur each, so the tranceivers costs more than switch

17

I am in a 10G switch with both NIC cards

18

Still waiting for the
ethtool <server nic name> | grep -i speed

Is your switch manageable ?
If yes, what does the ports show (connect speed)

Edit:
You can find the server nic name with a:
ip a

Locate the IF with the matching ip address

19

where am I supposed to run that command in the system shell it just returns a bunch of ~

20

Hmmm … I have no idea atm.
I just ssh into the box.