Can you post the output of sudo dmesg | grep ixgbe?
Maybe something in the boot log will give a hint as to what is going on.
@Alister Swapped the cards. NIC is now showing…
LnkCap: Port #0, Speed 8GT/s, Width x4,
But still isnt connecting to network.
@neofusion This is me booting into Mint mind you (with the NIC and GPU swapped ports)
[ 9.999105] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver
[ 9.999110] ixgbe: Copyright (c) 1999-2016 Intel Corporation.
[ 10.735860] ixgbe 0000:2b:00.0: Multiqueue Enabled: Rx Queue count = 32, Tx Queue count = 32 XDP Queue count = 0
[ 10.842703] ixgbe 0000:2b:00.0: 31.504 Gb/s available PCIe bandwidth (8.0 GT/s PCIe x4 link)
[ 10.967976] ixgbe 0000:2b:00.0: MAC: 4, PHY: 0, PBA No: H86377-007
[ 10.967987] ixgbe 0000:2b:00.0: ec:e7:a7:12:3e:80
[ 11.134665] ixgbe 0000:2b:00.0: Intel(R) 10 Gigabit Network Connection
[ 11.908184] ixgbe 0000:2b:00.1: Multiqueue Enabled: Rx Queue count = 32, Tx Queue count = 32 XDP Queue count = 0
[ 12.016332] ixgbe 0000:2b:00.1: 31.504 Gb/s available PCIe bandwidth (8.0 GT/s PCIe x4 link)
[ 12.142008] ixgbe 0000:2b:00.1: MAC: 4, PHY: 0, PBA No: H86377-007
[ 12.142019] ixgbe 0000:2b:00.1: ec:e7:a7:12:3e:81
[ 12.318593] ixgbe 0000:2b:00.1: Intel(R) 10 Gigabit Network Connection
[ 12.320739] ixgbe 0000:2b:00.0 enp43s0f0: renamed from eth0
[ 12.330695] ixgbe 0000:2b:00.1 enp43s0f1: renamed from eth1
[ 43.269010] ixgbe 0000:2b:00.0: registered PHC device on enp43s0f0
[ 43.562965] ixgbe 0000:2b:00.1: registered PHC device on enp43s0f1
[ 48.705144] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
[ 55.989419] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Down
[ 59.486612] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
[ 86.022234] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Down
[ 89.529389] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
[ 116.054033] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Down
[ 119.958615] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
[ 146.087067] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Down
[ 150.006610] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
[ 176.119239] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Down
[ 179.502610] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
[ 206.151834] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Down
[ 210.046700] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
[ 236.161864] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Down
[ 239.601409] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
[ 266.194301] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Down
[ 270.145392] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
[ 296.227212] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Down
[ 300.214504] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
[ 326.259030] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Down
[ 330.233879] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
[ 356.291732] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Down
[ 359.729496] ixgbe 0000:2b:00.0 enp43s0f0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
It’s possible that it’s trying to negotiate a 2.5 connection?
try forcing 1 GB
Assuming your interface is enp0s3
sudo ethtool -s enp0s3 autoneg off
sudo ethtool -s enp0s3 speed 1000 duplex full
sudo ethtool -S enp0s3 # Verify the settings
enp0s3 is actually the onboard NIC. That one works in Mint, but not in TrueNas (the entire reason i got the Intel NIC)
enp43s0f0 is the Intel interface
change enp0s3 to enp43s0f0 in those commands
I wanted to do another test, so i removed it and put it in my primary PC (the one im writing this with) which is running Windows on an x570 platform. Only change i made in BIOS was forcing that slot on Gen3 (which i may try on the server). After leaving my onboard connected so it could pull drivers, i switched the cable over and here i am. So this at least confirms the card DOES work (both ports) and working at 2.5Gb (what my current switch is).
Oh, to add, its also pulling the static IP’s i set on my router, so that works as well.
I’ll try turning off the auto negotiation as well when i get it back in the server.
So it’s negotiating 1 Gbps and then dropping entirely 20-30 seconds later.
It will be interesting to see the output of ethtool -S.
Also sudo ethtool enp43s0f0 to see what ethtool thinks about the card’s capabilities.
@Alister ethtool command not found
sudo ethtool
Wait, what version of TrueNAS is this?
Oh i was running this under Mint, not truenas because thats what @Alister had me to initially to pull the logs.
Okay, maybe ethtool is not part of the default mint installation.
I would suggest you move the card back to the TN system, but maybe there is a pressing reason not to.
The 3.0 x1 slot should let it transfer at 1GB/s at least.
Another option would be to get an m.2 to 10GbE adapter… a bit of a hack. (You can probably definitely pick up a no-name variant for less)
@Alister ran those setting in TNS linux shell. While it did set the speed to 1000 duplex, it doesnt seem to want to shut off auto-negotiation.
@neofusion seems to read what the card can do correctly. all the way up to 10gig
Could you please post the output of:
sudo ethtool -S enp43s0f0
and
sudo ethtool enp43s0f0
?
is there an easy way to do that without me typing it all out?
The only way would be to have the usb NIC plugged in and ssh into the server on that (using putty)
then you can copy and paste
Can we summarise what we’ve discovered
- Card under Windows works in the x570 at 2.5G on both ports
- Card in B550 is seen in x16 Elec at x4
- Card in B550 is seen in x1 Elec at x1
- Card in B550 in x4 elec connects @ 1G for 20 secs before dropping - in mint
So we have a working Card but a problem in the B550 motherboard
There a Wifi card in the B550 which can be removed - you unscrew the antenna shroud and pull
This might be conflicting with the x550?
Apologies, I bungled the B, I meant to write Gb/s.
I was aware it could be removed, and i forgot to mentioned that i did disable that card in BIOS. But i could try removing it, but it will take time.
Also to this point
- Card in B550 in x4 elec connects @ 1G for 20 secs before dropping - in mint
the card behaves the same in the x1 slot as well. I’ve seen no change in behavior in either PCI slot.
Completely removed the Wifi card, even went in the bios and disabled the onboard Ethernet. No difference.
In fact the on board Ethernet still shows up under TNS network interface configure screen.
I’m starting to think the X550 just isn’t compatible with TNS.
I moved it to my old hardware and booted up TNS, same thing. No connection, even when forced to 1gb.
Old hardware runs a 2500K cpu, which i know is only PCIe 2.0, which would slow the connection, but right now im just trying to get any connection.
[edit]
I finally got it to work. LOOONG story short, for some reason, Linux/TNS + Intel x550 + This 2.5gb Switch just DO NOT like each other.