I’m new to TrueNAS scale 25.04, but relatively competent in Linux and Debian generally. I have a new build that I’m trying to get on my 2.5GB switch, but the X520 DA2 with an RJ45 SFP+ module isn’t working and doesn’t show an active link. I do not see any issue in dmesg and the card is showing properly with lspci. I can boot in to a Debian live USB and everything is working fine, so it’s something specific to TrueNAS.
The card itself is being detected and I’ve done some troubleshooting across several threads. The bit that enables non-Intel modules was already flipped and the module is working in other operating systems. I saw reference to “enabling” the network interface in TrueNAS but haven’t found anything configurable in the UI that speaks to “enablement”.
Any thoughts on what I might need to do to get the card with module working in TrueNAS Scale? I tried adding links to this post from the various forums I’ve used for troubleshooting, but get an error when attempting to post.
You need to post better details on what you have tried. Be verbose on the steps you have taken. What is working fine using the Debian live USB? Do you just get a link or are you getting 2.5Gb connections.
You should be able to post images and such after viewing a few other forum threads and doing the Bot tutorial
TrueNAS-Bot
Type this in a new reply and send to bring up the tutorial, if you haven’t done it already.
@TrueNAS-Bot start tutorial
1 Like
So the module works in live cd’s and this is isolated to truenas, which is based on Debian? Seems janky if the card needs encouragement to use the sfp.
Thanks for engaging and apologies for the lack of verbosity. I’ll try to detail what I’ve done.
In Debian, I get a link and a 2.5GB connection. This required no setup on my part.
In TrueNAS, the card is detected and both ports are detected. However, there is no link and neither end of the ethernet show link lights.
I have confirmed the availability of the card and lack of errors with the following:
lspci -v | grep -i SFP
0a:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 560SFP+ Adapter
0a:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 560SFP+ Adapter
lspci -k -s 0a:00.0
0a:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 560SFP+ Adapter
Kernel driver in use: ixgbe
Kernel modules: ixgbe
lspci -k -s 0a:00.1
0a:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 560SFP+ Adapter
Kernel driver in use: ixgbe
Kernel modules: ixgbe
dmesg | grep -i gbe
[ 0.991774] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver
[ 0.991776] ixgbe: Copyright (c) 1999-2016 Intel Corporation.
[ 2.138717] ixgbe 0000:0a:00.0: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0
[ 2.139023] ixgbe 0000:0a:00.0: 16.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x4 link at 0000:00:1d.0 (capable of 32.000 Gb/s with 5.0 GT/s PCIe x8 link)
[ 2.139349] ixgbe 0000:0a:00.0: MAC: 2, PHY: 1, PBA No: G43260-012
[ 2.139356] ixgbe 0000:0a:00.0: 8c:dc:d4:af:c0:d4
[ 2.142611] ixgbe 0000:0a:00.0: Intel(R) 10 Gigabit Network Connection
[ 2.306247] ixgbe 0000:0a:00.1: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0
[ 2.306543] ixgbe 0000:0a:00.1: 16.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x4 link at 0000:00:1d.0 (capable of 32.000 Gb/s with 5.0 GT/s PCIe x8 link)
[ 2.306862] ixgbe 0000:0a:00.1: MAC: 2, PHY: 17, SFP+: 5, PBA No: G43260-012
[ 2.306868] ixgbe 0000:0a:00.1: 8c:dc:d4:af:c0:d5
[ 2.309183] ixgbe 0000:0a:00.1: Intel(R) 10 Gigabit Network Connection
[ 2.314600] ixgbe 0000:0a:00.1 enp10s0f1: renamed from eth1
[ 2.314924] ixgbe 0000:0a:00.0 enp10s0f0: renamed from eth0
I have confirmed the EEPROM bit for enabling non-Intel modules is correct with
ethtool -e enp10s0f1 offset 0x58 length 1
Offset Values
0x0058: fd
1 Like
From what little I know dealing with my own SFP+ interfaces in BSD and Linux, it looks really happy with the card, so I can only blame the SFP, especially because it’s so picky with modules you gotta toggle a bit. Figured that would be something you fixed in firmware on the card, not something an OS needs to know about. Is there another way to make your card less picky, or to test a different SFP module?
I lucked out with my 10 pack of generitrons (joke brand) SFP’s and some Mellanox Connectx-3’s and some Lenovo Intel 710’s. Neither card cares what plugs in.
Also you might try just a straight up little chiggety-China 2.5g pci-e ethernet card. I started with those and they were dirt cheap (and of course worked in Core under BSD and Scale under Linux). I dug it up from my order history, searching for this should land it: 2.5GBase-T PCIe 3.1 Network Adapter with Intel I225-V. It’s a $25 test you can return if it doesn’t work or keep a cheap solution, maybe even spend a few bucks more and get the double header.
I’ll grab a PCIe NIC and test it. I already have the x520 and have tried two separate brands of modules with no luck.
Seems unlikely that it’s the NIC specifically since it works just fine in Debian (and OPNSense).
I will report back here. Thanks for the assist!
1 Like
Good luck, sounds like you got a unicorn that works anywhere you don’t need it lol. Hope the test passes. Those cards worked for me.
The 2.5GB card was having the same experience, but I have figured out the issue (user error). Link lights and link will show down until a static IP is configured or DHCP is turned on for only the desired card. For those looking in the future:
TrueNAS install has the user select a default NIC and that is the only NIC that has DHCP. No other card, regardless of connection or active link, will show as active until an IP address or DHCP is configured for that specific link (and disabled for the default selected during install).
This is not a good user experience and I’m sure there are valid reasons to configure it in such a way.
Thanks for the help, @afrosheen! Appreciate you looking at it. I guess this is what another user meant when they said they “enabled” the card.
1 Like
What an odd design choice. No link until it has an ip is wild. Link state always told me it was plugged in or not, because it wasn’t evaluating anything beyond the physical link. Glad you got it solved even if it didn’t want to be solved that way lol.
