Only 100baseTX instead of 1000G

Hi there,I use am ASUS P12R‑E/10G‑2T/ASMB10 with 2x Intel X710‑AT2 on board.Each NIC is connected with Cat.7 cable to separate 1 GB switches.The used ports of the switch are set to AUTO (HP switches).Running TrueNAS Core TrueNAS-13.0-U6.7.

But both interfaces are running with 100baseTX:

ifconfig -m
ixl0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN
        options=4a500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,NOMAP>
        capabilities=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        ether 10:7c:61:56:12:41
        inet 192.168.130.200 netmask 0xffffff00 broadcast 192.168.130.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        supported media:
                media autoselect
                media 10Gbase-T
                media 5000Base-T
                media 2500Base-T
                media 1000baseT
                media 100baseTX
        nd6 options=9<PERFORMNUD,IFDISABLED>
ixl1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: SAN
        options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        capabilities=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        ether 10:7c:61:56:12:42
        inet 192.168.110.200 netmask 0xffffff00 broadcast 192.168.110.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        supported media:
                media autoselect
                media 10Gbase-T
                media 5000Base-T
                media 2500Base-T
                media 1000baseT
                media 100baseTX
        nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        capabilities=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
        groups: pflog
ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 61:36:33:63:32:61
        inet 192.168.1.22 netmask 0xffffff00 broadcast 192.168.1.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 58:9c:fc:10:aa:2d
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 7 priority 128 path cost 2000
        member: ixl0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 200000
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.1: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: ServiceTools as nic: epair0b
        options=8<VLAN_MTU>
        capabilities=8<VLAN_MTU>
        ether 12:7c:61:d2:ca:47
        hwaddr 02:c3:63:d3:d4:0a
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        supported media:
                media 10Gbase-T
        nd6 options=9<PERFORMNUD,IFDISABLED>


Have changed the cables and also the ports of the switches. Gluing on 100baseTX.
Any suggestions?

Use fixed speed (on both sides!) instead of autoconfiguration?

Put media 1000baseT into the options field on the TrueNAS/FreeBSD side. For the HP switch you probably know.

Thanks, have set this option and after test of the seeing saved.
There are no changes done:

 ifconfig -m ixl0
ixl0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN
        options=4a500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,NOMAP>
        capabilities=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        ether 10:7c:61:56:12:41
        inet 192.168.130.200 netmask 0xffffff00 broadcast 192.168.130.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        supported media:
                media autoselect
                media 10Gbase-T
                media 5000Base-T
                media 2500Base-T
                media 1000baseT
                media 100baseTX
        nd6 options=9<PERFORMNUD,IFDISABLED>

Then have set the port on switch to Auto-1000 .
Other settings would be: 10HDx, 100 HDx, 10FDx, 100 FDx, Auto, Auto-10, Auto 100, Auto-1000, Auto-10-100

Then the connection breaks. After switching back to Auto, interface is reachable again with 100tx.

Or my I have to reboot TrueNAS after changing network setting?
Thanks for your help.

What happens when you enter

 ifconfig ixl0 media 1000baseT

on the command line? Preferably on the console or connected over a different interface.

I’d typically force the connection speed on one side of the connection (switch or server), not both.

Also, how long is the cable you’re using? You shouldn’t go over 50m if you want 10Gb/s. Over 50m you should only expect 1Gb/s, though this is still more than what you’re seeing.

Have you tried different cables?

That said, there seem to be a lot of people online having similar issues with this NIC chipset on linux-based systems, so is likely a driver issue.

In that case if both sides implement autoconfiguration correctly, the side that is still on “auto” might drop to 10 Mbit/s half-duplex.

Autoconfiguration is not like a modem training on the line. It’s “Does the other side speak the autonegotiation protocol? If it doesn’t drop to 10 M”. And many devices do not speak the protocol when statically configured.

Not my experience with Cisco switches, but good to know for other scenarios :+1:

The only time I have seem this, is when either the cable or the equipment side jack have a problem with 1 of the 2 pairs needed for upgrade to 1Gbit/ps.

Both 10Mbit/ps and 100Mbit/ps use only 2 of the 4 pairs in a Cat 5/5e/6/7 cable. However, every speed above requires all 4 pairs.

Since it has been mentioned that other cables were tried, I would look at the ports on both sides to see if their is a bent pin preventing 1 of the additional pairs to function.

Now I do find it strange that this seems to affect both server side ports. So I consider the above a long shot, but should take less than 1 minute per port for examination, (all 4 ports, server side & switch side).

The only other thing I can think of, is if @digidax made their own cables without testing all 4 pairs. I can imagine that cheap RJ-45 / 8p8c connector heads might have implemented only the lower 2 lower speed pair. Thus, selling the connectors cheaper for 10/100Mbits/ps usage only.

1 Like
root@Storage4[~]# ifconfig ixl0 media 1000baseT
ifconfig: SIOCSIFMEDIA (media): Operation not supported by device

Maybe this would be the reason why the option, I set in GUI, was not executed?

dmesg | grep ixl0
ixl0: Media change is not supported.

Additional some infos from dmesg.boot:

cat /var/run/dmesg.boot | grep ixl0
ixl0: <Intel(R) Ethernet Controller X710 for 10GBASE-T - 2.3.1-k> mem 0xa6000000-0xa6ffffff,0xa7108000-0xa710ffff at device 0.0 on pci5
ixl0: fw 8.13.63341 api 1.12 nvm 8.15 etid 8000a3e0 oem 1.267.0
ixl0: PF-ID[0]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, MDIO shared
ixl0: Using 1024 TX descriptors and 1024 RX descriptors
ixl0: Using 8 RX queues 8 TX queues
ixl0: Using MSI-X interrupts with 9 vectors
ixl0: Ethernet address: 10:7c:61:56:12:41
ixl0: Allocating 8 queues for PF LAN VSI; 8 queues active
ixl0: PCI Express Bus: Speed 8.0GT/s Width x4
ixl0: SR-IOV ready
ixl0: Link is up, 100 Mbps Full Duplex, Requested FEC: None, Negotiated FEC: CL74 FC-FEC/BASE-R, Autoneg: True, Flow Control:
ixl0: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from ixl0 (ifp 0xfffff8000293c000), ignoring.
ixl0: link state changed to DOWN

Cable length is 2 meters. Both switches (one HP and one Netgear) are supporting only up to 1000baseT. 10G are not needed at the moment.

Why the problem is with both onboard NIC’s with different switches and also changed and tested CAT 7 Patch cables, the problem must be in relation of the TrueNAS with this hardware. I’m running 3 other TrueNAS’es with same OS version but different NIC’s, connected also on this switches, all up on 1000baseT. Have also changed the port of the switches and checked the configuration of the port set to AUTO.

maybe this bug is possible here:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=264253

Also interesting:

sysctl -d dev.ixl.0.advertise_speed
dev.ixl.0.advertise_speed:
Control advertised link speed.
Flags:
         0x1 - advertise 100M
         0x2 - advertise 1G
         0x4 - advertise 10G
         0x8 - advertise 20G
        0x10 - advertise 25G
        0x20 - advertise 40G
        0x40 - advertise 2.5G
        0x80 - advertise 5G

and

sysctl dev.ixl.0.advertise_speed
dev.ixl.0.advertise_speed: 199

executing this

sysctl dev.ixl.0.advertise_speed=2
dev.ixl.0.advertise_speed: 199 -> 2

will make the interface unreachable, changing back

sysctl dev.ixl.0.advertise_speed=199
dev.ixl.0.advertise_speed: 2 -> 199

doesn’t bring it back, also not after reboot

ifconfig -m ixl0
ixl0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN
        options=4a500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,NOMAP>
        capabilities=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        ether 10:7c:61:56:12:41
        inet 192.168.130.200 netmask 0xffffff00 broadcast 192.168.130.255
        media: Ethernet autoselect
        status: no carrier
        supported media:
                media autoselect
                media 10Gbase-T
                media 5000Base-T
                media 2500Base-T
                media 1000baseT
                media 100baseTX
        nd6 options=9<PERFORMNUD,IFDISABLED>

and now, the other onboard NIC is going up to 1000baseT:

ifconfig -m ixl1
ixl1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: SAN
        options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        capabilities=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
        ether 10:7c:61:56:12:42
        inet 192.168.110.200 netmask 0xffffff00 broadcast 192.168.110.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        supported media:
                media autoselect
                media 10Gbase-T
                media 5000Base-T
                media 2500Base-T
                media 1000baseT
                media 100baseTX
        nd6 options=9<PERFORMNUD,IFDISABLED>

Have add a D-Link NIC I had here on spare but there is no driver available but surprise:

 ifconfig ixl0
ixl0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN
        options=4a500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,NOMAP>
        ether 10:7c:61:56:12:41
        inet 192.168.130.200 netmask 0xffffff00 broadcast 192.168.130.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=9<PERFORMNUD,IFDISABLED>

Both onboard NIC’s are now running with 1000baseT.

I should be happy but the question at the end is: WHY?

I would try auto on the switch, not auto-1000.

with AUTO the link was set to 100tx, now to 1G

Confusing! ethtool might give you more information.