10GigE NIC Swap - A journey

TrueNAS General
, ,
1

I am lucky enough to be able to grab old gear from work when it is EOL. To that, I managed to score 3 HPE DL380 servers last year. They have been working great with a cheap SFP+ Amazon switch and 4x1GB LAGG groups.

Recently, I thought it would be a great idea to buy a Cisco 3850x UPOE 10G copper switch off Ebay. Annnnd, buy some HPE 10G copper NICS.

Happily surprised when the LOM and PCI-e NICS were recognized by CORE (BXE driver). So that is cool. Took a bit of time to update all the firmware of the NICs, and to swap over to 10G, and to get my Cisco working well. But I made it!!

Anyway, if anyone else has DL380 G9 servers, the HP FlexFabric 10Gb 2-port 533FLR-T Adapter (LOM) and the HP Ethernet 10Gb 2-port 530T Adapter (PCI-e) cards work great with CORE.

The one question that I have is how in pluperfect hell do I get my old BGE adapters (4 of them) to go away from my dashboard?

image
image1086×794 74.5 KB

I have tried the tunable hint.bge.0.disabled (4 tuneable entries, 1 for each nic) and they do not go away from the dashboard after a reboot. Anyone have another method of getting rid of the interfaces that are not being used? The 1G interfaces are not removable physically on the G9s that I have.

Happy NAS’ing.

Cheers,

2

Disable them in BIOS/UEFI?

3

Not a bad idea - not sure if I can (honesty never poked around the bios).

Cheers,

4

While I have not tried this because I only have one NIC, can you restrict this in the Dashboard using that little sprocket thingy?

If that fails, a pair of pliers will remove it for good. WARNING: This is a high risk operation, pluck it out like you were to pluck a hot boiling egg out of the boiling water (FAST). It is less painful that way.

In seriousness, if the BGE NICs are on the motherboard, the BIOS is the best way if you just disable the NICs you do not want operational. And a note of caution: If you have IPMI, that may not work over your add-on NIC. Something you should look into so you are not surprised when you need it.

1 Like
5

You should be able to mark them as PCIe pass through….

Ie add them to “pptdevs”

https://wiki.freebsd.org/bhyve/pci_passthru

(There might be simpler more appropriate ways to achieve your goal :wink: )

1 Like
6

Yeah, I will take a look in the BIOS - Just rebooted enough times today - reached my quota.

And luckily, the HPE iLO (fancy IPMI) has its own port.

image
image2366×1292 234 KB

Cheers,