Ive been running a TN Scale box for about a year and it has been happily running along without a hiccup but the itch to expand its capability has been biting. Looking around for 10G cards i saw that both Synology and QNAP offer a NIC/M.2 NVME combo card and it got me thinking what is stopping manufactures from adding an HBA and high-speed networking to a single 16x PCIE card? ITX users would benefit greatly from it and less compromises would need to be made between networking, storage and video/AI acceleration if that’s your thing. Thoughts?
What’s stopping them? Demand I suspect, plus maybe the cost of PLX switches as seen by the target market. Might also be a reliable combo driver nightmare, at least to get it off the ground.
The number of users for 10Gb+ networking in pcie expansion (rather than TB/USB etc) who are using itx machines probably rounds into the zeros. Same for HBA combos - the cheap sata expanders and M.2 sata expansion stuff probably takes care of majority of that market. Everyone else who wants an HBA is already throwing lots of pcie lanes at the problem, meaning they’ve got plenty of expansion slots available. If you look at broadcom’s advertising for HBA’s, they’re not exactly consumer friendly.
It’s likely to be technically possible but will only happen when someone with the skills and finances sees money to be made from it. It might happen in future if the worlds makers of server and workstation boards decide that they’ve had enough of edge connectors and want all 128 epyc lanes to go through a TB cable instead, but it’s unlikely short term.
It’s pretty much an identical question to “why can’t i find an m/itx board with bifurcation”?
2 Likes
If you’re short on PCIe-slots you could try experimenting with an M.2 to 10GbE/SFP+ adapter.
No promises as to how reliable you’ll find it though…
or just find a board with 10Gb built in and use the slot for the HBA
I have another thought as well - current (broadcom at least) HBAs generally state rather high cooling requirements - of the order of 100-200 linear feet per minute. That should give you a rather good clue about the 'home user in a tiny box with limited cooling" market that the OEM expects to entertain.
A hybrid card with a NIC might well exceed the pcie power limit (25W only. Only GPUs are permitted to go up to 75W (PCI Express - Wikipedia)) and will need a probably not long term reliable single card height compatible cooler plus enough general case cooling to feed it.
1 Like
Thats an old standard and there’s nothing in the standard that limits only GPUs to 75W. Full height x16 cards can draw up to 75W, irrespective of the type.
1 Like
Combo cards are niche products, so even if they existed, many people would say they are too expensive. However, some NAS hardware vendors may make a combo card because they only include 1 or 2 slots. And if the chips are common, then Linux support would be present. But, those chips selected may not be suitable for TrueNAS SCALE…
Sun Microsystems used to make combo cards. I remember the old SBus with 10/100Mb/ps Ethernet and SE 16bit parallel SCSI. They were popular because some servers or desktops had limited expansion slots. Sun even made a dual 10/100Mb/ps Ethernet card, (and I vaguely recall a quad 10/100Mb/ps Ethernet card).
Perhaps 15 years back I looked for a USB 3 & SATA expansion card for my PC that it into a 1 lane PCIe slot. But, most options were internal SATA ports to external connectors with USB. Not an actual SATA controller chip in addition to the USB controller chip.
What about this? Mellanox connectX3 DATA DIRECT NETWORKS INC 04-00232-614 REV 2 614 REV A0 3914
Some discussion about it running on Linux etc here, not sure if they ever got it working, but clearly such items do exist and may work with some dedication.
HBAs and 10gbe+ NICs are all server hardware, not niche homelab gear. The market for combo cards would be basically zero. ITX cases aren’t their focus.
Craft Computing did some interesting things with a nvme to sata device, check his video out.
Easiest path forward would be to buy a larger chassis and get a board that has enough PCIe slots and lanes for the devices you want to use.