The first Supermicro card i linked above, the official site says its a retimer based card and the second Supermicro card says its a redriver. Whereas, the Broadcom site says its a switch. My question is what’s the difference between the three and what is more viable and faster option with low latency. I’m asking this because, from various reads on the forum, it is suggested that using an HBA Card like TriMode Adapter is bad cause it adds an extra latency and makes the NVMe work slow with few other downsides.
Secondly, the Broadcom document says “out of band management”. What does it mean exactly? Can someone explain in simple terms?
I know the difference between U.2 and M,2 Drives. So, i thought to have some solution for the M.2 and i found these two:
Does your homelab have to be on PCIe 4.0? You’ll find a lot more options, and cheaper options, with 3.0.
A switch switches: It takes whatever lanes it gets from the PCIe slot and redistribute as needed to the attached devices—i.e. it manages bifurcation, can work with less than a full x16 slot and serve more device lanes than what it gets.
A retimer retimes: It adjusts the timing of the signals to remain within specification; it does NOT redistributes the lanes—i.e. it relies on bifurcation by the motherboard, and needs a x4x4x4x4 slot to serve four x4 devices.
(Incidentally, the switch does retime signals; but the retimer does not switch.)
As stated, what type of card you get will be partially dependant on what you have for a motherboard and CPU. If you can support bifurcation, then you don’t need to spend much money, however if you cannot support bifurcation then you are looking to spend some money. You are doing your research which is very good.
Yes, this Active NAS i’m starting to build requires PCIe 4.0 cause this way, i can have less storage, yet double the Gen3 speeds.
Gotcha!
Oh, so whatever card that requires bifurcation support by the motherboard, does that means that specific card is using a retimer technology?
Also, what is redriver here? Is it the same retimer? The #3 Supermicro link i posted have supposed to have redriver. Is it different than the switch and retimer?
Also, when talking about the latency, what is more preferred choice? The switch based cards or the retimer based cards?
Oh, yeah, i figured this out mate. For a card that doesn’t require bifurcation, the Motherboard is cheap but the card is costly and for a card that requires bifurcation, the card is cheap and the motherboard is costly. xD
Yes, Thanks. I think better to do this than spending money and later realizing that these part won’t work or ain’t that good.
Yup, sure does. I had to pick all my components using my brain. If I could do it again, I would want a motherboard that had one more PCIe port, x8 or x16 and on it’s own lanes.
The part that sucks about using an NVMe adapter like mine, if the computer fails, I can’t just take my card of NVMe drive to any old computer and plug it in, I must ensure it supports bifurcation. If you get one of the other boards which costs a bit of money, you can take that card and shove it into practically any computer and have access to your data.
Keep that in the back of your mind, if your data is important to have access to, that is a factor to consider. And it opens you up to a lot more motherboard options for that new system you are building.
I bought a Jeyi brand quad-nvme card for <$50. I see one on ebay for $15. It requires bifurcation. It’s not a brand I had heard of, however nvme directly uses PCIE lanes so is a very simple card and I assumed hard to screw up. I have two that have been working fine for >1 year.
Part of why I went with Epyc to replace my old server was for all the juicy PCIE lanes.
“Requires the motherboard PCIe x16 slot to support splitting to x4 + x4 + x4 + x4. If the PCIe x16 slot only supports splitting to x8 + x4 + x4, only three disks can be recognized. If the PCIe x16 slot only supports splitting to x8 + x8, only two disks will be recognized.”
On the product page, this tells you it requires bifurcation and is a passive card.
You’ll also notice the PCBs look much more sparsely populated with components, because they are essentially an adapter.
The trick isn’t the CPU now, it is the motherboard and ensuring each PCIe slot has it’s own dedicated lanes. I’m sure you have seen a x16 physical slot but it is only wired for x8 lanes. That is another one of those things to really pay attention to. Sometimes it is difficult to locate detailed information like that, or if the BIOS supports the configuration you want. None of this is a walk in the park, it can be damn confusing even for some of us who have been doing this for decades (since 1974).
I have Asus one and it works perfectly. As pointed out your bios need to support bifurcation, which basically means that it will turn your x16 slot into four independent x4 slots (on the card itself there are no semiconductors except for fan control, so there is no switch and I am not sure retiming is used for this solution - whatever retiming is, I have no experience with it). Also, be sure that your slot actually is x16, having x16 PCIe connector is not proof enough (couple of people in the thread mentioned that).
Redrivers and retimers are for signal integrity. When using cables, some active solution is probably required. With short traces from the PCIe slots, these cards probably manage PCIe 4.0 while being passive.
There’s certainly no retimer—no consumer would buy these cards at >$250—and I doubt there are redrivers.
