LSI 9305-16i only seeing 1 drive per channel not 4

TrueNAS General
,
1

Hello from Australia :smiley: - the land of over priced PC hardware and few used Server parts :stuck_out_tongue:

My new server build to learn TrueNAS and backup my media server is giving me grief. An exciting undertaking becoming a PITA.

A DOA motherboard and need to purchase a more expensive model once trapped into the AM5 line (couldn’t return CPU) meant sunk cost started kicking in :/.

I purchased all new hardware except the LSI HBA which is from a server reseller (I need to check the serial given my issues to see if its a clone).

Current hardware

  1. Asus x870 TUF AM5 motherboard (wifi, BT, Audio off, OC disabled).
  2. Ryzen 8600g CPU (I wanted the GPU given no IPMI, and ECC support)
  3. Kingston DDR5, 5200MHZ ECC UDIMM 2x16GB.
  4. Intel x550-T1 10GBe NIC (chipset PCIE Slot).
  5. Seagate Ironwolf 8TB x5 HDD’s (intend RAID6 or ZRAID2).
  6. SSD 250GB x2 for TrueNAS in RAID1. Connected with mb SATA.
  7. LSI 9305-16i HBA with 60mm Noctua fan fitted for cooling (CPU PCIE slot).
  8. PSU Thermaltake Gold 80, 550W which seems ok on jgreko’s list.
  9. Fractal Meshify case.

Two Computers are discussed - “Server” (being constructed) and “PC” (my desktop, which I use to flash and test the HBA).

I got the server hardware installed and tried installing TrueNAS Scale using current ISO to 2x 250GB SSD’s in RAID1 through install process… I discovered that things weren’t going ok with the HBA.

The LSI 9305-16i HBA will only recognise 2 of 5 HDD. I connected two SFF-8643 to SATA breakout cables for the 5 drives in my system and only 1 HDD on each channel is ‘seen’.

These are direct connect - no backplane.

Have you had this issue and do you know how I can correct it?

I put the HBA in my PC and updated firmware to 16.0.1.0 … the last Broadcom issued. I also installed the EFI bios (sas3mpt.rom).

I had to set HBA bios to “Legacy disabled” for it to show up at all in Server bios. Now it is seen in my server bios - but still not seeing drives.

I tried both server and PC with the HDD’s and HBA; Neither sees more than the 2 drives.

I disconnected 2 of 5 thinking it could be power but no change. Different sata power … no change.

Change sas cable channel on HBA, and the sata connector uses - no change.

I think it’s one of 3 issues:

  1. I need to reflash the bios to legacy from EFI for some reason.
  2. My sas cables are duds ($9 off amazon)
  3. The card is a dud.

The seller stated he tests them on a dell server. I don’t know if that means power on and see it’s in bios or connects drive array and confirms it actually functions. I’ve written to him to ask for more info and help.

The cables I’m returning to Amazon and have bought a different pair to check.

This forum is full of experts and I am hoping someone here can help me rule out something esoteric that is causing this in the instance it’s not an actual hardware fault.

As additional notes:

  1. Asus states their AM5 boards support ECC RAM (not just DDR5 “ecc”). My desktop is non ECC so I don’t think this is an issue.
  2. Until I disabled legacy bios with the EFI one installed the HBA would not show in the Asus TUF Bios or my TrueNAS dashboard.

Thanks for reading if you have - and my appreciation for any help or advice that comes my way :slight_smile:

BIOS HBA info in image and shot of the server assembled.

IMG_0367
IMG_03671920×1440 202 KB

IMG_0364
IMG_03641920×1440 352 KB

2

Suspect this.

I’ve had cheap cables from Amazon that weren’t even electrically connected!

Not sure what happens exactly if you use a reverse breakout cable, but make sure you don’t have a reverse breakout cable :wink:

3

Thank you. It helps to have some sort of external voice at this point. I will see what the new cables do. They are more expensive though I know that isn’t representative of value - especially on Amazon.

If they don’t help I’m going to be stumped and presume the card is a dud.

My first mainboard - an Asus Prime x670 P Wifi was DOA and had a raft of compatibility issues I discovered online.

Warranty issues here for computing gear are not grand… you lose the item for weeks or months to be “tested” so end up purchasing a replacement due to time factors. When I receive the repaired/replacement item I offload it on FB but at a loss… so you get a DOA item and end up out of pocket because retailers won’t exchange them anymore. This time I may get a refund I’m told so that’s a bonus at least as I don’t lose money on the RMA item.

Sunk cost is biting me at this point and I regret trying to pick good cheaper items instead of just sucking up the cost on a server board - with xeon-D or i3 etc but nice Supermicro quality. I have a 600mm rack which has made the notion of used server harder due to size. I could have gotten a 12bay Synology Rackmount RS3612xs+ for about $1000 and was too dumb to consider I could run anything on it including DSM. With quiet fans it would be a good solid option with no hardware compatibility risks.
I regret I didn’t know about the IX models before embarking on this. A silent home 12 bay looks very nice for TrueNAS.

4

I have the same mainboard as your Node 304 build :smiley:
It’s in the discontinued but ever so awesome Lian Li PC-26q

IMG_0243
IMG_02431920×1440 294 KB

6

UPDATE: Problem solved - might be factor for more people as 9305-16i becomes the cheap option like 9211-7i has been for years.
Onboard bios of card was interfering - regardless of settings.

Sometimes DIY = massive loss of opportunity cost (Time) that writes off the $ saving vs purchasing a Commercial Off The Shelf Solution (COTS) like the TrueNAS offerings.

I received new cables of a different type and source. No change.

I began to look at what I had changed from the first time I used it.

Note image in OP shows 6 disk attached. At that point it was showing discs but not seen by TrueNAS, and my MB was flaky - it booted only 5-6 times before going dead.

The changes I had made were to the Firmware and Bios.

I decided to remove the bios. Should be simple using sas3flash but in windows it will NOT allow erase function. / sigh … more hours messing around since I have no linux machine and can’t boot to dos on today’s secureboot PC’s.

I eventually worked out how to construct a boot disc to an EFI shell environment.

I used sas3flash.efi to reset the HBA to factory and then applied the latest firmware. No BIOS.

Naturally no HBA shows on boot or in bios anymore. It does however show in TrueNAS with all of the drives - success!!!

It also, on my main PC shows up in IBM MegaRAID storage APP correctly with drives.

Incidental benefit - boots far quicker than when BIOS has to be processed too.

I’m happy the problem is solved yet frustrated it took so long to sort out with a string of retail issues confounding me on top of the card…

  1. Dead MB (New Asus Prime x670 -P wifi) worked initially and so not apparent as cause when it died (fans would run but mb has no lighting in default bios which is “Stealth mode”… utterly stupid design decision, on a list of many.

  2. Misrepresented RAM (Stated ECC when not ECC - 3 week wait due no local server ram, and found it was standard DDR5).

  3. 3hrs “proving” mb is dead at store to start RMA. Purchase new MB - Asus x870 TUF [RMA will take 4-6 weeks so need to buy new MB.
    Also told Prime x670 P Wifi has high failure/return rate. Read it is incompatible with LSI 9305/9300 according to techs and multiple forums… sigh).

7

Ryzen APUs do not support ECC, unless they are of the “PRO” variety.
Most or all Ryzen 7000 or 9000 would support ECC even if non-PRO, but not the 8000.

8

Ugh that bites. I checked everywhere I could and not once did I see anything about withdrawal of that promise. There are plenty of comments when I search specifically though.

This type of crap is not ok.

I can’t find a cpu with gpu that is ecc enabled for am5. :sob:

9

Try looking at the series 9000x models. Mine 9600x has graphics and, I think, supports ECC

10

I’ll check that - I thought none of the x models have gpu but I’ll happily be wrong!

11

Is there a way in truenas I can confirm if ecc is working correctly?

12

I have the 9600x and no graphics card for my Win 11 computer

screenshot from AMD showing graphics core counts

image
image1374×722 12.8 KB

13

Yes I used the website to check my choices. I also used it to check before… unsure if I made an error or if AMD changed their page content after.

I’ve a zen 5 7600 on the way to me.

Interesting to note - earliest Asus bios had option that let me enable ecc. It was gone later versions and leaves me wondering if this is entirely cpu or if amd is playing games like Intel does with walled gardens.

14

ECC and Ryzen is kind of a mess to figure out what processors support it and what motherboards and even what is considered ‘supported’. The different generations have different answers.

15

That were the AM4 generations… And most non-X can do ECC as well.
Generally:

  • CPU (desktop, chiplets) => ECC capable
  • APU (laptops, monolithic) => ECC only for “PRO” variants

APUs always had iGPU; AM4 CPUs had no iGPU. But the real difference is in the die arrangement: Monolithic vs. chiplets. Now even “desktop” CPUs have an iGPU, and they are still ECC-capable.
Except that AMD muddied the waters by releasing some “CPUs” which actually are APUs with a disabled (or defective) iGPU, like Intel ‘F’ parts, and do not support ECC even though they look like other desktop-intended parts. Ryzen 5500 is one such offender.

All or almost all in Ryzen 7000 and 9000 ranges. But no 8000.
And EPYC 4004 of course, but that possibly brings us to “finding server components in Oz”.

16

I thought I’d close this out by a quick follow up.

  1. The LSI 9300-16i is now working. I had to remove the latest efi bios from the card for it to see drives and be seen by any of the relevant utilities or OS.

  2. I bought a Ryzen 5 7600 and now have ECC functioning.

It’s been a painful build this one but I’m sorted and now able to get it configured and holding data.