Unable to add new Disks

Problem adding drives. Can anyone offer advice: I put my TrueNAS system together in 2013. Since 2018, I have been using a Dell H310 6Gbps SAS HBA LSI 9211-8i P20 IT Mode card I bought from ArtofServer to be able to use more drives. Everything has worked OK until the last year, when I have tried drive after drive to replace 2 failed drives in my Z3 array (7 disks any 3 can fail I believe). I plug the new disk into the card, and TrueNAS flags errors and won’t let me use the drive. I have tried like 5 drives so far and had this problem, 2 of the drives were new 4Tb Western Digital Red SATA drives. Windows never has a problem with the drives when I have connected them to my daily driver desktop. I have tried buying new SATA cables and swapping them in the NAS to no avail. I’m still on TrueNAS-13.3-U1.2 and know I need to migrate off of it but I’ve been afraid to do it with 2 of the 3 redundant drives not working and the 3rd one marked as ‘degraded’ by TrueNAS.

What strategy would you use to try to recover? I have 2 backups to larger single-drive pools, one of which is off site. I can print error logs of what happens when I try to attach a drive. I’m willing to just try adding a new card if that seems like the best way to proceed, they don’t cost much. I’m not very good with drive errors and am looking for a way to spend the least time recovering the redundancy and moving to a modern version of TrueNAS. I have a bunch of jails that won’t migrate.

Do you have any other posts with this problem so you don’t have to repeat a lot of information? Can you link to them, if so?

Guessing with no other info, I would ask if you have have excellent cooling airflow over your HBA card. Manufacturer specs are in the 150-200 linear feet per minute range as they are meant for use in a server chassis.

Full hardware details on your system may help with advice. Posting the error logs is probably a good idea along with a summary of what you have tried, etc.

b4 buying another card check whether the HBA or the pool side is rejecting them.

when u plug a “bad” drive paste:

dmesg | tail -80
camcontrol devlist
smartctl -a /dev/daX
zpool status -v

common gotchas on old CORE + H310:

  1. leftover freebsd/zfs labels. zpool labelclear -f /dev/daX (only on the NEW empty disk never a pool member)
  2. HBA firmware not actually P20 IT. sas2flash -list and check it says IT and P20
  3. bad port/cable. move a known-good pool disk onto that same port once
  4. airflow over the H310. those cards cook in a desktop case and start throwing random drive errors

u already have offsite backups so: labelclear a blank disk → replace one degraded member → scrub → then deal with the CORE→SCALE/jails move separately.