my hdds have been removed from a pool without me exporting them but i can’t re-import them as it fails with I-O error? but says 6 hhds with exported pool, i’ve tried removing 1 hdd at a time and importing but i get the same error.
how do i fix this without losing data?
Server hardware -
Intel(R) Core™ i7-3770K CPU @ 3.50GHz
16gb ddr4 ram
Dell PERC H200 PCI-E x8 SAS SATA RAID Controller
120gb ssd (Truenas install/boot)
x8 sas (x6 Main_Temp pool / x2 backup pool)
x1 1tb sata (Storage pool)
We will need a bit more information. For example, what does the output of zpool import show. Please run the command without adding pool name, and put the output in CODE tags. CODE tags are the “</>” symbols in the forum editor.
This command will show us the pool configuration, and potentially which disks might be having problems.
This is of concern. ZFS does not normally work well with RAID controllers. But, I don’t know if the H200 can be set to HBA mode, (HBA = Host Bus Adapter, basically a plain disk controller with out any RAID).
ZFS pools are made of vDevs, (Virtual Devices) and those vDevs are made up of 1 or more disks, in various supported redundancy levels. You can’t generally import a pool unless all its Data vDev disks are available.
However, if their is proper redundancy available, yes, you can import a pool with a failed disk, (or disks if the redundancy level supports more than 1).
This is where we need to see the output of zpool import.
root@truenas[~]# zpool import
pool: Main_Temp
id: 17780023292427352798
state: ONLINE
status: One or more devices were being resilvered.
action: The pool can be imported using its name or numeric identifier.
config:
Main_Temp ONLINE
raidz1-0 ONLINE
a0ac776a-ba7d-4851-aba9-acb875448836 ONLINE
45eed3c3-8281-4999-9ee9-a4f21647b44e ONLINE
6bc23015-2f6e-4073-87d1-61cd4a6382db ONLINE
4064698c-bd25-43da-99d8-27c38279540d ONLINE
9d4a995c-1387-4b68-b693-fe6cb7b4b6dc ONLINE
fd267fd5-9973-440a-b197-24588f3289c2 ONLINE
root@truenas[~]#
Google says The Dell PERC H200 supports an HBA (Host Bus Adapter) mode where it acts as a simple pass-through controller, presenting disks directly to the OS without RAID, ideal for software RAID like ZFS/[TrueNAS]
It also says flashing to standard LSI IT (Initiator Target) firmware is often preferred for full compatibility and features in homelab/NAS setups.
I don’t know it that would help?
I checked and there is no option for HBA mode but there are no RAIDs setup so i assume its in HBA mode
root@truenas[~]# sas2flash -list
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved
Adapter Selected is a LSI SAS: SAS2008(B2)
Controller Number : 0
Controller : SAS2008(B2)
PCI Address : 00:02:00:00
SAS Address : 5c81f66-0-d11d-da00
NVDATA Version (Default) : 07.00.00.19
NVDATA Version (Persistent) : 07.00.00.19
Firmware Product ID : 0x2713 (IR)
Firmware Version : 07.15.08.00
NVDATA Vendor : Dell
NVDATA Product ID : H200A
BIOS Version : 07.11.10.00
UEFI BSD Version : 07.02.04.00
FCODE Version : N/A
Board Name : PERC H200A
Board Assembly : N/A
Board Tracer Number : N/A
Finished Processing Commands Successfully.
Exiting SAS2Flash.
I think unfortunately i have multiply hdds failures, i put a monitor on my server and as truenas boots i get I-O errors on 2 hdds sdd & sdg which are both in my Main_Temp pool
your Controller seems to run in IR Mode (RAID), not in IT Mode (HBA).
I don’t know, if this is the cause for the problems, nor if it will be an issue for further recovery of your Pool.
I think the Pool was created in this configuration, and at the moment this should not be touched (do not flash the controller yet).
Hmmm, the H200 does not seem to be an immediate problem.
However, 2 failing disks in a single parity setup, (RAID-Z1), means that unless you can get 1 disk functional, you probably have lost the pool, (and any data on it).
My suggestion is to attempt to get 1 of the 2 failing disks back to normal. Was it just bad blocks? Or something else… If bad blocks, then ZFS can possibly compensate by loosing a file or so. The rest would be okay. Using smartctl -x on the failing disks can help determine if it is bad blocks or something worse.
If you fail on getting 1 of the 2 failing disks back, then it is possible upgrading the firmware on the H200 might help in error recovery. In which case, perhaps that will allow you to get 1 of the 2 failing disks back to usability.
Using single parity, (RAID-Z1 or RAID-5), can be okay if good backups are available. And if scrubbing is done regularly, (checking the results too).
Adapter Selected is a LSI SAS: SAS2008(B2)
Controller Number : 0
Controller : SAS2008(B2)
PCI Address : 00:02:00:00
SAS Address : 5c81f66-0-d11d-da00
NVDATA Version (Default) : 14.01.00.08
NVDATA Version (Persistent) : 14.01.00.08
Firmware Product ID : 0x2213 (IT)
Firmware Version : 20.00.07.00
NVDATA Vendor : LSI
NVDATA Product ID : SAS9211-8i
BIOS Version : 07.39.02.00
UEFI BSD Version : 07.02.04.00
FCODE Version : N/A
Board Name : 6Gbps SAS HBA
Board Assembly : N/A
Board Tracer Number : N/A
Finished Processing Commands Successfully.
Exiting SAS2Flash.
I think both disks a fried as they flooded the logs with i-o errors about bad sectors i removed the 2 drives and wiped the remaining drives and now have them in RAID-Z2 setup (I have the important data backed up so was just movies/tv show that were lost).
I have also flashed the H200 card to IT mode so maybe that will help in the future if problems arise.
