hello,
today i noticed that my smb was unreachable and realized that Truenas had crashed and spitted this error in on the monitor :pool ‘boot-pool’ has encountered an uncorrectable I/O failure and has been suspended
i could not soft shutdown via IPMI and had to RESET system via IPMI. it rebboted fine, but a couple hours later it did it again. so i went into the shell and ran sudo smartctl -a /dev/sdi
smartctl 7.4 2023-08-01 r5530 \[x86_64-linux-6.12.91-production+truenas\] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Crucial/Micron Client SSDs
Device Model: CT120BX500SSD1
Serial Number: 2013E3F287ED
LU WWN Device Id: 0 000000 000000000
Firmware Version: M6CR013
User Capacity: 120,034,123,776 bytes \[120 GB\]
Sector Size: 512 bytes logical/physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
TRIM Command: Available, deterministic, zeroed
Device is: In smartctl database 7.3/5528
ATA Version is: ACS-2 T13/2015-D revision 3
SATA Version is: SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Sat Jul 11 17:54:21 2026 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART Status not supported: Incomplete response, ATA output registers missing
SMART overall-health self-assessment test result: PASSED
Warning: This result is based on an Attribute check.
See vendor-specific Attribute list for marginal Attributes.
General SMART Values:
Offline data collection status: (0x02) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 120) seconds.
Offline data collection
capabilities: (0x11) SMART execute Offline immediate.
No Auto Offline data collection support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
No Selective Self-test supported.
SMART capabilities: (0x0002) Does not save SMART data before
entering power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 10) minutes.
SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 100 100 050 Pre-fail Always - 0
5 Reallocate_NAND_Blk_Cnt 0x0032 100 100 010 Old_age Always - 0
9 Power_On_Hours 0x0032 100 100 050 Old_age Always - 43393
12 Power_Cycle_Count 0x0032 100 100 050 Old_age Always - 193
171 Program_Fail_Count 0x0032 100 100 050 Old_age Always - 0
172 Erase_Fail_Count 0x0032 100 100 050 Old_age Always - 0
173 Ave_Block-Erase_Count 0x0032 100 100 050 Old_age Always - 549
174 Unexpect_Power_Loss_Ct 0x0032 100 100 050 Old_age Always - 155
180 Unused_Reserve_NAND_Blk 0x0032 100 100 050 Old_age Always - 100
183 SATA_Interfac_Downshift 0x0032 100 100 050 Old_age Always - 0
184 Error_Correction_Count 0x0032 100 100 050 Old_age Always - 0
187 Reported_Uncorrect 0x0032 100 100 050 Old_age Always - 0
194 Temperature_Celsius 0x0022 060 041 050 Old_age Always In_the_past 40 (Min/Max 23/59)
196 Reallocated_Event_Count 0x0032 100 100 050 Old_age Always - 0
197 Current_Pending_ECC_Cnt 0x0032 100 100 050 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 100 050 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 100 100 050 Old_age Always - 1488
202 Percent_Lifetime_Remain 0x0030 064 064 001 Old_age Offline - 64
206 Write_Error_Rate 0x002e 100 100 050 Old_age Always - 0
210 Success_RAIN_Recov_Cnt 0x0032 100 100 050 Old_age Always - 0
246 Total_LBAs_Written 0x0032 100 100 050 Old_age Always - 71948838701
247 Host_Program_Page_Count 0x0032 100 100 050 Old_age Always - 2248401209
248 FTL_Program_Page_Count 0x0032 100 100 050 Old_age Always - 3418587208
SMART Error Log Version: 1
Warning: ATA error count 0 inconsistent with error log pointer 4
ATA Error Count: 0
CR = Command Register \[HEX\]
FR = Features Register \[HEX\]
SC = Sector Count Register \[HEX\]
SN = Sector Number Register \[HEX\]
CL = Cylinder Low Register \[HEX\]
CH = Cylinder High Register \[HEX\]
DH = Device/Head Register \[HEX\]
DC = Device Command Register \[HEX\]
ER = Error register \[HEX\]
ST = Status register \[HEX\]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It “wraps” after 49.710 days.
Error 0 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
---
84 51 00 00 00 00 40 Error: ICRC, ABRT at LBA = 0x00000000 = 0
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
---
35 00 08 60 e0 66 40 00 00:00:00.000 WRITE DMA EXT
25 00 b0 c0 58 ce 40 00 00:00:00.000 READ DMA EXT
b0 d0 01 00 4f c2 00 00 00:00:00.000 SMART READ DATA
b0 d5 01 06 4f c2 00 00 00:00:00.000 SMART READ LOG
b0 d5 01 01 4f c2 00 00 00:00:00.000 SMART READ LOG
Error -1 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
---
84 51 00 00 00 00 40 Error: ICRC, ABRT at LBA = 0x00000000 = 0
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
---
35 00 08 00 49 f9 40 00 00:00:00.000 WRITE DMA EXT
ea 00 00 00 00 00 00 00 00:00:00.000 FLUSH CACHE EXT
35 00 08 00 19 10 40 00 00:00:00.000 WRITE DMA EXT
35 00 08 00 1b 10 40 00 00:00:00.000 WRITE DMA EXT
35 00 08 00 47 f9 40 00 00:00:00.000 WRITE DMA EXT
Error -2 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
---
84 51 00 00 00 00 40 Error: ICRC, ABRT at LBA = 0x00000000 = 0
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
---
35 00 30 78 e1 6a 40 00 00:00:00.000 WRITE DMA EXT
ea 00 00 00 00 00 00 00 00:00:00.000 FLUSH CACHE EXT
35 00 08 f8 19 10 40 00 00:00:00.000 WRITE DMA EXT
35 00 08 f8 1b 10 40 00 00:00:00.000 WRITE DMA EXT
35 00 08 f8 47 f9 40 00 00:00:00.000 WRITE DMA EXT
Error -3 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
---
84 51 00 00 00 00 40 Error: ICRC, ABRT at LBA = 0x00000000 = 0
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
---
35 00 08 30 1b 10 40 00 00:00:00.000 WRITE DMA EXT
35 00 08 30 47 f9 40 00 00:00:00.000 WRITE DMA EXT
35 00 08 30 49 f9 40 00 00:00:00.000 WRITE DMA EXT
ea 00 00 00 00 00 00 00 00:00:00.000 FLUSH CACHE EXT
35 00 08 30 19 10 40 00 00:00:00.000 WRITE DMA EXT
Error -4 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
---
04 51 00 00 00 00 40 Error: ABRT at LBA = 0x00000000 = 0
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
---
35 00 48 08 45 3e 40 00 00:00:00.000 WRITE DMA EXT
35 00 28 50 45 3e 40 00 00:00:00.000 WRITE DMA EXT
35 00 70 e0 2f b8 40 00 00:00:00.000 WRITE DMA EXT
35 00 28 a0 53 35 40 00 00:00:00.000 WRITE DMA EXT
35 00 38 68 53 35 40 00 00:00:00.000 WRITE DMA EXT
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 43375 -
# 2 Short offline Completed without error 00% 43351 -
# 3 Short offline Completed without error 00% 43327 -
# 4 Short offline Completed without error 00% 43303 -
# 5 Short offline Completed without error 00% 43279 -
# 6 Short offline Completed without error 00% 43255 -
# 7 Short offline Completed without error 00% 43231 -
# 8 Short offline Completed without error 00% 43207 -
# 9 Short offline Completed without error 00% 43183 -
#10 Short offline Completed without error 00% 43160 -
#11 Extended offline Completed without error 00% 43138 -
#12 Short offline Completed without error 00% 43136 -
#13 Short offline Completed without error 00% 43112 -
#14 Short offline Completed without error 00% 43088 -
#15 Short offline Completed without error 00% 43064 -
#16 Short offline Completed without error 00% 43016 -
#17 Short offline Completed without error 00% 42992 -
#18 Short offline Completed without error 00% 42968 -
#19 Short offline Completed without error 00% 42944 -
#20 Short offline Completed without error 00% 42920 -
#21 Short offline Completed without error 00% 42896 -
Selective Self-tests/Logging not supported
The above only provides legacy SMART information - try ‘smartctl -x’ for more
truenas_admin@FREENAS\[\~\]$
from what i could googled the smart info do not indicate any failure or heavily worn drive.
i have a backup of my config. should i reinstalled the os and restore the config, could it be a cable or something else or the way it failed it is definitively a premature drive failure (manufactured in March 2020)