Let me expound a little bit about Multi-Report. This is a script that you run from CRON, once a day. With it’s companion script it will tst all your drives and generate a nice easy to read report. You would see an email generated each day and the subject line is all you need to look at. It will say “All Good” is there are no issues, this saves you from having to read the entire report. If there is a problem, the Subject line changes which will clue you in to open and read the email report.
In addition to that stuff is a check for SMR drives (yes, some people are surprised they accidentally got an SMR drive), emails you daily with the report, weekly it also appends your TrueNAS configuration file and your Multi-Report configuration file. You have the option to -dump all the smart drive data to email as well vice individually performing CLI commands. It does a heck of a lot, but it is not for everyone and I would never force it on someone. If you did want to entertain this script, read the first and last message in the thread. The first I keep updated, the last is good to see how others are doing with the scripts.
But try Scrutiny, it may be all you want.
It was a typo. I didn’t mean to quote that line.
2 Likes
Josh1
June 10, 2025, 4:23am
23
Updated results:
root@truenas[~]# smartctl -a /dev/sde
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.6.44-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Western Digital Ultrastar (He10/12)
Device Model: WDC WD80EMAZ-00WJTA0
Serial Number: 7HKJ8SMN
LU WWN Device Id: 5 000cca 257f1ad57
Firmware Version: 83.H0A83
User Capacity: 8,001,563,222,016 bytes [8.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5400 rpm
Form Factor: 3.5 inches
Device is: In smartctl database 7.3/5706
ATA Version is: ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is: SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Mon Jun 9 21:21:26 2025 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 93) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: (1114) minutes.
SCT capabilities: (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000b 100 100 016 Pre-fail Always - 0
2 Throughput_Performance 0x0004 129 129 054 Old_age Offline - 112
3 Spin_Up_Time 0x0007 148 148 024 Pre-fail Always - 447 (Average 444)
4 Start_Stop_Count 0x0012 100 100 000 Old_age Always - 231
5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0
7 Seek_Error_Rate 0x000a 100 100 067 Old_age Always - 0
8 Seek_Time_Performance 0x0004 128 128 020 Old_age Offline - 18
9 Power_On_Hours 0x0012 093 093 000 Old_age Always - 50482
10 Spin_Retry_Count 0x0012 100 100 060 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 231
22 Helium_Level 0x0023 100 100 025 Pre-fail Always - 100
192 Power-Off_Retract_Count 0x0032 099 099 000 Old_age Always - 2318
193 Load_Cycle_Count 0x0012 099 099 000 Old_age Always - 2318
194 Temperature_Celsius 0x0002 166 166 000 Old_age Always - 39 (Min/Max 19/50)
196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0
197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x000a 200 200 000 Old_age Always - 1
SMART Error Log Version: 1
ATA Error Count: 77 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 77 occurred at disk power-on lifetime: 45231 hours (1884 days + 15 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 41 00 00 00 00 00 Error: UNC at LBA = 0x00000000 = 0
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 00 b0 4c 36 40 00 6d+12:23:28.544 READ FPDMA QUEUED
61 00 10 b0 34 36 40 00 6d+12:23:21.606 WRITE FPDMA QUEUED
61 00 08 b0 24 36 40 00 6d+12:23:21.606 WRITE FPDMA QUEUED
61 00 10 b0 3b 36 40 00 6d+12:23:21.604 WRITE FPDMA QUEUED
61 00 08 b0 3a 36 40 00 6d+12:23:21.604 WRITE FPDMA QUEUED
Error 76 occurred at disk power-on lifetime: 45231 hours (1884 days + 15 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 41 00 00 00 00 00 Error: UNC at LBA = 0x00000000 = 0
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 b0 34 36 40 00 6d+12:23:21.225 READ FPDMA QUEUED
60 10 28 10 26 81 40 00 6d+12:23:14.284 READ FPDMA QUEUED
60 10 20 10 24 81 40 00 6d+12:23:14.284 READ FPDMA QUEUED
60 10 18 10 12 00 40 00 6d+12:23:14.284 READ FPDMA QUEUED
60 00 10 b0 3c 36 40 00 6d+12:23:14.284 READ FPDMA QUEUED
Error 75 occurred at disk power-on lifetime: 45231 hours (1884 days + 15 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 41 00 00 00 00 00 Error: UNC at LBA = 0x00000000 = 0
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 b0 24 36 40 00 6d+12:23:14.281 READ FPDMA QUEUED
60 00 00 b0 1c 36 40 00 6d+12:23:00.429 READ FPDMA QUEUED
60 00 10 b0 14 36 40 00 6d+12:23:00.257 READ FPDMA QUEUED
60 00 08 b0 0c 36 40 00 6d+12:23:00.257 READ FPDMA QUEUED
60 00 00 b0 04 36 40 00 6d+12:23:00.257 READ FPDMA QUEUED
Error 74 occurred at disk power-on lifetime: 41056 hours (1710 days + 16 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 41 00 00 00 00 00 Error: UNC at LBA = 0x00000000 = 0
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 08 78 f1 08 40 00 23d+11:50:54.576 READ FPDMA QUEUED
60 00 08 78 e9 08 40 00 23d+11:50:46.927 READ FPDMA QUEUED
60 10 00 10 12 00 40 00 23d+11:50:46.927 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 23d+11:50:46.924 READ LOG EXT
2f 00 01 10 00 00 00 00 23d+11:50:46.924 READ LOG EXT
Error 73 occurred at disk power-on lifetime: 41056 hours (1710 days + 16 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 41 00 00 00 00 00 Error: UNC at LBA = 0x00000000 = 0
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
60 00 00 78 e1 08 40 00 23d+11:50:46.924 READ FPDMA QUEUED
60 10 08 10 12 00 40 00 23d+11:50:39.774 READ FPDMA QUEUED
2f 00 01 10 00 00 00 00 23d+11:50:39.750 READ LOG EXT
2f 00 01 10 00 00 00 00 23d+11:50:39.750 READ LOG EXT
61 10 00 10 26 81 40 00 23d+11:50:32.827 WRITE FPDMA QUEUED
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 50476 -
# 2 Extended offline Completed without error 00% 37379 -
# 3 Short offline Completed without error 00% 21887 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
The above only provides legacy SMART information - try 'smartctl -x' for more
imo - still nothing fatal; just that CRC error. Up to you if you want to just clear the zfs errors after a scrub and monitor for a bit, or if you want to check connections etc.
1 Like
I agree. Nothing wrong with that drive and the CRC error is old, could have happened years ago.
Do as @Fleshmauler said, run a scrub and check the status again zpool status -v. Run a zpool clear and monitor.
Question… how do you know it was sde? You counded confident so I didn’t push it. With the drive tested results being very good, I have to question it.
3 Likes
Josh1
June 10, 2025, 6:31pm
26
The GUI told me
Would post a screenshot if I could. I went to the storage tab then expanded the VDEV of the faulted pool so I could see both devices. It says FAULTED under status for sde.
Josh1
June 10, 2025, 6:34pm
27
I installed my HBA card in February '24. Thinking back, I remember seeing the error then, and it’s part of the reason I bought the 9207-8i.
I haven’t had any issues with the drive or pool since moving the connection to the 9207-8i but never got around to fully investigating the error. I should have been more proactive, I know.
Good airflow is the big important thing people don’t think about. Sounds like you has accounted for that.
1 Like
