Gone for the weekend, returned to pool missing, possible drive failure, no idea what to do

TrueNAS General
1

I’ve used TrueNAS for years, and FreeNAS before that, but I’ve never had such a catastrophic failure. It has always just worked, and yes, I’m not a total noob but pretty dang close. I’m using Truenas Scale 25.04.1, I upgraded on May 27th from 24.04.2.5

When I got to my machine and tried to access my TrueNAS drive from my windows machine it showed the drives were offline. I looked at error message in the GUI and see:

Device: /dev/sdg [SAT], 96 Currently unreadable (pending) sectors.
2025-06-02 18:08:52
Device: /dev/sdg [SAT], 25 Offline uncorrectable sectors.
2025-06-02 18:08:52
Device: /dev/sdb [SAT], FAILED SMART self-check. BACK UP DATA NOW!.
2025-06-13 07:01:06
Device: /dev/sdb [SAT], Failed SMART usage Attribute: 7 Seek_Error_Rate…
2025-06-13 07:01:06
Device: /dev/sdk [SAT], Read SMART Self-Test Log Failed.
2025-06-13 08:31:28

I’m guessing that I lost two drives.
Is there any hope of recovering my data? I’m going to lose a bunch of family stuff and work stuff I really don’t want to lose. I have some backups but as I’m reading, I should have probably configured this differently.

My hardware is:
ASUS TUF X299 Mark2 Motherboard
i7-7820X CPU
32GB Corsair Vengeance DDR4 RAM
Cooler Master MWE Gold 850 V2 Full Modular, 850W
Patriot P300 M.2 PCIe Gen 3 x4 128GB Low-Power Consumption SSD (boot drive)
10 WD Ultrastar DC HC530 14TB SATA 6G 3.5" 7200RPM
2.5G PCIe Network Card, YuanLey 2.5 Gigabit PCI Express


Name       Serial                                      Disk Size         Pool
nvme0n1	P300HHBB240902000059	119.24 GiB	boot-pool
Disk Type:
SSD
Description:
Model:
Patriot M.2 P300 128GB
Transfer Mode:
Auto
Rotation Rate (RPM):
HDD Standby:
Always On
Adv. Power Management:
Disabled
Enable S.M.A.R.T.:
Yes

sdi	000M6KKV	12.73 TiB	Valkyrie (Exported)
Disk Type:
HDD
Description:
Model:
OOS14000G
Transfer Mode:
Auto
Rotation Rate (RPM):
7200
HDD Standby:
Always On
Adv. Power Management:
Disabled
Enable S.M.A.R.T.:
Yes

sdj	0000AMD2	12.73 TiB	Valkyrie (Exported)
Disk Type:
HDD
Description:
Model:
OOS14000G
Transfer Mode:
Auto
Rotation Rate (RPM):
7200
HDD Standby:
Always On
Adv. Power Management:
Disabled
Enable S.M.A.R.T.:
Yes

	sdh	00002LD2	12.73 TiB	Valkyrie (Exported)
Disk Type:
HDD
Description:
Model:
OOS14000G
Transfer Mode:
Auto
Rotation Rate (RPM):
7200
HDD Standby:
Always On
Adv. Power Management:
Disabled
Enable S.M.A.R.T.:
Yes

sdg	000Q4DRE	12.73 TiB	Valkyrie (Exported)
Disk Type:
HDD
Description:
Model:
OOS14000G
Transfer Mode:
Auto
Rotation Rate (RPM):
7200
HDD Standby:
Always On
Adv. Power Management:
Disabled
Enable S.M.A.R.T.:
Yes

sde	7LG8BEGK	12.73 TiB	Valkyrie (Exported)
Disk Type:
HDD
Description:
Model:
WDC_WUH721414ALE604
Transfer Mode:
Auto
Rotation Rate (RPM):
7200
HDD Standby:
Always On
Adv. Power Management:
Disabled
Enable S.M.A.R.T.:
Yes

sdf	9JHJR4NT	12.73 TiB	Valkyrie (Exported)
Disk Type:
HDD
Description:
Model:
WDC_WUH721414ALE604
Transfer Mode:
Auto
Rotation Rate (RPM):
7200
HDD Standby:
Always On
Adv. Power Management:
Disabled
Enable S.M.A.R.T.:
Yes

sdc	7LG8HXLK	12.73 TiB	Valkyrie (Exported)
Disk Type:
HDD
Description:
Model:
WDC_WUH721414ALE604
Transfer Mode:
Auto
Rotation Rate (RPM):
7200
HDD Standby:
Always On
Adv. Power Management:
Disabled
Enable S.M.A.R.T.:
Yes

sdb	7LG8S3HK	12.73 TiB	Valkyrie (Exported)
Disk Type:
HDD
Description:
Model:
WDC_WUH721414ALE604
Transfer Mode:
Auto
Rotation Rate (RPM):
7200
HDD Standby:
Always On
Adv. Power Management:
Disabled
Enable S.M.A.R.T.:
Yes

sda	7LG93A2K	12.73 TiB	Valkyrie (Exported)
Disk Type:
HDD
Description:
Model:
WDC_WUH721414ALE604
Transfer Mode:
Auto
Rotation Rate (RPM):
7200
HDD Standby:
Always On
Adv. Power Management:
Disabled
Enable S.M.A.R.T.:
Yes

sdd	7LG81ZSK	12.73 TiB	Valkyrie (Exported)
Disk Type:
HDD
Description:
Model:
WDC_WUH721414ALE604
Transfer Mode:
Auto
Rotation Rate (RPM):
7200
HDD Standby:
Always On
Adv. Power Management:
Disabled
Enable S.M.A.R.T.:
Yes

Here is what I got with a zpool import command in the shell:

root@bigtruenas[~]# zpool import
  pool: Valkyrie
    id: 14937449896484207154
 state: ONLINE
status: One or more devices were being resilvered.
action: The pool can be imported using its name or numeric identifier.
config:

        Valkyrie    ONLINE
          raidz2-0  ONLINE
            sda2    ONLINE
            sdb2    ONLINE
            sdc2    ONLINE
            sdd2    ONLINE
            sde2    ONLINE
            sdf2    ONLINE
            sdg2    ONLINE
            sdh2    ONLINE
            sdj2    ONLINE
            sdi2    ONLINE

When I try to use the GUI to import a pool, there is nothing in the dropdown menu.

Under ZFS Health, there’s a red circle with an X and the pool status is offline.

But Disk Health Shows a Green circle with a checkmark.

When I restarted the machine, I got a BIOS error saying that there was an error “AMERICAN MEGATRENDS warning! your hdd/sdd might crash at any moment” and it indicated Port 2’s drive. I disabled the SMART so that I could get the machine to start up.

I’m hoping that someone might take pity on me and at least tell me where to go from here. Thanks in advance.

2

@etorix must be on vacation, he usually grabs these pretty quick.

Don’t do anything else to it besides take it offline until the smarter folks arrive. We don’t know what state it’s in, says it’s resilvering but who knows. One drive resilvering shouldn’t take it out in a raidz2 unless you lost both drives in a mirror maybe.

If you can remember how you structured the vdevs and pool, it would be helpful, but we can use commands to learn that at some point.

One last thing. Find the thread here from @joeschmuck for monitoring hard drive health with reporting, it’s in his signature. Next time you’ll see this coming a mile away, most drives don’t just have a heart attack, they choke a little first.

1 Like
3

The good news is that your pool design looks sensible - RAIDZ2 gives you dual redundancy, so a single drive failure shouldn’t cause you to lose your data.

It is not uncommon for a pool to drop out when there is a serious error (though there is some evidence to suggest that in recent versions of TrueNAS/ZFS it is currently more common than it should be).

I am not a ZFS expert like the iX folks, but here is how I would proceed with this:

  1. I would turn on SMART in the BIOS and physically remove the single disk it is complaining about. A very poorly disk can cause more problems than simply being individually offline in ZFS.

  2. Reboot and see what happens. It is possible that TrueNAS will be able to reimport the pool on boot, and then you can add a replacement drive and resilver.

  3. If the pool doesn’t get imported on reboot then you can try the following commands in a command shell:

  • sudo zpool import -R /mnt Valkyrie

and if that gives an error message (no message is an indication that it might have worked):

  • sudo zpool import -R /mnt -f Valkyrie

Let us know the results, and we can see what you might be able to do next.

4

This has only 6 SATA ports, so something is missing in your system inventory.

Full SMART reports for all drives, each in its own CODE block </> would be in order.
If you have not already, write down the serial numbers to keep track of drives, as letters may change across reboots.

You have a raidz2 but (the drives that were…) sdb and sdk have failed SMART tests and sdg is in poor health. You’ll need to keep at least one of these three alive to have a chance of rescuing the pool.
Get replacement drives and/or extra storage to make emergency backup if the occasion arise.

5

You are absolutely correct. I have 4 drives connected to a SATA board as well. I missed this on my list. Thank you for knowing a heck of a lot more than me.

ACTIMED PCIE SATA Card 8 Port with 8 SATA Cable, 6 Gbps SATA 3.0 Controller PCI Express

6

I have 2 drives coming in the mail. I ordered them when I discovered this. Thank you. I will work on the SMART reports as well. And I’ll capture all of the serial numbers for each too. Thank you for your advice.

7

Thank you. I will test that and let you know.

8

Thank you!! I think I’m having the heart attack right now too. I’m reading @joeschmuck right now.

9

I don’t mean to be that guy, and I’m not really adding anything of use in regards to getting your pool or data back, but that thing looks sketchy to me. Given the price that I’m seeing, it may be worth considering to drop ~$30 USD on a used, tried and true LSI card in the future…

Maybe I’m way off & this thing is great, but we’ve seen too many horror stories on the forums when it comes to less well known HBAs (or worse sata multipliers).

1 Like
10

I was about to say something similar to this.

See also Multiply your problems with SATA Port Multipliers and cheap SATA controllers

3 Likes
11

I just looked the board up, like I “DuckDuckWent” your Actimed 8 SATA card. PCIe x1? :fearful:
It appears to use Marvell 88SE9215 :fearful:, which is a PCIe 2.0x1 controller with 4 ports. So the card hides port multipliers and puts you pool at risk. :scream: (That, or it sports two Marvell on a PCIe switch, but port multipliers are less expensive and the heatsink is not that big. Anyway, Marvell controllers don’t have a good track record with ZFS.)

I’d suggest to also get a SAS HBA (on latest IT firmware) right now. X299 has the lanes for it.

12

Yeah, you’re not wrong. Didn’t I mention I was not the sharpest knife in the drawer? I will upgrade as I go and I’m thankful for your advice. Nothing but humility from me.

1 Like
13

You’re golden, you’ll be surprised how much the forums will sharpen you up! I’ve basically gone from clueless to getting straight up promotions by just repeating what I’ve been reading here… I’m still clueless, but the point is there are smart folks in this community and in my experience it straight up pays (literally) to listen to them.

14

The version I’m seeing of that card is saying an ASM1064 and a JMB575 port multiplier :grimacing:

image
image1500×1500 222 KB

I’m going to second the recommendation to replace it with a non-multiplying card. The ASMedia itself isn’t necessarily bad (although the newer ASM11xx chips are better) but that port multiplier is likely going to give you a headache.

3 Likes
15

Sorry about the noob question, but if I got the SAS HBA does that mean I need SAS drives or can I continue to use the SATA? Would this be a good card to get? LSI 9300-16i 16-Port 12Gb/s SAS Controller HBA Card with P16 IT Mode?

16

Yes, I can see that I will learn more quickly. I’ve muddled my way though so far but clearly it’s not enough.

17

Sure, with a fanout cable you can go from a single port on the HBA to 4 SATA connectors.

Whatever your source, they made sure to hit the keywords potential buyers would recognise. It’s difficutl to say if it’s a good source for hardware or not though. I have purchased mine from places selling old enterprise gear, as opposed to the rather prolific sellers in China.

I will say that the 16i cards run especially hot. Even the 8i models need good fan coverage, ideally a dedicated fan, the 16i models even more so.

2 Likes
18

Sata drives work with SAS. SAS drives don’t work with Sata. So any Sata drives will work perfectly with a SAS HBA.

I have no affiliation, but if you want to minimize risk of knock-off products, I’ve always heard good things with about:

Can you find cheaper deals on the same hardware? Yes. Is this the only source for good products with good support? No. Would I recommend them if you’re willing to spend an extra few $ & don’t want to deal with trying to figure out if the HBA you’re buying is already on the latest firmware & not a knockoff? Yes.

100% slap a fan on any HBA you end up buying (unless your NAS is in a server rack & you need hearing production due to the amount of noise your fans make).

Will buying an HBA instead of continuing to use the port multiplier magically recover your data? Maybe, but likely not. Will it significantly reduce future risks? Yes.

3 Likes
19

Hi @etorix. Here are the SMART test reports. To me it looks like serial number 7LG8S3HK is the only one failing (/dev/sdb). Am I interpreting that correctly?

root@bigtruenas[~]# sudo smartctl -a /dev/sba    
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

Smartctl open device: /dev/sba failed: No such device
root@bigtruenas[~]# sudo smartctl -a /dev/sda
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Ultrastar DC HC530
Device Model:     WDC  WUH721414ALE604
Serial Number:    7LG93A2K
LU WWN Device Id: 5 000cca 2dbc423d2
Firmware Version: LDGNW400
User Capacity:    14,000,519,643,136 bytes [14.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database 7.3/5706
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jun 18 12:26:19 2025 MDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x80) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1540) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   001    Pre-fail  Always       -       0
  2 Throughput_Performance  0x0005   136   136   054    Pre-fail  Offline      -       96
  3 Spin_Up_Time            0x0007   081   081   001    Pre-fail  Always       -       392 (Average 366)
  4 Start_Stop_Count        0x0012   100   100   000    Old_age   Always       -       31
  5 Reallocated_Sector_Ct   0x0033   100   100   001    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000b   100   100   001    Pre-fail  Always       -       0
  8 Seek_Time_Performance   0x0005   128   128   020    Pre-fail  Offline      -       18
  9 Power_On_Hours          0x0012   100   100   000    Old_age   Always       -       5115
 10 Spin_Retry_Count        0x0013   100   100   001    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       31
 22 Helium_Level            0x0023   100   100   025    Pre-fail  Always       -       100
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       312
193 Load_Cycle_Count        0x0012   100   100   000    Old_age   Always       -       312
194 Temperature_Celsius     0x0002   053   053   000    Old_age   Always       -       40 (Min/Max 19/54)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x000a   100   100   000    Old_age   Always       -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more


root@bigtruenas[~]# sudo smartctl -a /dev/sdb
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Ultrastar DC HC530
Device Model:     WDC  WUH721414ALE604
Serial Number:    7LG8S3HK
LU WWN Device Id: 5 000cca 2dbc3f9bb
Firmware Version: LDGNW400
User Capacity:    14,000,519,643,136 bytes [14.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database 7.3/5706
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jun 18 12:21:48 2025 MDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: FAILED!
Drive failure expected in less than 24 hours. SAVE ALL DATA.
See vendor-specific Attribute list for failed Attributes.

General SMART Values:
Offline data collection status:  (0x80) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1277) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   001    Pre-fail  Always       -       0
  2 Throughput_Performance  0x0005   137   137   054    Pre-fail  Offline      -       92
  3 Spin_Up_Time            0x0007   093   093   001    Pre-fail  Always       -       162 (Average 140)
  4 Start_Stop_Count        0x0012   100   100   000    Old_age   Always       -       42
  5 Reallocated_Sector_Ct   0x0033   100   100   001    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000b   001   001   001    Pre-fail  Always   FAILING_NOW 6160424
  8 Seek_Time_Performance   0x0005   128   128   020    Pre-fail  Offline      -       18
  9 Power_On_Hours          0x0012   100   100   000    Old_age   Always       -       5115
 10 Spin_Retry_Count        0x0013   100   100   001    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       42
 22 Helium_Level            0x0023   100   100   025    Pre-fail  Always       -       100
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       329
193 Load_Cycle_Count        0x0012   100   100   000    Old_age   Always       -       329
194 Temperature_Celsius     0x0002   056   056   000    Old_age   Always       -       38 (Min/Max 19/51)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x000a   100   100   000    Old_age   Always       -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

root@bigtruenas[~]# sudo smartctl -a /dev/sdc
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Ultrastar DC HC530
Device Model:     WDC  WUH721414ALE604
Serial Number:    7LG8HXLK
LU WWN Device Id: 5 000cca 2dbc3debd
Firmware Version: LDGNW400
User Capacity:    14,000,519,643,136 bytes [14.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database 7.3/5706
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jun 18 12:21:59 2025 MDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x80) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1508) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   001    Pre-fail  Always       -       0
  2 Throughput_Performance  0x0005   137   137   054    Pre-fail  Offline      -       92
  3 Spin_Up_Time            0x0007   082   082   001    Pre-fail  Always       -       352 (Average 395)
  4 Start_Stop_Count        0x0012   100   100   000    Old_age   Always       -       29
  5 Reallocated_Sector_Ct   0x0033   100   100   001    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000b   100   100   001    Pre-fail  Always       -       0
  8 Seek_Time_Performance   0x0005   128   128   020    Pre-fail  Offline      -       18
  9 Power_On_Hours          0x0012   100   100   000    Old_age   Always       -       5115
 10 Spin_Retry_Count        0x0013   100   100   001    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       29
 22 Helium_Level            0x0023   100   100   025    Pre-fail  Always       -       100
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       321
193 Load_Cycle_Count        0x0012   100   100   000    Old_age   Always       -       321
194 Temperature_Celsius     0x0002   055   055   000    Old_age   Always       -       39 (Min/Max 19/53)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x000a   100   100   000    Old_age   Always       -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

root@bigtruenas[~]# sudo smartctl -a /dev/sdd
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Ultrastar DC HC530
Device Model:     WDC  WUH721414ALE604
Serial Number:    7LG81ZSK
LU WWN Device Id: 5 000cca 2dbc3aa72
Firmware Version: LDGNW400
User Capacity:    14,000,519,643,136 bytes [14.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database 7.3/5706
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jun 18 12:22:08 2025 MDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x80) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1516) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   001    Pre-fail  Always       -       0
  2 Throughput_Performance  0x0005   137   137   054    Pre-fail  Offline      -       92
  3 Spin_Up_Time            0x0007   091   091   001    Pre-fail  Always       -       294 (Average 86)
  4 Start_Stop_Count        0x0012   014   014   000    Old_age   Always       -       34712
  5 Reallocated_Sector_Ct   0x0033   100   100   001    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000b   100   100   001    Pre-fail  Always       -       0
  8 Seek_Time_Performance   0x0005   128   128   020    Pre-fail  Offline      -       18
  9 Power_On_Hours          0x0012   100   100   000    Old_age   Always       -       5106
 10 Spin_Retry_Count        0x0013   100   100   001    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   001   001   000    Old_age   Always       -       34712
 22 Helium_Level            0x0023   100   100   025    Pre-fail  Always       -       100
192 Power-Off_Retract_Count 0x0032   095   095   000    Old_age   Always       -       35109
193 Load_Cycle_Count        0x0012   095   095   000    Old_age   Always       -       35109
194 Temperature_Celsius     0x0002   055   055   000    Old_age   Always       -       39 (Min/Max 19/50)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x000a   100   100   000    Old_age   Always       -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

root@bigtruenas[~]# sudo smartctl -a /dev/sde
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Ultrastar DC HC530
Device Model:     WDC  WUH721414ALE604
Serial Number:    7LG8BEGK
LU WWN Device Id: 5 000cca 2dbc3cde4
Firmware Version: LDGNW400
User Capacity:    14,000,519,643,136 bytes [14.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database 7.3/5706
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jun 18 12:22:16 2025 MDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1565) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   001    Pre-fail  Always       -       0
  2 Throughput_Performance  0x0005   137   137   054    Pre-fail  Offline      -       92
  3 Spin_Up_Time            0x0007   092   092   001    Pre-fail  Always       -       260 (Average 70)
  4 Start_Stop_Count        0x0012   070   070   000    Old_age   Always       -       12278
  5 Reallocated_Sector_Ct   0x0033   100   100   001    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000b   100   100   001    Pre-fail  Always       -       0
  8 Seek_Time_Performance   0x0005   128   128   020    Pre-fail  Offline      -       18
  9 Power_On_Hours          0x0012   100   100   000    Old_age   Always       -       5112
 10 Spin_Retry_Count        0x0013   100   100   001    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   001   001   000    Old_age   Always       -       12278
 22 Helium_Level            0x0023   100   100   025    Pre-fail  Always       -       100
192 Power-Off_Retract_Count 0x0032   099   099   000    Old_age   Always       -       12575
193 Load_Cycle_Count        0x0012   099   099   000    Old_age   Always       -       12575
194 Temperature_Celsius     0x0002   050   050   000    Old_age   Always       -       43 (Min/Max 18/52)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0022   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0008   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x000a   100   100   000    Old_age   Always       -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

root@bigtruenas[~]# sudo smartctl -a /dev/sdf
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Ultrastar DC HC530
Device Model:     WDC  WUH721414ALE604
Serial Number:    9JHJR4NT
LU WWN Device Id: 5 000cca 258d5b097
Firmware Version: LDGSW2L0
User Capacity:    14,000,519,643,136 bytes [14.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database 7.3/5706
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jun 18 12:22:24 2025 MDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x80) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1381) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   001    Pre-fail  Always       -       1
  2 Throughput_Performance  0x0005   137   137   054    Pre-fail  Offline      -       92
  3 Spin_Up_Time            0x0007   082   082   001    Pre-fail  Always       -       387 (Average 360)
  4 Start_Stop_Count        0x0012   100   100   000    Old_age   Always       -       58
  5 Reallocated_Sector_Ct   0x0033   100   100   001    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000b   100   100   001    Pre-fail  Always       -       0
  8 Seek_Time_Performance   0x0005   128   128   020    Pre-fail  Offline      -       18
  9 Power_On_Hours          0x0012   095   095   000    Old_age   Always       -       40231
 10 Spin_Retry_Count        0x0013   100   100   001    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       43
 22 Helium_Level            0x0023   100   100   025    Pre-fail  Always       -       100
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       1551
193 Load_Cycle_Count        0x0012   100   100   000    Old_age   Always       -       1551
194 Temperature_Celsius     0x0002   049   049   000    Old_age   Always       -       44 (Min/Max 18/52)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0022   100   100   000    Old_age   Always       -       96
198 Offline_Uncorrectable   0x0008   100   100   000    Old_age   Offline      -       37
199 UDMA_CRC_Error_Count    0x000a   100   100   000    Old_age   Always       -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

root@bigtruenas[~]# sudo smartctl -a /dev/sdg 
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     OOS14000G
Serial Number:    000Q4DRE
LU WWN Device Id: 5 000c50 0e6507f86
Firmware Version: OOS1
User Capacity:    14,000,519,643,136 bytes [14.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database 7.3/5706
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jun 18 12:22:31 2025 MDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  567) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1243) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x50bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   080   064   044    Pre-fail  Always       -       107761462
  3 Spin_Up_Time            0x0003   092   090   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       153
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   082   060   045    Pre-fail  Always       -       152994317
  9 Power_On_Hours          0x0032   092   092   000    Old_age   Always       -       7165
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       41
 18 Unknown_Attribute       0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   063   045   040    Old_age   Always       -       37 (Min/Max 34/40)
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       19
193 Load_Cycle_Count        0x0032   098   098   000    Old_age   Always       -       5277
194 Temperature_Celsius     0x0022   037   055   000    Old_age   Always       -       37 (0 18 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       6347 (118 125 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       44601084403
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       33634822858

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

root@bigtruenas[~]# sudo smartctl -a /dev/sdh
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     OOS14000G
Serial Number:    00002LD2
LU WWN Device Id: 5 000c50 0b574aa3d
Firmware Version: OOS1
User Capacity:    14,000,519,643,136 bytes [14.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database 7.3/5706
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jun 18 12:22:43 2025 MDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  567) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1270) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x50bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   082   064   044    Pre-fail  Always       -       175545950
  3 Spin_Up_Time            0x0003   094   090   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   099   099   020    Old_age   Always       -       1968
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   082   060   045    Pre-fail  Always       -       158576692
  9 Power_On_Hours          0x0032   092   092   000    Old_age   Always       -       7537
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   099   099   020    Old_age   Always       -       1878
 18 Unknown_Attribute       0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   064   049   040    Old_age   Always       -       36 (Min/Max 34/40)
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       1859
193 Load_Cycle_Count        0x0032   094   094   000    Old_age   Always       -       13896
194 Temperature_Celsius     0x0022   036   051   000    Old_age   Always       -       36 (0 18 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       6084 (243 92 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       44948202507
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       33915854181

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

root@bigtruenas[~]# sudo smartctl -a /dev/sdi
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     OOS14000G
Serial Number:    000M6KKV
LU WWN Device Id: 5 000c50 0dcb1bef2
Firmware Version: OOS1
User Capacity:    14,000,519,643,136 bytes [14.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database 7.3/5706
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jun 18 12:22:48 2025 MDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  567) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1243) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x50bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   083   064   044    Pre-fail  Always       -       184462822
  3 Spin_Up_Time            0x0003   094   090   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   095   095   020    Old_age   Always       -       5342
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   082   060   045    Pre-fail  Always       -       153005798
  9 Power_On_Hours          0x0032   092   092   000    Old_age   Always       -       7407
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   095   095   020    Old_age   Always       -       5211
 18 Unknown_Attribute       0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   063   048   040    Old_age   Always       -       37 (Min/Max 35/37)
192 Power-Off_Retract_Count 0x0032   098   098   000    Old_age   Always       -       5177
193 Load_Cycle_Count        0x0032   095   095   000    Old_age   Always       -       11216
194 Temperature_Celsius     0x0022   037   052   000    Old_age   Always       -       37 (0 18 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       6297 (7 36 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       47663058937
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       34376067628

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

root@bigtruenas[~]# sudo smartctl -a /dev/sdj
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     OOS14000G
Serial Number:    0000AMD2
LU WWN Device Id: 5 000c50 0b6ce40bc
Firmware Version: OOS1
User Capacity:    14,000,519,643,136 bytes [14.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database 7.3/5706
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jun 18 12:23:03 2025 MDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  575) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1308) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x50bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   100   064   044    Pre-fail  Always       -       1202230
  3 Spin_Up_Time            0x0003   094   090   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       843
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   082   060   045    Pre-fail  Always       -       159353225
  9 Power_On_Hours          0x0032   092   092   000    Old_age   Always       -       7519
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       718
 18 Unknown_Attribute       0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   098   000    Old_age   Always       -       8590065667
190 Airflow_Temperature_Cel 0x0022   063   046   040    Old_age   Always       -       37 (Min/Max 35/41)
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       688
193 Load_Cycle_Count        0x0032   096   096   000    Old_age   Always       -       9526
194 Temperature_Celsius     0x0022   037   054   000    Old_age   Always       -       37 (0 18 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       6257 (208 174 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       44975782562
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       33648852773

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more
20

The sdb (SN: 7LG8S3HK) has indeed tripped a pre-fail in the SMART report. Hopefully under warranty with only 5k-ish hours on it?

But sdf (SN: 9JHJR4NT) is also showing signs with a bunch of sectors pending, offline and a read error.

Some other drives, like sdd show wildly excessive Power_Cycle_Count of 34712. Is that value encoded? Compare to sdc that only has 29.

1 Like