Every Drive in my pool has degraded!

TrueNAS General
1

Hi Everyone,

I have had my server built for several years without any issues, i previously had used some second-hand drives and had to replace a drive on a few occasions. However, about 2 years ago, I rebuilt the server, using 7 16 TB WD Red Pro drives in a Z2 configuration. On Saturday, I logged onto the server to service some cloud sync tasks when I noticed the pool was degraded. Upon inspection, it appears all of my drives are degraded.

I ran a scrub, which resulted in 5 errors; all of the drives are showing checksum errors, but no read or write. I am not quite sure of my next steps. The server is built using old computer parts, bar the drives, so my main thinking is that another component is causing these issues. It would seem very odd for all 7 drives to have an issue at the same time. However, I am very much a novice and would appreciate any advice on what I should do next.

Thanks in advance

  • Asus MAXIMUS VIII HERO
  • Intel(R) Core™ i5-6600K CPU @ 3.50GHz
  • 128GB DDR4
2

Please provide the output of zpool status -v (prefixed by sudo on SCALE), formatted with the </> button.
Indeed it may not be the drives; possible suspects are:

  • overheating HBA (but I suspect there’s none);
  • failing RAM (run a few passes of memtest);
  • bad cables;
  • failing PSU.
1 Like
3

Your hardware spec leaves a lot to be desired.
There is no PSU, there is no boot drive, no GPU (but you may not have one).

How are the HDD’s connected to the motherboard

How are you powering the HDD’s, Molex to SATA splitters, SATA Splitters, a backplane

Details matter.

However in general all drives haveing issues generally indicates that something upstream is causing the issue.

4

Hello, thank you for the help. I believe I have done what you asked; however, I’m a novice with code, so please let me know if I have messed up. Below is the output.

pool: Broductions Server
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.

scan: scrub repaired 12.6M in 1 days 02:29:50 with 5 errors on Sun Sep 14 10:38:57 2025
config:

    NAME                                            STATE     READ WRITE CKSUM
    Broductions Server                              DEGRADED     0     0 0
      raidz2-0                                      DEGRADED     0     0 0
        gptid/fbe8d59e-f3c7-11ec-9328-a0369fda124c  DEGRADED     0     0   324  too many errors
        gptid/fc81adf8-f3c7-11ec-9328-a0369fda124c  DEGRADED     0     0   321  too many errors
        gptid/fcabb7fc-f3c7-11ec-9328-a0369fda124c  DEGRADED     0     0   321  too many errors
        gptid/fcae7142-f3c7-11ec-9328-a0369fda124c  DEGRADED     0     0   334  too many errors
        gptid/fcb16723-f3c7-11ec-9328-a0369fda124c  DEGRADED     0     0   314  too many errors
        gptid/fca1d259-f3c7-11ec-9328-a0369fda124c  DEGRADED     0     0   318  too many errors
        gptid/fca59322-f3c7-11ec-9328-a0369fda124c  DEGRADED     0     0   314  too many errors

errors: Permanent errors have been detected in the following files:

    /mnt/Broductions Server/Weddings 24/Aine & Ciaran/Speaches/B019C978_240623X9_RONAN.MXF
    /mnt/Broductions Server/Weddings 24/2025/Jess & Evan/Speaches/B_0024D049A250815_160830EJ_RONAN.MP4
    /mnt/Broductions Server/Videos/The Social Plug/ProPadz/A320D005_250819OZ_RONAN.MXF
    /mnt/Broductions Server/Videos/The Social Plug/ProPadz/A322C104_250820GX_RONAN.MXF

pool: boot-pool
state: ONLINE
scan: scrub repaired 0B in 00:00:19 with 0 errors on Tue Sep 9 03:45:19 2025

Regarding questions about the hardware, I apologise for the lack of detail.

Power supply is a Corsair CX600
All drives are powered via Molex to SATA
There is a 120 GB M.5 SSD as a boot drive
There is no GPU
There is a 10G PCI-E Network Card

Thanks again

5

A 600 W PSU should just fit (see “Xeon E3”)

but it’s possible that the PSU is ageing, or that one rail is overloaded.

Before considering replacing the PSU, run memtest86 to test your RAM.

And before that, let’s check the drives with smartctl -x /dev/XXX, where XXX is replaced by each drive in the pool (should be ada0, ada1, etc. on CORE or sda, sdb, etc. on SCALE).
All drives attached to the motherboard, right? Or is somekind of “SATA card” involved?
Again, for readbility please select each SMART output to format with the </> button, or paste between two lines of three backquotes
```
terminal output here
```

Some data is already lost. Do you have a backup?

6

chksum errors are often the result of cabling issues - they are not drive issues. Given that these are occurring on all devices in the pool then you (I suspect) need to look upstream.

One question that you have conspicuously failed to answer is how the HDD’s are connected to the motherboard. My initial google fu show 6 sata interfaces. You have 7 drives plus a boot drive

@etorix has mentioned a few possibilities.
Reseat the cables - easy job
Run memtest for 24 hours minimum - easy even if it takes a while

How many drives do you have connected to each PSU cable - are you perhaps overloading a cable (probably not).

1 Like
7

Another thing that irritates me:

You PSU has 6 SATA connectors - why would you power all drives with Molex to sata [lose all your data] adapters?

But of course it would be more important to know, how those HDDs are connected data-wise.

8

It is 100% possible that the PSU is ageing, as I have had the unit for several years. If that is what needs to be replaced, that would be understandable.

I have run the SMART tests and pasted the output below; hopefully, this is formatted correctly. I wasn’t able to include all of the drives due to a character limit, so I will split them in another comment.

All 7 drives are connected directly to the motherboard, no SATA card.

I will figure out how to run memtest86 in the meantime and will let you know the result.

Once that is complete, I will shut down the server and try to reseat all the cables in case that helps. I will also check the exact cabling setup on the power end. I am not sure of the exact split at the moment.

Thanks again for the help.


=== START OF INFORMATION SECTION ===
Device Model:     WDC WD161KFGX-68AFPN0
Serial Number:    3XG35WDU
LU WWN Device Id: 5 000cca 285c17333
Firmware Version: 83.00A83
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 published, ANSI INCITS 529-2018
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Sep 16 05:34:28 2025 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Disabled
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1758) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   001    -    0
  2 Throughput_Performance  --S---   136   136   054    -    96
  3 Spin_Up_Time            POS---   082   082   001    -    372 (Average 373)
  4 Start_Stop_Count        -O--C-   100   100   000    -    19
  5 Reallocated_Sector_Ct   PO--CK   100   100   001    -    0
  7 Seek_Error_Rate         -O-R--   100   100   001    -    0
  8 Seek_Time_Performance   --S---   140   140   020    -    15
  9 Power_On_Hours          -O--C-   097   097   000    -    27748
 10 Spin_Retry_Count        -O--C-   100   100   001    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    19
 22 Unknown_Attribute       PO---K   100   100   025    -    100
192 Power-Off_Retract_Count -O--CK   100   100   000    -    1178
193 Load_Cycle_Count        -O--C-   100   100   000    -    1178
194 Temperature_Celsius     -O----   062   062   000    -    33 (Min/Max 14/44)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O---K   100   100   000    -    0
198 Offline_Uncorrectable   ---R--   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O-R--   100   100   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      1  Comprehensive SMART error log
0x03       GPL     R/O      1  Ext. Comprehensive SMART error log
0x04       GPL     R/O    256  Device Statistics log
0x04       SL      R/O    255  Device Statistics log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x08       GPL     R/O      2  Power Conditions log
0x09           SL  R/W      1  Selective self-test log
0x0c       GPL     R/O  30001  Pending Defects log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x12       GPL     R/O      1  SATA NCQ Non-Data log
0x13       GPL     R/O      1  SATA NCQ Send and Receive log
0x15       GPL     R/W      1  Rebuild Assist log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x24       GPL     R/O    256  Current Device Internal Status Data log
0x25       GPL     R/O    256  Saved Device Internal Status Data log
0x2f       GPL     -        1  Set Sector Configuration
0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       256 (0x0100)
Device State:                        Active (0)
Current Temperature:                    33 Celsius
Power Cycle Min/Max Temperature:     23/43 Celsius
Lifetime    Min/Max Temperature:     14/44 Celsius
Under/Over Temperature Limit Count:   0/0
SMART Status:                        0xc24f (PASSED)
Minimum supported ERC Time Limit:    70 (7.0 seconds)

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -40/70 Celsius
Temperature History Size (Index):    128 (34)

Index    Estimated Time   Temperature Celsius
  35    2025-09-16 03:27    32  *************
 ...    ..( 76 skipped).    ..  *************
 112    2025-09-16 04:44    32  *************
 113    2025-09-16 04:45    33  **************
 114    2025-09-16 04:46    33  **************
 115    2025-09-16 04:47    32  *************
 116    2025-09-16 04:48    32  *************
 117    2025-09-16 04:49    33  **************
 118    2025-09-16 04:50    32  *************
 119    2025-09-16 04:51    32  *************
 120    2025-09-16 04:52    33  **************
 121    2025-09-16 04:53    33  **************
 122    2025-09-16 04:54    32  *************
 123    2025-09-16 04:55    33  **************
 124    2025-09-16 04:56    32  *************
 125    2025-09-16 04:57    33  **************
 ...    ..( 35 skipped).    ..  **************
  33    2025-09-16 05:33    33  **************
  34    2025-09-16 05:34    32  *************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP Log 0x04)
Page  Offset Size        Value Flags Description
0x01  =====  =               =  ===  == General Statistics (rev 1) ==
0x01  0x008  4              19  ---  Lifetime Power-On Resets
0x01  0x010  4           27748  ---  Power-on Hours
0x01  0x018  6     64370679322  ---  Logical Sectors Written
0x01  0x020  6       796280101  ---  Number of Write Commands
0x01  0x028  6    140641770836  ---  Logical Sectors Read
0x01  0x030  6      1464512025  ---  Number of Read Commands
0x01  0x038  6     99894058200  ---  Date and Time TimeStamp
0x03  =====  =               =  ===  == Rotating Media Statistics (rev 1) ==
0x03  0x008  4           27748  ---  Spindle Motor Power-on Hours
0x03  0x010  4           27747  ---  Head Flying Hours
0x03  0x018  4            1178  ---  Head Load Events
0x03  0x020  4               0  ---  Number of Reallocated Logical Sectors
0x03  0x028  4               0  ---  Read Recovery Attempts
0x03  0x030  4               0  ---  Number of Mechanical Start Failures
0x04  =====  =               =  ===  == General Errors Statistics (rev 1) ==
0x04  0x008  4               0  ---  Number of Reported Uncorrectable Errors
0x04  0x010  4               0  ---  Resets Between Cmd Acceptance and Completion
0x05  =====  =               =  ===  == Temperature Statistics (rev 1) ==
0x05  0x008  1              33  ---  Current Temperature
0x05  0x010  1              31  N--  Average Short Term Temperature
0x05  0x018  1              31  N--  Average Long Term Temperature
0x05  0x020  1              44  ---  Highest Temperature
0x05  0x028  1              14  ---  Lowest Temperature
0x05  0x030  1              42  N--  Highest Average Short Term Temperature
0x05  0x038  1              22  N--  Lowest Average Short Term Temperature
0x05  0x040  1              38  N--  Highest Average Long Term Temperature
0x05  0x048  1              24  N--  Lowest Average Long Term Temperature
0x05  0x050  4               0  ---  Time in Over-Temperature
0x05  0x058  1              60  ---  Specified Maximum Operating Temperature
0x05  0x060  4               0  ---  Time in Under-Temperature
0x05  0x068  1               0  ---  Specified Minimum Operating Temperature
0x06  =====  =               =  ===  == Transport Statistics (rev 1) ==
0x06  0x008  4              58  ---  Number of Hardware Resets
0x06  0x010  4               3  ---  Number of ASR Events
0x06  0x018  4               0  ---  Number of Interface CRC Errors
0xff  =====  =               =  ===  == Vendor Specific Statistics (rev 1) ==
0xff  0x040  7            1383  ---  Vendor Specific
0xff  0x048  7               0  ---  Vendor Specific
0xff  0x050  7               0  ---  Vendor Specific
0xff  0x058  7               0  ---  Vendor Specific
0xff  0x060  7               0  ---  Vendor Specific
0xff  0x068  7               0  ---  Vendor Specific
0xff  0x070  7            3270  ---  Vendor Specific
0xff  0x078  7               0  ---  Vendor Specific
0xff  0x080  7              14  ---  Vendor Specific
                                |||_ C monitored condition met
                                ||__ D supports DSN
                                |___ N normalized value

Pending Defects log (GP Log 0x0c)
No Defects Logged

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data F



=== START OF INFORMATION SECTION ===
Device Model:     WDC WD161KFGX-68AFPN0
Serial Number:    2PH9BB6T
LU WWN Device Id: 5 000cca 2c1d2597e
Firmware Version: 83.00A83
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 published, ANSI INCITS 529-2018
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Sep 16 05:20:11 2025 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Disabled
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1712) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   001    -    0
  2 Throughput_Performance  --S---   135   135   054    -    100
  3 Spin_Up_Time            POS---   082   082   001    -    367 (Average 360)
  4 Start_Stop_Count        -O--C-   100   100   000    -    22
  5 Reallocated_Sector_Ct   PO--CK   100   100   001    -    0
  7 Seek_Error_Rate         -O-R--   100   100   001    -    0
  8 Seek_Time_Performance   --S---   140   140   020    -    15
  9 Power_On_Hours          -O--C-   097   097   000    -    27748
 10 Spin_Retry_Count        -O--C-   100   100   001    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    22
 22 Unknown_Attribute       PO---K   100   100   025    -    100
192 Power-Off_Retract_Count -O--CK   100   100   000    -    1180
193 Load_Cycle_Count        -O--C-   100   100   000    -    1180
194 Temperature_Celsius     -O----   053   053   000    -    40 (Min/Max 12/52)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O---K   100   100   000    -    0
198 Offline_Uncorrectable   ---R--   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O-R--   100   100   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      1  Comprehensive SMART error log
0x03       GPL     R/O      1  Ext. Comprehensive SMART error log
0x04       GPL     R/O    256  Device Statistics log
0x04       SL      R/O    255  Device Statistics log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x08       GPL     R/O      2  Power Conditions log
0x09           SL  R/W      1  Selective self-test log
0x0c       GPL     R/O  30001  Pending Defects log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x12       GPL     R/O      1  SATA NCQ Non-Data log
0x13       GPL     R/O      1  SATA NCQ Send and Receive log
0x15       GPL     R/W      1  Rebuild Assist log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x24       GPL     R/O    256  Current Device Internal Status Data log
0x25       GPL     R/O    256  Saved Device Internal Status Data log
0x2f       GPL     -        1  Set Sector Configuration
0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       256 (0x0100)
Device State:                        Active (0)
Current Temperature:                    40 Celsius
Power Cycle Min/Max Temperature:     22/51 Celsius
Lifetime    Min/Max Temperature:     12/52 Celsius
Under/Over Temperature Limit Count:   0/0
SMART Status:                        0xc24f (PASSED)
Minimum supported ERC Time Limit:    70 (7.0 seconds)

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -40/70 Celsius
Temperature History Size (Index):    128 (102)

Index    Estimated Time   Temperature Celsius
 103    2025-09-16 03:13    39  ********************
 ...    ..(  8 skipped).    ..  ********************
 112    2025-09-16 03:22    39  ********************
 113    2025-09-16 03:23    40  *********************
 114    2025-09-16 03:24    39  ********************
 115    2025-09-16 03:25    39  ********************
 116    2025-09-16 03:26    40  *********************
 117    2025-09-16 03:27    39  ********************
 ...    ..(  2 skipped).    ..  ********************
 120    2025-09-16 03:30    39  ********************
 121    2025-09-16 03:31    40  *********************
 ...    ..(  5 skipped).    ..  *********************
 127    2025-09-16 03:37    40  *********************
   0    2025-09-16 03:38    39  ********************
   1    2025-09-16 03:39    40  *********************
   2    2025-09-16 03:40    40  *********************
   3    2025-09-16 03:41    39  ********************
   4    2025-09-16 03:42    40  *********************
 ...    ..( 13 skipped).    ..  *********************
  18    2025-09-16 03:56    40  *********************
  19    2025-09-16 03:57    39  ********************
  20    2025-09-16 03:58    40  *********************
 ...    ..( 80 skipped).    ..  *********************
 101    2025-09-16 05:19    40  *********************
 102    2025-09-16 05:20    39  ********************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP Log 0x04)
Page  Offset Size        Value Flags Description
0x01  =====  =               =  ===  == General Statistics (rev 1) ==
0x01  0x008  4              22  ---  Lifetime Power-On Resets
0x01  0x010  4           27748  ---  Power-on Hours
0x01  0x018  6     64372492138  ---  Logical Sectors Written
0x01  0x020  6       795199791  ---  Number of Write Commands
0x01  0x028  6    140668572182  ---  Logical Sectors Read
0x01  0x030  6      1457170771  ---  Number of Read Commands
0x01  0x038  6     99895867900  ---  Date and Time TimeStamp
0x03  =====  =               =  ===  == Rotating Media Statistics (rev 1) ==
0x03  0x008  4           27748  ---  Spindle Motor Power-on Hours
0x03  0x010  4           27748  ---  Head Flying Hours
0x03  0x018  4            1180  ---  Head Load Events
0x03  0x020  4               0  ---  Number of Reallocated Logical Sectors
0x03  0x028  4               0  ---  Read Recovery Attempts
0x03  0x030  4               0  ---  Number of Mechanical Start Failures
0x04  =====  =               =  ===  == General Errors Statistics (rev 1) ==
0x04  0x008  4               0  ---  Number of Reported Uncorrectable Errors
0x04  0x010  4               0  ---  Resets Between Cmd Acceptance and Completion
0x05  =====  =               =  ===  == Temperature Statistics (rev 1) ==
0x05  0x008  1              40  ---  Current Temperature
0x05  0x010  1              38  N--  Average Short Term Temperature
0x05  0x018  1              39  N--  Average Long Term Temperature
0x05  0x020  1              52  ---  Highest Temperature
0x05  0x028  1              12  ---  Lowest Temperature
0x05  0x030  1              50  N--  Highest Average Short Term Temperature
0x05  0x038  1              25  N--  Lowest Average Short Term Temperature
0x05  0x040  1              43  N--  Highest Average Long Term Temperature
0x05  0x048  1              25  N--  Lowest Average Long Term Temperature
0x05  0x050  4               0  ---  Time in Over-Temperature
0x05  0x058  1              60  ---  Specified Maximum Operating Temperature
0x05  0x060  4               0  ---  Time in Under-Temperature
0x05  0x068  1               0  ---  Specified Minimum Operating Temperature
0x06  =====  =               =  ===  == Transport Statistics (rev 1) ==
0x06  0x008  4              54  ---  Number of Hardware Resets
0x06  0x010  4               6  ---  Number of ASR Events
0x06  0x018  4               0  ---  Number of Interface CRC Errors
0xff  =====  =               =  ===  == Vendor Specific Statistics (rev 1) ==
0xff  0x040  7            1359  ---  Vendor Specific
0xff  0x048  7               0  ---  Vendor Specific
0xff  0x050  7               0  ---  Vendor Specific
0xff  0x058  7               0  ---  Vendor Specific
0xff  0x060  7               0  ---  Vendor Specific
0xff  0x068  7               0  ---  Vendor Specific
0xff  0x070  7             847  ---  Vendor Specific
0xff  0x078  7               0  ---  Vendor Specific
0xff  0x080  7              14  ---  Vendor Specific
                                |||_ C monitored condition met
                                ||__ D supports DSN
                                |___ N normalized value

Pending Defects log (GP Log 0x0c)
No Defects Logged

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0008  2            0  Device-to-host non-data FIS retries
0x0009  2            2  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            3  Device-to-host registe



=== START OF INFORMATION SECTION ===
Device Model:     WDC WD161KFGX-68AFPN0
Serial Number:    2PH9XGHT
LU WWN Device Id: 5 000cca 2c1d299b5
Firmware Version: 83.00A83
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 published, ANSI INCITS 529-2018
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Sep 16 05:20:50 2025 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Disabled
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1676) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   001    -    0
  2 Throughput_Performance  --S---   134   134   054    -    104
  3 Spin_Up_Time            POS---   083   083   001    -    353 (Average 353)
  4 Start_Stop_Count        -O--C-   100   100   000    -    22
  5 Reallocated_Sector_Ct   PO--CK   100   100   001    -    0
  7 Seek_Error_Rate         -O-R--   100   100   001    -    0
  8 Seek_Time_Performance   --S---   140   140   020    -    15
  9 Power_On_Hours          -O--C-   097   097   000    -    27748
 10 Spin_Retry_Count        -O--C-   100   100   001    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    22
 22 Unknown_Attribute       PO---K   100   100   025    -    100
192 Power-Off_Retract_Count -O--CK   100   100   000    -    1180
193 Load_Cycle_Count        -O--C-   100   100   000    -    1180
194 Temperature_Celsius     -O----   044   044   000    -    48 (Min/Max 13/60)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O---K   100   100   000    -    0
198 Offline_Uncorrectable   ---R--   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O-R--   100   100   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      1  Comprehensive SMART error log
0x03       GPL     R/O      1  Ext. Comprehensive SMART error log
0x04       GPL     R/O    256  Device Statistics log
0x04       SL      R/O    255  Device Statistics log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x08       GPL     R/O      2  Power Conditions log
0x09           SL  R/W      1  Selective self-test log
0x0c       GPL     R/O  30001  Pending Defects log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x12       GPL     R/O      1  SATA NCQ Non-Data log
0x13       GPL     R/O      1  SATA NCQ Send and Receive log
0x15       GPL     R/W      1  Rebuild Assist log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x24       GPL     R/O    256  Current Device Internal Status Data log
0x25       GPL     R/O    256  Saved Device Internal Status Data log
0x2f       GPL     -        1  Set Sector Configuration
0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       256 (0x0100)
Device State:                        Active (0)
Current Temperature:                    48 Celsius
Power Cycle Min/Max Temperature:     23/58 Celsius
Lifetime    Min/Max Temperature:     13/60 Celsius
Under/Over Temperature Limit Count:   0/0
SMART Status:                        0xc24f (PASSED)
Minimum supported ERC Time Limit:    70 (7.0 seconds)

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -40/70 Celsius
Temperature History Size (Index):    128 (83)

Index    Estimated Time   Temperature Celsius
  84    2025-09-16 03:13    47  ****************************
 ...    ..( 90 skipped).    ..  ****************************
  47    2025-09-16 04:44    47  ****************************
  48    2025-09-16 04:45    48  *****************************
  49    2025-09-16 04:46    48  *****************************
  50    2025-09-16 04:47    47  ****************************
  51    2025-09-16 04:48    48  *****************************
  52    2025-09-16 04:49    47  ****************************
  53    2025-09-16 04:50    47  ****************************
  54    2025-09-16 04:51    48  *****************************
  55    2025-09-16 04:52    47  ****************************
  56    2025-09-16 04:53    47  ****************************
  57    2025-09-16 04:54    48  *****************************
 ...    ..( 24 skipped).    ..  *****************************
  82    2025-09-16 05:19    48  *****************************
  83    2025-09-16 05:20    47  ****************************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP Log 0x04)
Page  Offset Size        Value Flags Description
0x01  =====  =               =  ===  == General Statistics (rev 1) ==
0x01  0x008  4              22  ---  Lifetime Power-On Resets
0x01  0x010  4           27748  ---  Power-on Hours
0x01  0x018  6     64370972842  ---  Logical Sectors Written
0x01  0x020  6       795727612  ---  Number of Write Commands
0x01  0x028  6    140670537984  ---  Logical Sectors Read
0x01  0x030  6      1457413868  ---  Number of Read Commands
0x01  0x038  6     99893536950  ---  Date and Time TimeStamp
0x03  =====  =               =  ===  == Rotating Media Statistics (rev 1) ==
0x03  0x008  4           27747  ---  Spindle Motor Power-on Hours
0x03  0x010  4           27747  ---  Head Flying Hours
0x03  0x018  4            1180  ---  Head Load Events
0x03  0x020  4               0  ---  Number of Reallocated Logical Sectors
0x03  0x028  4              59  ---  Read Recovery Attempts
0x03  0x030  4               0  ---  Number of Mechanical Start Failures
0x04  =====  =               =  ===  == General Errors Statistics (rev 1) ==
0x04  0x008  4               0  ---  Number of Reported Uncorrectable Errors
0x04  0x010  4               0  ---  Resets Between Cmd Acceptance and Completion
0x05  =====  =               =  ===  == Temperature Statistics (rev 1) ==
0x05  0x008  1              48  ---  Current Temperature
0x05  0x010  1              46  N--  Average Short Term Temperature
0x05  0x018  1              46  N--  Average Long Term Temperature
0x05  0x020  1              60  ---  Highest Temperature
0x05  0x028  1              13  ---  Lowest Temperature
0x05  0x030  1              58  N--  Highest Average Short Term Temperature
0x05  0x038  1              25  N--  Lowest Average Short Term Temperature
0x05  0x040  1              53  N--  Highest Average Long Term Temperature
0x05  0x048  1              25  N--  Lowest Average Long Term Temperature
0x05  0x050  4               0  ---  Time in Over-Temperature
0x05  0x058  1              60  ---  Specified Maximum Operating Temperature
0x05  0x060  4               0  ---  Time in Under-Temperature
0x05  0x068  1               0  ---  Specified Minimum Operating Temperature
0x06  =====  =               =  ===  == Transport Statistics (rev 1) ==
0x06  0x008  4              51  ---  Number of Hardware Resets
0x06  0x010  4               6  ---  Number of ASR Events
0x06  0x018  4               0  ---  Number of Interface CRC Errors
0xff  =====  =               =  ===  == Vendor Specific Statistics (rev 1) ==
0xff  0x040  7            1382  ---  Vendor Specific
0xff  0x048  7               0  ---  Vendor Specific
0xff  0x050  7               0  ---  Vendor Specific
0xff  0x058  7               0  ---  Vendor Specific
0xff  0x060  7               0  ---  Vendor Specific
0xff  0x068  7           43868  ---  Vendor Specific
0xff  0x070  7               0  ---  Vendor Specific
0xff  0x078  7               0  ---  Vendor Specific
0xff  0x080  7              14  ---  Vendor Specific
                                |||_ C monitored condition met
                                ||__ D supports DSN
                                |___ N normalized value

Pending Defects log (GP Log 0x0c)
No Defects Logged

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device


=== START OF INFORMATION SECTION ===
Device Model:     WDC WD161KFGX-68AFPN0
Serial Number:    2PH9ZHBT
LU WWN Device Id: 5 000cca 2c1d2a152
Firmware Version: 83.00A83
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 published, ANSI INCITS 529-2018
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Sep 16 05:51:47 2025 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Disabled
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1707) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   001    -    0
  2 Throughput_Performance  --S---   136   136   054    -    96
  3 Spin_Up_Time            POS---   082   082   001    -    367 (Average 361)
  4 Start_Stop_Count        -O--C-   100   100   000    -    22
  5 Reallocated_Sector_Ct   PO--CK   100   100   001    -    0
  7 Seek_Error_Rate         -O-R--   100   100   001    -    0
  8 Seek_Time_Performance   --S---   140   140   020    -    15
  9 Power_On_Hours          -O--C-   097   097   000    -    27749
 10 Spin_Retry_Count        -O--C-   100   100   001    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    22
 22 Unknown_Attribute       PO---K   100   100   025    -    100
192 Power-Off_Retract_Count -O--CK   100   100   000    -    1184
193 Load_Cycle_Count        -O--C-   100   100   000    -    1184
194 Temperature_Celsius     -O----   039   039   000    -    52 (Min/Max 13/62)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O---K   100   100   000    -    0
198 Offline_Uncorrectable   ---R--   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O-R--   100   100   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      1  Comprehensive SMART error log
0x03       GPL     R/O      1  Ext. Comprehensive SMART error log
0x04       GPL     R/O    256  Device Statistics log
0x04       SL      R/O    255  Device Statistics log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x08       GPL     R/O      2  Power Conditions log
0x09           SL  R/W      1  Selective self-test log
0x0c       GPL     R/O  30001  Pending Defects log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x12       GPL     R/O      1  SATA NCQ Non-Data log
0x13       GPL     R/O      1  SATA NCQ Send and Receive log
0x15       GPL     R/W      1  Rebuild Assist log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x24       GPL     R/O    256  Current Device Internal Status Data log
0x25       GPL     R/O    256  Saved Device Internal Status Data log
0x2f       GPL     -        1  Set Sector Configuration
0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       256 (0x0100)
Device State:                        Active (0)
Current Temperature:                    52 Celsius
Power Cycle Min/Max Temperature:     23/61 Celsius
Lifetime    Min/Max Temperature:     13/62 Celsius
Under/Over Temperature Limit Count:   0/0
SMART Status:                        0xc24f (PASSED)
Minimum supported ERC Time Limit:    70 (7.0 seconds)

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -40/70 Celsius
Temperature History Size (Index):    128 (22)

Index    Estimated Time   Temperature Celsius
  23    2025-09-16 03:44    51  ********************************
 ...    ..(108 skipped).    ..  ********************************
   4    2025-09-16 05:33    51  ********************************
   5    2025-09-16 05:34    52  *********************************
   6    2025-09-16 05:35    51  ********************************
   7    2025-09-16 05:36    51  ********************************
   8    2025-09-16 05:37    52  *********************************
 ...    ..(  3 skipped).    ..  *********************************
  12    2025-09-16 05:41    52  *********************************
  13    2025-09-16 05:42    51  ********************************
  14    2025-09-16 05:43    52  *********************************
 ...    ..(  6 skipped).    ..  *********************************
  21    2025-09-16 05:50    52  *********************************
  22    2025-09-16 05:51    51  ********************************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP Log 0x04)
Page  Offset Size        Value Flags Description
0x01  =====  =               =  ===  == General Statistics (rev 1) ==
0x01  0x008  4              22  ---  Lifetime Power-On Resets
0x01  0x010  4           27749  ---  Power-on Hours
0x01  0x018  6     64373614002  ---  Logical Sectors Written
0x01  0x020  6       795092999  ---  Number of Write Commands
0x01  0x028  6    140677461307  ---  Logical Sectors Read
0x01  0x030  6      1457577704  ---  Number of Read Commands
0x01  0x038  6     99897521700  ---  Date and Time TimeStamp
0x03  =====  =               =  ===  == Rotating Media Statistics (rev 1) ==
0x03  0x008  4           27748  ---  Spindle Motor Power-on Hours
0x03  0x010  4           27748  ---  Head Flying Hours
0x03  0x018  4            1184  ---  Head Load Events
0x03  0x020  4               0  ---  Number of Reallocated Logical Sectors
0x03  0x028  4               1  ---  Read Recovery Attempts
0x03  0x030  4               0  ---  Number of Mechanical Start Failures
0x04  =====  =               =  ===  == General Errors Statistics (rev 1) ==
0x04  0x008  4               0  ---  Number of Reported Uncorrectable Errors
0x04  0x010  4               0  ---  Resets Between Cmd Acceptance and Completion
0x05  =====  =               =  ===  == Temperature Statistics (rev 1) ==
0x05  0x008  1              52  ---  Current Temperature
0x05  0x010  1              50  N--  Average Short Term Temperature
0x05  0x018  1              50  N--  Average Long Term Temperature
0x05  0x020  1              62  ---  Highest Temperature
0x05  0x028  1              13  ---  Lowest Temperature
0x05  0x030  1              61  N--  Highest Average Short Term Temperature
0x05  0x038  1              25  N--  Lowest Average Short Term Temperature
0x05  0x040  1              55  N--  Highest Average Long Term Temperature
0x05  0x048  1              25  N--  Lowest Average Long Term Temperature
0x05  0x050  4            5265  ---  Time in Over-Temperature
0x05  0x058  1              60  ---  Specified Maximum Operating Temperature
0x05  0x060  4               0  ---  Time in Under-Temperature
0x05  0x068  1               0  ---  Specified Minimum Operating Temperature
0x06  =====  =               =  ===  == Transport Statistics (rev 1) ==
0x06  0x008  4              52  ---  Number of Hardware Resets
0x06  0x010  4               6  ---  Number of ASR Events
0x06  0x018  4               0  ---  Number of Interface CRC Errors
0xff  =====  =               =  ===  == Vendor Specific Statistics (rev 1) ==
0xff  0x040  7            1383  ---  Vendor Specific
0xff  0x048  7               0  ---  Vendor Specific
0xff  0x050  7               0  ---  Vendor Specific
0xff  0x058  7               0  ---  Vendor Specific
0xff  0x060  7               0  ---  Vendor Specific
0xff  0x068  7          216117  ---  Vendor Specific
0xff  0x070  7            3751  ---  Vendor Specific
0xff  0x078  7               0  ---  Vendor Specific
0xff  0x080  7              14  ---  Vendor Specific
                                |||_ C monitored condition met
                                ||__ D supports DSN
                                |___ N normalized value

Pending Defects log (GP Log 0x0c)
No Defects Logged

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data F```
9 


=== START OF INFORMATION SECTION ===
Device Model:     WDC WD161KFGX-68AFPN0
Serial Number:    2PH9ZJMT
LU WWN Device Id: 5 000cca 2c1d2a179
Firmware Version: 83.00A83
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 published, ANSI INCITS 529-2018
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Sep 16 05:30:57 2025 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Disabled
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1738) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   001    -    0
  2 Throughput_Performance  --S---   136   136   054    -    96
  3 Spin_Up_Time            POS---   082   082   001    -    361 (Average 359)
  4 Start_Stop_Count        -O--C-   100   100   000    -    22
  5 Reallocated_Sector_Ct   PO--CK   100   100   001    -    0
  7 Seek_Error_Rate         -O-R--   100   100   001    -    0
  8 Seek_Time_Performance   --S---   140   140   020    -    15
  9 Power_On_Hours          -O--C-   097   097   000    -    27748
 10 Spin_Retry_Count        -O--C-   100   100   001    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    22
 22 Unknown_Attribute       PO---K   100   100   025    -    100
192 Power-Off_Retract_Count -O--CK   100   100   000    -    1180
193 Load_Cycle_Count        -O--C-   100   100   000    -    1180
194 Temperature_Celsius     -O----   056   056   000    -    38 (Min/Max 13/50)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O---K   100   100   000    -    0
198 Offline_Uncorrectable   ---R--   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O-R--   100   100   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      1  Comprehensive SMART error log
0x03       GPL     R/O      1  Ext. Comprehensive SMART error log
0x04       GPL     R/O    256  Device Statistics log
0x04       SL      R/O    255  Device Statistics log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x08       GPL     R/O      2  Power Conditions log
0x09           SL  R/W      1  Selective self-test log
0x0c       GPL     R/O  30001  Pending Defects log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x12       GPL     R/O      1  SATA NCQ Non-Data log
0x13       GPL     R/O      1  SATA NCQ Send and Receive log
0x15       GPL     R/W      1  Rebuild Assist log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x24       GPL     R/O    256  Current Device Internal Status Data log
0x25       GPL     R/O    256  Saved Device Internal Status Data log
0x2f       GPL     -        1  Set Sector Configuration
0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       256 (0x0100)
Device State:                        Active (0)
Current Temperature:                    38 Celsius
Power Cycle Min/Max Temperature:     22/49 Celsius
Lifetime    Min/Max Temperature:     13/50 Celsius
Under/Over Temperature Limit Count:   0/0
SMART Status:                        0xc24f (PASSED)
Minimum supported ERC Time Limit:    70 (7.0 seconds)

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -40/70 Celsius
Temperature History Size (Index):    128 (99)

Index    Estimated Time   Temperature Celsius
 100    2025-09-16 03:23    37  ******************
 ...    ..( 70 skipped).    ..  ******************
  43    2025-09-16 04:34    37  ******************
  44    2025-09-16 04:35    38  *******************
  45    2025-09-16 04:36    38  *******************
  46    2025-09-16 04:37    37  ******************
  47    2025-09-16 04:38    38  *******************
 ...    ..( 50 skipped).    ..  *******************
  98    2025-09-16 05:29    38  *******************
  99    2025-09-16 05:30    37  ******************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP Log 0x04)
Page  Offset Size        Value Flags Description
0x01  =====  =               =  ===  == General Statistics (rev 1) ==
0x01  0x008  4              22  ---  Lifetime Power-On Resets
0x01  0x010  4           27748  ---  Power-on Hours
0x01  0x018  6     64369840370  ---  Logical Sectors Written
0x01  0x020  6       795713343  ---  Number of Write Commands
0x01  0x028  6    140662895634  ---  Logical Sectors Read
0x01  0x030  6      1457613739  ---  Number of Read Commands
0x01  0x038  6     99896052850  ---  Date and Time TimeStamp
0x03  =====  =               =  ===  == Rotating Media Statistics (rev 1) ==
0x03  0x008  4           27748  ---  Spindle Motor Power-on Hours
0x03  0x010  4           27748  ---  Head Flying Hours
0x03  0x018  4            1180  ---  Head Load Events
0x03  0x020  4               0  ---  Number of Reallocated Logical Sectors
0x03  0x028  4               0  ---  Read Recovery Attempts
0x03  0x030  4               0  ---  Number of Mechanical Start Failures
0x04  =====  =               =  ===  == General Errors Statistics (rev 1) ==
0x04  0x008  4               0  ---  Number of Reported Uncorrectable Errors
0x04  0x010  4               0  ---  Resets Between Cmd Acceptance and Completion
0x05  =====  =               =  ===  == Temperature Statistics (rev 1) ==
0x05  0x008  1              38  ---  Current Temperature
0x05  0x010  1              36  N--  Average Short Term Temperature
0x05  0x018  1              36  N--  Average Long Term Temperature
0x05  0x020  1              50  ---  Highest Temperature
0x05  0x028  1              13  ---  Lowest Temperature
0x05  0x030  1              48  N--  Highest Average Short Term Temperature
0x05  0x038  1              25  N--  Lowest Average Short Term Temperature
0x05  0x040  1              43  N--  Highest Average Long Term Temperature
0x05  0x048  1              25  N--  Lowest Average Long Term Temperature
0x05  0x050  4               0  ---  Time in Over-Temperature
0x05  0x058  1              60  ---  Specified Maximum Operating Temperature
0x05  0x060  4               0  ---  Time in Under-Temperature
0x05  0x068  1               0  ---  Specified Minimum Operating Temperature
0x06  =====  =               =  ===  == Transport Statistics (rev 1) ==
0x06  0x008  4              51  ---  Number of Hardware Resets
0x06  0x010  4               6  ---  Number of ASR Events
0x06  0x018  4               0  ---  Number of Interface CRC Errors
0xff  =====  =               =  ===  == Vendor Specific Statistics (rev 1) ==
0xff  0x040  7            1333  ---  Vendor Specific
0xff  0x048  7               0  ---  Vendor Specific
0xff  0x050  7               0  ---  Vendor Specific
0xff  0x058  7               0  ---  Vendor Specific
0xff  0x060  7               0  ---  Vendor Specific
0xff  0x068  7               0  ---  Vendor Specific
0xff  0x070  7               0  ---  Vendor Specific
0xff  0x078  7               0  ---  Vendor Specific
0xff  0x080  7              14  ---  Vendor Specific
                                |||_ C monitored condition met
                                ||__ D supports DSN
                                |___ N normalized value

Pending Defects log (GP Log 0x0c)
No Defects Logged

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-dat



=== START OF INFORMATION SECTION ===
Device Model:     WDC WD161KFGX-68AFPN0
Serial Number:    2BK9VU1R
LU WWN Device Id: 5 000cca 295eec27c
Firmware Version: 83.00A83
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 published, ANSI INCITS 529-2018
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Sep 16 05:31:43 2025 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM level is:     164 (intermediate level without standby)
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, NOT FROZEN [SEC1]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (2040) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   001    -    0
  2 Throughput_Performance  --S---   135   135   054    -    100
  3 Spin_Up_Time            POS---   085   085   001    -    229 (Average 371)
  4 Start_Stop_Count        -O--C-   100   100   000    -    26
  5 Reallocated_Sector_Ct   PO--CK   100   100   001    -    0
  7 Seek_Error_Rate         -O-R--   100   100   001    -    0
  8 Seek_Time_Performance   --S---   140   140   020    -    15
  9 Power_On_Hours          -O--C-   097   097   000    -    27748
 10 Spin_Retry_Count        -O--C-   100   100   001    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    26
 22 Unknown_Attribute       PO---K   100   100   025    -    100
192 Power-Off_Retract_Count -O--CK   100   100   000    -    1183
193 Load_Cycle_Count        -O--C-   100   100   000    -    1183
194 Temperature_Celsius     -O----   038   038   000    -    53 (Min/Max 13/65)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O---K   100   100   000    -    0
198 Offline_Uncorrectable   ---R--   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O-R--   100   100   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      1  Comprehensive SMART error log
0x03       GPL     R/O      1  Ext. Comprehensive SMART error log
0x04       GPL     R/O    256  Device Statistics log
0x04       SL      R/O    255  Device Statistics log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x08       GPL     R/O      2  Power Conditions log
0x09           SL  R/W      1  Selective self-test log
0x0c       GPL     R/O  30001  Pending Defects log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x12       GPL     R/O      1  SATA NCQ Non-Data log
0x13       GPL     R/O      1  SATA NCQ Send and Receive log
0x15       GPL     R/W      1  Rebuild Assist log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x24       GPL     R/O    256  Current Device Internal Status Data log
0x25       GPL     R/O    256  Saved Device Internal Status Data log
0x2f       GPL     -        1  Set Sector Configuration
0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       256 (0x0100)
Device State:                        Active (0)
Current Temperature:                    53 Celsius
Power Cycle Min/Max Temperature:     49/60 Celsius
Lifetime    Min/Max Temperature:     13/65 Celsius
Under/Over Temperature Limit Count:   0/0
SMART Status:                        0xc24f (PASSED)
Minimum supported ERC Time Limit:    70 (7.0 seconds)

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -40/70 Celsius
Temperature History Size (Index):    128 (121)

Index    Estimated Time   Temperature Celsius
 122    2025-09-16 03:24    52  *********************************
 ...    ..( 44 skipped).    ..  *********************************
  39    2025-09-16 04:09    52  *********************************
  40    2025-09-16 04:10    53  **********************************
  41    2025-09-16 04:11    53  **********************************
  42    2025-09-16 04:12    53  **********************************
  43    2025-09-16 04:13    52  *********************************
  44    2025-09-16 04:14    53  **********************************
  45    2025-09-16 04:15    53  **********************************
  46    2025-09-16 04:16    53  **********************************
  47    2025-09-16 04:17    52  *********************************
  48    2025-09-16 04:18    52  *********************************
  49    2025-09-16 04:19    53  **********************************
 ...    ..( 70 skipped).    ..  **********************************
 120    2025-09-16 05:30    53  **********************************
 121    2025-09-16 05:31    52  *********************************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP Log 0x04)
Page  Offset Size        Value Flags Description
0x01  =====  =               =  ===  == General Statistics (rev 1) ==
0x01  0x008  4              26  ---  Lifetime Power-On Resets
0x01  0x010  4           27748  ---  Power-on Hours
0x01  0x018  6     64370973850  ---  Logical Sectors Written
0x01  0x020  6       795871969  ---  Number of Write Commands
0x01  0x028  6    140666410595  ---  Logical Sectors Read
0x01  0x030  6      1458030551  ---  Number of Read Commands
0x01  0x038  6     99895864400  ---  Date and Time TimeStamp
0x03  =====  =               =  ===  == Rotating Media Statistics (rev 1) ==
0x03  0x008  4           27748  ---  Spindle Motor Power-on Hours
0x03  0x010  4           27748  ---  Head Flying Hours
0x03  0x018  4            1183  ---  Head Load Events
0x03  0x020  4               0  ---  Number of Reallocated Logical Sectors
0x03  0x028  4               0  ---  Read Recovery Attempts
0x03  0x030  4               0  ---  Number of Mechanical Start Failures
0x04  =====  =               =  ===  == General Errors Statistics (rev 1) ==
0x04  0x008  4               0  ---  Number of Reported Uncorrectable Errors
0x04  0x010  4               0  ---  Resets Between Cmd Acceptance and Completion
0x05  =====  =               =  ===  == Temperature Statistics (rev 1) ==
0x05  0x008  1              53  ---  Current Temperature
0x05  0x010  1              51  N--  Average Short Term Temperature
0x05  0x018  1              52  N--  Average Long Term Temperature
0x05  0x020  1              65  ---  Highest Temperature
0x05  0x028  1              13  ---  Lowest Temperature
0x05  0x030  1              63  N--  Highest Average Short Term Temperature
0x05  0x038  1              25  N--  Lowest Average Short Term Temperature
0x05  0x040  1              58  N--  Highest Average Long Term Temperature
0x05  0x048  1              25  N--  Lowest Average Long Term Temperature
0x05  0x050  4           34399  ---  Time in Over-Temperature
0x05  0x058  1              60  ---  Specified Maximum Operating Temperature
0x05  0x060  4               0  ---  Time in Under-Temperature
0x05  0x068  1               0  ---  Specified Minimum Operating Temperature
0x06  =====  =               =  ===  == Transport Statistics (rev 1) ==
0x06  0x008  4              51  ---  Number of Hardware Resets
0x06  0x010  4               5  ---  Number of ASR Events
0x06  0x018  4               0  ---  Number of Interface CRC Errors
0xff  =====  =               =  ===  == Vendor Specific Statistics (rev 1) ==
0xff  0x040  7            1339  ---  Vendor Specific
0xff  0x048  7               0  ---  Vendor Specific
0xff  0x050  7               0  ---  Vendor Specific
0xff  0x058  7               0  ---  Vendor Specific
0xff  0x060  7               0  ---  Vendor Specific
0xff  0x068  7          676525  ---  Vendor Specific
0xff  0x070  7           32229  ---  Vendor Specific
0xff  0x078  7               0  ---  Vendor Specific
0xff  0x080  7              14  ---  Vendor Specific
                                |||_ C monitored condition met
                                ||__ D supports DSN
                                |___ N normalized value

Pending Defects log (GP Log 0x0c)
No Defects Logged

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data



=== START OF INFORMATION SECTION ===
Device Model:     WDC WD161KFGX-68AFPN0
Serial Number:    2PH9B83T
LU WWN Device Id: 5 000cca 2c1d2591e
Firmware Version: 83.00A83
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 published, ANSI INCITS 529-2018
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Sep 16 05:32:45 2025 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Disabled
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  101) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (1929) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     PO-R--   100   100   001    -    0
  2 Throughput_Performance  --S---   135   135   054    -    100
  3 Spin_Up_Time            POS---   082   082   001    -    359 (Average 355)
  4 Start_Stop_Count        -O--C-   100   100   000    -    22
  5 Reallocated_Sector_Ct   PO--CK   100   100   001    -    0
  7 Seek_Error_Rate         -O-R--   100   100   001    -    0
  8 Seek_Time_Performance   --S---   140   140   020    -    15
  9 Power_On_Hours          -O--C-   097   097   000    -    27749
 10 Spin_Retry_Count        -O--C-   100   100   001    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    22
 22 Unknown_Attribute       PO---K   100   100   025    -    100
192 Power-Off_Retract_Count -O--CK   100   100   000    -    1179
193 Load_Cycle_Count        -O--C-   100   100   000    -    1179
194 Temperature_Celsius     -O----   052   052   000    -    41 (Min/Max 13/52)
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O---K   100   100   000    -    0
198 Offline_Uncorrectable   ---R--   100   100   000    -    0
199 UDMA_CRC_Error_Count    -O-R--   100   100   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      1  Comprehensive SMART error log
0x03       GPL     R/O      1  Ext. Comprehensive SMART error log
0x04       GPL     R/O    256  Device Statistics log
0x04       SL      R/O    255  Device Statistics log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x08       GPL     R/O      2  Power Conditions log
0x09           SL  R/W      1  Selective self-test log
0x0c       GPL     R/O  30001  Pending Defects log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x12       GPL     R/O      1  SATA NCQ Non-Data log
0x13       GPL     R/O      1  SATA NCQ Send and Receive log
0x15       GPL     R/W      1  Rebuild Assist log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x24       GPL     R/O    256  Current Device Internal Status Data log
0x25       GPL     R/O    256  Saved Device Internal Status Data log
0x2f       GPL     -        1  Set Sector Configuration
0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (1 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       256 (0x0100)
Device State:                        Active (0)
Current Temperature:                    41 Celsius
Power Cycle Min/Max Temperature:     22/51 Celsius
Lifetime    Min/Max Temperature:     13/52 Celsius
Under/Over Temperature Limit Count:   0/0
SMART Status:                        0xc24f (PASSED)
Minimum supported ERC Time Limit:    70 (7.0 seconds)

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -40/70 Celsius
Temperature History Size (Index):    128 (98)

Index    Estimated Time   Temperature Celsius
  99    2025-09-16 03:25    40  *********************
 ...    ..( 83 skipped).    ..  *********************
  55    2025-09-16 04:49    40  *********************
  56    2025-09-16 04:50    41  **********************
  57    2025-09-16 04:51    40  *********************
  58    2025-09-16 04:52    41  **********************
  59    2025-09-16 04:53    41  **********************
  60    2025-09-16 04:54    40  *********************
  61    2025-09-16 04:55    40  *********************
  62    2025-09-16 04:56    41  **********************
 ...    ..( 34 skipped).    ..  **********************
  97    2025-09-16 05:31    41  **********************
  98    2025-09-16 05:32    40  *********************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP Log 0x04)
Page  Offset Size        Value Flags Description
0x01  =====  =               =  ===  == General Statistics (rev 1) ==
0x01  0x008  4              22  ---  Lifetime Power-On Resets
0x01  0x010  4           27749  ---  Power-on Hours
0x01  0x018  6     64379259397  ---  Logical Sectors Written
0x01  0x020  6       790857193  ---  Number of Write Commands
0x01  0x028  6    140749292035  ---  Logical Sectors Read
0x01  0x030  6      1437359872  ---  Number of Read Commands
0x01  0x038  6     99896609350  ---  Date and Time TimeStamp
0x03  =====  =               =  ===  == Rotating Media Statistics (rev 1) ==
0x03  0x008  4           27748  ---  Spindle Motor Power-on Hours
0x03  0x010  4           27748  ---  Head Flying Hours
0x03  0x018  4            1179  ---  Head Load Events
0x03  0x020  4               0  ---  Number of Reallocated Logical Sectors
0x03  0x028  4               0  ---  Read Recovery Attempts
0x03  0x030  4               0  ---  Number of Mechanical Start Failures
0x04  =====  =               =  ===  == General Errors Statistics (rev 1) ==
0x04  0x008  4               0  ---  Number of Reported Uncorrectable Errors
0x04  0x010  4               0  ---  Resets Between Cmd Acceptance and Completion
0x05  =====  =               =  ===  == Temperature Statistics (rev 1) ==
0x05  0x008  1              41  ---  Current Temperature
0x05  0x010  1              39  N--  Average Short Term Temperature
0x05  0x018  1              39  N--  Average Long Term Temperature
0x05  0x020  1              52  ---  Highest Temperature
0x05  0x028  1              13  ---  Lowest Temperature
0x05  0x030  1              50  N--  Highest Average Short Term Temperature
0x05  0x038  1              25  N--  Lowest Average Short Term Temperature
0x05  0x040  1              43  N--  Highest Average Long Term Temperature
0x05  0x048  1              25  N--  Lowest Average Long Term Temperature
0x05  0x050  4               0  ---  Time in Over-Temperature
0x05  0x058  1              60  ---  Specified Maximum Operating Temperature
0x05  0x060  4               0  ---  Time in Under-Temperature
0x05  0x068  1               0  ---  Specified Minimum Operating Temperature
0x06  =====  =               =  ===  == Transport Statistics (rev 1) ==
0x06  0x008  4              51  ---  Number of Hardware Resets
0x06  0x010  4               3  ---  Number of ASR Events
0x06  0x018  4               0  ---  Number of Interface CRC Errors
0xff  =====  =               =  ===  == Vendor Specific Statistics (rev 1) ==
0xff  0x040  7            1369  ---  Vendor Specific
0xff  0x048  7               0  ---  Vendor Specific
0xff  0x050  7               0  ---  Vendor Specific
0xff  0x058  7               0  ---  Vendor Specific
0xff  0x060  7               0  ---  Vendor Specific
0xff  0x068  7               0  ---  Vendor Specific
0xff  0x070  7             127  ---  Vendor Specific
0xff  0x078  7               0  ---  Vendor Specific
0xff  0x080  7             602  ---  Vendor Specific
                                |||_ C monitored condition met
                                ||__ D supports DSN
                                |___ N normalized value

Pending Defects log (GP Log 0x0c)
No Defects Logged

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0
10

Took a look at the specs myself now. The board has 6+2 SATA ports - the two ports are on an ASM1061 which should be fine for two drives I think.

Just from a quick glance the SMART data seems to indicate no drive failures (would have been a real surprise if all failed simultaneously) but the temperatures seem high on at least some of them - some have max temps of 44°C and averages of ~33°C but others average over 50°C and have gotten up to 65°C which is above the drive’s recommended temps. That probably won’t benefit those drive’s longevity.

1 Like
11

At a glance all the drives have crc errors logged & an nearly identical amount of zfs errors. Smart tests imply that drives themselves aren’t the fault.

Something caused a blip on all them, psu, ram, maybe the chipset on the motherboard, etc.

The good news is you can in theory use this as an excuse for a new build?

12

As said, some of the drives are really hot, so there’s a cooling issue. That should not be the cause of this issue, but this is something to improve.
You haven’t been running periodic SMART tests, but, as expected the drives look fine.
All drives being equally affected would tend to exclude a cabling issue. Two controllers being involved (Z170 and ASM106x) would tend to excuse both.
That leaves RAM or PSU…

Passmark memtest86 runs directly from a USB thumbdrive. Of course, you can’t use the NAS meanwhile.

1 Like
13

This evening, I got the chance to get memtest running. Within 10 minutes, I already seem to have a selection of errors. I have left it running and will check again in the morning; however, I am assuming we have found the root of the issue. Bad RAM?

Just to clarify, while I had the server off, I checked the exact power situation. I am in fact using the 6 SATA connections on the power supply, and the remaining drives are powered via Molex to SATA.

Thanks again for all the help.

1 Like
14

…no need to leave it running - it has already failed. RMA if available/time relevant, or buy new ram. Optional is testing each dim to see if any of them work. More optional; try and play with voltages and timings (god I hate ram OC) to try to find something stable - I wouldn’t go this route on a machine meant for 24/7.

15

Hi Everyone,

So as expected, there were many errors resulting from the memtest.

I am simply going to replace the RAM. Are there any suggestions on good RAM, or something to look out for? I am using a consumer board, so I just have to stick with DDR4, but anything else to look out for?

Finally, to restore the status of the server, do I simply need to delete or replace the affected files and scrub the pool again?

Thanks

16

I would like to point out something to you that IMHO should not be overlooked at all.
You have a kaby lake processor, with a z170 motherboard… they are not officially capable to run with all of the RAM you put on It

Having explored for myself some time ago how to “squeeze” this chipset (got one too), i obviously came across discussions where other people trying that thing and for the most was a total failure (system neither boot), other come into PCI-EX lanes problem, someone could achieve that with a BIOS mod (injecting a particoular firmware probably coming from coffe lake motherboard).
Just for say that you are overloading a lot your CPU memory controller, and errors can come from that not only from a defected RAM stick

1 Like
17

Good points. Skylake (Kaby Lake would be 7xxx) is officially limited to 64 GB, but this

suggests that the CPU can actually handle denser RAM modules than what was available when it launched.

Whatever you chose, run memtest86 and burn-in you new modules before putting them into “production”.

1 Like
18

@winnielinnie strikes again!

1 Like
19

FTFY:

2 Likes
20

Hi Everyone,

Back again, I bought some new RAM, as suggested, and ran memtest on it. To my surprise, the test has failed. I assume the next step is to RMA my RAM and start again, but is there anything else I should look into before this?

Thanks
Ronan