How crucial are a couple read write errors?

Near on 50 hrs spent now returning to TrueNAS rebuilding a TrueNAS server with the latest TNScale CE- (having left the space around a decade ago), gotta ask the obvious question just how crucial are a couple of read write errors!?

RAIDZ2 8 discs upgrading the pool slowly and laboriously from 2TB (Old Hitachi) to 4 TB (New WD RED plus) discs - put in a new disc into the pool and an hr or so later KABOOM… it’s saying the pool is degraded due to that new added disc! (A brand spanking new disc!)…would ya’ll tolerate just a few read write erros? I mean the scrutiny app is just saying a 3% chance of failure…

I know from years back just how picky FreeNAS was and I get why ya’ll are using it (how accurate and powerful the disc reporting error is with this awesome software) but in the real world rough and tumble of getting on with your lives (like doing something else with your precious time!!!) …what is a general accepted level of failure tolerance running this software using a RAIDZ2 pool? Are there any settings in Scrutiny (or 25.04.1 itself) that tones down the drive failure aka “gotta replace that drive right now (or your pool is toast) !” panic level? thanks!

Brand new? I’d RMA/return to store asap & not even think about it. Otherwise? I’d investigate what could be causing the errors & try to mitigate. Maybe it isn’t the disk itself at fault.

IMO Raidz2 is there so you have extra time to resolve vs the ‘drop everything & restore redundancy’ of raidz1. It ain’t an ‘oh I’ll just wait for a 2nd one to start having issues, since I have 2 redundant disks’.

Waiting for second disk to get errors/die & then resilvering a replacement drive is 100%, without any doubt, when you’ll have a third disk fail during the resilver (due to Murphy’s law).

That being said, not mission critical data? You got backups? It isn’t a production system? Send it.

I’d also argue there isn’t any generally agreed upon risk tolerance for personal NAS use. Too much of a personal thing, one that has to include the individual’s budget (money or time) vs attachment to data in question.

1 Like

Well unfortunately you have not provided enough data to give you a proper answer. This may have nothing to do with the new drive or maybe it is the new drive.

Here is what you need to do:

  1. What exactly does Kaboom mean?
  2. Post the output of zpool status -v
  3. Post the output of smartctl -x /dev/sdX where X id the Device ID for the drive.
  4. We can tell you what to do next.

OR better yet, take a look at the Drive Troubleshooting Flowcharts I created that is in the Resources of our forums, or the link in my signature. It will take you step by step to diagnose if you have a drive problem or possibly a ZFS/computer problem. Way too many people jump to the wrong conclusion that if an error has occurred, the drive must be at fault. In your case it could be the data cable, or an unstable system. Yes, a perfectly good system last year could be unstable now.

Good Luck and feel free to post those outputs requested. While you are at it, maybe some of your system specs as well. These little things might not actually be little and contribute to problems.

EDIT: I forgot to say, are you sure the drive is not SMR? Posting the SMART output will confirm that.

3 Likes

I have been told they are CMR. ^Sure will …I will post all these logs as soon as I can.

I have no doubt about that and you listed them as WD Red Plus, so those “should” be CMR, but what is actually in there? It may be all good but I’d rather explore all the possibilities that I can and sooner than later so you can have a proper answer to your concerns and then move forward. But do not replace the drive without knowing if it is bad, right now it is speculation.

Modern drives (everything build within the last 30 years) have internal defect management. Under normal circumstances they appear always as error-free. Any error is a sign of something wrong. This isn’t necessarily the disk itself, it can also be cabling, connectors, hotplug bays, controllers, power supplies, mainboards. I’ve seen strange pool errors that were caused by an insuffcient power supply.

I hope you have a good backup.

2 Likes

The disk concerned is throwing up multiple errors “power on or device reset occurred” like every couple of seconds it scrolls down the GUI

this is the Long SMART log…

smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
Sending command: “Execute SMART Extended self-test routine immediately in off-line mode”.
Command “Execute SMART Extended self-test routine immediately in off-line mode” failed: Input/output error

I have good backup.
The pool is also replicated on another stripped 8TB larger disc in one of the SATA only bays
The faulty pool is “degraded but functional”… it’s 8X ZH2 and just the one drive/bay etc is down…
I am actually re adding the last problem drive to the pool now in a SATA only bay
Another drive has the read/write errors now…
I suspect this is a drive bay issue/power supply/wire/ HBA issue???
This is 12-14 yr old gear I am rebuilding here… (See my other thread)
Functional and still powerful, hardly ever used, hasn’t been knocked around much but it’s 12 year old enterprise gear (now with new SSD’s for the TN OS and new 4TB WD drives)
Once I have the entire pool up and expanded I will share some more logs
Thanks everyone for being so helpful !

smartctl -x /dev/sdX

smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

Read Device Identity failed: Input/output error

:point_up_2:

The capitol “X”, not the lowercase “x”. So if the drive is sda, then the command would be smartctl -x /dev/sda

Maybe in the future I should use a different letter or symbol.

1 Like

For the same result.
Reportedly, the message “Press any key” has some users looking for the “ANY” key… :roll_eyes:

2 Likes

Seen that one many times, amazing.

Storytime: My ex-brother-in-law use to work at Microsoft Tech Support for MS-DOS and Windows 3.11. He took a call from a woman (it could happen with a man as well) and she told him her plight. He told her to double click on the button on the screen. She said it didn’t work. They talked for a few moments and he told her to try it again, double click on the box on the screen. [a faint tap, tap heard in the background], again nothing worked. He asked her what that noise was, she said it was the sound of the mouse hitting the screen (glass those days) as she double clicked the box. He shook he head and told her nicely that she should box the computer up and return it from where she bought it. Okay, he wanted to say that but he did the right thing and told her exactly what to do, but that was the kind of calls he would get.

Sounds like the “Any Key” problem as well.

2 Likes

smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.15-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red (CMR)
Device Model:     WDC WD40EFPX-68C6CN0
Serial Number:    WD-WXA2D15MT04L
LU WWN Device Id: 5 0014ee 2c16481bb
Firmware Version: 81.00A81
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database 7.3/5706
ATA Version is:   ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Mon Jun  2 06:33:36 2025 AEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Unavailable
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, NOT FROZEN [SEC1]
Read SCT Status failed: Input/output error
Wt Cache Reorder: Unknown (SCT Feature Control command failed)

Read SMART Thresholds failed: Input/output error

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (42960) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 448) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3039) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR-K   100   253   ---    -    0
  3 Spin_Up_Time            POS--K   205   205   ---    -    2716
  4 Start_Stop_Count        -O--CK   100   100   ---    -    7
  5 Reallocated_Sector_Ct   PO--CK   200   200   ---    -    0
  7 Seek_Error_Rate         -OSR-K   200   200   ---    -    0
  9 Power_On_Hours          -O--CK   100   100   ---    -    21
 10 Spin_Retry_Count        -O--CK   100   253   ---    -    0
 11 Calibration_Retry_Count -O--CK   100   253   ---    -    0
 12 Power_Cycle_Count       -O--CK   100   100   ---    -    7
192 Power-Off_Retract_Count -O--CK   200   200   ---    -    4
193 Load_Cycle_Count        -O--CK   200   200   ---    -    2
194 Temperature_Celsius     -O---K   126   113   ---    -    21
196 Reallocated_Event_Count -O--CK   200   200   ---    -    0
197 Current_Pending_Sector  -O--CK   200   200   ---    -    0
198 Offline_Uncorrectable   ----CK   100   253   ---    -    0
199 UDMA_CRC_Error_Count    -O--CK   200   199   ---    -    26
200 Multi_Zone_Error_Rate   ---R--   100   253   ---    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      5  Comprehensive SMART error log
0x03       GPL     R/O      6  Ext. Comprehensive SMART error log
0x04       GPL     R/O    256  Device Statistics log
0x04       SL      R/O    255  Device Statistics log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x09           SL  R/W      1  Selective self-test log
0x0c       GPL     R/O   2048  Pending Defects log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x24       GPL     R/O    307  Current Device Internal Status Data log
0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xa0-0xa7  GPL,SL  VS      16  Device vendor specific log
0xa8-0xb6  GPL,SL  VS       1  Device vendor specific log
0xb7       GPL,SL  VS      78  Device vendor specific log
0xbd       GPL,SL  VS       1  Device vendor specific log
0xc0       GPL,SL  VS       1  Device vendor specific log
0xc1       GPL     VS      93  Device vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

ATA_READ_LOG_EXT (addr=0x03:0x00, page=0, n=1) failed: Input/output error
Read SMART Extended Comprehensive Error Log failed

SMART Error Log Version: 1
No Errors Logged

ATA_READ_LOG_EXT (addr=0x07:0x00, page=0, n=1) failed: Input/output error
Read SMART Extended Self-test Log failed

Read SMART Self-test Log failed: Input/output error

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       258 (0x0102)
Device State:                        Active (0)
Current Temperature:                    22 Celsius
Power Cycle Min/Max Temperature:     22/22 Celsius
Lifetime    Min/Max Temperature:     16/23 Celsius
Under/Over Temperature Limit Count:   0/0
Vendor specific:
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Write SCT Data Table failed: Input/output error
Read SCT Temperature History failed

Write SCT (Get) Error Recovery Control Command failed: Input/output error
SCT (Get) Error Recovery Control command failed

ATA_READ_LOG_EXT (addr=0x04:0x00, page=0, n=1) failed: Input/output error
Read Device Statistics page 0x00 failed

ATA_READ_LOG_EXT (addr=0x0c:0x00, page=0, n=1) failed: Input/output error
Read Pending Defects log page 0x00 failed

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2           15  Command failed due to ICRC error
0x0002  2           34  R_ERR response for data FIS
0x0003  2           22  R_ERR response for device-to-host data FIS
0x0004  2           12  R_ERR response for host-to-device data FIS
0x0005  2        14966  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2        14966  R_ERR response for host-to-device non-data FIS
0x0008  2            0  Device-to-host non-data FIS retries
0x0009  2        29662  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2        29736  Device-to-host register FISes sent due to a COMRESET
0x000b  2           11  CRC errors within host-to-device FIS
0x000d  2        14967  Non-CRC errors within host-to-device FIS
0x000f  2           12  R_ERR response for host-to-device data FIS, CRC
0x0012  2        65535+ R_ERR response for host-to-device non-data FIS, CRC
0x8000  4        28939  Vendor specific

Please pay your respects. We lost him when he was still so young. :cry: I really do miss him.

1 Like

pool:
state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Replace the faulted device, or use ‘zpool clear’ to mark the device
repaired.
scan: resilvered 750G in 02:09:59 with 0 errors on Mon Jun 2 00:49:27 2025
config:

    NAME                                      STATE     READ WRITE CKSUM
                                                   DEGRADED     0     0     0
      raidz2-0                                DEGRADED     0     0     0
        9f481dbc-eb46-42a3-aaea-e8a7b8d729b1  ONLINE       0     0     0
        923d0676-6909-427c-ad96-63e89c727b3e  ONLINE       0     0     0
        ec9d535d-e362-4d34-9e33-a74b99b718b2  ONLINE       0     0     0
        8d488d0c-d286-4173-b570-6832e09bc81c  ONLINE       0     0     0
        96416399-1c3a-4609-b995-7f9a9162bd6a  ONLINE       0     0     0
        3807936544339984491                   FAULTED      2     4     0  was /dev/disk/by-partuuid/2ff34d7f-c357-4aa1-a427-a5c5fc9efbcc
        584aedb4-43ff-433f-adfd-262819b2a3fe  ONLINE       0     0     0
        35bcfc2b-b259-4ff3-b971-149bf0134337  ONLINE       0     0     0

errors: No known data errors

pool: boot-pool
state: ONLINE
scan: scrub repaired 0B in 00:00:24 with 0 errors on Sun Jun 1 03:45:26 2025
config:

    NAME        STATE     READ WRITE CKSUM
    boot-pool   ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        sdh3    ONLINE       0     0     0
        sdi3    ONLINE       0     0     0

errors: No known data errors

Model WD40EFPX is a CMR drive. That rules out any SMR problems.

Not sure why the SMART output has so many input/output errors. How are these drives attached?

See my other post - I am refurbishing/repairing an old supermicro server

It’s a 12 yr old 12 bay

The Boot drives are SSD’s mirrored in the directly attached SATA bays
The Z2Pool drives are in bays connected via the HBA

Thanks

really appreciate the help… it’s 7am here, cold, dark and I’ve gotta get to work earn some real dollars hit the road before the sun gets up
The traffic starts to get real bad real soon…
Will be back when I can in around 8-10 hrs :grinning:

Please help those who try to help you:
COMPLETE hardware description, including motherboard, CPU, RAM, HBA and its firmware version, etc.
And, please, properly format terminal output with the </> button or by pasting the text between two lines of three backquotes
```
output
```