Drive Failure

Good Evening

im looking for some guidance on recovering my truenas Pool today i logged in and got the dreaded error “[EZFS_POOLUNAVAIL]: zfs_open() failed - cannot open ‘Raw’: pool I/O is currently suspended”

doing a bit more digging i found

zpool status Raw

pool: Raw
state: SUSPENDED
status: One or more devices are faulted in response to IO failures.
action: Make sure the affected devices are connected, then run 'zpool clear'.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-JQ
scan: resilvered 0B in 00:00:05 with 0 errors on Wed May 20 19:16:11 2026
config:

NAME STATE READ WRITE CKSUM
Raw DEGRADED 0 0 0
646b002a-31f9-4c5f-806f-660ad6142315 ONLINE 0 0 0
572c878e-e195-48a4-bb51-31af41f3c1b8 ONLINE 0 0 0
ac365c94-bf2e-4929-a6b7-7fc12998ce0d DEGRADED 3 4 8.21K too many errors

errors: 100 data errors, use '-v' for a list

these are my smartctl Results which show the drive is failing

smartctl -a /dev/sdh
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.33-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Seagate Barracuda 2.5 5400
Device Model: ST2000LM015-2E8174
Serial Number: WY22NCTH
LU WWN Device Id: 5 000c50 0f218a714
Firmware Version: 0001
User Capacity: 2,000,398,934,016 bytes [2.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Rotation Rate: 5400 rpm
Form Factor: 2.5 inches
TRIM Command: Available
Device is: In smartctl database 7.3/5528
ATA Version is: ACS-3 T13/2161-D revision 3b
SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Tue May 26 19:27:16 2026 SAST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART Status not supported: Incomplete response, ATA output registers missing
SMART overall-health self-assessment test result: FAILED!
Drive failure expected in less than 24 hours. SAVE ALL DATA.
Warning: This result is based on an Attribute check.
See vendor-specific Attribute list for failed Attributes.

General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 0) seconds.
Offline data collection
capabilities: (0x71) SMART execute Offline immediate.
No Auto Offline data collection support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 331) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x3035) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 039 039 006 Pre-fail Always - 115927409
3 Spin_Up_Time 0x0003 100 100 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 406
5 Reallocated_Sector_Ct 0x0033 001 001 036 Pre-fail Always FAILING_NOW 57288
7 Seek_Error_Rate 0x000f 084 060 045 Pre-fail Always - 240481344
9 Power_On_Hours 0x0032 089 089 000 Old_age Always - 9671 (151 207 0)
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 406
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 099 099 000 Old_age Always - 1
188 Command_Timeout 0x0032 095 071 000 Old_age Always - 463863808118
189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0022 072 057 040 Old_age Always - 28 (Min/Max 21/32)
191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 4
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 368
193 Load_Cycle_Count 0x0032 099 099 000 Old_age Always - 3835
194 Temperature_Celsius 0x0022 028 043 000 Old_age Always - 28 (0 16 0 0 0)
197 Current_Pending_Sector 0x0012 099 099 000 Old_age Always - 120
198 Offline_Uncorrectable 0x0010 099 099 000 Old_age Offline - 120
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 9664 (142 14 0)
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 27020998696
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 10698261107
254 Free_Fall_Sensor 0x0032 100 100 000 Old_age Always - 0

SMART Error Log Version: 1
ATA Error Count: 1
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 1 occurred at disk power-on lifetime: 9665 hours (402 days + 17 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: WP at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
61 00 08 ff ff ff 4f 00 1d+04:38:28.570 WRITE FPDMA QUEUED
61 00 38 ff ff ff 4f 00 1d+04:38:28.569 WRITE FPDMA QUEUED
61 00 10 ff ff ff 4f 00 1d+04:38:28.569 WRITE FPDMA QUEUED
61 00 28 ff ff ff 4f 00 1d+04:38:28.569 WRITE FPDMA QUEUED
61 00 10 ff ff ff 4f 00 1d+04:38:28.569 WRITE FPDMA QUEUED

SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 9629 -
# 2 Short offline Completed without error 00% 9607 -
# 3 Short offline Completed without error 00% 9585 -
# 4 Short offline Completed without error 00% 9562 -
# 5 Short offline Completed without error 00% 9538 -
# 6 Short offline Completed without error 00% 9523 -
# 7 Short offline Completed without error 00% 9500 -
# 8 Short offline Completed without error 00% 9476 -
# 9 Short offline Completed without error 00% 9452 -
#10 Short offline Completed without error 00% 9428 -
#11 Short offline Completed without error 00% 9404 -
#12 Short offline Completed without error 00% 9380 -
#13 Short offline Completed without error 00% 9356 -
#14 Short offline Completed without error 00% 9332 -
#15 Short offline Completed without error 00% 9308 -
#16 Short offline Completed without error 00% 9284 -
#17 Short offline Completed without error 00% 9260 -
#18 Short offline Completed without error 00% 9237 -
#19 Short offline Completed without error 00% 9213 -
#20 Short offline Completed without error 00% 9193 -
#21 Short offline Completed without error 00% 9169 -

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

My issue is because i used stripe when setting up the pool there is no redundancy all the drives were new approximately one year ago

im looking for guidance in recovering the data if possible and getting it up and running. Is there any way to copy the data to another drive?

You could try software like Klennet Recovery to see if you can, possibly, recover some of the data. Read about it on their website. The scan and preliminary findings are free but you have to purchase to recover.

smartctl -a /dev/sdh -v 1,raw48:54 -v 5,raw48:54 -v 7,raw48:54 -v 191,raw48:54 -v 197,raw48:54 -v 198,raw48:54

Mind trying the following since it is a seagate drive to translate some of these values? Is drive connected directly to the motherboard? Anything happens when you try to zpool import ? If you manually rub a scrub or clear the errors are you able to try to import & get some of your data recovered?

here’s the results. I did see the option of zpool import and scrub but i was nervous to try as i read it could damage the drive more.

smartctl -a /dev/sdf -v  1,raw48:54  -v 5,raw48:54  -v 7,raw48:54 -v 191,raw48:54  -v 197,raw48:54  -v 198,raw48:54


smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.33-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 2.5 5400
Device Model:     ST2000LM015-2E8174
Serial Number:    WY22NCTH
LU WWN Device Id: 5 000c50 0f218a714
Firmware Version: 0001
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Form Factor:      2.5 inches
TRIM Command:     Available
Device is:        In smartctl database 7.3/5528
ATA Version is:   ACS-3 T13/2161-D revision 3b
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue May 26 20:49:27 2026 SAST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART Status not supported: Incomplete response, ATA output registers missing
SMART overall-health self-assessment test result: FAILED!
Drive failure expected in less than 24 hours. SAVE ALL DATA.
Warning: This result is based on an Attribute check.
See vendor-specific Attribute list for failed Attributes.

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (    0) seconds.
Offline data collection
capabilities:                    (0x71) SMART execute Offline immediate.
                                        No Auto Offline data collection support.
                                        Suspend Offline collection upon new
                                        command.
                                        No Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        ( 331) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   039   039   006    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0003   100   100   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       407
  5 Reallocated_Sector_Ct   0x0033   001   001   036    Pre-fail  Always   FAILING_NOW 0
  7 Seek_Error_Rate         0x000f   084   060   045    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   089   089   000    Old_age   Always       -       9672 (108 3 0)
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       407
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   099   099   000    Old_age   Always       -       1
188 Command_Timeout         0x0032   099   071   000    Old_age   Always       -       463863808118
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   073   057   040    Old_age   Always       -       27 (Min/Max 27/29)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       369
193 Load_Cycle_Count        0x0032   099   099   000    Old_age   Always       -       3836
194 Temperature_Celsius     0x0022   027   043   000    Old_age   Always       -       27 (0 16 0 0 0)
197 Current_Pending_Sector  0x0012   099   099   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   099   099   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       9665 (37 197 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       27020998696
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       10698270698
254 Free_Fall_Sensor        0x0032   100   100   000    Old_age   Always       -       0

SMART Error Log Version: 1
ATA Error Count: 1
        CR = Command Register [HEX]
        FR = Features Register [HEX]
        SC = Sector Count Register [HEX]
        SN = Sector Number Register [HEX]
        CL = Cylinder Low Register [HEX]
        CH = Cylinder High Register [HEX]
        DH = Device/Head Register [HEX]
        DC = Device Command Register [HEX]
        ER = Error register [HEX]
        ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 1 occurred at disk power-on lifetime: 9665 hours (402 days + 17 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: WP at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  61 00 08 ff ff ff 4f 00   1d+04:38:28.570  WRITE FPDMA QUEUED
  61 00 38 ff ff ff 4f 00   1d+04:38:28.569  WRITE FPDMA QUEUED
  61 00 10 ff ff ff 4f 00   1d+04:38:28.569  WRITE FPDMA QUEUED
  61 00 28 ff ff ff 4f 00   1d+04:38:28.569  WRITE FPDMA QUEUED
  61 00 10 ff ff ff 4f 00   1d+04:38:28.569  WRITE FPDMA QUEUED

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%      9629         -
# 2  Short offline       Completed without error       00%      9607         -
# 3  Short offline       Completed without error       00%      9585         -
# 4  Short offline       Completed without error       00%      9562         -
# 5  Short offline       Completed without error       00%      9538         -
# 6  Short offline       Completed without error       00%      9523         -
# 7  Short offline       Completed without error       00%      9500         -
# 8  Short offline       Completed without error       00%      9476         -
# 9  Short offline       Completed without error       00%      9452         -
#10  Short offline       Completed without error       00%      9428         -
#11  Short offline       Completed without error       00%      9404         -
#12  Short offline       Completed without error       00%      9380         -
#13  Short offline       Completed without error       00%      9356         -
#14  Short offline       Completed without error       00%      9332         -
#15  Short offline       Completed without error       00%      9308         -
#16  Short offline       Completed without error       00%      9284         -
#17  Short offline       Completed without error       00%      9260         -
#18  Short offline       Completed without error       00%      9237         -
#19  Short offline       Completed without error       00%      9213         -
#20  Short offline       Completed without error       00%      9193         -
#21  Short offline       Completed without error       00%      9169         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

thanks for the idea. im going to try see if i can find some cheaper alternatives first and its a bit of a pain to try install on a windows machine and get all the drives mounted on that machine but ill def give it a try if all else fails.

Is that a striped pool? I don’t see a RAID level indicated. If it is a stripe, then running software recovery against it is a bad idea. You need to make a clone or an image file of the faulty disk first.

it is a stripe pool

Well I checked vs my seagate drive which have values of 100 (value) - 100 (worst) - thresh, so I’m guess you being at 001 - 001 is indeed bad.

I’ll leave the recovery parts to the more knowledgeable

You are also using SMR disks, which are not recommended for use with ZFS.

This might mean that your pool is in better shape that it appears. SMR disks are known to cause write delays because of the need to shuffle the overlapping tracks around. While you might have 1 bad sector, which should at most affect just 1 file. And if that bad sector is in Metadata, by default their are 2 copies of such.

The other problems might be overcome by rebooting and importing the pool. Though, it may be needed to use manual methods to import the pool in error recovery mode.

not sure if this is of any use but if i run zpool clear Raw i get most of hese seem to be docker files and VMS which i think i can manage without. Rebooting i had no change

zpool status Raw -v
pool: Raw
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
scan: resilvered 0B in 00:00:05 with 0 errors on Wed May 20 19:16:11 2026
config:

NAME STATE READ WRITE CKSUM
Raw ONLINE 0 0 0
646b002a-31f9-4c5f-806f-660ad6142315 ONLINE 0 0 0
572c878e-e195-48a4-bb51-31af41f3c1b8 ONLINE 0 0 0
ac365c94-bf2e-4929-a6b7-7fc12998ce0d ONLINE 0 0 6

errors: Permanent errors have been detected in the following files:

Raw/ix-apps/docker:/overlay2/72af987364b884fbdd4e7038b8b30472e3952ec6052bec58779eb41c06a390c2/diff/app/node_modules/semver/functions/eq.js
Raw/ix-apps/docker:/overlay2/7ceeada4a5baa99712cdbd4f2a80663f44822fcf85a7b370b8df024fbfd3cbdf/diff/opt/venv/lib/python3.11/site-packages/pydantic/_migration.py
Raw/ix-apps/docker:/overlay2/3c7b33a0cc38df29796124cae5ea81caa725571c466d3106b955a08f6b4a5ca6/diff/jellyfin/Microsoft.AspNetCore.Components.dll
Raw/ix-apps/docker:/overlay2/cd318c78502df459d0dfe410cd240f33e598760e782fdf299893434797dfd7a5/diff/usr/bin/python3.11
Raw/ix-apps/docker:/overlay2/24cd5ea1fcf17490022c2e7b9fe7188a9398a1f80a5b3aff87be7752ab32e6c3/diff/usr/local/bin/node
Raw/ix-apps/docker:/overlay2/06f1bc334847646ce15e755bad4441799743f2e8d1b43096f1dcbc21f537b5ad/diff/usr/bin/perl5.36.0
Raw/ix-apps/docker:/overlay2/ea395e4d1b451b865143239d9d9e0a34399cfeea05927386b25b68b325cb4a82/diff/etc/ld.so.cache
Raw/ix-apps/docker:/overlay2/a60769dc1274080453efef9e5355fd24b655311d05140ccd8ff4dcd01a20173a/diff/usr/src/node-red/node_modules/semver/classes/comparator.js
Raw/ix-apps/docker:/overlay2/f5453b2b3054c1b7f1febb19262b463e654fe4a59c021d686f85db0198f63302/diff/usr/lib/x86_64-linux-gnu/libcrypto.so.3
Raw/ix-apps/docker:/overlay2/3c7b33a0cc38df29796124cae5ea81caa725571c466d3106b955a08f6b4a5ca6/diff/jellyfin/Jellyfin.Api.dll
Raw/ix-apps/docker:/overlay2/8bb0c7188b8a9bf87a8aa7f56ecb14895f5b4c716156ff377594f405167e7f7f/diff/usr/lib/x86_64-linux-gnu/libcrypto.so.3
Raw/ix-apps/docker:/overlay2/e248f29ad213b9b64a0ffd936aa3a4d9dc0d3b810dce5216537cdb749b6186cb/diff/usr/local/lib/libjxl.so.0.11.1
Raw/ix-apps/docker:/overlay2/96e37707151205b6454b1dacb2c6e32732e589ce0cebadbc9d7f7a4f3d02c382/diff/usr/lib/postgresql/15/lib/vchord.so
Raw/ix-apps/docker:/overlay2/071c025b39cb36d37ad146d6ae33b41e4176fe929d6206f7e856f6b9c32239ed/diff/usr/local/bin/cloudflared
Raw/ix-apps/docker:/overlay2/150b21600e1058b752b9e74f062b9eed9be5f77240316bd4436f4f8d5d858334/diff/lib/ld-musl-x86_64.so.1
Raw/ix-apps/docker:/overlay2/8b56f9a26346d45c1ef9d8b5669bf409cc250a889d3d32977a6cfe7687534ec7/diff/usr/local/bin/valkey-cli
Raw/ix-apps/docker:/overlay2/72af987364b884fbdd4e7038b8b30472e3952ec6052bec58779eb41c06a390c2/diff/app/node_modules/date-fns/nextDay.js
Raw/ix-apps/docker:/overlay2/a60769dc1274080453efef9e5355fd24b655311d05140ccd8ff4dcd01a20173a/diff/usr/src/node-red/node_modules/semver/ranges/subset.js
Raw/ix-apps/docker:/image/overlay2/imagedb/content/sha256/dae0c92b7b63e2efb8ff54d1b95141eb34b8eda08d2c7c8003755277b60022ed
Raw/ix-apps/docker:/overlay2/06f1bc334847646ce15e755bad4441799743f2e8d1b43096f1dcbc21f537b5ad/diff/usr/lib/x86_64-linux-gnu/perl-base/Socket.pm
Raw/ix-apps/docker:/overlay2/7ceeada4a5baa99712cdbd4f2a80663f44822fcf85a7b370b8df024fbfd3cbdf/diff/opt/venv/lib/python3.11/site-packages/urllib3/__pycache__/_c
ollections.cpython-311.pyc
Raw/ix-apps/docker:/overlay2/672652a8ebd1e8026f9d5855715c59d8bf4a45615efbbc47bb537eafcca8aa8a/diff/usr/local/lib/python3.11/dist-packages/rapidfuzz/distance/me
trics_cpp.cpython-311-x86_64-linux-gnu.so
Raw/ix-apps/docker:/overlay2/7ceeada4a5baa99712cdbd4f2a80663f44822fcf85a7b370b8df024fbfd3cbdf/diff/opt/venv/lib/python3.11/site-packages/pydantic/__init__.py
Raw/ix-apps/docker:/overlay2/7ceeada4a5baa99712cdbd4f2a80663f44822fcf85a7b370b8df024fbfd3cbdf/diff/opt/venv/lib/python3.11/site-packages/pydantic/__pycache__/_
migration.cpython-311.pyc
Raw/ix-apps/docker:/overlay2/aca9354b9f090bbd23b95b13dfa25f97cbbc1dcb0023aa67e7cf0df9f593b6d2/lower
Raw/ix-apps/docker:/overlay2/a60769dc1274080453efef9e5355fd24b655311d05140ccd8ff4dcd01a20173a/diff/usr/src/node-red/node_modules/unpipe/package.json
Raw/ix-apps/docker:/overlay2/11de4331a803d5d8d50e1405df0556551ce1afa862ba8427aaafec67d68b1824/diff/usr/src/app/server/bin/start.sh
Raw/ix-apps/docker:/overlay2/72af987364b884fbdd4e7038b8b30472e3952ec6052bec58779eb41c06a390c2/diff/app/node_modules/date-fns/subMonths.js
Raw/ix-apps/docker:/overlay2/72af987364b884fbdd4e7038b8b30472e3952ec6052bec58779eb41c06a390c2/diff/app/node_modules/vary/package.json
Raw/ix-apps/docker:/overlay2/a60769dc1274080453efef9e5355fd24b655311d05140ccd8ff4dcd01a20173a/diff/usr/src/node-red/node_modules/express/package.json
Raw/ix-apps/docker:/overlay2/72af987364b884fbdd4e7038b8b30472e3952ec6052bec58779eb41c06a390c2/diff/app/node_modules/axios/lib/helpers/toURLEncodedForm.js
Raw/ix-apps/docker:/image/overlay2/imagedb/content/sha256/f2de8ed54c7b02ee05e42abaa454978c76fb552a883877b37588ebb62f31e41a
Raw/ix-apps/docker:/overlay2/72af987364b884fbdd4e7038b8b30472e3952ec6052bec58779eb41c06a390c2/diff/app/node_modules/hasown/index.js
Raw/ix-apps/docker:/overlay2/7ceeada4a5baa99712cdbd4f2a80663f44822fcf85a7b370b8df024fbfd3cbdf/diff/opt/venv/lib/python3.11/site-packages/urllib3/util/__pycache
__/__init__.cpython-311.pyc
Raw/ix-apps/docker:/overlay2/cd318c78502df459d0dfe410cd240f33e598760e782fdf299893434797dfd7a5/diff/usr/lib/ffmpeg/7.0/bin/ffmpeg
Raw/ix-apps/docker:/image/overlay2/imagedb/content/sha256/bcba9cf0611a4656bd0bbf27e062fa2a9eec4f3c36ea6c2f7d9cce30e2f35266
Raw/ix-apps/docker:/image/overlay2/imagedb/content/sha256/ecd3ede65dfca0cc8f3bb796c2833f0a3f0b8bdacd1a9153c8641652935c9a96
Raw/ix-apps/docker:/overlay2/a60769dc1274080453efef9e5355fd24b655311d05140ccd8ff4dcd01a20173a/diff/usr/src/node-red/node_modules/side-channel-map/package.json
Raw/ix-apps/docker:/overlay2/7ceeada4a5baa99712cdbd4f2a80663f44822fcf85a7b370b8df024fbfd3cbdf/diff/opt/venv/lib/python3.11/site-packages/urllib3/util/__pycache
__/util.cpython-311.pyc
Raw/ix-apps/docker:/overlay2/72af987364b884fbdd4e7038b8b30472e3952ec6052bec58779eb41c06a390c2/diff/app/node_modules/date-fns/previousMonday.js
Raw/ix-apps/docker:/overlay2/58f9a142ff88812b9fcd23b2738e35dbd0540d7eb59ef2727f524da316330e5b/diff/app/src/app-gocardless/banks/abanca_caglesmm.js
Raw/ix-apps/docker:/overlay2/6b90ef99e8f5818ce6614ffdd8c14f86cbdd7767ac9bd11a909b9cf4b37548e9/diff/app/package.json
Raw/ix-apps/docker:/overlay2/06f1bc334847646ce15e755bad4441799743f2e8d1b43096f1dcbc21f537b5ad/diff/usr/lib/x86_64-linux-gnu/libresolv.so.2
Raw/ix-apps/docker:/overlay2/72af987364b884fbdd4e7038b8b30472e3952ec6052bec58779eb41c06a390c2/diff/app/node_modules/keyv/package.json
Raw/ix-apps/docker:/overlay2/72af987364b884fbdd4e7038b8b30472e3952ec6052bec58779eb41c06a390c2/diff/app/node_modules/date-fns/previousTuesday.js
Raw/ix-apps/docker:/overlay2/02f54d3a6b5eef746d1ab6ef20e32ca1b244dabeae533ed23fe458371c99ca29/diff/usr/lib/x86_64-linux-gnu/libm.so.6
Raw/ix-apps/docker:/overlay2/a60769dc1274080453efef9e5355fd24b655311d05140ccd8ff4dcd01a20173a/diff/usr/src/node-red/node_modules/type-is/index.js
Raw/ix-apps/docker:/image/overlay2/imagedb/content/sha256/83cb3e4ff8399001af5b572e34a04d49ebada00820ff2a9bf4eecb64194e5c71
Raw/ix-apps/docker:/overlay2/72af987364b884fbdd4e7038b8b30472e3952ec6052bec58779eb41c06a390c2/diff/app/node_modules/date-fns/secondsToHours.js
Raw/ix-apps/docker:/overlay2/a60769dc1274080453efef9e5355fd24b655311d05140ccd8ff4dcd01a20173a/diff/usr/src/node-red/node_modules/@node-red/runtime/lib/api/sett
ings.js
Raw/ix-apps/docker:/overlay2/11de4331a803d5d8d50e1405df0556551ce1afa862ba8427aaafec67d68b1824/diff/usr/src/app/server/node_modules/.pnpm/jose@5.10.0/node_modul
es/jose/dist/node/cjs/runtime/aeskw.js
Raw/ix-apps/docker:/overlay2/a60769dc1274080453efef9e5355fd24b655311d05140ccd8ff4dcd01a20173a/diff/usr/src/node-red/node_modules/@node-red/runtime/lib/nodes/in
dex.js
Raw/ix-apps/docker:/containers/deeda835fd6835f4bb7be85b48137a9eef2cf9899bc4761750ac6fab22304749/deeda835fd6835f4bb7be85b48137a9eef2cf9899bc4761750ac6fab2230474
9-json.log
Raw/ix-apps/docker:/overlay2/3c7b33a0cc38df29796124cae5ea81caa725571c466d3106b955a08f6b4a5ca6/diff/jellyfin/System.Text.Encoding.CodePages.dll
Raw/ix-apps/docker:/image/overlay2/imagedb/content/sha256/cb8cfd826db1b9501e6e742ec2b00209d6a3476a330d7c0a6e191fda1116f4ae
Raw/ix-apps/docker:/overlay2/b18a0874adcf4c389f7a5f2b4af809d9d53d830cdbaf8e19bb487444f40d1674/diff/usr/lib/x86_64-linux-gnu/libpthread.so.0
Raw/.ix-virt/deleted/images/930ae31c4bf995d5a0ab534749c96e08d1fb0ebf08e89b8779a0af84e19fbe9f:/rootfs/usr/lib/locale/C.utf8/LC_CTYPE
Raw/.ix-virt/deleted/images/930ae31c4bf995d5a0ab534749c96e08d1fb0ebf08e89b8779a0af84e19fbe9f:/rootfs/usr/bin/perl5.36.0
Raw/.ix-virt/virtual-machines/noble24:/agent-client.key
Raw/ix-apps/app_mounts/jellyfin/config:/data/jellyfin.db
Raw/.ix-virt/deleted/images/930ae31c4bf995d5a0ab534749c96e08d1fb0ebf08e89b8779a0af84e19fbe9f@readonly:/rootfs/usr/lib/locale/C.utf8/LC_CTYPE
Raw/.ix-virt/deleted/images/930ae31c4bf995d5a0ab534749c96e08d1fb0ebf08e89b8779a0af84e19fbe9f@readonly:/rootfs/usr/bin/perl5.36.0
/var/lib/incus/storage-pools/Raw/containers/pteradactl/rootfs/var/log/journal/89bc59fb6675403abfb408a415477a92/system@2eae12b218694566abe83cadcd7e94af-00000000
000026cd-00065242f9444805.journal
/var/lib/incus/storage-pools/Raw/containers/pteradactl/rootfs/var/cache/apt/pkgcache.bin
Raw/.ix-virt/deleted/images/ba711673de502c882e12d388cf53800811ad2ab350e0ee9314fbfc859ca6774c:/rootfs/usr/lib/x86_64-linux-gnu/libm.so.6
Raw/.ix-virt/containers/odoo18:/rootfs/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
Raw/.ix-virt/containers/NginxProxyManager:/rootfs/usr/lib/x86_64-linux-gnu/libm.so.6
/mnt/Raw/rawstorage/frigate/config/frigate.db
Raw/ix-apps/app_mounts/qbittorrent/config:/qBittorrent/torrents.db-wal
/var/lib/incus/storage-pools/Raw/containers/GnuHospital/rootfs/var/log/journal/807a7b42d0a64eff8c230af8551f1cf8/system@00064a3dc135803d-ecdb616029a687ac.journa
l~
/var/lib/incus/storage-pools/Raw/containers/GnuHospital/rootfs/usr/lib/locale/C.utf8/LC_CTYPE
/var/lib/incus/storage-pools/Raw/containers/GnuHospital/rootfs/usr/bin/perl5.36.0
Raw/.ix-virt/deleted/images/cdd8385ee955aa07e91ec2f312710d39ca67c7d4237803c9209729ce07130f6d:/rootfs/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
Raw/.ix-virt/virtual-machines/noble24.block:<0x1>
Raw/.ix-virt/deleted/images/cdd8385ee955aa07e91ec2f312710d39ca67c7d4237803c9209729ce07130f6d@readonly:/rootfs/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
Raw/.ix-virt/deleted/images/ba711673de502c882e12d388cf53800811ad2ab350e0ee9314fbfc859ca6774c@readonly:/rootfs/usr/lib/x86_64-linux-gnu/libm.so.6

There’s much more than 1 bad sector, based on the first smartctl dump

5 Reallocated_Sector_Ct 0x0033 001 001 036 Pre-fail Always FAILING_NOW 57288

57288

Good for you!
As you have managed to mount the pool again, backup everything you can NOW.

Ah, I missed that. In one place I saw only one sector problem.

@Jordan1 You also look to only have run a SMART short test (yea, I’m the guy who talks about SMART tests). This is fine for a “general” idea if things are sort-of working fine, however a SMART extended (aka long) test is required to test if the entire media surface is readable, it is a minimal test but valuable. This will not fix the SMR issue you have nor any failed sectors, but it is a good indicator of drive failure.

I am not asking you to run a SMART test, the previous advice of “Copy everything you can NOW” is the best thing you can do. Once that is completed, then you can do something with the SMR drives, like place them in a different computer, or I believe @Arwen has used one as a backup drive for data on her NAS, not an active NAS source drive. I have used an external SMR drive for the same purpose, until I dropped it, Killed In Action.

When you replace the drives, use CMR drives, also think about if your data is important to you as this will drive you into redundancy for your Zpool design. If the files on your server are not important, then a Stripe is fine, provided you have no problems rebuilding what you have created (which often takes a lot of time). And if this was just a test to see what TrueNAS is like, then you have learned part of the downside to SMR drives the hard way, as many do.

Good luck and I hope your data recovery is successful.