Discs exported? How do I access the data on them?

TrueNAS General
1

Falling on the good will of the community for help here. I have a pool in my TrueNAS Scale 22.12.2 install which has exported itself and I have no idea how to get it back.

In short, yesterday I noticed my truenas and nextcloud instances being uber slow/non-responsive, eventually I restarted the truenas machine as I was rushing out the door.

I come home later that day to find that one of my SMB shares isn’t accessible, and looking at the truenas GUI my main pool “Manako” looks like it has 0 discs attached, and the discs which should be there are all reading as “Exported” in disc manager. For reference the pool was set up in RAID Z1 with a mirrored pair of Log VDEVs and an SSD Cache VDEV.

A while back one of the HDDS was starting to fail, so I replaced it and noticed one of the other discs was starting to develop errors in that process. So I pulled a bunch of data off the Manako pool and onto another pool (“AreMoe” - which is happy with no errors) but there is still stuff on there that I’d love to retrieve if possible.

I know just enough about this stuff to be dangerous, so any help would be massively appreciated! I have searched the forums for an answer, but haven’t come across anything that quite fits my situation. Perhaps it’s my lack of understanding.

Do I just add the discs back to the pool? I’m presuming that would wipe them?

Thanks heaps to anyone who takes the time to look at this post.

2

Detailed hardware listing would help.

Run the following in a Shell and paste the results back using Preformatted Text (</>) or Ctrl+e. This will let us see your pool status.

sudo zpool status -v
3

So the three accessible pools I typically have are;
AreMoe (cold storage)
Manako (hot storage)
Knoppbox (Dedicated nextcloud pool)

And of course the boot pool.

But I can’t see Manako in this list.

admin@knoppnas[~]$ sudo zpool status -v
  pool: AreMoe
 state: ONLINE
  scan: scrub canceled on Mon Feb 24 05:53:12 2025
config:

        NAME                                      STATE     READ WRITE CKSUM
        AreMoe                                    ONLINE       0     0     0
          raidz1-0                                ONLINE       0     0     0
            df7d77de-0941-4b6f-9f40-cd7036b1b537  ONLINE       0     0     0
            012a109d-f8c4-4c42-9a86-2bf9f8b94b33  ONLINE       0     0     0
            acf8b2be-f5cd-4ee1-a678-e2981042f950  ONLINE       0     0     0
            876cc5c3-e435-4b31-aaf0-4ff231b9dc3e  ONLINE       0     0     0
            46aba996-3101-42c1-8078-baa510e44e66  ONLINE       0     0     0
            1ef59f91-d22f-4e16-8709-329f44df1f92  ONLINE       0     0     0

errors: No known data errors

  pool: KnoppBox
 state: ONLINE
  scan: scrub repaired 0B in 00:36:17 with 0 errors on Sun Feb 23 00:36:19 2025
config:

        NAME                                      STATE     READ WRITE CKSUM
        KnoppBox                                  ONLINE       0     0     0
          raidz1-0                                ONLINE       0     0     0
            d9137431-0525-4c65-aa0d-3cbf30618a42  ONLINE       0     0     0
            4b122768-2356-438c-8290-144a60fdaff8  ONLINE       0     0     0
            4f2e5eef-c7bc-4686-a214-ac2f9c34272a  ONLINE       0     0     0

errors: No known data errors

  pool: boot-pool
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 00:00:12 with 0 errors on Sun Feb 23 03:45:13 2025
config:

        NAME           STATE     READ WRITE CKSUM
        boot-pool      ONLINE       0     0     0
          mirror-0     ONLINE       0     0     0
            nvme0n1p3  ONLINE       0     0     0
            nvme1n1p3  ONLINE       0     0     0

errors: No known data errors
1 Like
4

How are the disks attached? Detailed hardware listing?

Try running this to see the status of any pools for import. Are you showing your mirror log VDEV and L2ARC SSD?

sudo zpool import
5

Thanks for your quick reply (I have tentative hope!)

Response from Sudo zpool import is;

   pool: Manako
     id: 12445446880718311994
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

        Manako                                    ONLINE
          raidz1-0                                ONLINE
            254d49d0-b32e-4a41-afbd-d33853addc84  ONLINE
            be1e78d2-980f-4b5d-b107-efcb6f5c85a4  ONLINE
            734f63a6-28d8-4bc7-b12f-747c28a6310a  ONLINE
            2fb41522-5622-4019-b9b2-be72f210b2ae  ONLINE
        cache
          sdm1
        logs
          mirror-4                                ONLINE
            6b62561d-2aef-4bca-a324-0d30967a9ec0  ONLINE
            e0d488d4-9e86-4b95-92b3-5571eadbfb2e  ONLINE

As far as the hardware set up goes, I’ll have to have a good look (likely involve opening the machine up and having a look). I’ll let you know as soon as I can how I get on with that.

For reference re zpool importing, I have tried via the GUI to import the pool, but there is nothing in the dropdown when I select it.

Thanks again.

1 Like
6

Have you tried a shutdown, waiting a bit and then looking in the GUI for importing?

It looks good that it is recognized.

If you believe you can see all your disks, You can try
sudo zpool import Manako

7

I can’t seem to put screenshots in this reply, unsure why… But there we are.

Just to be clear re which disks I can see.

In the GUI the disk management space is showing 7 disks unassigned (which are all various VDEVs from Manako) and the Manako Pool is showing as being there, but offline with no VDEVs.

Worth trying the sudo import in this state? I’m guessing that reassigning the 7 disks to the Manako pool will wipe them… and result in tears?

8

Don’t reassign them. If a cold reboot or the CLI import doesn’t work, I will try to get someone else to look at this thread. I am just going by simple fix. I can try asking for second opinions in Chat. I just don’t know why it exported the pool but, otherwise, you are looking good here with it being seen.

Browse some other threads and do the Tutorial by the Bot to get your forum trust level up and post images. If browsing a few other thread doesn’t get you up, try the Bot

TrueNAS-Bot
Type this in a new reply and send to bring up the tutorial, if you haven’t done it already.

@TrueNAS-Bot start tutorial
9

Well… The light in the tunnel is fading slightly. The sudo zpool import command returned with;

admin@knoppnas[~]$ sudo zpool import Manako
cannot import 'Manako': I/O error
        Destroy and re-create the pool from
        a backup source.
10

We may have to wait a little bit for a follow up from more experience users. @HoneyBadger I think we need your expertise

11

Two drives with errors in a RAIDZ1 does put data at risk.

Can you run the command below against the /dev/sdX devices that make up your pool?

sudo smartctl -a /dev/sda

1 Like
12

…and past each individual result between two lines of three backquotes, or as formatted text (</> button), thanks!

```
SMART result here
```

1 Like
13

Hopefully I get the formatting right on this. Thanks again people, you’re all amazing!

I’m preparing myself for the possibility I might have borked it. Own goal and all.

Silly idea, but if I were to build another temporary nas with the same drives and attempt to import the pool on that, would that be a possible avenue?

Thanks again team!

=== START OF INFORMATION SECTION ===
Model Family:     Apacer AS340/350 SSDs
Device Model:     Apacer AS350 128GB
Serial Number:    AFJK1812800134
Firmware Version: V0823A0
User Capacity:    128,035,676,160 bytes [128 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Feb 25 19:32:41 2025 NZDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  120) seconds.
Offline data collection
capabilities:                    (0x11) SMART execute Offline immediate.
                                        No Auto Offline data collection support.
                                        Suspend Offline collection upon new
                                        command.
                                        No Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        No Selective Self-test supported.
SMART capabilities:            (0x0002) Does not save SMART data before
                                        entering power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (  10) minutes.
SCT capabilities:              (0x0001) SCT Status supported.

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x0032   100   100   050    Old_age   Always       -       0
  5 Reallocated_Sector_Ct   0x0032   100   100   050    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   100   100   050    Old_age   Always       -       14818
 12 Power_Cycle_Count       0x0032   100   100   050    Old_age   Always       -       92
160 Unknown_Attribute       0x0032   100   100   050    Old_age   Always       -       0
161 Unknown_Attribute       0x0033   100   100   050    Pre-fail  Always       -       100
163 Max_Erase_Count         0x0032   100   100   050    Old_age   Always       -       3
164 Average_Erase_Count     0x0032   100   100   050    Old_age   Always       -       15857
165 Unknown_Attribute       0x0032   100   100   050    Old_age   Always       -       52
166 Later_Bad_Block_Count   0x0032   100   100   050    Old_age   Always       -       4
167 SSD_Protect_Mode        0x0032   100   100   050    Old_age   Always       -       34
168 SATA_PHY_Error_Count    0x0032   100   100   050    Old_age   Always       -       5050
169 Unknown_Attribute       0x0032   100   100   050    Old_age   Always       -       100
175 Bad_Cluster_Table_Count 0x0032   100   100   050    Old_age   Always       -       0
176 Erase_Fail_Count_Chip   0x0032   100   100   050    Old_age   Always       -       0
177 Wear_Leveling_Count     0x0032   100   100   050    Old_age   Always       -       0
178 Used_Rsvd_Blk_Cnt_Chip  0x0032   100   100   050    Old_age   Always       -       0
181 Program_Fail_Cnt_Total  0x0032   100   100   050    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   050    Old_age   Always       -       0
192 Unexpect_Power_Loss_Ct  0x0032   100   100   050    Old_age   Always       -       6
194 Temperature_Celsius     0x0022   100   100   050    Old_age   Always       -       36
195 Hardware_ECC_Recovered  0x0032   100   100   050    Old_age   Always       -       0
196 Reallocated_Event_Count 0x0032   100   100   050    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   100   100   050    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0032   100   100   050    Old_age   Always       -       0
199 UDMA_CRC_Error_Count    0x0032   100   100   050    Old_age   Always       -       0
232 Available_Reservd_Space 0x0032   100   100   050    Old_age   Always       -       100
241 Total_LBAs_Written      0x0030   100   100   050    Old_age   Offline      -       24107
242 Total_LBAs_Read         0x0030   100   100   050    Old_age   Offline      -       1870
245 Unknown_Attribute       0x0032   100   100   050    Old_age   Always       -       125721

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     14797         -
# 2  Short offline       Completed without error       00%     14225         -
# 3  Short offline       Completed without error       00%     13483         -
# 4  Short offline       Completed without error       00%     12741         -
# 5  Short offline       Completed without error       00%     12022         -
# 6  Short offline       Completed without error       00%     11280         -
# 7  Short offline       Completed without error       00%     10577         -
# 8  Short offline       Completed without error       00%      9834         -
# 9  Short offline       Completed without error       00%      9092         -
#10  Short offline       Completed without error       00%      8374         -
#11  Short offline       Completed without error       00%      7659         -
#12  Short offline       Completed without error       00%      6939         -
#13  Short offline       Completed without error       00%      6395         -
#14  Short offline       Completed without error       00%      5775         -
#15  Short offline       Completed without error       00%      5032         -
#16  Short offline       Completed without error       00%      4290         -
#17  Short offline       Completed without error       00%      3581         -
#18  Short offline       Completed without error       00%      2895         -
#19  Short offline       Completed without error       00%      2177         -
#20  Short offline       Completed without error       00%      1443         -
#21  Short offline       Completed without error       00%       704         -

Selective Self-tests/Logging not supported

admin@knoppnas[~]$ sudo smartctl -a /dev/sdb
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.15.79+truenas] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Apacer AS340/350 SSDs
Device Model:     Apacer AS350 128GB
Serial Number:    AFJK1812803297
Firmware Version: V0823A0
User Capacity:    128,035,676,160 bytes [128 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Feb 25 19:33:22 2025 NZDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  120) seconds.
Offline data collection
capabilities:                    (0x11) SMART execute Offline immediate.
                                        No Auto Offline data collection support.
                                        Suspend Offline collection upon new
                                        command.
                                        No Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        No Selective Self-test supported.
SMART capabilities:            (0x0002) Does not save SMART data before
                                        entering power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (  10) minutes.
SCT capabilities:              (0x0001) SCT Status supported.

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x0032   100   100   050    Old_age   Always       -       2
  5 Reallocated_Sector_Ct   0x0032   100   100   050    Old_age   Always       -       2
  9 Power_On_Hours          0x0032   100   100   050    Old_age   Always       -       14847
 12 Power_Cycle_Count       0x0032   100   100   050    Old_age   Always       -       91
160 Unknown_Attribute       0x0032   100   100   050    Old_age   Always       -       0
161 Unknown_Attribute       0x0033   100   100   050    Pre-fail  Always       -       93
163 Max_Erase_Count         0x0032   100   100   050    Old_age   Always       -       11
164 Average_Erase_Count     0x0032   100   100   050    Old_age   Always       -       7487
165 Unknown_Attribute       0x0032   100   100   050    Old_age   Always       -       29
166 Later_Bad_Block_Count   0x0032   100   100   050    Old_age   Always       -       3
167 SSD_Protect_Mode        0x0032   100   100   050    Old_age   Always       -       16
168 SATA_PHY_Error_Count    0x0032   100   100   050    Old_age   Always       -       5050
169 Unknown_Attribute       0x0032   100   100   050    Old_age   Always       -       100
175 Bad_Cluster_Table_Count 0x0032   100   100   050    Old_age   Always       -       0
176 Erase_Fail_Count_Chip   0x0032   100   100   050    Old_age   Always       -       0
177 Wear_Leveling_Count     0x0032   100   100   050    Old_age   Always       -       0
178 Used_Rsvd_Blk_Cnt_Chip  0x0032   100   100   050    Old_age   Always       -       2
181 Program_Fail_Cnt_Total  0x0032   100   100   050    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   050    Old_age   Always       -       0
192 Unexpect_Power_Loss_Ct  0x0032   100   100   050    Old_age   Always       -       6
194 Temperature_Celsius     0x0022   100   100   050    Old_age   Always       -       37
195 Hardware_ECC_Recovered  0x0032   100   100   050    Old_age   Always       -       0
196 Reallocated_Event_Count 0x0032   100   100   050    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   100   100   050    Old_age   Always       -       2
198 Offline_Uncorrectable   0x0032   100   100   050    Old_age   Always       -       0
199 UDMA_CRC_Error_Count    0x0032   100   100   050    Old_age   Always       -       0
232 Available_Reservd_Space 0x0032   100   100   050    Old_age   Always       -       93
241 Total_LBAs_Written      0x0030   100   100   050    Old_age   Offline      -       20531
242 Total_LBAs_Read         0x0030   100   100   050    Old_age   Offline      -       640
245 Unknown_Attribute       0x0032   100   100   050    Old_age   Always       -       47853

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     14253         -
# 2  Short offline       Completed without error       00%     13509         -
# 3  Short offline       Completed without error       00%     12765         -
# 4  Short offline       Completed without error       00%     12045         -
# 5  Short offline       Completed without error       00%     11301         -
# 6  Short offline       Completed without error       00%     10597         -
# 7  Short offline       Completed without error       00%     10530         -
# 8  Extended offline    Completed without error       00%     10529         -
# 9  Short offline       Completed without error       00%      9853         -
#10  Short offline       Completed without error       00%      9108         -
#11  Short offline       Completed without error       00%      8389         -
#12  Short offline       Completed without error       00%      7673         -
#13  Short offline       Completed without error       00%      6953         -
#14  Short offline       Completed without error       00%      6406         -
#15  Short offline       Completed without error       00%      5785         -
#16  Short offline       Completed without error       00%      5041         -
#17  Short offline       Completed without error       00%      4297         -
#18  Short offline       Completed without error       00%      3587         -
#19  Short offline       Completed without error       00%      2900         -
#20  Short offline       Completed without error       00%      2181         -
#21  Short offline       Completed without error       00%      1445         -

Selective Self-tests/Logging not supported

admin@knoppnas[~]$ sudo smartctl -a /dev/sdc
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.15.79+truenas] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  559) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1025) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x50bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   082   064   044    Pre-fail  Always       -       167140983
  3 Spin_Up_Time            0x0003   092   090   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       109
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   086   060   045    Pre-fail  Always       -       386938710
  9 Power_On_Hours          0x0032   083   083   000    Old_age   Always       -       14942
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       109
 18 Head_Health             0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   060   038   000    Old_age   Always       -       40 (Min/Max 39/44)
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       2
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       1177
194 Temperature_Celsius     0x0022   040   062   000    Old_age   Always       -       40 (0 14 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Pressure_Limit          0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   100   000    Old_age   Offline      -       14921h+03m+01.844s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       39386225893
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       350948051079

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     14347         -
# 2  Short offline       Completed without error       00%     13603         -
# 3  Short offline       Completed without error       00%     12859         -
# 4  Short offline       Completed without error       00%     12139         -
# 5  Short offline       Completed without error       00%     11395         -
# 6  Short offline       Completed without error       00%     10689         -
# 7  Short offline       Completed without error       00%      9945         -
# 8  Short offline       Completed without error       00%      9201         -
# 9  Short offline       Completed without error       00%      8481         -
#10  Short offline       Completed without error       00%      7763         -
#11  Short offline       Completed without error       00%      7042         -
#12  Short offline       Completed without error       00%      6482         -
#13  Short offline       Completed without error       00%      5860         -
#14  Short offline       Completed without error       00%      5116         -
#15  Short offline       Completed without error       00%      4372         -
#16  Short offline       Completed without error       00%      3661         -
#17  Short offline       Completed without error       00%      2970         -
#18  Short offline       Completed without error       00%      2251         -
#19  Short offline       Completed without error       00%      1514         -
#20  Short offline       Completed without error       00%       773         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

admin@knoppnas[~]$ sudo smartctl -a /dev/sdd
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.15.79+truenas] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate IronWolf
Device Model:     ST12000VN0008-2PH103
Serial Number:    ZTN0ZTAW
LU WWN Device Id: 5 000c50 0e3dccd81
Firmware Version: SC61
User Capacity:    12,000,138,625,024 bytes [12.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Tue Feb 25 19:33:44 2025 NZDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
See vendor-specific Attribute list for marginal Attributes.

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  567) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1088) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x50bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   082   064   044    Pre-fail  Always       -       171563054
  3 Spin_Up_Time            0x0003   097   090   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   037   037   020    Old_age   Always       -       65535
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   087   060   045    Pre-fail  Always       -       522109230
  9 Power_On_Hours          0x0032   067   067   000    Old_age   Always       -       29564
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   045   037   020    Old_age   Always       -       56875
 18 Head_Health             0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   060   038   040    Old_age   Always   In_the_past 40 (Min/Max 40/40 #299)
192 Power-Off_Retract_Count 0x0032   039   039   000    Old_age   Always       -       122303
193 Load_Cycle_Count        0x0032   039   039   000    Old_age   Always       -       123428
194 Temperature_Celsius     0x0022   040   062   000    Old_age   Always       -       40 (0 14 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Pressure_Limit          0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       14320h+19m+07.988s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       38753543120
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       324616266553

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Interrupted (host reset)      00%     14347         -
# 2  Short offline       Interrupted (host reset)      00%     13603         -
# 3  Short offline       Interrupted (host reset)      00%     12859         -
# 4  Short offline       Interrupted (host reset)      00%     12139         -
# 5  Short offline       Interrupted (host reset)      00%     11395         -
# 6  Short offline       Completed without error       00%     11069         -
# 7  Short offline       Completed without error       00%     10689         -
# 8  Short offline       Completed without error       00%      9945         -
# 9  Short offline       Completed without error       00%      9201         -
#10  Short offline       Completed without error       00%      8481         -
#11  Short offline       Completed without error       00%      7763         -
#12  Short offline       Completed without error       00%      7042         -
#13  Short offline       Completed without error       00%      6482         -
#14  Short offline       Completed without error       00%      5860         -
#15  Short offline       Completed without error       00%      5116         -
#16  Short offline       Completed without error       00%      4372         -
#17  Short offline       Completed without error       00%      3661         -
#18  Short offline       Completed without error       00%      2970         -
#19  Short offline       Completed without error       00%      2251         -
#20  Short offline       Completed without error       00%      1514         -
#21  Short offline       Completed without error       00%       773         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

admin@knoppnas[~]$ sudo smartctl -a /dev/sdm
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.15.79+truenas] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Samsung based SSDs
Device Model:     Samsung SSD 870 EVO 500GB
Serial Number:    S7BWNJ0W629157M
LU WWN Device Id: 5 002538 f33620c76
Firmware Version: SVT02B6Q
User Capacity:    500,107,862,016 bytes [500 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available, deterministic, zeroed
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-4 T13/BSR INCITS 529 revision 5
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Feb 25 19:33:56 2025 NZDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (    0) seconds.
Offline data collection
capabilities:                    (0x53) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        No Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        (  85) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   097   097   000    Old_age   Always       -       14024
 12 Power_Cycle_Count       0x0032   099   099   000    Old_age   Always       -       58
177 Wear_Leveling_Count     0x0013   098   098   000    Pre-fail  Always       -       25
179 Used_Rsvd_Blk_Cnt_Tot   0x0013   100   100   010    Pre-fail  Always       -       0
181 Program_Fail_Cnt_Total  0x0032   100   100   010    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   010    Old_age   Always       -       0
183 Runtime_Bad_Block       0x0013   100   100   010    Pre-fail  Always       -       0
187 Uncorrectable_Error_Cnt 0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0032   063   050   000    Old_age   Always       -       37
195 ECC_Error_Rate          0x001a   200   200   000    Old_age   Always       -       0
199 CRC_Error_Count         0x003e   100   100   000    Old_age   Always       -       0
235 POR_Recovery_Count      0x0012   099   099   000    Old_age   Always       -       49
241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -       26647761150
252 Added_Bad_Flash_Blk_Ct  0x0032   100   100   000    Old_age   Always       -       2

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     13429         -
# 2  Short offline       Completed without error       00%     12685         -
# 3  Short offline       Completed without error       00%     11941         -
# 4  Short offline       Completed without error       00%     11221         -
# 5  Short offline       Completed without error       00%     10477         -
# 6  Short offline       Completed without error       00%      9771         -
# 7  Short offline       Completed without error       00%      9027         -
# 8  Short offline       Completed without error       00%      8283         -
# 9  Short offline       Completed without error       00%      7564         -
#10  Short offline       Completed without error       00%      6846         -
#11  Short offline       Completed without error       00%      6125         -
#12  Short offline       Completed without error       00%      5565         -
#13  Short offline       Completed without error       00%      4943         -
#14  Short offline       Completed without error       00%      4198         -
#15  Short offline       Completed without error       00%      3454         -
#16  Short offline       Completed without error       00%      2744         -
#17  Short offline       Completed without error       00%      2053         -
#18  Short offline       Completed without error       00%      1334         -
#19  Short offline       Completed without error       00%       597         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
  256        0    65535  Read_scanning was never started
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

admin@knoppnas[~]$ sudo smartctl -a /dev/sdo
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.15.79+truenas] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate IronWolf
Device Model:     ST12000VN0008-2YS101
Serial Number:    ZRT0SWTY
LU WWN Device Id: 5 000c50 0e6e5d961
Firmware Version: SC60
User Capacity:    12,000,138,625,024 bytes [12.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Feb 25 19:34:03 2025 NZDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  567) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1058) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x50bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   081   064   044    Pre-fail  Always       -       132262345
  3 Spin_Up_Time            0x0003   096   090   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       115
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   086   060   045    Pre-fail  Always       -       401135560
  9 Power_On_Hours          0x0032   083   083   000    Old_age   Always       -       14942
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       115
 18 Head_Health             0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   059   038   000    Old_age   Always       -       41 (Min/Max 40/45)
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       9
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       1183
194 Temperature_Celsius     0x0022   041   062   000    Old_age   Always       -       41 (0 14 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Pressure_Limit          0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   100   000    Old_age   Offline      -       14920h+47m+34.866s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       39366989668
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       351033752858

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     14347         -
# 2  Short offline       Completed without error       00%     13603         -
# 3  Short offline       Completed without error       00%     12859         -
# 4  Short offline       Completed without error       00%     12139         -
# 5  Short offline       Completed without error       00%     11395         -
# 6  Short offline       Completed without error       00%     10689         -
# 7  Short offline       Completed without error       00%      9945         -
# 8  Short offline       Completed without error       00%      9201         -
# 9  Short offline       Completed without error       00%      8481         -
#10  Short offline       Completed without error       00%      7763         -
#11  Short offline       Completed without error       00%      7042         -
#12  Short offline       Completed without error       00%      6482         -
#13  Short offline       Completed without error       00%      5860         -
#14  Short offline       Completed without error       00%      5116         -
#15  Short offline       Completed without error       00%      4371         -
#16  Short offline       Completed without error       00%      3661         -
#17  Short offline       Completed without error       00%      2970         -
#18  Short offline       Completed without error       00%      2250         -
#19  Short offline       Completed without error       00%      1514         -
#20  Short offline       Completed without error       00%       773         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

admin@knoppnas[~]$ sudo smartctl -a /dev/sdp
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.15.79+truenas] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     ST16000NT001-3LV101
Serial Number:    ZRS1LYTP
LU WWN Device Id: 5 000c50 0e8cb2135
Firmware Version: EN01
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 1.5 Gb/s)
Local Time is:    Tue Feb 25 19:34:09 2025 NZDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (  567) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1402) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x50bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   100   100   044    Pre-fail  Always       -       1048843
  3 Spin_Up_Time            0x0003   096   096   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   097   097   020    Old_age   Always       -       3180
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   100   253   045    Pre-fail  Always       -       664907
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       106
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   097   097   020    Old_age   Always       -       3230
 18 Unknown_Attribute       0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   061   055   000    Old_age   Always       -       39 (Min/Max 39/39)
192 Power-Off_Retract_Count 0x0032   099   099   000    Old_age   Always       -       3143
193 Load_Cycle_Count        0x0032   099   099   000    Old_age   Always       -       3998
194 Temperature_Celsius     0x0022   039   045   000    Old_age   Always       -       39 (0 36 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       5 (187 75 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       4064
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       1044779

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%        84         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
14

Picking at the SMART results, it looks like the only one that’s kinda sus is one of your SSDs (sdb with non-zero values in Reallocated_Sector_Ct but it’s a member of a mirror and shouldn’t be critical at all in that sense)

One of your drives though seems to be connected at a lower speed:

Device Model:     ST16000NT001-3LV101
Serial Number:    ZRS1LYTP
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 1.5 Gb/s)

You’ve also got a 12T drive at 3Gb/s:

Model Family:     Seagate IronWolf
Device Model:     ST12000VN0008-2PH103
Serial Number:    ZTN0ZTAW
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 3.0 Gb/s)

Which isn’t totally unusual if you have a SATA controller that can only do a limited number of ports at 6Gbps, but the 1.5Gb/s is odd to me.

15

Hmm very weird. Time for a new controller perhaps?

Or build a new system and import the pool?

16

Sigh… Looks like the disc is properly dead taking the pool with it. I suspect the one I replaced a while back didn’t “take”, which will be down the user error and nothing else. So, sadly I think I need to kiss good bye to the data and rework my storage protocols to avoid losses in the future.

Smaller pools with more redundancy is the key I think.

In the meantime a different pool seems to have had become degraded! Which is a little scary as that’s the one with my Nextcloud on it (Luckily all the data is backed up in multiple places as it’s mission critical… apart from the mass of passwords in NC Password).

So I’ll be adding another redundancy drive to that pool as soon as my NAS has finished scrubbing the one remaining healthy pool.

Who ever said IT work wasn’t fun right!

Thanks again for all your help!