Help understanding raid HDD issue reported

[xbmcgotham]

Hi, I am having difficulty understanding how the raid issues reported should be handled, hope someone can provide me an easy explanation as I am getting confused with the way the raid disc naming is reported. Thanks so much.

I have the following reported issues:

see attached

I have the following configuration:

see attached

I am getting confused with the two places that mention “spare”. How do I read this now correctly? And what process flow should I follow replacing the discs? Please let me know in steps to better understand and learn from it.

Thanks so much. And have a nice weekend.

[etorix]

Joes Rules to Asking for Help

Purpose: To assist people posting clear and informative requests for help and to define the absolute minimum information required in order to provide the helper what is needed and to reduce assumptions. EDIT: I have moved the specific areas of interest to separate entries below. Please read Joes Rules before posting anything Drive Failures Drive Speed Tests Network Issues Software Issues Build Advice Joes Rules Be considerate. This rule applies to both the person asking a question a…

To upload pictures, read the forum rules and follow the bot tutorial.
But preferably use SSH and copy paste command output as formatted text (</> button).

[xbmcgotham]

Thnaks etorix. Please find output below:

Status pool:

pool1 0 0 0 ONLINE
RAIDZ3  0 0 0 ONLINE
da10p2  0 0 0 ONLINE  more_vert
da14p2  0 0 0 ONLINE  more_vert
da15p2  0 0 0 ONLINE  more_vert
da13p2  0 0 0 ONLINE  more_vert
da11p2  0 0 0 ONLINE  more_vert
da3p2 0 0 0 ONLINE  more_vert
da2p2 0 0 0 ONLINE  more_vert
da12p2  0 0 0 ONLINE  more_vert
da8p2 0 0 0 ONLINE  more_vert
da6p2 0 0 0 ONLINE  more_vert
da4p2 0 0 0 ONLINE  more_vert
da1p2 0 0 0 ONLINE  more_vert
da9p2 0 0 0 ONLINE  more_vert
da5p2 0 0 0 ONLINE  more_vert
da7p2 0 0 0 ONLINE  more_vert
da16p2  0 0 0 ONLINE  more_vert
da23p2  0 0 0 ONLINE  more_vert
da17p2  0 0 0 ONLINE  more_vert
da0p2 0 0 0 ONLINE  more_vert
SPARE 0 0 0 ONLINE
da18p2  0 0 0 ONLINE  more_vert
da22p2  0 0 0 ONLINE  more_vert
da19p2  0 0 0 ONLINE  more_vert
da20p2  0 0 0 ONLINE  more_vert
log       
ada3p1  0 0 0 ONLINE  more_vert
cache       
ada2p1  0 0 0 ONLINE  more_vert
spare       
da22p2  0 0 0 UNAVAIL more_vert
da21p2  0 0 0 ONLINE  more_vert

zpool status -v:

Last login: Fri Feb 14 19:55:50 on pts/1
FreeBSD 12.2-RELEASE-p9 2ee62d665f0(HEAD) TRUENAS

        TrueNAS (c) 2009-2021, iXsystems, Inc.
        All rights reserved.
        TrueNAS code is released under the modified BSD license with some
        files copyrighted by (c) iXsystems, Inc.

        For more information, documentation, help or support, go here:
        http://truenas.com
Welcome to FreeNAS

Warning: settings changed through the CLI are not written to
the configuration database and will be reset on reboot.

root@freenas[~]# zpool status -v
  pool: freenas-boot
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(5) for details.
  scan: scrub repaired 0B in 00:00:55 with 0 errors on Sun Feb 23 03:45:55 2025
config:

        NAME          STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          mirror-0    ONLINE       0     0     0
            ada0p2    ONLINE       0     0     0
            ada1p2    ONLINE       0     0     0

errors: No known data errors

  pool: pool1
 state: ONLINE
  scan: scrub repaired 0B in 16:27:23 with 0 errors on Sun Feb 23 19:27:37 2025
config:

        NAME                                              STATE     READ WRITE CKSUM
        pool1                                             ONLINE       0     0   0
          raidz3-0                                        ONLINE       0     0   0
            gptid/b12a2dae-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/b171800e-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/b8a98827-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/b34a7b2a-a554-11ec-b4aa-002590f392a6    ONLINE       0     0   0
            gptid/b9034d67-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/b95eb56b-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/b9dbe959-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/ba627fa9-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/ba976478-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/baef899b-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/bb718858-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/bb33a0c9-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/bb902ffb-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/bc18476b-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/bbc94c83-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/bc0e056f-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/c26cde3e-a2bb-11ec-ba73-002590f392a6    ONLINE       0     0   0
            gptid/bc98ab84-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/beaa063c-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            spare-19                                      ONLINE       0     0   0
              gptid/c3929f90-0970-11eb-a40e-002590f392a6  ONLINE       0     0   0
              gptid/9dd03cb8-211e-11eb-9b98-002590f392a6  ONLINE       0     0   0
            gptid/c46d7308-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
            gptid/c4cd2490-0970-11eb-a40e-002590f392a6    ONLINE       0     0   0
        logs
          gptid/bd8a38ef-0970-11eb-a40e-002590f392a6      ONLINE       0     0   0
        cache
          gptid/bd81c392-0970-11eb-a40e-002590f392a6      ONLINE       0     0   0
        spares
          gptid/9dd03cb8-211e-11eb-9b98-002590f392a6      INUSE     currently in use
          gptid/9de1842c-211e-11eb-9b98-002590f392a6      AVAIL

errors: No known data errors
root@freenas[~]#

I have a warning of:

Device: /dev/da21, SMART Failure: HARDWARE IMPENDING FAILURE GENERAL HARD DRIVE FAILURE.

and

Device: /dev/da18, SMART Failure: DATA CHANNEL IMPENDING FAILURE DATA ERROR RATE TOO HIGH.

The Glabel Status:

root@freenas[~]# glabel status
                                      Name  Status  Components
gptid/1d94025e-096c-11eb-9c76-002590f392a6     N/A  ada0p1
gptid/1dd6e937-096c-11eb-9c76-002590f392a6     N/A  ada1p1
gptid/bd81c392-0970-11eb-a40e-002590f392a6     N/A  ada2p1
gptid/bd8a38ef-0970-11eb-a40e-002590f392a6     N/A  ada3p1
gptid/beaa063c-0970-11eb-a40e-002590f392a6     N/A  da0p2
gptid/bb33a0c9-0970-11eb-a40e-002590f392a6     N/A  da1p2
gptid/b9dbe959-0970-11eb-a40e-002590f392a6     N/A  da2p2
gptid/b95eb56b-0970-11eb-a40e-002590f392a6     N/A  da3p2
gptid/bb718858-0970-11eb-a40e-002590f392a6     N/A  da4p2
gptid/bc18476b-0970-11eb-a40e-002590f392a6     N/A  da5p2
gptid/baef899b-0970-11eb-a40e-002590f392a6     N/A  da6p2
gptid/bbc94c83-0970-11eb-a40e-002590f392a6     N/A  da7p2
gptid/ba976478-0970-11eb-a40e-002590f392a6     N/A  da8p2
gptid/bb902ffb-0970-11eb-a40e-002590f392a6     N/A  da9p2
gptid/b12a2dae-0970-11eb-a40e-002590f392a6     N/A  da10p2
gptid/b9034d67-0970-11eb-a40e-002590f392a6     N/A  da11p2
gptid/ba627fa9-0970-11eb-a40e-002590f392a6     N/A  da12p2
gptid/b34a7b2a-a554-11ec-b4aa-002590f392a6     N/A  da13p2
gptid/b171800e-0970-11eb-a40e-002590f392a6     N/A  da14p2
gptid/b8a98827-0970-11eb-a40e-002590f392a6     N/A  da15p2
gptid/bc0e056f-0970-11eb-a40e-002590f392a6     N/A  da16p2
gptid/bc98ab84-0970-11eb-a40e-002590f392a6     N/A  da17p2
gptid/c3929f90-0970-11eb-a40e-002590f392a6     N/A  da18p2
gptid/c46d7308-0970-11eb-a40e-002590f392a6     N/A  da19p2
gptid/c4cd2490-0970-11eb-a40e-002590f392a6     N/A  da20p2
gptid/9de1842c-211e-11eb-9b98-002590f392a6     N/A  da21p2
gptid/9dd03cb8-211e-11eb-9b98-002590f392a6     N/A  da22p2
gptid/c26cde3e-a2bb-11ec-ba73-002590f392a6     N/A  da23p2
gptid/b913e554-0970-11eb-a40e-002590f392a6     N/A  da3p1
gptid/b9777f2b-0970-11eb-a40e-002590f392a6     N/A  da2p1
gptid/baa775d6-0970-11eb-a40e-002590f392a6     N/A  da1p1
gptid/be91ddd1-0970-11eb-a40e-002590f392a6     N/A  da0p1
gptid/1de77d83-096c-11eb-9c76-002590f392a6     N/A  ada1p3
gptid/1da48998-096c-11eb-9c76-002590f392a6     N/A  ada0p3
root@freenas[~]#

[etorix]

That raidz3 is way too large for comfort.
Now let’s look at the SMART reports:
smartctl -x /dev/da21
smartctl -x /dev/da18

Please use the formatted text button </>, or paste the results between two lines of three backquotes for readbility.
```
report here
```

[xbmcgotham]

Thanks @etorix

Please find below:

root@freenas[~]# smartctl -x /dev/da21
smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               SEAGATE
Product:              ST4000NM0023
Revision:             A001
Compliance:           SPC-4
User Capacity:        4,000,787,030,016 bytes [4.00 TB]
Logical block size:   512 bytes
LU is fully provisioned
Rotation Rate:        7200 rpm
Form Factor:          3.5 inches
Logical Unit id:      0x5000c50055e501db
Serial number:        Z1Z07WCQ0000S311BUKE
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Sat Mar  1 11:28:39 2025 GMT
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled
Read Cache is:        Enabled
Writeback Cache is:   Disabled

=== START OF READ SMART DATA SECTION ===
SMART Health Status: HARDWARE IMPENDING FAILURE GENERAL HARD DRIVE FAILURE [asc=5d, ascq=10]

Current Drive Temperature:     40 C
Drive Trip Temperature:        68 C

Manufactured in week @8 of year 20#
Specified cycle count over device lifetime:  10000
Accumulated start-stop cycles:  132
Specified load-unload count over device lifetime:  300000
Accumulated load-unload cycles:  132
Elements in grown defect list: 65

Vendor (Seagate Cache) information
  Blocks sent to initiator = 359626571
  Blocks received from initiator = 3401526252
  Blocks read from cache and sent to initiator = 677356887
  Number of read and write commands whose size <= segment size = 57221988
  Number of read and write commands whose size > segment size = 0

Vendor (Seagate/Hitachi) factory information
  number of hours powered up = 13729.25
  number of minutes until next internal SMART test = 27

Error counter log:
           Errors Corrected by           Total   Correction     Gigabytes    Total
               ECC          rereads/    errors   algorithm      processed    uncorrected
           fast | delayed   rewrites  corrected  invocations   [10^9 bytes]  errors
read:   1309035598       25         0  1309035623         35       6781.199     10
write:         0        0         0         0         21       1770.083 18
verify:      445        0         0       445          0          0.000  0

Non-medium error count:       18


[GLTSD (Global Logging Target Save Disable) set. Enable Save with '-S on']
SMART Self-test log
Num  Test              Status                 segment  LifeTime  LBA_first_err [SK ASC ASQ]
     Description                              number   (hours)
# 1  Background long   Failed in segment -->       -   13723        5472081256 [0x3 0x11 0x0]
# 2  Background short  Completed                   -   13550                 - [-   -    -]
# 3  Background short  Completed                   -   13382                 - [-   -    -]
# 4  Background short  Completed                   -   13235                 - [-   -    -]
# 5  Background long   Failed in segment -->       -   13072        5472081256 [0x3 0x11 0x0]
# 6  Background short  Completed                   -   12899                 - [-   -    -]
# 7  Background short  Completed                   -   12750                 - [-   -    -]
# 8  Background short  Completed                   -   12582                 - [-   -    -]
# 9  Background short  Completed                   -   12419                 - [-   -    -]
#10  Background short  Completed                   -   12251                 - [-   -    -]
#11  Background short  Completed                   -   12099                 - [-   -    -]
#12  Background long   Failed in segment -->       -   11960        5472081256 [0x3 0x11 0x0]
#13  Background short  Completed                   -   11931                 - [-   -    -]
#14  Background short  Completed                   -   11763                 - [-   -    -]
#15  Background short  Completed                   -   11595                 - [-   -    -]
#16  Background short  Completed                   -   11427                 - [-   -    -]
#17  Background short  Completed                   -   11259                 - [-   -    -]
#18  Background long   Failed in segment -->       -   11240        5472081256 [0x3 0x11 0x0]
#19  Background short  Completed                   -   11090                 - [-   -    -]
#20  Background short  Completed                   -   10922                 - [-   -    -]

Long (extended) Self-test duration: 32700 seconds [545.0 minutes]

Background scan results log
  Status: waiting until BMS interval timer expires
    Accumulated power on time, hours:minutes 13729:15 [823755 minutes]
    Number of background scans performed: 1259,  scan progress: 0.00%
    Number of background medium scans performed: 1259

   #  when        lba(hex)    [sk,asc,ascq]    reassign_status
   1 4875:42  00000000ddf0c6f3  [1,16,0]   Recovered via rewrite in-place
   2 30635:38  00000000ddf0c6f3  [1,16,0]   Recovered via rewrite in-place
   3 37268:29  000000016f7fd219  [1,17,1]   Recovered via rewrite in-place
   4 66886:12  00000001462903a2  [1,18,1]   Recovered via rewrite in-place
   5 66886:12  00000001462903a3  [1,18,1]   Recovered via rewrite in-place
   6 66886:13  00000001462903a5  [1,17,1]   Recovered via rewrite in-place
   7 66886:13  00000001462906d0  [1,17,1]   Recovered via rewrite in-place
   8 66886:13  00000001462906d1  [1,17,1]   Recovered via rewrite in-place
   9 66886:13  00000001462906d3  [1,17,1]   Recovered via rewrite in-place
  10 66886:13  00000001462908af  [1,18,1]   Recovered via rewrite in-place
  11 66886:14  00000001462908b8  [3,11,0]   Recovered via rewrite in-place
  12 66886:18  00000001462908b9  [3,11,0]   Recovered via rewrite in-place
  13 66886:18  00000001462908ba  [1,18,1]   Recovered via rewrite in-place
  14 66886:18  00000001462908be  [3,11,0]   Recovered via rewrite in-place
  15 66886:18  00000001462908bf  [3,11,0]   Recovered via rewrite in-place
  16 66886:18  00000001462908c0  [3,11,0]   Recovered via rewrite in-place
  17 66886:19  00000001462908c1  [3,11,0]   Recovered via rewrite in-place
  18 66886:19  00000001462908c2  [3,11,0]   Recovered via rewrite in-place
  19 66886:19  00000001462908c3  [3,11,0]   Recovered via rewrite in-place
  20 66886:19  00000001462908c4  [1,17,1]   Recovered via rewrite in-place
  21 66886:20  00000001462908c5  [3,11,0]   Recovered via rewrite in-place
  22 66886:20  00000001462908c6  [3,11,0]   Recovered via rewrite in-place
  23 66886:20  00000001462908c7  [1,18,1]   Recovered via rewrite in-place
  24 66886:20  00000001462908c8  [3,11,0]   Recovered via rewrite in-place
  25 66886:20  00000001462908c9  [3,11,0]   Recovered via rewrite in-place
  26 66886:21  00000001462908ca  [3,11,0]   Recovered via rewrite in-place
  27 66886:21  00000001462908cb  [3,11,0]   Recovered via rewrite in-place
  28 66886:21  00000001462908cc  [3,16,0]   Recovered via rewrite in-place
  29 66886:21  00000001462908cd  [3,11,0]   Recovered via rewrite in-place
  30 66886:22  00000001462908ce  [3,11,0]   Recovered via rewrite in-place
  31 66886:23  00000001462908cf  [3,11,0]   Recovered via rewrite in-place
  32 66886:24  00000001462908d0  [3,11,0]   Recovered via rewrite in-place
  33 66886:25  00000001462908d1  [3,11,0]   Recovered via rewrite in-place
  34 66886:26  00000001462908d2  [3,11,0]   Recovered via rewrite in-place
  35 66886:28  00000001462908d3  [3,11,0]   Recovered via rewrite in-place
  36 66886:29  00000001462908d4  [3,11,0]   Recovered via rewrite in-place
  37 66886:29  00000001462908d5  [3,11,0]   Recovered via rewrite in-place
  38 66886:31  00000001462908d6  [3,11,0]   Recovered via rewrite in-place
  39 66886:31  00000001462908d7  [3,11,0]   Recovered via rewrite in-place
  40 66886:32  00000001462908d8  [3,11,0]   Recovered via rewrite in-place
  41 66886:32  00000001462908d9  [3,11,0]   Recovered via rewrite in-place
  42 66886:32  00000001462908da  [3,11,0]   Recovered via rewrite in-place
  43 66886:33  00000001462908db  [3,11,0]   Recovered via rewrite in-place
  44 67020:17  000000014629555d  [1,17,1]   Recovered via rewrite in-place
  45 67020:17  000000014629555e  [1,18,1]   Recovered via rewrite in-place
  46 67020:17  0000000146295568  [3,11,0]   Require Write or Reassign Blocks command
  47 67020:17  0000000146295569  [3,11,0]   Require Write or Reassign Blocks command
  48 67020:17  000000014629556a  [3,11,0]   Require Write or Reassign Blocks command
  49 67020:17  000000014629556b  [1,17,1]   Recovered via rewrite in-place
  50 67020:17  000000014629556c  [1,17,1]   Recovered via rewrite in-place
  51 67020:17  000000014629556d  [1,17,1]   Recovered via rewrite in-place
  52 67020:17  000000014629556f  [3,11,0]   Require Write or Reassign Blocks command
  53 67020:17  0000000146295de1  [3,11,0]   Require Write or Reassign Blocks command
  54 67020:17  0000000146295de3  [3,11,0]   Require Write or Reassign Blocks command
  55 67020:17  0000000146295de4  [3,11,0]   Require Write or Reassign Blocks command
  56 67020:17  0000000146295de5  [3,11,0]   Require Write or Reassign Blocks command
  57 67020:17  0000000146295de6  [3,11,0]   Require Write or Reassign Blocks command
  58 67020:18  0000000146295de7  [3,11,0]   Require Write or Reassign Blocks command
  59 67020:18  0000000146295de9  [3,11,0]   Require Write or Reassign Blocks command
  60 67020:18  0000000146295dea  [3,11,0]   Require Write or Reassign Blocks command
  61 67020:18  0000000146295deb  [3,11,0]   Require Write or Reassign Blocks command
  62 67020:18  0000000146295dec  [3,11,0]   Require Write or Reassign Blocks command
  63 67020:18  0000000146295ded  [3,11,0]   Require Write or Reassign Blocks command
  64 67020:18  0000000146295dee  [1,17,1]   Recovered via rewrite in-place
  65 67020:18  0000000146295def  [3,11,0]   Require Write or Reassign Blocks command
  66 67020:18  0000000146295df0  [3,11,0]   Require Write or Reassign Blocks command
  67 67020:18  0000000146295df1  [1,17,1]   Recovered via rewrite in-place
  68 67020:18  0000000146295df2  [1,17,1]   Recovered via rewrite in-place
  69 67020:18  0000000146295df3  [1,17,1]   Recovered via rewrite in-place
  70 67020:18  0000000146295df4  [1,17,3]   Recovered via rewrite in-place
  71 67020:18  0000000146295df5  [1,18,1]   Recovered via rewrite in-place
  72 67020:18  0000000146295df6  [1,17,1]   Recovered via rewrite in-place
  73 67020:18  000000014629665b  [3,11,0]   Require Write or Reassign Blocks command
  74 67020:18  000000014629665c  [1,17,1]   Recovered via rewrite in-place
  75 67020:18  000000014629665d  [3,11,0]   Require Write or Reassign Blocks command
  76 67020:18  000000014629665e  [1,17,1]   Recovered via rewrite in-place
  77 67020:18  000000014629665f  [3,11,0]   Require Write or Reassign Blocks command
  78 67020:18  0000000146296661  [3,11,0]   Require Write or Reassign Blocks command
  79 67020:18  0000000146296662  [1,18,1]   Recovered via rewrite in-place
  80 67020:18  0000000146296663  [3,11,0]   Require Write or Reassign Blocks command
  81 67020:19  0000000146296664  [3,11,0]   Require Write or Reassign Blocks command
  82 67020:19  0000000146296ec0  [1,18,1]   Recovered via rewrite in-place
  83 67020:19  0000000146296ec4  [1,17,1]   Recovered via rewrite in-place
  84 67020:19  0000000146296ec5  [1,17,1]   Recovered via rewrite in-place
  85 67020:19  0000000146296ec6  [3,11,0]   Require Write or Reassign Blocks command
  86 67020:19  0000000146296ec7  [3,11,0]   Require Write or Reassign Blocks command
  87 67020:19  0000000146296ec8  [1,18,1]   Recovered via rewrite in-place
  88 67020:19  0000000146296ecb  [1,17,1]   Recovered via rewrite in-place
  89 67020:19  0000000146296ecc  [1,17,1]   Recovered via rewrite in-place
  90 67020:19  0000000146296ecd  [1,18,1]   Recovered via rewrite in-place
  91 67020:19  0000000146296ece  [3,11,0]   Require Write or Reassign Blocks command
  92 67020:19  0000000146296ecf  [3,11,0]   Require Write or Reassign Blocks command
  93 67020:19  0000000146296ed0  [3,11,0]   Require Write or Reassign Blocks command
  94 67020:19  0000000146296ed1  [3,11,0]   Require Write or Reassign Blocks command
  95 67020:19  0000000146296ed2  [3,11,0]   Require Write or Reassign Blocks command
  96 67020:19  0000000146296ed4  [3,11,0]   Require Write or Reassign Blocks command
  97 67020:19  0000000146296ed5  [3,11,0]   Require Write or Reassign Blocks command
  98 67020:19  0000000146296ed6  [3,11,0]   Require Write or Reassign Blocks command
  99 67020:19  0000000146296ed7  [3,11,0]   Require Write or Reassign Blocks command
 100 67020:19  0000000146296ed8  [3,11,0]   Require Write or Reassign Blocks command
 101 67020:19  0000000146296ed9  [3,11,0]   Require Write or Reassign Blocks command
 102 67020:20  0000000146296eda  [3,11,0]   Require Write or Reassign Blocks command
 103 67020:20  0000000146296edb  [1,17,1]   Recovered via rewrite in-place
 104 67020:20  0000000146296edc  [3,11,0]   Require Write or Reassign Blocks command
 105 67020:20  0000000146296edd  [1,18,1]   Recovered via rewrite in-place
 106 67020:20  0000000146296edf  [1,17,1]   Recovered via rewrite in-place
 107 67020:20  0000000146296ee2  [1,17,1]   Recovered via rewrite in-place
 108 67020:20  0000000146297743  [1,17,1]   Recovered via rewrite in-place
 109 67020:20  000000014629774e  [1,18,1]   Recovered via rewrite in-place
 110 67020:20  000000014629774f  [3,11,0]   Recovered via rewrite in-place
 111 67020:20  0000000146297750  [3,11,0]   Recovered via rewrite in-place
 112 67020:20  0000000146297752  [3,11,0]   Recovered via rewrite in-place
 113 67020:20  0000000146297753  [3,11,0]   Recovered via rewrite in-place
 114 67020:20  0000000146297754  [1,17,1]   Recovered via rewrite in-place
 115 67020:20  0000000146297755  [3,11,0]   Recovered via rewrite in-place
 116 67020:20  0000000146297756  [3,11,0]   Recovered via rewrite in-place
 117 67020:20  0000000146297757  [1,17,1]   Recovered via rewrite in-place
 118 67020:20  0000000146297fbf  [3,11,0]   Require Write or Reassign Blocks command
 119 67020:20  0000000146297fc0  [1,17,1]   Recovered via rewrite in-place
 120 67020:20  0000000146297fc1  [1,17,1]   Recovered via rewrite in-place
 121 67020:20  0000000146297fc2  [1,17,1]   Recovered via rewrite in-place
 122 67020:20  0000000146297fc3  [3,11,0]   Require Write or Reassign Blocks command
 123 67020:20  0000000146297fc5  [3,11,0]   Require Write or Reassign Blocks command
 124 67020:20  0000000146297fc6  [3,11,0]   Require Write or Reassign Blocks command
 125 67020:20  0000000146297fc7  [3,11,0]   Require Write or Reassign Blocks command
 126 67020:20  0000000146297fc8  [3,11,0]   Require Write or Reassign Blocks command
 127 67020:21  0000000146297fc9  [3,11,0]   Require Write or Reassign Blocks command
 128 67020:21  0000000146297fca  [3,11,0]   Require Write or Reassign Blocks command
 129 67020:21  0000000146297fcb  [3,11,0]   Require Write or Reassign Blocks command
 130 67020:21  0000000146297fcc  [3,11,0]   Require Write or Reassign Blocks command
 131 67020:21  0000000146297fcd  [3,16,0]   Require Write or Reassign Blocks command
 132 67020:21  0000000146297fce  [3,11,0]   Require Write or Reassign Blocks command
 133 67020:21  0000000146297fcf  [3,11,0]   Require Write or Reassign Blocks command
 134 67020:21  0000000146297fd0  [3,11,0]   Require Write or Reassign Blocks command
 135 67020:21  0000000146297fd1  [3,11,0]   Require Write or Reassign Blocks command
 136 67020:21  0000000146297fd2  [1,17,1]   Recovered via rewrite in-place
 137 67020:21  0000000146297fd3  [1,17,1]   Recovered via rewrite in-place
 138 67020:21  0000000146297fd5  [1,18,1]   Recovered via rewrite in-place
 139 67020:21  000000014629883d  [1,17,1]   Recovered via rewrite in-place
 140 67020:21  000000014629883e  [1,18,1]   Recovered via rewrite in-place
 141 67020:21  000000014629883f  [1,17,1]   Recovered via rewrite in-place
 142 67020:21  0000000146298840  [1,18,1]   Recovered via rewrite in-place
 143 67020:21  0000000146298841  [1,17,1]   Recovered via rewrite in-place
 144 67020:21  0000000146298842  [3,11,0]   Require Write or Reassign Blocks command
 145 67020:21  0000000146298843  [1,17,1]   Recovered via rewrite in-place
 146 67020:21  0000000146298844  [3,11,0]   Require Write or Reassign Blocks command
 147 67020:21  0000000146298845  [1,17,1]   Recovered via rewrite in-place
 148 67020:21  0000000146298846  [1,17,1]   Recovered via rewrite in-place
 149 67020:22  0000000146298848  [3,11,0]   Require Write or Reassign Blocks command
 150 67020:22  0000000146298849  [1,18,1]   Recovered via rewrite in-place
 151 67020:22  000000014629884a  [1,17,1]   Recovered via rewrite in-place
 152 67020:22  000000014629884b  [3,11,0]   Require Write or Reassign Blocks command
 153 67020:22  000000014629884c  [1,18,1]   Recovered via rewrite in-place
 154 67020:22  00000001462990b5  [3,11,0]   Require Write or Reassign Blocks command
 155 67020:22  00000001462990b6  [1,18,1]   Recovered via rewrite in-place
 156 67020:22  00000001462990b8  [3,11,0]   Require Write or Reassign Blocks command
 157 67020:22  00000001462990b9  [3,11,0]   Require Write or Reassign Blocks command
 158 67020:22  00000001462990bb  [3,11,0]   Require Write or Reassign Blocks command
 159 67020:22  00000001462990bc  [3,11,0]   Require Write or Reassign Blocks command
 160 67020:22  00000001462990bd  [1,17,1]   Recovered via rewrite in-place
 161 67020:22  00000001462990be  [3,11,0]   Require Write or Reassign Blocks command
 162 67020:22  00000001462990bf  [3,11,0]   Require Write or Reassign Blocks command
 163 67020:22  000000014629992e  [1,17,1]   Recovered via rewrite in-place
 164 67020:22  000000014629992f  [1,17,1]   Recovered via rewrite in-place
 165 67020:22  0000000146299930  [1,17,1]   Recovered via rewrite in-place
 166 67020:22  0000000146299931  [3,11,0]   Recovered via rewrite in-place
 167 67020:22  0000000146299932  [3,11,0]   Recovered via rewrite in-place
 168 67020:22  0000000146299933  [1,18,1]   Recovered via rewrite in-place
 169 67020:22  0000000146299934  [1,17,1]   Recovered via rewrite in-place
 170 67020:22  0000000146299935  [3,11,0]   Recovered via rewrite in-place
 171 67020:22  0000000146299936  [3,11,0]   Recovered via rewrite in-place
 172 67020:22  0000000146299937  [1,17,1]   Recovered via rewrite in-place
 173 67020:23  0000000146299938  [3,11,0]   Recovered via rewrite in-place
 174 67020:23  0000000146299939  [3,11,0]   Recovered via rewrite in-place
 175 67020:23  000000014629993a  [1,17,1]   Recovered via rewrite in-place
 176 67020:23  000000014629993b  [3,11,0]   Recovered via rewrite in-place
 177 67020:23  000000014629993c  [3,11,0]   Recovered via rewrite in-place
 178 67020:23  000000014629993d  [1,18,1]   Recovered via rewrite in-place
 179 67020:23  000000014629993e  [3,11,0]   Recovered via rewrite in-place
 180 67020:23  000000014629993f  [1,17,1]   Recovered via rewrite in-place
 181 67020:23  0000000146299941  [1,17,1]   Recovered via rewrite in-place
 182 67020:23  0000000146299942  [1,17,1]   Recovered via rewrite in-place
 183 67020:23  000000014629a1ab  [1,17,1]   Recovered via rewrite in-place
 184 67020:23  000000014629a1ac  [1,17,1]   Recovered via rewrite in-place
 185 67020:23  000000014629a1af  [1,17,1]   Recovered via rewrite in-place
 186 67020:23  000000014629a1b2  [1,18,1]   Recovered via rewrite in-place
 187 67020:23  000000014629a1b3  [1,18,1]   Recovered via rewrite in-place
 188 67020:23  000000014629aa1c  [1,17,1]   Recovered via rewrite in-place
 189 67020:23  000000014629aa1d  [1,17,3]   Recovered via rewrite in-place
 190 67020:23  000000014629aa1f  [1,18,1]   Recovered via rewrite in-place
 191 67020:23  000000014629aa20  [1,18,1]   Recovered via rewrite in-place
 192 67020:23  000000014629aa21  [1,17,1]   Recovered via rewrite in-place
 193 67020:23  000000014629aa22  [1,17,1]   Recovered via rewrite in-place
 194 67020:23  000000014629aa23  [1,17,1]   Recovered via rewrite in-place
 195 67020:23  000000014629aa24  [3,11,0]   Require Write or Reassign Blocks command
 196 67020:23  000000014629aa25  [3,11,0]   Require Write or Reassign Blocks command
 197 67020:23  000000014629aa26  [3,11,0]   Require Write or Reassign Blocks command
 198 67020:23  000000014629aa27  [3,11,0]   Require Write or Reassign Blocks command
 199 67020:23  000000014629aa28  [3,11,0]   Require Write or Reassign Blocks command
 200 67020:24  000000014629aa29  [3,11,0]   Require Write or Reassign Blocks command
 201 67020:24  000000014629aa2a  [3,11,0]   Require Write or Reassign Blocks command
 202 67020:24  000000014629aa2b  [3,11,0]   Require Write or Reassign Blocks command
 203 67020:24  000000014629aa2c  [3,11,0]   Require Write or Reassign Blocks command
 204 67020:24  000000014629aa2d  [3,11,0]   Require Write or Reassign Blocks command
 205 67020:24  000000014629aa2e  [3,11,0]   Require Write or Reassign Blocks command
 206 67020:24  000000014629aa2f  [1,17,1]   Recovered via rewrite in-place
 207 67020:24  000000014629aa32  [1,17,1]   Recovered via rewrite in-place
 208 67020:24  000000014629aa33  [1,17,2]   Recovered via rewrite in-place
 209 67020:24  000000014629aa34  [1,17,1]   Recovered via rewrite in-place
 210 67020:24  000000014629aa35  [1,17,1]   Recovered via rewrite in-place
 211 67020:24  000000014629aa36  [3,11,0]   Require Write or Reassign Blocks command
 212 67020:24  000000014629aa37  [3,11,0]   Require Write or Reassign Blocks command
 213 67020:24  000000014629aa38  [1,17,1]   Recovered via rewrite in-place
 214 67020:24  000000014629aa39  [1,18,1]   Recovered via rewrite in-place
 215 67020:24  000000014629aa3b  [1,17,1]   Recovered via rewrite in-place
 216 67020:24  000000014629b29d  [1,18,1]   Recovered via rewrite in-place
 217 67020:24  000000014629b29e  [1,18,1]   Recovered via rewrite in-place
 218 67020:24  000000014629b29f  [1,17,1]   Recovered via rewrite in-place
 219 67020:24  000000014629b2a0  [1,17,1]   Recovered via rewrite in-place
 220 67020:24  000000014629b2a1  [3,11,0]   Require Write or Reassign Blocks command
 221 67020:24  000000014629b2a2  [1,17,1]   Recovered via rewrite in-place
 222 67020:24  000000014629b2a3  [1,17,1]   Recovered via rewrite in-place
 223 67020:24  000000014629b2a4  [1,17,1]   Recovered via rewrite in-place
 224 67020:24  000000014629b2a5  [1,18,1]   Recovered via rewrite in-place
 225 67020:24  000000014629b2a6  [3,11,0]   Require Write or Reassign Blocks command
 226 67020:24  000000014629b2a7  [1,17,1]   Recovered via rewrite in-place
 227 67020:25  000000014629b2a8  [1,17,1]   Recovered via rewrite in-place
 228 67020:25  000000014629b2a9  [1,17,1]   Recovered via rewrite in-place
 229 67020:25  000000014629b2aa  [1,17,1]   Recovered via rewrite in-place
 230 67020:25  000000014629bb13  [1,17,1]   Recovered via rewrite in-place
 231 67020:25  000000014629bb14  [3,11,0]   Require Write or Reassign Blocks command
 232 67020:25  000000014629bb15  [1,18,1]   Recovered via rewrite in-place
 233 67020:25  000000014629bb16  [1,17,1]   Recovered via rewrite in-place
 234 67020:25  000000014629bb17  [3,11,0]   Require Write or Reassign Blocks command
 235 67020:25  000000014629bb18  [1,17,1]   Recovered via rewrite in-place
 236 67020:25  000000014629bb19  [1,17,1]   Recovered via rewrite in-place
 237 67020:25  000000014629bb1a  [3,11,0]   Require Write or Reassign Blocks command
 238 67020:25  000000014629bb1b  [3,11,0]   Require Write or Reassign Blocks command
 239 67020:25  000000014629bb1c  [3,11,0]   Require Write or Reassign Blocks command
 240 67020:25  000000014629bb1d  [3,11,0]   Require Write or Reassign Blocks command
 241 67020:25  000000014629bb1e  [3,11,0]   Require Write or Reassign Blocks command
 242 67020:25  000000014629bb1f  [1,18,1]   Recovered via rewrite in-place
 243 67020:25  000000014629bb20  [1,17,1]   Recovered via rewrite in-place
 244 67020:25  000000014629bb21  [1,17,1]   Recovered via rewrite in-place
 245 67020:25  000000014629bb22  [3,11,0]   Require Write or Reassign Blocks command
 246 67020:25  000000014629bb23  [3,11,0]   Require Write or Reassign Blocks command
 247 67020:25  000000014629bb25  [3,11,0]   Require Write or Reassign Blocks command
 248 67020:25  000000014629bb26  [1,17,1]   Recovered via rewrite in-place
 249 67020:25  000000014629bb27  [1,18,1]   Recovered via rewrite in-place
 250 67020:25  000000014629bb28  [1,17,1]   Recovered via rewrite in-place
 251 67020:25  000000014629bb29  [1,17,1]   Recovered via rewrite in-place
 252 67020:25  000000014629bb2b  [1,17,1]   Recovered via rewrite in-place
 253 67020:26  000000014629c38b  [1,17,1]   Recovered via rewrite in-place
 254 67020:26  000000014629c38d  [1,17,1]   Recovered via rewrite in-place
 255 67020:26  000000014629c38f  [1,17,1]   Recovered via rewrite in-place
 256 67020:26  000000014629c390  [1,17,1]   Recovered via rewrite in-place
 257 67020:26  000000014629c392  [3,11,0]   Require Write or Reassign Blocks command
 258 67020:26  000000014629c393  [3,11,0]   Require Write or Reassign Blocks command
 259 67020:26  000000014629c394  [3,11,0]   Require Write or Reassign Blocks command
 260 67020:26  000000014629c395  [3,11,0]   Require Write or Reassign Blocks command
 261 67020:26  000000014629c396  [3,11,0]   Require Write or Reassign Blocks command
 262 67020:26  000000014629c397  [3,11,0]   Require Write or Reassign Blocks command
 263 67020:26  000000014629c398  [3,11,0]   Require Write or Reassign Blocks command
 264 67020:26  000000014629c399  [3,11,0]   Require Write or Reassign Blocks command
 265 67020:26  000000014629c39a  [3,11,0]   Require Write or Reassign Blocks command
 266 67020:26  000000014629c39b  [3,11,0]   Require Write or Reassign Blocks command
 267 67020:26  000000014629c39c  [3,11,0]   Require Write or Reassign Blocks command
 268 67020:26  000000014629c39d  [3,11,0]   Require Write or Reassign Blocks command
 269 67020:26  000000014629c39e  [3,11,0]   Require Write or Reassign Blocks command
 270 67020:26  000000014629c39f  [1,17,1]   Recovered via rewrite in-place
 271 67020:26  000000014629c3a0  [1,17,1]   Recovered via rewrite in-place
 272 67020:26  000000014629c3a1  [1,17,1]   Recovered via rewrite in-place
 273 67020:26  000000014629c3a3  [1,17,1]   Recovered via rewrite in-place
 274 67020:26  000000014629c3a4  [3,11,0]   Require Write or Reassign Blocks command
 275 67020:26  000000014629c3a5  [1,18,1]   Recovered via rewrite in-place
 276 67020:27  000000014629c3a6  [1,17,1]   Recovered via rewrite in-place
 277 67020:27  000000014629c3a7  [1,18,1]   Recovered via rewrite in-place
 278 67020:27  000000014629cbfa  [1,18,1]   Recovered via rewrite in-place
 279 67020:27  000000014629cbfc  [1,17,1]   Recovered via rewrite in-place
 280 67020:27  000000014629cbfd  [3,11,0]   Require Write or Reassign Blocks command
 281 67020:27  000000014629cbfe  [1,17,1]   Recovered via rewrite in-place
 282 67020:27  000000014629cbff  [1,17,1]   Recovered via rewrite in-place
 283 67020:27  000000014629cc01  [3,11,0]   Require Write or Reassign Blocks command
 284 67020:27  000000014629cc02  [1,17,1]   Recovered via rewrite in-place
 285 67020:27  000000014629cc04  [1,17,1]   Recovered via rewrite in-place
 286 67020:27  000000014629cc06  [1,17,1]   Recovered via rewrite in-place
 287 67020:27  000000014629cc07  [1,17,1]   Recovered via rewrite in-place
 288 67020:27  000000014629cc08  [1,18,1]   Recovered via rewrite in-place
 289 67020:27  000000014629cc09  [1,18,1]   Recovered via rewrite in-place
 290 67020:27  000000014629cc0b  [1,17,1]   Recovered via rewrite in-place
 291 67020:27  000000014629cc0c  [3,11,0]   Require Write or Reassign Blocks command
 292 67020:27  000000014629cc0d  [3,11,0]   Require Write or Reassign Blocks command
 293 67020:27  000000014629cc0e  [3,11,0]   Require Write or Reassign Blocks command
 294 67020:27  000000014629cc0f  [1,17,1]   Recovered via rewrite in-place
 295 67020:27  000000014629cc18  [1,17,1]   Recovered via rewrite in-place
 296 67020:27  000000014629cc19  [1,17,1]   Recovered via rewrite in-place
 297 67020:27  000000014629cc1a  [1,17,1]   Recovered via rewrite in-place
 298 67020:27  000000014629d480  [1,18,1]   Recovered via rewrite in-place
 299 67020:27  000000014629d481  [1,18,1]   Recovered via rewrite in-place
 300 67020:27  000000014629d484  [1,17,1]   Recovered via rewrite in-place
 301 67020:27  000000014629d485  [1,17,1]   Recovered via rewrite in-place
 302 67020:27  000000014629d486  [1,17,1]   Recovered via rewrite in-place
 303 67020:27  000000014629d487  [1,17,1]   Recovered via rewrite in-place
 304 67020:27  000000014629d488  [3,11,0]   Require Write or Reassign Blocks command
 305 67020:27  000000014629d489  [1,18,1]   Recovered via rewrite in-place
 306 67020:28  000000014629d48a  [1,17,1]   Recovered via rewrite in-place
 307 67020:28  000000014629d48b  [3,11,0]   Require Write or Reassign Blocks command
 308 67020:28  000000014629d48c  [1,17,1]   Recovered via rewrite in-place
 309 67020:28  000000014629d48d  [1,17,1]   Recovered via rewrite in-place
 310 67020:28  000000014629d48e  [1,17,1]   Recovered via rewrite in-place
 311 67020:28  000000014629d48f  [1,18,1]   Recovered via rewrite in-place
 312 67020:28  000000014629dcf8  [1,18,1]   Recovered via rewrite in-place
 313 67020:28  000000014629dcfa  [1,17,1]   Recovered via rewrite in-place
 314 67020:28  000000014629dcfc  [1,18,1]   Recovered via rewrite in-place
 315 67020:28  000000014629dcfd  [1,17,1]   Recovered via rewrite in-place
 316 67020:28  000000014629dcfe  [1,17,1]   Recovered via rewrite in-place
 317 67020:28  000000014629dcff  [1,17,1]   Recovered via rewrite in-place
 318 67020:28  000000014629dd00  [1,17,1]   Recovered via rewrite in-place
 319 67020:28  000000014629dd01  [1,17,1]   Recovered via rewrite in-place
 320 67020:28  000000014629dd02  [1,17,1]   Recovered via rewrite in-place
 321 67020:28  000000014629dd03  [1,17,1]   Recovered via rewrite in-place
 322 67020:28  000000014629dd04  [1,17,1]   Recovered via rewrite in-place
 323 67020:28  000000014629dd05  [1,17,1]   Recovered via rewrite in-place
 324 67020:28  000000014629e576  [1,17,1]   Recovered via rewrite in-place
 325 67020:28  000000014629e577  [1,17,1]   Recovered via rewrite in-place
 326 67020:28  000000014629e578  [1,17,1]   Recovered via rewrite in-place
 327 67020:28  000000014629e57a  [1,17,1]   Recovered via rewrite in-place
 328 67020:28  000000014629e57b  [1,17,3]   Recovered via rewrite in-place
 329 67020:28  000000014629e57c  [1,17,1]   Recovered via rewrite in-place
 330 67020:28  000000014629e57d  [1,17,1]   Recovered via rewrite in-place
 331 67020:28  000000014629e57e  [3,11,0]   Require Write or Reassign Blocks command
 332 67020:28  000000014629e57f  [1,18,1]   Recovered via rewrite in-place
 333 67020:28  000000014629e586  [1,18,1]   Recovered via rewrite in-place
 334 67020:28  000000014629f66b  [1,17,1]   Recovered via rewrite in-place
 335 67020:28  000000014629f66c  [1,17,1]   Recovered via rewrite in-place
 336 67020:28  000000014629f66d  [1,17,1]   Recovered via rewrite in-place
 337 67020:28  000000014629f66e  [1,17,1]   Recovered via rewrite in-place
 338 67020:28  000000014629f670  [1,17,1]   Recovered via rewrite in-place
 339 67020:28  000000014629f671  [1,17,1]   Recovered via rewrite in-place
 340 67020:28  000000014629f672  [1,18,1]   Recovered via rewrite in-place
 341 67020:28  000000014629f673  [1,17,1]   Recovered via rewrite in-place
 342 67020:28  000000014629fee1  [1,17,1]   Recovered via rewrite in-place
 343 67020:29  000000014629fee3  [1,17,1]   Recovered via rewrite in-place
 344 67020:29  000000014629fee4  [1,17,1]   Recovered via rewrite in-place
 345 67020:29  000000014629fee5  [1,17,1]   Recovered via rewrite in-place
 346 67020:29  000000014629fee6  [1,17,2]   Recovered via rewrite in-place
 347 67020:29  000000014629fee7  [1,17,1]   Recovered via rewrite in-place
 348 67020:29  000000014629feed  [1,18,1]   Recovered via rewrite in-place
 349 67020:29  000000014629feef  [1,18,1]   Recovered via rewrite in-place
 350 67020:29  00000001462a0752  [1,17,1]   Recovered via rewrite in-place
 351 67020:29  00000001462a0753  [1,17,1]   Recovered via rewrite in-place
 352 67020:29  00000001462a0754  [1,17,1]   Recovered via rewrite in-place
 353 67020:29  00000001462a0756  [1,17,1]   Recovered via rewrite in-place
 354 67020:29  00000001462a0757  [1,18,1]   Recovered via rewrite in-place
 355 67020:29  00000001462a0758  [1,18,1]   Recovered via rewrite in-place
 356 67020:29  00000001462a0759  [1,17,1]   Recovered via rewrite in-place
 357 67020:29  00000001462a075a  [1,17,1]   Recovered via rewrite in-place
 358 67020:29  00000001462a075c  [1,18,1]   Recovered via rewrite in-place
 359 67020:29  00000001462a075f  [1,17,1]   Recovered via rewrite in-place
 360 67020:29  00000001462a0760  [3,11,0]   Require Write or Reassign Blocks command
 361 67020:29  00000001462a0761  [3,11,0]   Require Write or Reassign Blocks command
 362 67020:29  00000001462a0762  [1,18,1]   Recovered via rewrite in-place
 363 67020:29  00000001462a0763  [1,18,1]   Recovered via rewrite in-place
 364 67020:29  00000001462a0764  [3,11,0]   Require Write or Reassign Blocks command
 365 67020:29  00000001462a0765  [3,11,0]   Require Write or Reassign Blocks command
 366 67020:29  00000001462a0766  [3,11,0]   Require Write or Reassign Blocks command
 367 67020:29  00000001462a0767  [1,17,1]   Recovered via rewrite in-place
 368 67020:29  00000001462a0768  [3,11,0]   Require Write or Reassign Blocks command
 369 67020:29  00000001462a0769  [1,18,1]   Recovered via rewrite in-place
 370 67020:29  00000001462a076a  [1,17,1]   Recovered via rewrite in-place
 371 67020:29  00000001462a076c  [1,17,1]   Recovered via rewrite in-place
 372 67020:29  00000001462a076e  [1,17,1]   Recovered via rewrite in-place
 373 67020:29  00000001462a076f  [1,18,1]   Recovered via rewrite in-place
 374 67020:29  00000001462a0fd5  [1,18,1]   Recovered via rewrite in-place
 375 67020:29  00000001462a0fd6  [1,17,1]   Recovered via rewrite in-place
 376 67020:29  00000001462a0fd7  [1,18,1]   Recovered via rewrite in-place
 377 67020:29  00000001462a0fd9  [1,17,1]   Recovered via rewrite in-place
 378 67020:29  00000001462a0fdb  [1,17,1]   Recovered via rewrite in-place
 379 67020:29  00000001462a0fdc  [1,18,1]   Recovered via rewrite in-place
 380 67020:29  00000001462a0fdd  [1,18,1]   Recovered via rewrite in-place
 381 67020:29  00000001462a0fdf  [1,18,1]   Recovered via rewrite in-place
 382 67020:29  00000001462a0fe0  [1,17,1]   Recovered via rewrite in-place
 383 67020:30  00000001462a0fe1  [1,17,1]   Recovered via rewrite in-place
 384 67020:30  00000001462a2993  [1,17,1]   Recovered via rewrite in-place
 385 67020:30  00000001462a2995  [1,18,1]   Recovered via rewrite in-place
 386 67020:30  00000001462a2997  [1,17,2]   Recovered via rewrite in-place
 387 67020:30  00000001462a2999  [1,17,1]   Recovered via rewrite in-place
 388 67020:30  00000001462a299b  [1,18,1]   Recovered via rewrite in-place
 389 67020:30  00000001462a299d  [1,17,1]   Recovered via rewrite in-place
 390 67020:30  00000001462a299e  [1,18,1]   Recovered via rewrite in-place
 391 67020:30  00000001462a3220  [1,17,1]   Recovered via rewrite in-place
 392 67132:36  0000000146290c1e  [1,18,1]   Recovered via rewrite in-place
 393 67132:36  0000000146295ddf  [1,17,1]   Recovered via rewrite in-place
 394 67132:36  0000000146296668  [1,17,2]   Recovered via rewrite in-place
 395 67132:36  000000014629666a  [1,18,1]   Recovered via rewrite in-place
 396 67132:36  0000000146296eca  [1,17,1]   Recovered via rewrite in-place
 397 67132:36  0000000146296ed3  [1,17,1]   Recovered via rewrite in-place
 398 67132:36  000000014629883a  [1,17,1]   Recovered via rewrite in-place
 399 67132:36  000000014629a1ae  [1,17,1]   Recovered via rewrite in-place
 400 67132:36  000000014629a1b1  [1,18,1]   Recovered via rewrite in-place
 401 67132:36  000000014629a1b7  [1,17,1]   Recovered via rewrite in-place
 402 67132:36  000000014629aa3a  [1,18,1]   Recovered via rewrite in-place
 403 67132:36  000000014629bb2c  [1,17,3]   Recovered via rewrite in-place
 404 67132:36  000000014629c391  [1,17,1]   Recovered via rewrite in-place
 405 67132:36  000000014629cc00  [1,18,1]   Recovered via rewrite in-place
 406 67132:37  000000014629cc03  [1,18,1]   Recovered via rewrite in-place
 407 67132:37  000000014629d47e  [1,18,1]   Recovered via rewrite in-place
 408 67132:37  000000014629e584  [1,17,1]   Recovered via rewrite in-place
 409 67132:37  000000014629e587  [1,17,1]   Recovered via rewrite in-place
 410 67132:37  000000014629e588  [1,17,1]   Recovered via rewrite in-place
 411 67132:37  000000014629f66a  [1,18,1]   Recovered via rewrite in-place
 412 67132:37  000000014629f674  [1,17,3]   Recovered via rewrite in-place
 413 67132:37  00000001462a0fd2  [1,17,1]   Recovered via rewrite in-place
 414 67132:37  00000001462a2119  [1,17,3]   Recovered via rewrite in-place
 415 67169:15  000000014629883c  [1,17,1]   Recovered via rewrite in-place
 416 67169:15  00000001462990b7  [1,17,1]   Recovered via rewrite in-place
 417 67169:15  000000014629aa30  [1,17,1]   Recovered via rewrite in-place
 418 67169:15  000000014629aa31  [1,17,1]   Recovered via rewrite in-place
 419 67169:15  000000014629d47f  [1,17,1]   Recovered via rewrite in-place
 420 67169:15  000000014629e585  [1,17,1]   Recovered via rewrite in-place
 421 67169:15  00000001462a075b  [1,18,1]   Recovered via rewrite in-place
 422 67169:15  00000001462a075d  [1,17,1]   Recovered via rewrite in-place
 423 67169:15  00000001462a075e  [1,18,1]   Recovered via rewrite in-place
 424 67169:15  00000001462a076d  [1,17,1]   Recovered via rewrite in-place
 425 67246:20  0000000146295de8  [1,17,3]   Recovered via rewrite in-place
 426 67246:20  000000014629dcf9  [1,18,1]   Recovered via rewrite in-place
 427 67246:20  00000001462a076b  [1,17,1]   Recovered via rewrite in-place
 428 67322:58  0000000146290f4a  [1,18,1]   Recovered via rewrite in-place
 429 67322:58  000000014629c38a  [1,18,1]   Recovered via rewrite in-place
 430 67322:58  000000014629d490  [1,17,1]   Recovered via rewrite in-place
 431 67416:19  00000001462a0fe2  [1,17,1]   Recovered via rewrite in-place
 432 67416:19  00000001462a3219  [1,17,1]   Recovered via rewrite in-place
 433 67416:19  00000001462a321a  [1,18,1]   Recovered via rewrite in-place
 434 67416:19  00000001462a321e  [1,17,1]   Recovered via rewrite in-place
 435 67449:29  0000000146297fbe  [1,17,3]   Recovered via rewrite in-place
 436 67449:30  000000014629aa18  [1,17,1]   Recovered via rewrite in-place
 437 67449:30  000000014629b297  [1,17,1]   Recovered via rewrite in-place
 438 67449:55  000000015bfaad5a  [1,17,1]   Successfully reassigned
 439 67521:29  000000014629556e  [1,17,2]   Recovered via rewrite in-place
 440 67521:29  000000014629bb2a  [1,18,1]   Recovered via rewrite in-place
 441 67521:29  000000014629dd0a  [1,18,1]   Recovered via rewrite in-place
 442 67521:29  00000001462a0fd3  [1,18,1]   Recovered via rewrite in-place
 443 67593:29  00000001462910ca  [1,17,1]   Recovered via rewrite in-place
 444 67593:29  00000001462a185f  [1,18,1]   Recovered via rewrite in-place
 445 67665:29  0000000146290c1d  [1,18,1]   Recovered via rewrite in-place
 446 67665:29  0000000146295de0  [1,18,1]   Recovered via rewrite in-place
 447 67665:29  000000014629aa1b  [1,18,1]   Recovered via rewrite in-place
 448 67665:29  000000014629dd07  [1,17,3]   Recovered via rewrite in-place
 449 67665:29  000000014629e572  [1,17,3]   Recovered via rewrite in-place
 450 67665:29  00000001462a321f  [1,17,1]   Recovered via rewrite in-place
 451 67737:29  0000000146296ebf  [1,17,1]   Recovered via rewrite in-place
 452 67737:29  000000014629b29a  [1,18,1]   Recovered via rewrite in-place
 453 67737:29  00000001462a211b  [1,17,1]   Recovered via rewrite in-place
 454 67809:29  000000014629a1aa  [1,18,1]   Recovered via rewrite in-place
 455 67809:29  00000001462a211c  [1,17,1]   Recovered via rewrite in-place
 456 67809:29  00000001462a211f  [1,18,1]   Recovered via rewrite in-place
 457 67881:28  000000014629555a  [1,17,1]   Recovered via rewrite in-place
 458 67881:28  0000000146296ec9  [1,18,1]   Recovered via rewrite in-place
 459 67881:29  00000001462a2996  [1,17,1]   Recovered via rewrite in-place
 460 67953:29  0000000146296ee0  [1,17,1]   Recovered via rewrite in-place
 461 67953:29  000000014629aa15  [1,17,1]   Recovered via rewrite in-place
 462 67953:29  000000014629aa17  [1,17,1]   Recovered via rewrite in-place
 463 67953:29  000000014629b2ab  [1,18,1]   Recovered via rewrite in-place
 464 67953:29  000000014629bb11  [1,17,3]   Recovered via rewrite in-place
 465 67953:29  000000014629bb12  [1,17,1]   Recovered via rewrite in-place
 466 67953:29  000000014629edfa  [1,18,1]   Recovered via rewrite in-place
 467 68025:29  0000000146290c1c  [1,17,1]   Recovered via rewrite in-place
 468 68097:28  000000014629d482  [1,17,1]   Recovered via rewrite in-place
 469 68169:28  00000001462a3213  [1,18,1]   Recovered via rewrite in-place
 470 68385:28  000000014629aa13  [1,18,1]   Recovered via rewrite in-place
 471 68529:28  0000000146297fbd  [1,18,1]   Recovered via rewrite in-place
 472 68529:28  000000014629aa14  [1,17,3]   Recovered via rewrite in-place
 473 68601:28  00000001462a0fda  [1,18,1]   Recovered via rewrite in-place
 474 68673:27  0000000146298830  [1,18,1]   Recovered via rewrite in-place
 475 68673:27  000000014629e583  [1,18,1]   Recovered via rewrite in-place
 476 68745:27  000000014629aa16  [1,18,1]   Recovered via rewrite in-place
 477 68745:27  000000014629fee0  [1,17,1]   Recovered via rewrite in-place
 478 68745:27  00000001462a074f  [1,17,1]   Recovered via rewrite in-place
 479 68889:27  00000001462a211a  [1,17,2]   Recovered via rewrite in-place
 480 69033:27  0000000146297fb5  [1,18,1]   Recovered via rewrite in-place
 481 69177:27  00000001462a211e  [1,17,1]   Recovered via rewrite in-place
 482 69396:08  000000014629e575  [1,17,1]   Recovered via rewrite in-place
 483 69540:08  00000001462a2998  [1,17,1]   Recovered via rewrite in-place
 484 69756:08  000000014629edf7  [1,17,1]   Recovered via rewrite in-place
 485 69828:07  00000001462990b1  [1,18,1]   Recovered via rewrite in-place
 486 69828:08  00000001462990b2  [1,18,1]   Recovered via rewrite in-place
 487 69828:08  00000001462990b3  [1,18,1]   Recovered via rewrite in-place
 488 69879:43  0000000146295de2  [1,17,1]   Recovered via rewrite in-place
 489 69879:43  000000014629aa3c  [1,17,1]   Recovered via rewrite in-place
 490 70023:43  000000014629cbf4  [1,17,1]   Recovered via rewrite in-place
 491 70389:59  000000014629c386  [1,18,1]   Recovered via rewrite in-place
 492 70461:59  00000001462a074e  [1,17,1]   Recovered via rewrite in-place
 493 72953:32  000000014629f677  [1,17,1]   Recovered via rewrite in-place
 494 73335:23  00000001462a2991  [1,18,1]   Recovered via rewrite in-place
 495 73479:23  0000000146290f4e  [1,18,1]   Recovered via rewrite in-place
 496 73479:23  000000014629883b  [1,17,1]   Recovered via rewrite in-place
 497 73983:23  0000000146297fb8  [1,17,1]   Recovered via rewrite in-place
 498 74374:28  000000014629e582  [1,18,1]   Recovered via rewrite in-place
 499 75239:22  000000014629e581  [1,17,1]   Recovered via rewrite in-place
 500 75741:56  00000001462a0fd4  [1,17,1]   Recovered via rewrite in-place
 501 77841:55  000000014629d491  [1,17,1]   Recovered via rewrite in-place
 502 77963:01  00000000000b4b1f  [4,9,0]   Successfully reassigned
 503 77963:01  00000000000b4b20  [4,9,0]   Successfully reassigned
 504 77963:01  00000000000b4b21  [4,9,0]   Successfully reassigned
 505 77963:01  00000000000b4b22  [4,9,0]   Successfully reassigned
 506 77963:01  00000000000b4b23  [4,9,0]   Successfully reassigned
 507 77963:01  00000000000b4b24  [4,9,0]   Successfully reassigned
 508 77963:01  00000000000b4b25  [4,9,0]   Successfully reassigned
 509 77963:01  00000000000b4b26  [4,9,0]   Successfully reassigned
 510 77963:01  00000000000b4b27  [4,9,0]   Successfully reassigned
 511 77963:01  00000000000b4b28  [4,9,0]   Successfully reassigned
 512 77963:01  00000000000b4b29  [4,9,0]   Successfully reassigned
 513 77963:01  00000000000b4b2a  [4,9,0]   Successfully reassigned
 514 77963:01  00000000000b4b2b  [4,9,0]   Successfully reassigned
 515 77963:01  00000000000b4b2c  [4,9,0]   Successfully reassigned
 516 77963:01  00000000000b4b2d  [4,9,0]   Successfully reassigned
 517 77963:01  00000000000b626a  [4,9,0]   Successfully reassigned
 518 77963:01  00000000000b626b  [4,9,0]   Successfully reassigned
 519 77963:01  00000000000b626c  [4,9,0]   Successfully reassigned
 520 77963:01  00000000000b626d  [4,9,0]   Successfully reassigned
 521 77963:01  00000000000b626e  [4,9,0]   Successfully reassigned
 522 77963:01  00000000000b626f  [4,9,0]   Successfully reassigned
 523 77963:01  00000000000b6270  [4,9,0]   Successfully reassigned
 524 77963:01  00000000000b6271  [4,9,0]   Successfully reassigned
 525 77963:01  00000000000b6272  [4,9,0]   Successfully reassigned
 526 77963:01  00000000000b6273  [4,9,0]   Successfully reassigned
 527 77963:01  00000000000b6274  [4,9,0]   Successfully reassigned
 528 77963:01  00000000000b6275  [4,9,0]   Successfully reassigned
 529 77963:01  00000000000b6276  [4,9,0]   Successfully reassigned
 530 77963:01  00000000000b6277  [4,9,0]   Successfully reassigned
 531 77963:02  00000000000b6278  [4,9,0]   Successfully reassigned
 532 77963:02  00000000000bc98d  [4,9,0]   Successfully reassigned
 533 77963:02  00000000000bc98e  [4,9,0]   Successfully reassigned
 534 77963:02  00000000000bc98f  [4,9,0]   Successfully reassigned
 535 77963:02  00000000000bc990  [4,9,0]   Successfully reassigned
 536 77963:02  00000000000bc991  [4,9,0]   Successfully reassigned
 537 77963:02  00000000000bc992  [4,9,0]   Successfully reassigned
 538 77963:02  00000000000bc993  [4,9,0]   Successfully reassigned
 539 77963:02  00000000000bc99b  [4,9,0]   Successfully reassigned
 540 77963:02  00000000000bc99c  [4,9,0]   Successfully reassigned
 541 77963:02  00000000000bc99d  [4,9,0]   Successfully reassigned
 542 77963:02  00000000000bc9a9  [4,9,0]   Successfully reassigned
 543 77963:02  00000000000bc9aa  [4,9,0]   Successfully reassigned
 544 77963:02  00000000000bc9ab  [4,9,0]   Successfully reassigned
 545 77963:02  00000000000bc9ac  [4,9,0]   Successfully reassigned
 546 77963:02  00000000000bc9ad  [4,9,0]   Successfully reassigned
 547 77963:02  00000000000bc9ae  [4,9,0]   Successfully reassigned
 548 77963:02  00000000000bc9af  [4,9,0]   Successfully reassigned
 549 77963:02  00000000000bc9b0  [4,9,0]   Successfully reassigned
 550 77963:02  00000000000bca3c  [4,9,0]   Successfully reassigned
 551 77985:55  0000000146296ec3  [1,17,1]   Recovered via rewrite in-place
 552 78129:55  000000014629a1b5  [1,17,1]   Recovered via rewrite in-place
 553   14:25  000000014629d483  [1,17,1]   Recovered via rewrite in-place
 554  158:25  0000000146297fbc  [1,17,1]   Recovered via rewrite in-place
 555  374:25  000000014629882e  [1,17,1]   Recovered via rewrite in-place
 556  518:25  0000000146290f20  [1,18,1]   Recovered via rewrite in-place
 557  851:35  000000014629a1b4  [1,18,1]   Recovered via rewrite in-place
 558  995:35  0000000146298831  [1,17,2]   Recovered via rewrite in-place
 559  995:35  00000001462a2992  [1,17,1]   Recovered via rewrite in-place
 560 1139:35  0000000146295df7  [1,17,1]   Recovered via rewrite in-place
 561 1427:34  00000001462a321b  [1,17,2]   Recovered via rewrite in-place
 562 2363:33  000000014629c38e  [1,17,1]   Recovered via rewrite in-place
 563 2741:04  0000000146296659  [1,17,1]   Recovered via rewrite in-place
 564 3461:03  0000000146295b91  [1,17,1]   Recovered via rewrite in-place
 565 3677:03  000000014629c38c  [1,18,1]   Recovered via rewrite in-place
 566 5261:01  000000014629e574  [1,17,1]   Recovered via rewrite in-place
 567 6700:59  000000014629e580  [1,17,1]   Recovered via rewrite in-place
 568 6700:59  00000001462a0fe7  [1,18,1]   Recovered via rewrite in-place
 569 6844:59  000000014629bb10  [1,17,1]   Recovered via rewrite in-place
 570 6916:59  000000014629cc1b  [1,17,1]   Recovered via rewrite in-place
 571 6988:59  000000014629f665  [1,18,1]   Recovered via rewrite in-place
 572 7060:59  000000014629dd09  [1,18,1]   Recovered via rewrite in-place
 573 7060:59  00000001462a0751  [1,18,1]   Recovered via rewrite in-place
 574 7204:59  00000001462a0fe4  [1,18,1]   Recovered via rewrite in-place
 575 7276:59  000000014629cc1c  [1,17,1]   Recovered via rewrite in-place
 576 7276:59  000000014629f676  [1,18,1]   Recovered via rewrite in-place
 577 7492:58  000000014629bb24  [1,17,1]   Recovered via rewrite in-place
 578 7708:58  0000000146295dc3  [1,18,1]   Recovered via rewrite in-place
 579 7708:58  0000000146297fd6  [1,17,1]   Recovered via rewrite in-place
 580 7853:28  00000001462a210a  [1,17,1]   Recovered via rewrite in-place
 581 7925:28  0000000146296ebe  [1,18,1]   Recovered via rewrite in-place
 582 7997:28  000000014629882f  [1,17,1]   Recovered via rewrite in-place
 583 8069:28  000000014629d492  [1,17,1]   Recovered via rewrite in-place
 584 8069:28  00000001462a210f  [1,18,1]   Recovered via rewrite in-place
 585 8146:28  000000014629555c  [1,17,1]   Recovered via rewrite in-place
 586 8146:28  000000014629664f  [1,18,1]   Recovered via rewrite in-place
 587 8146:28  0000000146296669  [1,18,1]   Recovered via rewrite in-place
 588 8146:28  00000001462a210e  [1,17,3]   Recovered via rewrite in-place
 589 8357:28  0000000146296654  [1,17,1]   Recovered via rewrite in-place
 590 8357:28  0000000146296666  [1,18,1]   Recovered via rewrite in-place
 591 8429:28  000000014629c389  [1,18,1]   Recovered via rewrite in-place
 592 8543:14  0000000146290c18  [1,17,1]   Recovered via rewrite in-place
 593 8615:14  0000000146297fc4  [1,17,1]   Recovered via rewrite in-place
 594 8615:14  000000014629f666  [1,17,1]   Recovered via rewrite in-place
 595 8687:14  000000014629884d  [1,17,1]   Recovered via rewrite in-place
 596 8759:14  00000001462a210d  [1,18,1]   Recovered via rewrite in-place
 597 8975:14  000000014629edf0  [1,17,1]   Recovered via rewrite in-place
 598 8975:14  00000001462a2118  [1,18,1]   Recovered via rewrite in-place
 599 9472:47  0000000147042400  [1,18,1]   Recovered via rewrite in-place
 600 9832:45  00000001462a299c  [1,18,1]   Recovered via rewrite in-place
 601 10032:49  000000014629aa1a  [1,18,1]   Recovered via rewrite in-place
 602 10176:49  000000014629a1ad  [1,18,1]   Recovered via rewrite in-place
 603 10176:49  000000014629dd08  [1,18,1]   Recovered via rewrite in-place
 604 10608:49  00000001462a0750  [1,17,1]   Recovered via rewrite in-place
 605 10680:49  00000001462910c8  [1,18,1]   Recovered via rewrite in-place
 606 11184:48  000000014629555b  [1,17,1]   Recovered via rewrite in-place
 607 11256:48  000000014629fedc  [1,18,1]   Recovered via rewrite in-place
 608 11688:47  000000014629cc1e  [1,18,1]   Recovered via rewrite in-place
 609 12106:50  000000014629fee9  [1,17,1]   Recovered via rewrite in-place
 610 12322:56  000000014629edf9  [1,18,1]   Recovered via rewrite in-place
 611 12394:50  0000000146296655  [1,17,1]   Recovered via rewrite in-place
 612 12394:50  000000014629dcef  [1,17,1]   Recovered via rewrite in-place
 613 12394:50  000000014629fedf  [1,18,1]   Recovered via rewrite in-place
 614 12435:26  000000014629feea  [1,18,1]   Recovered via rewrite in-place
 615 12580:18  0000000172afcc93  [1,17,2]   Recovered via rewrite in-place
 616 12580:25  00000001787993ee  [1,18,1]   Recovered via rewrite in-place
 617 12581:06  0000000198516995  [1,18,1]   Recovered via rewrite in-place
 618 12581:11  000000019b84208e  [1,18,1]   Recovered via rewrite in-place
 619 12581:31  00000001a9ac9405  [1,18,1]   Successfully reassigned
 620 12581:31  00000001a9ac9406  [1,18,1]   Successfully reassigned
 621 12581:31  00000001a9ac9407  [1,18,1]   Successfully reassigned
 622 12581:31  00000001a9ac9408  [1,18,1]   Successfully reassigned
 623 12581:31  00000001a9ac9409  [1,18,1]   Recovered via rewrite in-place
 624 12581:45  00000001b399ad91  [1,17,1]   Recovered via rewrite in-place
 625 12647:35  0000000058cce0ce  [1,18,1]   Successfully reassigned
 626 12647:35  0000000058cce0cf  [1,18,1]   Successfully reassigned
 627 12647:36  0000000058cce0d0  [1,18,1]   Successfully reassigned
 628 12647:36  0000000059deceff  [1,18,1]   Successfully reassigned
 629 12647:36  0000000059decf00  [1,17,1]   Recovered via rewrite in-place
 630 12647:38  000000005b59b23f  [1,17,1]   Recovered via rewrite in-place
 631 12648:05  000000007ab20376  [1,18,1]   Successfully reassigned
 632 12648:15  0000000085f470e8  [1,17,3]   Recovered via rewrite in-place
 633 12649:43  00000000e378fa9b  [1,18,1]   Successfully reassigned
 634 12651:05  0000000132d23579  [1,18,1]   Recovered via rewrite in-place
 635 12651:20  0000000141001ed8  [1,18,1]   Successfully reassigned
 636 12651:22  00000001424c7d42  [1,17,1]   Recovered via rewrite in-place
 637 12651:26  0000000146296eba  [1,17,1]   Recovered via rewrite in-place
 638 12651:26  0000000146296ec1  [1,17,1]   Recovered via rewrite in-place
 639 12651:26  0000000146296ec2  [1,18,1]   Recovered via rewrite in-place
 640 12651:26  000000014629f667  [1,17,1]   Recovered via rewrite in-place
 641 12651:31  000000014aa91d74  [1,17,1]   Recovered via rewrite in-place
 642 12651:44  0000000155d34539  [1,17,1]   Recovered via rewrite in-place
 643 12651:47  00000001585da0b5  [1,17,1]   Recovered via rewrite in-place
 644 12652:20  0000000173e459d9  [1,17,1]   Recovered via rewrite in-place
 645 12652:24  0000000176f55511  [1,17,2]   Recovered via rewrite in-place
 646 12719:37  000000005a3cfaba  [1,17,1]   Recovered via rewrite in-place
 647 12720:03  0000000078a34d64  [1,17,1]   Successfully reassigned
 648 12720:03  0000000078a34d65  [1,18,1]   Successfully reassigned
 649 12720:03  0000000078a34d66  [1,18,1]   Successfully reassigned
 650 12720:03  0000000078a34d67  [1,17,1]   Recovered via rewrite in-place
 651 12720:43  00000000a4946d2f  [1,16,0]   Recovered via rewrite in-place
 652 12721:21  00000000cd3c1571  [1,17,2]   Recovered via rewrite in-place
 653 12722:50  0000000125133729  [1,17,1]   Recovered via rewrite in-place
 654 12898:14  000000014629bb0c  [1,17,1]   Recovered via rewrite in-place
 655 12970:14  000000014629edef  [1,17,1]   Recovered via rewrite in-place
 656 12970:14  000000014629feda  [1,18,1]   Recovered via rewrite in-place
 657 12972:40  00000001b7d5b718  [1,18,1]   Successfully reassigned
 658 13041:23  0000000116853e8c  [1,18,1]   Successfully reassigned
 659 13041:23  0000000116853e8d  [1,17,1]   Recovered via rewrite in-place
 660 13042:14  000000014629cbfb  [1,18,1]   Recovered via rewrite in-place
 661 13043:06  0000000172579ff8  [1,17,2]   Recovered via rewrite in-place
 662 13043:35  0000000189a32888  [1,18,1]   Recovered via rewrite in-place
 663 13043:48  0000000193528096  [1,17,3]   Recovered via rewrite in-place
 664 13044:03  000000019e44414b  [1,18,1]   Successfully reassigned
 665 13044:03  000000019e44414c  [1,18,1]   Successfully reassigned
 666 13044:03  000000019e44414d  [1,18,1]   Recovered via rewrite in-place
 667 13044:05  000000019fc7b8e3  [1,18,1]   Successfully reassigned
 668 13044:05  000000019fc7b8e4  [1,17,1]   Recovered via rewrite in-place
 669 13114:22  000000014629aa19  [1,17,1]   Recovered via rewrite in-place
 670 13116:59  00000001bd3ed913  [1,18,1]   Recovered via rewrite in-place
 671 13116:59  00000001bd3f5dc9  [1,18,1]   Successfully reassigned
 >>>> log truncated, fetched 16124 of 16964 available bytes

Protocol Specific port log page for SAS SSP
relative target port id = 1
  generation code = 0
  number of phys = 1
  phy identifier = 0
    attached device type: expander device
    attached reason: SMP phy control function
    reason: power on
    negotiated logical link rate: phy enabled; 6 Gbps
    attached initiator port: ssp=0 stp=0 smp=0
    attached target port: ssp=0 stp=0 smp=1
    SAS address = 0x5000c50055e501d9
    attached SAS address = 0x50030480011d203f
    attached phy identifier = 32
    Invalid DWORD count = 0
    Running disparity error count = 0
    Loss of DWORD synchronization = 0
    Phy reset problem = 0
    Phy event descriptors:
     Invalid word count: 0
     Running disparity error count: 0
     Loss of dword synchronization count: 0
     Phy reset problem count: 0
relative target port id = 2
  generation code = 0
  number of phys = 1
  phy identifier = 1
    attached device type: no device attached
    attached reason: unknown
    reason: unknown
    negotiated logical link rate: phy enabled; unknown
    attached initiator port: ssp=0 stp=0 smp=0
    attached target port: ssp=0 stp=0 smp=0
    SAS address = 0x5000c50055e501da
    attached SAS address = 0x0
    attached phy identifier = 0
    Invalid DWORD count = 0
    Running disparity error count = 0
    Loss of DWORD synchronization = 0
    Phy reset problem = 0
    Phy event descriptors:
     Invalid word count: 0
     Running disparity error count: 0
     Loss of dword synchronization count: 0
     Phy reset problem count: 0

root@freenas[~]#

[xbmcgotham]

And

root@freenas[~]# smartctl -x /dev/da18
smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               SEAGATE
Product:              ST4000NM0023
Revision:             0003
Compliance:           SPC-4
User Capacity:        4,000,787,030,016 bytes [4.00 TB]
Logical block size:   512 bytes
LU is fully provisioned
Rotation Rate:        7200 rpm
Form Factor:          3.5 inches
Logical Unit id:      0x5000c50058c25dd7
Serial number:        Z1Z4CWCR00009444VDPP
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Sat Mar  1 11:30:02 2025 GMT
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Enabled
Read Cache is:        Enabled
Writeback Cache is:   Disabled

=== START OF READ SMART DATA SECTION ===
SMART Health Status: DATA CHANNEL IMPENDING FAILURE DATA ERROR RATE TOO HIGH [asc=5d, ascq=32]

Current Drive Temperature:     42 C
Drive Trip Temperature:        40 C

Manufactured in week 21 of year 2014
Specified cycle count over device lifetime:  10000
Accumulated start-stop cycles:  203
Specified load-unload count over device lifetime:  300000
Accumulated load-unload cycles:  3535
Elements in grown defect list: 16367

Vendor (Seagate Cache) information
  Blocks sent to initiator = 2926444274
  Blocks received from initiator = 2181297869
  Blocks read from cache and sent to initiator = 3099646889
  Number of read and write commands whose size <= segment size = 2696453622
  Number of read and write commands whose size > segment size = 0

Vendor (Seagate/Hitachi) factory information
  number of hours powered up = 1525.18
  number of minutes until next internal SMART test = 1

Error counter log:
           Errors Corrected by           Total   Correction     Gigabytes    Total
               ECC          rereads/    errors   algorithm      processed    uncorrected
           fast | delayed   rewrites  corrected  invocations   [10^9 bytes]  errors
read:   4221751909    20006         0  4221771915      20006     555652.200      0
write:         0        0         0         0          0     570139.593  0
verify:     1547        0         0      1547          0          0.000  0

Non-medium error count:       66


[GLTSD (Global Logging Target Save Disable) set. Enable Save with '-S on']
SMART Self-test log
Num  Test              Status                 segment  LifeTime  LBA_first_err [SK ASC ASQ]
     Description                              number   (hours)
# 1  Background long   Completed                   -    1520                 - [-   -    -]
# 2  Background short  Completed                   -    1346                 - [-   -    -]
# 3  Background short  Completed                   -    1178                 - [-   -    -]
# 4  Background short  Completed                   -    1031                 - [-   -    -]
# 5  Background long   Completed                   -     870                 - [-   -    -]
# 6  Background short  Completed                   -     695                 - [-   -    -]
# 7  Background short  Completed                   -     546                 - [-   -    -]
# 8  Background short  Completed                   -     378                 - [-   -    -]
# 9  Background short  Completed                   -     215                 - [-   -    -]
#10  Background short  Completed                   -      47                 - [-   -    -]
#11  Background short  Completed                   -   65535                 - [-   -    -]
#12  Background long   Completed                   -   65535                 - [-   -    -]
#13  Background short  Completed                   -   65535                 - [-   -    -]
#14  Background short  Completed                   -   65535                 - [-   -    -]
#15  Background short  Completed                   -   65535                 - [-   -    -]
#16  Background short  Completed                   -   65535                 - [-   -    -]
#17  Background short  Completed                   -   65535                 - [-   -    -]
#18  Background long   Completed                   -   65535                 - [-   -    -]
#19  Background short  Completed                   -   65535                 - [-   -    -]
#20  Background short  Completed                   -   65535                 - [-   -    -]

Long (extended) Self-test duration: 32700 seconds [545.0 minutes]

Background scan results log
  Status: halted due to fatal error
    Accumulated power on time, hours:minutes 1525:11 [91511 minutes]
    Number of background scans performed: 1067,  scan progress: 0.01%
    Number of background medium scans performed: 1067

   #  when        lba(hex)    [sk,asc,ascq]    reassign_status
   1 1048:13  0000000147387c09  [1,18,1]   Recovered via rewrite in-place
   2 1048:13  00000001473885e6  [1,18,1]   Recovered via rewrite in-place
   3 1048:13  0000000147388f15  [1,18,1]   Recovered via rewrite in-place
   4 1048:14  00000001473899f3  [1,18,1]   Recovered via rewrite in-place
   5 1048:14  0000000147387204  [1,18,1]   Recovered via rewrite in-place
   6 1048:15  0000000147380e53  [1,18,1]   Recovered via rewrite in-place
   7 1051:01  00000001460b6169  [1,18,1]   Recovered via rewrite in-place
   8 1051:04  00000000daa42f5c  [1,18,1]   Recovered via rewrite in-place
   9 1053:24  00000001476539a0  [1,17,1]   Recovered via rewrite in-place
  10 1053:24  0000000147653fd8  [1,17,1]   Recovered via rewrite in-place
  11 1053:24  000000014765687b  [1,18,1]   Recovered via rewrite in-place
  12 1053:25  00000001476508d6  [1,18,1]   Recovered via rewrite in-place
  13 1053:25  0000000147651221  [1,18,1]   Recovered via rewrite in-place
  14 1053:25  0000000147651d79  [1,18,1]   Recovered via rewrite in-place
  15 1053:25  0000000147652dc3  [1,17,1]   Recovered via rewrite in-place
  16 1053:25  000000014764bb02  [1,18,1]   Recovered via rewrite in-place
  17 1053:25  000000014764d504  [1,17,1]   Recovered via rewrite in-place
  18 1053:25  000000014764ed09  [1,18,1]   Recovered via rewrite in-place
  19 1053:25  000000014764a5e5  [1,18,1]   Recovered via rewrite in-place
  20 1053:44  00000001475c1d81  [1,18,1]   Recovered via rewrite in-place
  21 1053:44  00000001475bd9f8  [1,18,1]   Recovered via rewrite in-place
  22 1053:45  00000001474ee8a2  [1,17,1]   Recovered via rewrite in-place
  23 1053:45  00000001474f0fc5  [1,17,1]   Recovered via rewrite in-place
  24 1054:16  000000014745f950  [1,17,1]   Recovered via rewrite in-place
  25 1054:17  0000000147463272  [1,18,1]   Recovered via rewrite in-place
  26 1054:17  000000014738b56c  [1,17,1]   Recovered via rewrite in-place
  27 1054:24  0000000147304d67  [1,18,1]   Recovered via rewrite in-place
  28 1054:24  000000014737924d  [1,17,1]   Recovered via rewrite in-place
  29 1054:25  00000001472f4914  [1,18,1]   Recovered via rewrite in-place
  30 1054:25  00000001472f66f3  [1,17,1]   Recovered via rewrite in-place
  31 1054:25  00000001472f7f4f  [1,18,1]   Recovered via rewrite in-place
  32 1054:25  00000001472f8ae0  [1,18,1]   Recovered via rewrite in-place
  33 1054:27  000000014717b054  [1,18,1]   Recovered via rewrite in-place
  34 1054:27  0000000147175914  [1,18,1]   Recovered via rewrite in-place
  35 1054:28  0000000147176557  [1,18,1]   Recovered via rewrite in-place
  36 1054:28  0000000147177fae  [1,18,1]   Recovered via rewrite in-place
  37 1054:28  0000000147173ba6  [1,18,1]   Recovered via rewrite in-place
  38 1054:28  0000000147173ea0  [1,18,1]   Recovered via rewrite in-place
  39 1054:28  000000014716d494  [1,18,1]   Recovered via rewrite in-place
  40 1054:28  000000014716dbaf  [1,18,1]   Recovered via rewrite in-place
  41 1054:28  000000014716e74d  [1,18,1]   Recovered via rewrite in-place
  42 1054:40  00000001465cce2d  [4,32,0]   Reassignment by disk failed
  43 1054:40  00000001465cd01c  [1,18,1]   Recovered via rewrite in-place
  44 1055:19  000000014684b31d  [1,17,1]   Recovered via rewrite in-place
  45 1055:51  0000000146e6083c  [1,18,1]   Recovered via rewrite in-place
  46 1055:51  0000000146e6a397  [1,18,1]   Recovered via rewrite in-place
  47 1055:52  0000000146ef12f0  [1,17,1]   Recovered via rewrite in-place
  48 1055:53  0000000146e72a28  [1,18,1]   Recovered via rewrite in-place
  49 1055:53  0000000146e642d8  [1,17,1]   Recovered via rewrite in-place
  50 1056:13  0000000146e64477  [1,17,1]   Recovered via rewrite in-place
  51 1056:13  0000000146e6748c  [1,18,1]   Recovered via rewrite in-place
  52 1056:13  0000000146e630c5  [1,18,1]   Recovered via rewrite in-place
  53 1056:13  0000000146d6e95f  [1,17,1]   Recovered via rewrite in-place
  54 1056:14  0000000146d6cc40  [1,18,1]   Recovered via rewrite in-place
  55 1056:14  0000000146d66b9f  [1,18,1]   Recovered via rewrite in-place
  56 1056:14  0000000146d64184  [1,18,1]   Recovered via rewrite in-place
  57 1056:14  0000000146be973b  [1,18,1]   Recovered via rewrite in-place
  58 1056:15  0000000146be17b1  [1,18,1]   Recovered via rewrite in-place
  59 1056:18  00000001466d3d9a  [1,17,1]   Recovered via rewrite in-place
  60 1056:20  000000014654ad61  [1,18,1]   Recovered via rewrite in-place
  61 1066:41  0000000146b6069a  [1,17,1]   Recovered via rewrite in-place
  62 1072:47  0000000146447bf8  [1,17,1]   Recovered via rewrite in-place
  63 1079:06  00000001474e86ac  [1,17,1]   Recovered via rewrite in-place
  64 1079:14  000000014738b3c7  [1,18,1]   Recovered via rewrite in-place
  65 1079:17  00000001474e5f3b  [1,18,1]   Recovered via rewrite in-place
  66 1079:17  00000001474e6ae2  [1,17,1]   Recovered via rewrite in-place
  67 1079:17  00000001474ec306  [1,18,1]   Recovered via rewrite in-place
  68 1079:17  00000001474e9c08  [1,18,1]   Recovered via rewrite in-place
  69 1079:18  000000014737918b  [1,17,1]   Recovered via rewrite in-place
  70 1079:18  00000001472fdafa  [1,18,1]   Recovered via rewrite in-place
  71 1079:18  00000001472fe448  [1,17,1]   Recovered via rewrite in-place
  72 1079:22  00000001474f381e  [1,18,1]   Recovered via rewrite in-place
  73 1079:37  000000014746eb60  [1,18,1]   Recovered via rewrite in-place
  74 1079:37  0000000147469be9  [1,18,1]   Recovered via rewrite in-place
  75 1079:38  000000014738901a  [1,18,1]   Recovered via rewrite in-place
  76 1079:38  00000001473852b2  [1,18,1]   Recovered via rewrite in-place
  77 1079:38  0000000147386843  [1,18,1]   Recovered via rewrite in-place
  78 1079:43  00000001474ea3d4  [1,18,1]   Recovered via rewrite in-place
  79 1079:50  000000014746a767  [1,18,1]   Recovered via rewrite in-place
  80 1079:50  00000001474692b2  [1,18,1]   Recovered via rewrite in-place
  81 1079:53  0000000147177f93  [1,17,1]   Recovered via rewrite in-place
  82 1079:54  0000000147176e78  [1,18,1]   Recovered via rewrite in-place
  83 1079:54  000000014716f026  [1,18,1]   Recovered via rewrite in-place
  84 1079:55  000000014716f813  [1,18,1]   Recovered via rewrite in-place
  85 1079:55  000000014716da1e  [1,18,1]   Recovered via rewrite in-place
  86 1080:07  00000001475c5574  [4,32,0]   Reassignment by disk failed
  87 1080:07  00000001475c5574  [1,17,1]   Recovered via rewrite in-place
  88 1080:07  00000001475c6100  [1,18,1]   Recovered via rewrite in-place
  89 1080:16  00000001475c6b7f  [1,18,1]   Recovered via rewrite in-place
  90 1080:16  00000001475c3193  [1,18,1]   Recovered via rewrite in-place
  91 1080:16  00000001475c4512  [1,17,1]   Recovered via rewrite in-place
  92 1080:16  00000001475bbcd2  [1,18,1]   Recovered via rewrite in-place
  93 1080:16  00000001474f483d  [1,18,1]   Recovered via rewrite in-place
  94 1080:18  0000000146ef43b5  [1,18,1]   Recovered via rewrite in-place
  95 1080:19  0000000146e6611c  [1,18,1]   Recovered via rewrite in-place
  96 1081:27  0000000147647d80  [1,17,1]   Recovered via rewrite in-place
  97 1081:46  00000001477254dc  [1,18,1]   Recovered via rewrite in-place
  98 1081:47  0000000147725598  [1,18,1]   Recovered via rewrite in-place
  99 1089:37  000000014772ac55  [1,18,1]   Recovered via rewrite in-place
 100 1089:41  000000014772b669  [1,18,1]   Recovered via rewrite in-place
 101 1090:05  0000000147733d9a  [1,17,1]   Recovered via rewrite in-place
 102 1090:06  00000001477346f6  [1,17,1]   Recovered via rewrite in-place
 103 1090:08  00000001477ad064  [1,17,1]   Recovered via rewrite in-place
 104 1090:08  0000000147735b03  [1,17,1]   Recovered via rewrite in-place
 105 1090:08  00000001477ad90b  [1,17,1]   Recovered via rewrite in-place
 106 1095:53  000000014745d007  [1,18,1]   Recovered via rewrite in-place
 107 1096:01  00000001477b032a  [1,18,1]   Recovered via rewrite in-place
 108 1096:02  00000001477b0b38  [1,18,1]   Recovered via rewrite in-place
 109 1096:03  00000001477aa6f8  [1,18,1]   Recovered via rewrite in-place
 110 1096:04  00000001477ab319  [1,18,1]   Recovered via rewrite in-place
 111 1096:04  00000001477b14e7  [1,17,1]   Recovered via rewrite in-place
 112 1096:07  00000001477b1f0c  [1,18,1]   Recovered via rewrite in-place
 113 1096:08  00000001477b2697  [4,32,0]   Reassignment by disk failed
 114 1096:08  00000001477b2765  [4,32,0]   Reassignment by disk failed
 115 1096:10  00000001477b28f8  [1,18,1]   Recovered via rewrite in-place
 116 1096:12  00000001477b3356  [1,18,1]   Recovered via rewrite in-place
 117 1096:15  00000001477b5c3f  [1,18,1]   Recovered via rewrite in-place
 118 1096:15  00000001477b62de  [4,32,0]   Reassignment by disk failed
 119 1096:15  00000001477b65ee  [1,18,1]   Recovered via rewrite in-place
 120 1096:16  00000001477b6b61  [4,32,0]   Reassignment by disk failed
 121 1096:16  00000001477b6be5  [4,32,0]   Reassignment by disk failed
 122 1096:16  00000001477b7028  [1,18,1]   Recovered via rewrite in-place
 123 1096:19  00000001477b765b  [4,32,0]   Reassignment by disk failed
 124 1096:19  00000001477b7673  [4,32,0]   Reassignment by disk failed
 125 1096:20  00000001477b7a10  [1,17,1]   Recovered via rewrite in-place
 126 1096:20  00000001477b7d14  [4,32,0]   Reassignment by disk failed
 127 1096:21  00000001477b7ef5  [4,32,0]   Reassignment by disk failed
 128 1096:21  00000001477b7ef7  [4,32,0]   Reassignment by disk failed
 129 1096:22  00000001477b8525  [1,18,1]   Recovered via rewrite in-place
 130 1096:24  00000001477b8e16  [1,18,1]   Recovered via rewrite in-place
 131 1096:25  00000001477b905c  [4,32,0]   Reassignment by disk failed
 132 1096:25  00000001477b90ee  [4,32,0]   Reassignment by disk failed
 133 1096:26  00000001477b917f  [4,32,0]   Reassignment by disk failed
 134 1096:27  00000001477b988f  [1,18,1]   Recovered via rewrite in-place
 135 1096:27  00000001477b9a63  [4,32,0]   Reassignment by disk failed
 136 1096:28  00000001477b9bfc  [4,32,0]   Reassignment by disk failed
 137 1096:30  00000001477ba271  [1,18,1]   Recovered via rewrite in-place
 138 1096:31  00000001477ba531  [4,32,0]   Reassignment by disk failed
 139 1096:31  00000001477ba554  [4,32,0]   Reassignment by disk failed
 140 1096:32  00000001477ba64b  [4,32,0]   Reassignment by disk failed
 141 1096:32  00000001477ba668  [4,32,0]   Reassignment by disk failed
 142 1096:32  00000001477ba66d  [4,32,0]   Reassignment by disk failed
 143 1096:33  00000001477bac12  [1,18,1]   Recovered via rewrite in-place
 144 1096:34  00000001477b4caa  [1,18,1]   Recovered via rewrite in-place
 145 1096:34  00000001477bace3  [1,18,1]   Recovered via rewrite in-place
 146 1096:36  00000001477bb5e4  [1,18,1]   Recovered via rewrite in-place
 147 1096:37  00000001477b514b  [1,18,1]   Recovered via rewrite in-place
 148 1096:37  00000001477b577d  [4,32,0]   Reassignment by disk failed
 149 1096:37  00000001477b593a  [4,32,0]   Reassignment by disk failed
 150 1096:37  00000001477b3dd2  [1,18,1]   Recovered via rewrite in-place
 151 1096:38  00000001477b42a3  [4,32,0]   Reassignment by disk failed
 152 1096:38  00000001477b42a7  [4,32,0]   Reassignment by disk failed
 153 1096:38  00000001477b42a8  [4,32,0]   Reassignment by disk failed
 154 1096:38  00000001477b459d  [4,32,0]   Reassignment by disk failed
 155 1096:38  00000001477b459e  [4,32,0]   Reassignment by disk failed
 156 1096:38  00000001477b459f  [4,32,0]   Reassignment by disk failed
 157 1096:38  00000001477b45a2  [4,32,0]   Reassignment by disk failed
 158 1096:38  00000001477b334a  [1,18,1]   Recovered via rewrite in-place
 159 1096:39  00000001477b38b4  [4,32,0]   Reassignment by disk failed
 160 1096:39  00000001477b38ca  [4,32,0]   Reassignment by disk failed
 161 1096:39  00000001477b72ec  [1,18,1]   Recovered via rewrite in-place
 162 1096:39  00000001477b72ed  [4,32,0]   Reassignment by disk failed
 163 1096:39  00000001477b73d9  [4,32,0]   Reassignment by disk failed
 164 1096:39  00000001477b73db  [4,32,0]   Reassignment by disk failed
 165 1096:40  00000001477b7d57  [1,18,1]   Recovered via rewrite in-place
 166 1096:40  00000001477b85f6  [1,18,1]   Recovered via rewrite in-place
 167 1096:40  00000001477b8600  [4,32,0]   Reassignment by disk failed
 168 1096:40  00000001477b8609  [4,32,0]   Reassignment by disk failed
 169 1096:40  00000001477b8675  [4,32,0]   Reassignment by disk failed
 170 1096:40  00000001477b8722  [4,32,0]   Reassignment by disk failed
 171 1096:40  00000001477b8975  [4,32,0]   Reassignment by disk failed
 172 1096:40  00000001477b9001  [1,18,1]   Recovered via rewrite in-place
 173 1096:40  00000001477b9971  [1,17,1]   Recovered via rewrite in-place
 174 1096:41  00000001478a6791  [4,32,0]   Reassignment by disk failed
 175 1096:41  00000001478a6791  [1,18,1]   Recovered via rewrite in-place
 176 1096:41  00000001478a67bb  [4,32,0]   Reassignment by disk failed
 177 1096:41  00000001478a67bc  [4,32,0]   Reassignment by disk failed
 178 1096:42  00000001478a69a6  [4,32,0]   Reassignment by disk failed
 179 1096:44  00000001478a71ac  [1,18,1]   Recovered via rewrite in-place
 180 1096:45  00000001477bc192  [1,18,1]   Recovered via rewrite in-place
 181 1096:46  00000001478a7273  [1,18,1]   Recovered via rewrite in-place
 182 1096:47  00000001478a734b  [4,32,0]   Reassignment by disk failed
 183 1096:47  00000001478a7359  [4,32,0]   Reassignment by disk failed
 184 1096:47  00000001478a7af8  [4,32,0]   Reassignment by disk failed
 185 1096:48  00000001478a7bb5  [1,18,1]   Recovered via rewrite in-place
 186 1096:53  00000001d1bba00d  [4,32,0]   Reassignment by disk failed
 187 1096:55  00000001478a85ac  [4,32,0]   Reassignment by disk failed
 188 1096:55  00000001478a85ac  [1,18,1]   Recovered via rewrite in-place
 189 1096:56  00000001478a862d  [4,32,0]   Reassignment by disk failed
 190 1096:56  00000001478a865a  [4,32,0]   Reassignment by disk failed
 191 1096:56  00000001478a86e2  [4,32,0]   Reassignment by disk failed
 192 1096:57  00000001478a8ef1  [4,32,0]   Reassignment by disk failed
 193 1096:57  00000001478a8fd7  [1,18,1]   Recovered via rewrite in-place
 194 1096:58  00000001478a973d  [4,32,0]   Reassignment by disk failed
 195 1096:58  00000001478a979b  [4,32,0]   Reassignment by disk failed
 196 1096:59  00000001478a9845  [4,32,0]   Reassignment by disk failed
 197 1096:59  00000001478a984d  [4,32,0]   Reassignment by disk failed
 198 1096:59  00000001478a985b  [4,32,0]   Reassignment by disk failed
 199 1096:59  00000001478a9877  [4,32,0]   Reassignment by disk failed
 200 1096:59  00000001478a988a  [4,32,0]   Reassignment by disk failed
 201 1096:59  00000001478a98ab  [4,32,0]   Reassignment by disk failed
 202 1097:00  00000001478a991f  [4,32,0]   Reassignment by disk failed
 203 1097:00  00000001478a9921  [4,32,0]   Reassignment by disk failed
 204 1097:00  00000001478a9927  [4,32,0]   Reassignment by disk failed
 205 1097:00  00000001478a9976  [4,32,0]   Reassignment by disk failed
 206 1097:01  00000001478a99a8  [4,32,0]   Reassignment by disk failed
 207 1097:01  00000001478a99bd  [4,32,0]   Reassignment by disk failed
 208 1097:01  00000001478a99be  [4,32,0]   Reassignment by disk failed
 209 1097:01  00000001478a99be  [1,18,1]   Recovered via rewrite in-place
 210 1097:01  00000001478a99da  [4,32,0]   Reassignment by disk failed
 211 1097:01  00000001478a99f7  [4,32,0]   Reassignment by disk failed
 212 1097:01  00000001478a9a06  [4,32,0]   Reassignment by disk failed
 213 1097:01  00000001478a9a07  [4,32,0]   Reassignment by disk failed
 214 1097:02  00000001478a9a95  [4,32,0]   Reassignment by disk failed
 215 1097:04  00000001478aa3d7  [1,18,1]   Recovered via rewrite in-place
 216 1097:06  00000001478aadd0  [1,18,1]   Recovered via rewrite in-place
 217 1097:07  00000001478ab5ea  [4,32,0]   Reassignment by disk failed
 218 1097:07  00000001477bacca  [1,17,1]   Recovered via rewrite in-place
 219 1097:08  00000001478a8fe0  [1,18,1]   Recovered via rewrite in-place
 220 1097:08  00000001478ab706  [1,18,1]   Recovered via rewrite in-place
 221 1097:08  00000001478ab715  [4,32,0]   Reassignment by disk failed
 222 1097:09  00000001478ab7da  [1,18,1]   Recovered via rewrite in-place
 223 1097:10  00000001478abe4e  [4,32,0]   Reassignment by disk failed
 224 1097:14  00000000da886bd4  [1,18,1]   Recovered via rewrite in-place
 225 1097:14  00000001d1c08164  [1,18,1]   Recovered via rewrite in-place
 226 1097:16  00000001d1bbb289  [4,32,0]   Reassignment by disk failed
 227 1097:19  00000001478a99ec  [1,18,1]   Recovered via rewrite in-place
 228 1097:19  00000001478a99f0  [4,32,0]   Reassignment by disk failed
 229 1097:19  00000001478aa1bb  [4,32,0]   Reassignment by disk failed
 230 1097:19  00000001478abf0c  [1,18,1]   Recovered via rewrite in-place
 231 1097:19  00000001478abf16  [4,32,0]   Reassignment by disk failed
 232 1097:19  00000001478abf24  [4,32,0]   Reassignment by disk failed
 233 1097:19  00000001478abf50  [4,32,0]   Reassignment by disk failed
 234 1097:19  00000001478abf51  [4,32,0]   Reassignment by disk failed
 235 1097:20  00000001478ac054  [4,32,0]   Reassignment by disk failed
 236 1097:20  00000001478ac099  [4,32,0]   Reassignment by disk failed
 237 1097:21  00000001478ac169  [4,32,0]   Reassignment by disk failed
 238 1097:21  00000001478ac1e7  [1,18,1]   Recovered via rewrite in-place
 239 1097:22  00000001478ac725  [4,32,0]   Reassignment by disk failed
 240 1097:22  00000001478ac83a  [4,32,0]   Reassignment by disk failed
 241 1097:22  00000001478ac848  [4,32,0]   Reassignment by disk failed
 242 1097:23  00000001478ac893  [4,32,0]   Reassignment by disk failed
 243 1097:23  00000001478ac9ec  [4,32,0]   Reassignment by disk failed
 244 1097:24  00000001478acacb  [4,32,0]   Reassignment by disk failed
 245 1097:24  00000001478acadd  [4,32,0]   Reassignment by disk failed
 246 1097:24  00000001478acaed  [4,32,0]   Reassignment by disk failed
 247 1097:24  00000001478acaf8  [4,32,0]   Reassignment by disk failed
 248 1097:24  00000001478acb06  [4,32,0]   Reassignment by disk failed
 249 1097:24  00000001478acb07  [4,32,0]   Reassignment by disk failed
 250 1097:24  00000001478acb08  [4,32,0]   Reassignment by disk failed
 251 1097:24  00000001478acb0a  [4,32,0]   Reassignment by disk failed
 252 1097:25  00000001478acb24  [4,32,0]   Reassignment by disk failed
 253 1097:25  00000001478acbeb  [4,32,0]   Reassignment by disk failed
 254 1097:25  00000001478acbeb  [1,18,1]   Recovered via rewrite in-place
 255 1097:26  00000001477bb680  [1,18,1]   Recovered via rewrite in-place
 256 1097:27  00000001477bb8d1  [4,32,0]   Reassignment by disk failed
 257 1097:27  00000001478aaaf1  [1,17,1]   Recovered via rewrite in-place
 258 1097:27  00000001478aab5c  [4,32,0]   Reassignment by disk failed
 259 1097:27  00000001478ab489  [1,17,1]   Recovered via rewrite in-place
 260 1097:27  00000001478abf1d  [1,18,1]   Recovered via rewrite in-place
 261 1097:27  00000001478ac7b4  [1,18,1]   Recovered via rewrite in-place
 262 1097:27  00000001478ad186  [1,18,1]   Recovered via rewrite in-place
 263 1097:28  00000001478ad2b2  [4,32,0]   Reassignment by disk failed
 264 1097:29  00000001478ad2eb  [4,32,0]   Reassignment by disk failed
 265 1097:29  00000001478ad2ec  [4,32,0]   Reassignment by disk failed
 266 1097:29  00000001478ad2fa  [4,32,0]   Reassignment by disk failed
 267 1097:29  00000001478ad317  [4,32,0]   Reassignment by disk failed
 268 1097:29  00000001478ad31d  [4,32,0]   Reassignment by disk failed
 269 1097:29  00000001478ad332  [4,32,0]   Reassignment by disk failed
 270 1097:29  00000001478ad399  [4,32,0]   Reassignment by disk failed
 271 1097:29  00000001478ad39f  [4,32,0]   Reassignment by disk failed
 272 1097:29  00000001478ad3b3  [4,32,0]   Reassignment by disk failed
 273 1097:30  00000001478ad444  [4,32,0]   Reassignment by disk failed
 274 1097:30  00000001478ad4f4  [4,32,0]   Reassignment by disk failed
 275 1097:30  00000001478ad522  [4,32,0]   Reassignment by disk failed
 276 1097:31  00000001478ad5f6  [1,18,1]   Recovered via rewrite in-place
 277 1097:33  00000001478adc00  [4,32,0]   Reassignment by disk failed
 278 1097:33  00000001478adc01  [4,32,0]   Reassignment by disk failed
 279 1097:33  00000001478adc08  [4,32,0]   Reassignment by disk failed
 280 1097:33  00000001478adc0d  [4,32,0]   Reassignment by disk failed
 281 1097:33  00000001478adc12  [4,32,0]   Reassignment by disk failed
 282 1097:33  00000001478adc1b  [4,32,0]   Reassignment by disk failed
 283 1097:33  00000001478adc36  [4,32,0]   Reassignment by disk failed
 284 1097:33  00000001478adc3f  [4,32,0]   Reassignment by disk failed
 285 1097:33  00000001478adc40  [4,32,0]   Reassignment by disk failed
 286 1097:34  00000001478adc6f  [4,32,0]   Reassignment by disk failed
 287 1097:34  00000001478adc82  [4,32,0]   Reassignment by disk failed
 288 1097:34  00000001478adc84  [4,32,0]   Reassignment by disk failed
 289 1097:34  00000001478adc8f  [4,32,0]   Reassignment by disk failed
 290 1097:34  00000001478adcb3  [4,32,0]   Reassignment by disk failed
 291 1097:34  00000001478adcc2  [4,32,0]   Reassignment by disk failed
 292 1097:34  00000001478adcc3  [4,32,0]   Reassignment by disk failed
 293 1097:34  00000001478adce8  [4,32,0]   Reassignment by disk failed
 294 1097:34  00000001478adcf1  [4,32,0]   Reassignment by disk failed
 295 1097:34  00000001478adcf4  [4,32,0]   Reassignment by disk failed
 296 1097:34  00000001478adcf5  [4,32,0]   Reassignment by disk failed
 297 1097:35  00000001478add9e  [4,32,0]   Reassignment by disk failed
 298 1097:35  00000001478addcb  [4,32,0]   Reassignment by disk failed
 299 1097:35  00000001478addcd  [4,32,0]   Reassignment by disk failed
 300 1097:35  00000001478addd1  [4,32,0]   Reassignment by disk failed
 301 1097:36  00000001478adde1  [4,32,0]   Reassignment by disk failed
 302 1097:36  00000001478addeb  [4,32,0]   Reassignment by disk failed
 303 1097:36  00000001478addff  [4,32,0]   Reassignment by disk failed
 304 1097:36  00000001478ade26  [4,32,0]   Reassignment by disk failed
 305 1097:36  00000001478ade2e  [4,32,0]   Reassignment by disk failed
 306 1097:36  00000001478ade71  [4,32,0]   Reassignment by disk failed
 307 1097:36  00000001478ade78  [4,32,0]   Reassignment by disk failed
 308 1097:38  00000001478adff3  [4,32,0]   Reassignment by disk failed
 309 1097:38  00000001478ae00e  [1,17,1]   Recovered via rewrite in-place
 310 1097:40  00000001478ae4af  [4,32,0]   Reassignment by disk failed
 311 1097:40  00000001478ae4ef  [4,32,0]   Reassignment by disk failed
 312 1097:40  00000001478ae539  [4,32,0]   Reassignment by disk failed
 313 1097:41  00000001478ae554  [4,32,0]   Reassignment by disk failed
 314 1097:42  00000001478ae6de  [4,32,0]   Reassignment by disk failed
 315 1097:42  00000001478ae6ea  [4,32,0]   Reassignment by disk failed
 316 1097:42  00000001478ae6ee  [4,32,0]   Reassignment by disk failed
 317 1097:42  00000001478ae6ef  [4,32,0]   Reassignment by disk failed
 318 1097:42  00000001478ae6f1  [4,32,0]   Reassignment by disk failed
 319 1097:42  00000001478ae706  [4,32,0]   Reassignment by disk failed
 320 1097:42  00000001478ae710  [4,32,0]   Reassignment by disk failed
 321 1097:42  00000001478ae737  [4,32,0]   Reassignment by disk failed
 322 1097:42  00000001478ae79a  [4,32,0]   Reassignment by disk failed
 323 1097:43  00000001478ae818  [4,32,0]   Reassignment by disk failed
 324 1097:43  00000001478aea52  [1,18,1]   Recovered via rewrite in-place
 325 1097:44  00000001478aedc4  [4,32,0]   Reassignment by disk failed
 326 1097:46  00000001478af17d  [4,32,0]   Reassignment by disk failed
 327 1097:46  00000001478af1e4  [4,32,0]   Reassignment by disk failed
 328 1097:47  00000001478af420  [1,18,1]   Recovered via rewrite in-place
 329 1097:47  00000001478af5e9  [4,32,0]   Reassignment by disk failed
 330 1097:48  00000001478af810  [4,32,0]   Reassignment by disk failed
 331 1097:51  00000001478afc36  [4,32,0]   Reassignment by disk failed
 332 1097:51  00000001478afe8d  [1,17,1]   Recovered via rewrite in-place
 333 1097:52  00000001478b024d  [4,32,0]   Reassignment by disk failed
 334 1097:53  00000001478b02eb  [4,32,0]   Reassignment by disk failed
 335 1097:53  00000001478b02ed  [4,32,0]   Reassignment by disk failed
 336 1097:53  00000001478b02f0  [4,32,0]   Reassignment by disk failed
 337 1097:53  00000001478b030b  [4,32,0]   Reassignment by disk failed
 338 1097:57  00000001478b04a3  [1,18,1]   Recovered via rewrite in-place
 339 1097:57  00000001478b04bc  [4,32,0]   Reassignment by disk failed
 340 1097:57  00000001478b04c2  [4,32,0]   Reassignment by disk failed
 341 1097:57  00000001478b04cb  [4,32,0]   Reassignment by disk failed
 342 1097:57  00000001478b04d0  [4,32,0]   Reassignment by disk failed
 343 1097:57  00000001478b04d1  [4,32,0]   Reassignment by disk failed
 344 1097:58  00000001478b05b3  [4,32,0]   Reassignment by disk failed
 345 1098:54  00000001478b05e1  [1,17,1]   Recovered via rewrite in-place
 346 1098:54  00000001478b0cbe  [1,18,1]   Recovered via rewrite in-place
 347 1098:54  0000000147732165  [1,17,1]   Recovered via rewrite in-place
 348 1098:55  00000001477ba5db  [1,18,1]   Recovered via rewrite in-place
 349 1099:08  000000014772be38  [1,18,1]   Recovered via rewrite in-place
 350 1099:17  000000014772ac76  [1,18,1]   Recovered via rewrite in-place
 351 1099:18  000000014716d343  [1,18,1]   Recovered via rewrite in-place
 352 1099:33  000000014772bf72  [1,18,1]   Recovered via rewrite in-place
 353 1099:33  000000014772c78e  [1,18,1]   Recovered via rewrite in-place
 354 1099:34  000000014772db27  [1,18,1]   Recovered via rewrite in-place
 355 1099:34  00000001477320f3  [1,17,1]   Recovered via rewrite in-place
 356 1099:34  00000001477329cb  [1,17,1]   Recovered via rewrite in-place
 357 1099:34  000000014772a299  [1,17,1]   Recovered via rewrite in-place
 358 1099:34  0000000147723e38  [1,18,1]   Recovered via rewrite in-place
 359 1099:34  00000001477249dc  [1,18,1]   Recovered via rewrite in-place
 360 1099:35  0000000147724b95  [4,32,0]   Reassignment by disk failed
 361 1099:35  00000001477254a7  [1,17,1]   Recovered via rewrite in-place
 362 1099:35  0000000147725d20  [1,17,1]   Recovered via rewrite in-place
 363 1099:35  0000000147725e45  [4,32,0]   Reassignment by disk failed
 364 1099:37  0000000147733275  [1,18,1]   Recovered via rewrite in-place
 365 1099:38  00000001477270aa  [1,18,1]   Recovered via rewrite in-place
 366 1099:38  0000000147727cbe  [1,17,1]   Recovered via rewrite in-place
 367 1099:38  000000014772865f  [4,32,0]   Reassignment by disk failed
 368 1099:38  0000000147728660  [4,32,0]   Reassignment by disk failed
 369 1099:38  0000000147728660  [1,18,1]   Recovered via rewrite in-place
 370 1099:38  0000000147728f91  [1,18,1]   Recovered via rewrite in-place
 371 1099:39  0000000147730216  [1,18,1]   Recovered via rewrite in-place
 372 1099:39  0000000147730b6e  [4,32,0]   Reassignment by disk failed
 373 1099:39  0000000147730c60  [1,17,1]   Recovered via rewrite in-place
 374 1099:39  0000000147731684  [1,17,1]   Recovered via rewrite in-place
 375 1099:39  000000014772b43a  [1,18,1]   Recovered via rewrite in-place
 376 1099:40  0000000147733f63  [1,17,1]   Recovered via rewrite in-place
 377 1099:42  000000005c78de57  [1,18,1]   Recovered via rewrite in-place
 378 1099:42  000000005c78eaba  [1,18,1]   Recovered via rewrite in-place
 379 1099:42  000000005c78f134  [1,18,1]   Recovered via rewrite in-place
 380 1099:58  000000005c78f70c  [1,17,1]   Recovered via rewrite in-place
 381 1099:58  000000005c790ed3  [1,18,1]   Recovered via rewrite in-place
 382 1099:58  000000005c792711  [1,18,1]   Recovered via rewrite in-place
 383 1099:58  000000005c8d1f21  [1,18,1]   Recovered via rewrite in-place
 384 1099:59  000000005c8d307b  [1,18,1]   Recovered via rewrite in-place
 385 1100:00  000000005c8d3c4e  [1,18,1]   Recovered via rewrite in-place
 386 1100:00  000000005c8d4752  [1,18,1]   Recovered via rewrite in-place
 387 1100:00  000000005c793ed7  [1,18,1]   Recovered via rewrite in-place
 388 1100:01  000000005c7948d8  [1,18,1]   Recovered via rewrite in-place
 389 1100:02  000000005c7956e5  [1,18,1]   Recovered via rewrite in-place
 390 1100:03  000000005c7961b2  [1,18,1]   Recovered via rewrite in-place
 391 1100:04  000000005c796ff4  [1,18,1]   Recovered via rewrite in-place
 392 1100:04  000000005c8d52eb  [1,18,1]   Recovered via rewrite in-place
 393 1100:05  000000005c8d1483  [1,18,1]   Recovered via rewrite in-place
 394 1100:05  000000005c8dcdae  [1,18,1]   Recovered via rewrite in-place
 395 1100:08  0000000098102fa9  [1,18,1]   Recovered via rewrite in-place
 396 1100:08  0000000098104931  [1,18,1]   Recovered via rewrite in-place
 397 1100:10  0000000147735a80  [4,32,0]   Reassignment by disk failed
 398 1100:10  0000000147735a80  [1,18,1]   Recovered via rewrite in-place
 399 1100:10  00000001477a947e  [1,18,1]   Recovered via rewrite in-place
 400 1100:10  00000001477a9d76  [1,18,1]   Recovered via rewrite in-place
 401 1100:11  00000001477aa76c  [1,18,1]   Recovered via rewrite in-place
 402 1100:11  00000001477aa787  [4,32,0]   Reassignment by disk failed
 403 1100:11  00000001477aa7be  [4,32,0]   Reassignment by disk failed
 404 1100:11  00000001477ab18c  [1,18,1]   Recovered via rewrite in-place
 405 1100:11  0000000147732a00  [1,18,1]   Recovered via rewrite in-place
 406 1100:11  0000000147733270  [1,18,1]   Recovered via rewrite in-place
 407 1100:11  0000000147733cc6  [1,18,1]   Recovered via rewrite in-place
 408 1100:11  0000000147734796  [1,18,1]   Recovered via rewrite in-place
 409 1100:11  00000001477350b4  [1,18,1]   Recovered via rewrite in-place
 410 1100:12  00000001477ac49e  [4,32,0]   Reassignment by disk failed
 411 1100:12  00000001477ac4b2  [1,18,1]   Recovered via rewrite in-place
 412 1100:12  00000001477ac56c  [4,32,0]   Reassignment by disk failed
 413 1100:12  00000001477ac56f  [4,32,0]   Reassignment by disk failed
 414 1100:12  00000001477ac57c  [4,32,0]   Reassignment by disk failed
 415 1100:12  000000014772ee8c  [1,18,1]   Recovered via rewrite in-place
 416 1100:12  00000001477302fb  [1,18,1]   Recovered via rewrite in-place
 417 1100:13  0000000147730321  [4,32,0]   Reassignment by disk failed
 418 1100:13  0000000147730ee1  [1,18,1]   Recovered via rewrite in-place
 419 1100:13  00000001477314f8  [1,18,1]   Recovered via rewrite in-place
 420 1100:13  000000014772b6e2  [1,18,1]   Recovered via rewrite in-place
 421 1100:13  000000014772be61  [1,17,1]   Recovered via rewrite in-place
 422 1100:13  000000014772c6e9  [1,17,1]   Recovered via rewrite in-place
 423 1100:14  000000014772d414  [1,18,1]   Recovered via rewrite in-place
 424 1100:14  000000014772a43a  [1,18,1]   Recovered via rewrite in-place
 425 1100:28  000000005c8d620c  [1,18,1]   Recovered via rewrite in-place
 426 1100:28  000000005c8dd0e6  [1,18,1]   Recovered via rewrite in-place
 427 1100:28  000000005c8d9fcf  [1,17,1]   Recovered via rewrite in-place
 428 1100:46  0000000146cde985  [1,17,1]   Recovered via rewrite in-place
 429 1101:29  00000000d6b12c23  [1,17,1]   Recovered via rewrite in-place
 430 1101:36  00000000d6b1650c  [1,17,1]   Recovered via rewrite in-place
 431 1102:22  000000014613f1cd  [1,18,1]   Recovered via rewrite in-place
 432 1102:28  0000000146230c86  [1,18,1]   Recovered via rewrite in-place
 433 1102:48  0000000147735a81  [1,18,1]   Recovered via rewrite in-place
 434 1102:50  00000001477b4f18  [1,18,1]   Recovered via rewrite in-place
 435 1104:03  0000000015c837a1  [1,18,1]   Recovered via rewrite in-place
 436 1104:03  0000000015c84451  [1,18,1]   Recovered via rewrite in-place
 437 1104:04  0000000015b3167e  [1,18,1]   Recovered via rewrite in-place
 438 1104:51  0000000015e9f3cf  [1,18,1]   Recovered via rewrite in-place
 439 1104:53  0000000015f4e729  [1,18,1]   Recovered via rewrite in-place
 440 1104:54  0000000015f5030c  [1,17,1]   Recovered via rewrite in-place
 441 1105:10  000000001500ff6b  [1,18,1]   Recovered via rewrite in-place
 442 1105:14  000000001609b0c5  [1,18,1]   Recovered via rewrite in-place
 443 1105:19  0000000015014d95  [1,18,1]   Recovered via rewrite in-place
 444 1105:31  00000000150c2521  [1,18,1]   Recovered via rewrite in-place
 445 1106:37  0000000015c7cfe0  [1,18,1]   Recovered via rewrite in-place
 446 1111:06  000000001609f22c  [1,17,1]   Recovered via rewrite in-place
 447 1111:32  00000000160b010a  [1,18,1]   Recovered via rewrite in-place
 448 1111:44  00000000160b27f8  [1,18,1]   Recovered via rewrite in-place
 449 1112:14  00000000d707970e  [1,18,1]   Recovered via rewrite in-place
 450 1113:08  0000000015c879e7  [1,18,1]   Recovered via rewrite in-place
 451 1119:25  000000005c8e24f2  [1,18,1]   Recovered via rewrite in-place
 452 1125:18  000000005c980068  [1,18,1]   Recovered via rewrite in-place
 453 1125:18  00000000dad32e3b  [1,18,1]   Recovered via rewrite in-place
 454 1125:47  00000000dad3d3ab  [1,17,1]   Recovered via rewrite in-place
 455 1125:53  00000000dad3df58  [1,18,1]   Recovered via rewrite in-place
 456 1125:55  00000000dad3eabd  [1,18,1]   Recovered via rewrite in-place
 457 1125:57  00000000dad3f622  [1,17,1]   Recovered via rewrite in-place
 458 1126:03  00000000dad40acd  [1,18,1]   Recovered via rewrite in-place
 459 1126:37  00000000dad461f1  [1,17,1]   Recovered via rewrite in-place
 460 1126:41  00000000dadd0551  [1,18,1]   Recovered via rewrite in-place
 461 1126:48  00000000dadd10b8  [1,18,1]   Recovered via rewrite in-place
 462 1127:15  00000000dadd4926  [1,18,1]   Recovered via rewrite in-place
 463 1127:51  00000000bb3988a0  [1,18,1]   Recovered via rewrite in-place
 464 1128:28  00000000bd059cd7  [1,18,1]   Recovered via rewrite in-place
 465 1128:28  00000000bd05a890  [1,18,1]   Recovered via rewrite in-place
 466 1128:29  00000000bd0e9afb  [1,18,1]   Recovered via rewrite in-place
 467 1128:30  00000000bd0ec793  [1,18,1]   Recovered via rewrite in-place
 468 1128:30  00000000bd0ee032  [1,18,1]   Recovered via rewrite in-place
 469 1128:31  00000000bd0f37b0  [1,18,1]   Recovered via rewrite in-place
 470 1128:31  00000000bd0f46c2  [1,18,1]   Recovered via rewrite in-place
 471 1128:31  00000000bd0f73d2  [1,18,1]   Recovered via rewrite in-place
 472 1128:32  00000000bd0f8c4b  [1,17,1]   Recovered via rewrite in-place
 473 1128:32  00000000bd0fa008  [1,17,1]   Recovered via rewrite in-place
 474 1129:35  00000000c7646685  [1,17,1]   Recovered via rewrite in-place
 475 1130:18  00000000bd22a948  [1,18,1]   Recovered via rewrite in-place
 476 1130:19  00000000bd229ee5  [1,18,1]   Recovered via rewrite in-place
 477 1130:19  00000000bd22d7c0  [1,18,1]   Recovered via rewrite in-place
 478 1130:20  00000000bd2292dc  [1,18,1]   Recovered via rewrite in-place
 479 1130:20  00000000bd22d972  [1,17,1]   Recovered via rewrite in-place
 480 1130:21  00000000bd22f171  [1,18,1]   Recovered via rewrite in-place
 481 1130:21  00000000bd230649  [1,17,1]   Recovered via rewrite in-place
 482 1130:21  00000000bd231f0a  [1,18,1]   Recovered via rewrite in-place
 483 1130:47  00000001d1bba00d  [1,18,1]   Recovered via rewrite in-place
 484 1131:59  00000001d1bbb288  [1,18,1]   Recovered via rewrite in-place
 485 1132:13  00000001d1bba00d  [1,18,1]   Recovered via rewrite in-place
 486 1132:15  000000005c59462a  [1,18,1]   Recovered via rewrite in-place
 487 1132:18  000000005c598091  [1,17,1]   Recovered via rewrite in-place
 488 1132:22  000000005c6ded44  [1,18,1]   Recovered via rewrite in-place
 489 1132:25  000000005c6e1199  [1,18,1]   Recovered via rewrite in-place
 490 1132:27  000000005c6d9aa0  [1,18,1]   Recovered via rewrite in-place
 491 1132:27  000000005c6d680d  [1,18,1]   Recovered via rewrite in-place
 492 1132:30  000000005c6dc8f7  [1,17,1]   Recovered via rewrite in-place
 493 1132:32  000000005c6dd5af  [1,18,1]   Recovered via rewrite in-place
 494 1132:32  000000005c6d8117  [1,18,1]   Recovered via rewrite in-place
 495 1132:32  000000005c6d501e  [1,18,1]   Recovered via rewrite in-place
 496 1132:39  000000005c6dbe7e  [1,18,1]   Recovered via rewrite in-place
 497 1132:53  000000005c6d5b6c  [1,18,1]   Recovered via rewrite in-place
 498 1132:55  00000001d1c0be78  [1,17,1]   Recovered via rewrite in-place
 499 1132:58  000000005c6e2ced  [1,18,1]   Recovered via rewrite in-place
 500 1132:58  000000005c6df994  [1,18,1]   Recovered via rewrite in-place
 501 1132:58  00000001d1c0b608  [1,17,1]   Recovered via rewrite in-place
 502 1132:59  000000005c6e739f  [1,18,1]   Recovered via rewrite in-place
 503 1133:24  00000001d1c084c1  [1,17,1]   Recovered via rewrite in-place
 504 1133:32  000000005c6e4515  [1,18,1]   Recovered via rewrite in-place
 505 1133:40  000000005c6e1f43  [1,18,1]   Recovered via rewrite in-place
 506 1135:31  000000005c784b52  [1,17,1]   Recovered via rewrite in-place
 507 1135:43  00000000db892c72  [1,18,1]   Recovered via rewrite in-place
 508 1135:43  00000000db893eba  [1,18,1]   Recovered via rewrite in-place
 509 1135:44  00000000db7ff72c  [1,18,1]   Recovered via rewrite in-place
 510 1136:13  00000000db896cfc  [1,18,1]   Recovered via rewrite in-place
 511 1136:13  00000000db898143  [1,18,1]   Recovered via rewrite in-place
 512 1136:14  00000000db898b04  [1,17,1]   Recovered via rewrite in-place
 513 1136:14  00000000db89abdb  [1,17,1]   Recovered via rewrite in-place
 514 1136:41  00000000db89c27f  [1,17,1]   Recovered via rewrite in-place
 515 1147:12  000000005cb7643f  [1,18,1]   Recovered via rewrite in-place
 516 1160:14  000000005c785690  [1,18,1]   Recovered via rewrite in-place
 517 1166:58  00000000000a6c86  [4,32,0]   Reassignment by disk failed
 518 1167:12  00000000d05ce504  [1,17,1]   Recovered via rewrite in-place
 519 1167:50  000000005bfa2356  [1,18,1]   Recovered via rewrite in-place
 520 1167:51  00000000000a6c5d  [4,32,0]   Reassignment by disk failed
 521 1171:59  00000000000a6c5d  [4,32,0]   Reassignment by disk failed
 522 1174:10  00000000daa42489  [1,17,1]   Recovered via rewrite in-place
 523 1174:20  000000005bf96192  [1,17,1]   Recovered via rewrite in-place
 524 1174:20  000000005bf96e79  [1,18,1]   Recovered via rewrite in-place
 525 1174:35  000000005bf917ed  [1,18,1]   Recovered via rewrite in-place
 526 1174:36  000000005bef9bde  [1,18,1]   Recovered via rewrite in-place
 527 1174:36  000000005bef5161  [1,18,1]   Recovered via rewrite in-place
 528 1174:36  000000005bef7637  [1,18,1]   Recovered via rewrite in-place
 529 1174:36  000000005bef8e0c  [1,18,1]   Recovered via rewrite in-place
 530 1178:42  000000005c0edb98  [1,17,1]   Recovered via rewrite in-place
 531 1178:53  000000005c0e9ef1  [1,17,1]   Recovered via rewrite in-place
 532 1178:57  000000005c192e5e  [1,18,1]   Recovered via rewrite in-place
 533 1178:57  000000005c194fc8  [1,18,1]   Recovered via rewrite in-place
 534 1179:29  000000005c0e6e05  [1,18,1]   Recovered via rewrite in-place
 535 1179:32  000000005c1a3b04  [1,18,1]   Recovered via rewrite in-place
 536 1179:55  000000005c2f2b44  [1,17,1]   Recovered via rewrite in-place
 537 1179:58  000000005c2f36bb  [1,17,1]   Recovered via rewrite in-place
 538 1180:00  000000005c2f5ae5  [1,18,1]   Recovered via rewrite in-place
 539 1180:01  000000005c2f7336  [1,18,1]   Recovered via rewrite in-place
 540 1182:06  00000000d59dbcb5  [1,18,1]   Recovered via rewrite in-place
 541 1182:12  00000000d59ddd57  [1,17,1]   Recovered via rewrite in-place
 542 1182:20  00000000d59dfd6f  [1,17,1]   Recovered via rewrite in-place
 543 1182:30  00000000d59e2b20  [1,18,1]   Recovered via rewrite in-place
 544 1375:30  0000000101d9fc9d  [1,18,1]   Recovered via rewrite in-place
 545 1375:35  000000010798acb6  [1,18,1]   Recovered via rewrite in-place
 546 1375:38  000000010798c03a  [1,18,1]   Recovered via rewrite in-place
 547 1375:50  000000010798f300  [1,18,1]   Recovered via rewrite in-place
 548 1376:02  0000000107a141be  [1,18,1]   Recovered via rewrite in-place
 549 1378:50  0000000107a2817d  [1,17,1]   Recovered via rewrite in-place
 550 1383:16  00000001068c65e6  [1,17,1]   Recovered via rewrite in-place
 551 1384:07  000000010502c1e6  [1,18,1]   Recovered via rewrite in-place
 552 1387:12  000000010694d3cc  [1,18,1]   Recovered via rewrite in-place
 553 1415:03  00000001080deef4  [1,17,1]   Recovered via rewrite in-place
 554 1446:26  000000010c5ab2b2  [1,18,1]   Recovered via rewrite in-place
 555 1447:11  000000010c90ae27  [1,18,1]   Recovered via rewrite in-place
 556 1455:21  0000000108f53357  [1,18,1]   Recovered via rewrite in-place
 557 1455:47  00000001d1bba00d  [1,18,1]   Recovered via rewrite in-place
 558 1461:44  00000000bef8b229  [1,18,1]   Recovered via rewrite in-place
 559 1461:45  00000000bef8c6f2  [1,18,1]   Recovered via rewrite in-place
 560 1463:47  00000000bf1f0f1b  [1,18,1]   Recovered via rewrite in-place
 561 1463:53  00000000bf163277  [1,18,1]   Recovered via rewrite in-place
 562 1467:25  00000000bf6e2b02  [1,18,1]   Recovered via rewrite in-place
 563 1469:25  00000000bfe324cc  [1,17,1]   Recovered via rewrite in-place
 564 1470:31  00000000c0012860  [1,18,1]   Recovered via rewrite in-place
 565 1474:29  00000001d1bba00d  [4,32,0]   Reassignment by disk failed
 566 1483:43  00000000b9bb8b91  [1,18,1]   Recovered via rewrite in-place
 567 1483:44  00000000b9bb98ea  [1,18,1]   Recovered via rewrite in-place
 568 1502:18  00000001d1bba00d  [1,18,1]   Recovered via rewrite in-place
 569 1503:34  00000000b9cf93c4  [1,18,1]   Recovered via rewrite in-place
 570 1510:37  00000000b9d944ca  [1,18,1]   Recovered via rewrite in-place
 571 1510:38  00000000b9d97c60  [1,18,1]   Recovered via rewrite in-place
 572  947:07  00000001bb851e02  [1,18,1]   Recovered via rewrite in-place
 573  947:07  00000001bb851e03  [1,18,1]   Successfully reassigned
 574  947:07  00000001bb851e04  [1,18,1]   Recovered via rewrite in-place
 575  947:07  00000001bb851e05  [1,18,1]   Successfully reassigned
 576  947:07  00000001bb851e06  [1,18,1]   Successfully reassigned
 577  947:07  00000001bb851e09  [1,18,1]   Successfully reassigned
 578  947:07  00000001bb851e0c  [1,18,1]   Successfully reassigned
 579  947:07  00000001bb851e0d  [1,18,1]   Successfully reassigned
 580  947:07  00000001bb851e0e  [1,18,1]   Successfully reassigned
 581  947:08  00000001bb851e0f  [1,18,1]   Successfully reassigned
 582  947:08  00000001bb851e10  [1,18,1]   Successfully reassigned
 583  947:08  00000001bb851e11  [1,17,1]   Successfully reassigned
 584  947:08  00000001bb851e12  [1,18,1]   Successfully reassigned
 585  947:08  00000001bb851e13  [1,18,1]   Successfully reassigned
 586  947:08  00000001bb851e15  [1,18,1]   Successfully reassigned
 587  947:08  00000001bb851e16  [1,18,1]   Recovered via rewrite in-place
 588  947:08  00000001bb851e17  [1,18,1]   Recovered via rewrite in-place
 589  947:08  00000001bb851e18  [1,18,1]   Successfully reassigned
 590  947:08  00000001bb851e1a  [1,18,1]   Successfully reassigned
 591  947:08  00000001bb851e1b  [1,18,1]   Successfully reassigned
 592  947:08  00000001bb851e1c  [1,18,1]   Recovered via rewrite in-place
 593  947:08  00000001bb851e1d  [1,18,1]   Successfully reassigned
 594  947:08  00000001bb851e1e  [1,18,1]   Recovered via rewrite in-place
 595  947:08  00000001bb851e1f  [1,18,1]   Recovered via rewrite in-place
 596  947:08  00000001bb851e24  [1,18,1]   Successfully reassigned
 597  947:08  00000001bb851e25  [1,18,1]   Recovered via rewrite in-place
 598  947:08  00000001bb851e28  [1,18,1]   Successfully reassigned
 599  947:08  00000001bb851e2e  [1,18,1]   Recovered via rewrite in-place
 600  947:08  00000001bb851e2f  [1,18,1]   Successfully reassigned
 601  947:08  00000001bb851e30  [1,18,1]   Successfully reassigned
 602  947:08  00000001bb851e31  [1,18,1]   Recovered via rewrite in-place
 603  947:08  00000001bb851e32  [1,18,1]   Successfully reassigned
 604  947:08  00000001bb851e33  [1,18,1]   Successfully reassigned
 605  947:08  00000001bb851e34  [1,18,1]   Successfully reassigned
 606  947:08  00000001bb851e35  [1,18,1]   Successfully reassigned
 607  947:08  00000001bb851e36  [1,18,1]   Successfully reassigned
 608  947:08  00000001bb851e38  [1,18,1]   Successfully reassigned
 609  947:08  00000001bb851e39  [1,18,1]   Successfully reassigned
 610  947:08  00000001bb851e3a  [1,18,1]   Successfully reassigned
 611  947:08  00000001bb851e3b  [1,18,1]   Successfully reassigned
 612  947:08  00000001bb851e3c  [1,17,1]   Successfully reassigned
 613  947:08  00000001bb851e3d  [1,18,1]   Successfully reassigned
 614  947:08  00000001bb851e3e  [1,18,1]   Successfully reassigned
 615  947:08  00000001bb851e3f  [1,18,1]   Successfully reassigned
 616  947:08  00000001bb851e40  [1,18,1]   Successfully reassigned
 617  947:08  00000001bb851e43  [1,18,1]   Successfully reassigned
 618  947:08  00000001bb851e44  [1,18,1]   Successfully reassigned
 619  947:08  00000001bb851e45  [1,18,1]   Successfully reassigned
 620  947:08  00000001bb851e46  [1,18,1]   Successfully reassigned
 621  947:08  00000001bb851e47  [1,18,1]   Recovered via rewrite in-place
 622  947:08  00000001bb851e48  [1,18,1]   Successfully reassigned
 623  947:08  00000001bb851e49  [1,18,1]   Successfully reassigned
 624  947:08  00000001bb851e53  [1,18,1]   Successfully reassigned
 625  947:08  00000001bb851e54  [1,18,1]   Recovered via rewrite in-place
 626  947:08  00000001bb851e55  [1,18,1]   Successfully reassigned
 627  947:08  00000001bb851e56  [1,18,1]   Successfully reassigned
 628  947:09  00000001bb851e57  [1,18,1]   Successfully reassigned
 629  947:09  00000001bb851e58  [1,18,1]   Successfully reassigned
 630  947:09  00000001bb851e5b  [1,18,1]   Successfully reassigned
 631  947:09  00000001bb851e5c  [1,18,1]   Recovered via rewrite in-place
 632  947:09  00000001bb851e5d  [1,18,1]   Successfully reassigned
 633  947:09  00000001bb851e5e  [1,18,1]   Successfully reassigned
 634  947:09  00000001bb851e5f  [1,18,1]   Successfully reassigned
 635  947:09  00000001bb851e60  [1,18,1]   Successfully reassigned
 636  947:09  00000001bb851e61  [1,18,1]   Recovered via rewrite in-place
 637  947:09  00000001bb851e62  [1,18,1]   Successfully reassigned
 638  947:09  00000001bb851e63  [1,18,1]   Successfully reassigned
 639  947:09  00000001bb851e64  [1,18,1]   Successfully reassigned
 640  947:09  00000001bb851e65  [1,18,1]   Recovered via rewrite in-place
 641  947:09  00000001bb851e6f  [1,18,1]   Recovered via rewrite in-place
 642  947:09  00000001bb851e70  [1,18,1]   Recovered via rewrite in-place
 643  947:09  00000001bb851e72  [1,18,1]   Recovered via rewrite in-place
 644  947:09  00000001bb851e73  [1,18,1]   Successfully reassigned
 645  947:09  00000001bb851e74  [1,18,1]   Successfully reassigned
 646  947:09  00000001bb851e75  [1,18,1]   Successfully reassigned
 647  947:09  00000001bb851e76  [1,18,1]   Recovered via rewrite in-place
 648  947:09  00000001bb851e78  [1,18,1]   Recovered via rewrite in-place
 649  947:09  00000001bb851e79  [1,18,1]   Recovered via rewrite in-place
 650  947:09  00000001bb851e7c  [1,18,1]   Recovered via rewrite in-place
 651  947:09  00000001bb851e7d  [1,18,1]   Successfully reassigned
 652  947:09  00000001bb851e7e  [1,18,1]   Successfully reassigned
 653  947:09  00000001bb851e7f  [1,18,1]   Successfully reassigned
 654  947:09  00000001bb851e80  [1,18,1]   Successfully reassigned
 655  947:09  00000001bb851e82  [1,18,1]   Recovered via rewrite in-place
 656  947:09  00000001bb851e83  [1,18,1]   Successfully reassigned
 657  947:09  00000001bb851e86  [1,18,1]   Successfully reassigned
 658  947:09  00000001bb851e87  [1,18,1]   Successfully reassigned
 659  947:09  00000001bb851e88  [1,18,1]   Successfully reassigned
 660  947:09  00000001bb851e89  [1,18,1]   Recovered via rewrite in-place
 661  947:09  00000001bb851e8a  [1,18,1]   Recovered via rewrite in-place
 662  947:09  00000001bb851e8b  [1,18,1]   Successfully reassigned
 663  947:09  00000001bb851e8c  [1,18,1]   Successfully reassigned
 664  947:09  00000001bb851e8d  [1,18,1]   Successfully reassigned
 665  947:09  00000001bb851e8e  [1,18,1]   Successfully reassigned
 666  947:09  00000001bb851e8f  [1,18,1]   Successfully reassigned
 667  947:09  00000001bb851e90  [1,18,1]   Successfully reassigned
 668  947:09  00000001bb851e91  [1,18,1]   Successfully reassigned
 669  947:09  00000001bb851e92  [1,18,1]   Successfully reassigned
 670  947:09  00000001bb851e95  [1,18,1]   Recovered via rewrite in-place
 671  947:09  00000001bb851e96  [1,18,1]   Successfully reassigned
 >>>> log truncated, fetched 16124 of 51476 available bytes

Protocol Specific port log page for SAS SSP
relative target port id = 1
  generation code = 0
  number of phys = 1
  phy identifier = 0
    attached device type: expander device
    attached reason: SMP phy control function
    reason: power on
    negotiated logical link rate: phy enabled; 6 Gbps
    attached initiator port: ssp=0 stp=0 smp=0
    attached target port: ssp=0 stp=0 smp=1
    SAS address = 0x5000c50058c25dd5
    attached SAS address = 0x50030480011d203f
    attached phy identifier = 33
    Invalid DWORD count = 0
    Running disparity error count = 0
    Loss of DWORD synchronization = 0
    Phy reset problem = 0
    Phy event descriptors:
     Invalid word count: 0
     Running disparity error count: 0
     Loss of dword synchronization count: 0
     Phy reset problem count: 0
relative target port id = 2
  generation code = 0
  number of phys = 1
  phy identifier = 1
    attached device type: no device attached
    attached reason: unknown
    reason: unknown
    negotiated logical link rate: phy enabled; unknown
    attached initiator port: ssp=0 stp=0 smp=0
    attached target port: ssp=0 stp=0 smp=0
    SAS address = 0x5000c50058c25dd6
    attached SAS address = 0x0
    attached phy identifier = 0
    Invalid DWORD count = 0
    Running disparity error count = 0
    Loss of DWORD synchronization = 0
    Phy reset problem = 0
    Phy event descriptors:
     Invalid word count: 0
     Running disparity error count: 0
     Loss of dword synchronization count: 0
     Phy reset problem count: 0

root@freenas[~]#

[joeschmuck]

@xbmcgotham

Z1Z07WCQ0000S311BUKE

Drive S/N (da21) Z1Z07WCQ0000S311BUKE is definitely failed. That is currently in SPARE status. RMA this drive if it is still under warranty.

Drive S/N (da18) Z1Z4CWCR00009444VDPP , run smartctl -l farm /dev/da18 since this is a relatively new drive. While it is only 4TB,maybe it was used previously in a data farm. I guess you could tell us if it was purchased new or refurbished as well.
Look at the section of data called “Background scan results log” and take a read of all the errors. You should RMA this drive.

What I found interesting was this drive was performing its own internal SMART testing, without being told to do so. Learn something new everyday.

[xbmcgotham]

Thanks @joeschmuck!

I will follow up with your advice, and for both drives, these were secondhand and used before.

To clarify, what is the flow of steps I need to do till full resolve?

I understand this code is related to checking for fake Seagate drives smartctl -l farm /dev/da18.

Additional, in what order and steps do I perform the replacement of these two discs to complete having a “health” pool again?

Thanks again, and happy to have contributed to you learning something new again.

[joeschmuck]

I am going to teach you something and I hope you find it useful as well as anyone reading this thread.

Go to the TrueNAS documentation webpage and in the upper left corner is the TrueNAS AI Powered Search

Next click on it and then enter a question, for this example enter “How do I replace a failed hard drive in CORE?”

The thing will provide you a nice detailed description of what to do. The only thing it does not specify is to power Off/On your system. Of course everyone “should know” to power off before removing/inserting a component, unless it is a USB type component.

As for “Healthy Pool”, as @etorix implied, the RAIDZ3 pool is a bit too wide. (I think that is the proper term). You should consider a different format, but you did not come here for that advice, just wanted to point this out.

Let me know if this works for you.

[xbmcgotham]

@joeschmuck

How do I replace a failed hard drive in CORE?

Thanks, that is a nice tool. Thanks for bringing it to my attention!

I do have a question which the AI tool is not able to answer. Do I first replace the DA21 or the DA18 disc?

I am also interested to know still why the pool status shows two sections called spares. this just confuses me.

Lastly I am interested in that other advice related to the pool being as it is. What do you mean and how would you advice it to be made instead? This pool was setup by an IT specialist a couple of years ago, so I am surprised that he did it this way if its not recommended.

[xbmcgotham]

Update:

I tried replacing the da21 first, but I received a message that it could not do this as its part of a hot spare. I could not find an answer with AI related to how I would need to progress, so I replaced da18 instead, and its now resilvering.

After this resilvering is completed, what is the correct procedure for replacing disc da21? Pleas keep in mind that da22 spare is currently in use according to the log.

Thanks for your assistance.

[joeschmuck]

First, let me be perfectly honest, I am not the Pool Layout Expert, not by a long shot.

Which drive to replace first, it doesn’t matter as far as I am aware. You have a RAIDZ3 so you are protected from 3 drive failures so makes no difference. I personally think I would replace the failing spare first, but that is purely a personal choice. And let it all resilver. Then I would run a Scrub, you should not need to, however it is a personal thing. Then replace the drive in the pool.

As for instructions, I entered “How do I remove a spare drive from core?” and it provided the answer.

I will say, the AI help was terrible in the beginning and it is getting much better over time. The information is likely in there but the wording of a question may need to be slightly different.

The pool layout, as I said, I’m not expert. At least your IT person did a RAIDZ3 for such a large amount of drives in one VDEV, in your case one VDEV=your pool. For now, get your system back to 100%. Then you can read what the AI will tell you if you ask it this “what is the best pool layout for 24 drives?” and it will tell you some stuff and provide a link. Here is a link: ZFS Pool Layout White Paper

[xbmcgotham]

Thanks.
I did try the AI, but it just fails to tell me how to do the replacement for a spare drive. I guess after I do the resilvering I will try a few things. As you mentioned, not much can go wrong if there are at least 3 drives needed to fail first.

[joeschmuck]

@xbmcgotham

As you mentioned, not much can go wrong if there are at least 3 drives needed to fail first.

Take your time.

@xbmcgotham

I did try the AI, but it just fails to tell me how to do the replacement for a spare drive.

You “remove” the spare drive first. Once it is removed, power down, pull the drive (use the serial number to verify you pulled the correct drive), then install a new drive and add it back as a SPARE. I’m sure that while it sounds easy to me, I’ve been there where what is obvious to one person, is not obvious to another. I still hate Python but I need to learn it for a little project I have going on.

From the AI:
To remove a spare drive from TrueNAS CORE, you can follow these general steps using the web interface:

Access the TrueNAS Web Interface: Log in to your TrueNAS CORE web interface.

Navigate to Storage: Go to the “Storage” section and select “Pools”.

Select the Pool: Click on the pool that contains the spare drive you want to remove.

Open Pool Status: Click on the settings icon next to the pool and select “Status” to view the pool status.

Remove Spare Drive: Locate the spare drive in the list of devices. Click on the options or settings icon next to the spare drive and select “Remove” or “Detach” to remove it from the pool.

Confirm Removal: Follow any prompts to confirm the removal of the spare drive.

Please note that these steps are based on TrueNAS CORE 13.0 documentation, and the exact steps may vary slightly depending on your version or specific configuration.

For more detailed instructions, you can refer to the TrueNAS Documentation Hub.

Your feedback helps make this bot better! Click the thumbs-up or thumbs-down icons to rate responses. Thanks!

[Jorsher]

I don’t power off my system during drive, PSU, or fan replacements, because these are hot-swappable parts Might be good for it to tell users to shut down if drives aren’t hot-swappable, though.

After the resilvering is complete, the spare will go back to ‘AVAIL’ from ‘IN-USE.’ The spare is needed until resilvering to the new drive is complete. All drives, spare or not, are replaced with the same steps. Maybe just let the resilver finish before you move to replacing the next one, even though you technically can go ahead and replace it – it seems like only one drive is replaced at a time.

[etorix]

@xbmcgotham

After this resilvering is completed, what is the correct procedure for replacing disc da21?

Just remove the drive from GUI and let the spare become a permanent member.

[joeschmuck]

@Jorsher

Might be good for it to tell users to shut down if drives aren’t hot-swappable, though.

While I know people can elect to do that, I do not. I have seen what happens when a drive is inserted into a powered rack and the SATA power connector does not align properly. I almost shit my pants. And this was a quality server from Supermicro. Shit happens and after that, Power Down if you can, especially in a home system.

It only melted the plastic on both sides, drive was toast, that one slot was toast. Good thing we had two empty slots to use and a good thing the wiring didn’t remain shorted out. This is why everyone should have a 10,000 Watt power supply, to burn away any metal bits.

We replaced the server later that day. Just something I will never forget.

[xbmcgotham]

Thanks so much all for your assistance. Resilvering is still running its taking some time still. I keep you posted.

[xbmcgotham]

Hi @joeschmuck, @etorix and @Jorsher update from my end.

After resilvering finished yesterday, this morning I removed drive da21 by clicking REMOVE from the Status page. After this I powered down the NAS and replaced the drive. I made sure to check if it was the correct serial. After rebooting, I notice that the new drive is added as da23 but in says N/A for pool instead of being in pool1.

I can also see the drive in the VDevs pool manager page.

When I try to Expend the pool in on the pool page I am getting the following failed message:

Blockquote
Error: Traceback (most recent call last):
File “/usr/local/lib/python3.9/site-packages/middlewared/job.py”, line 367, in run
await self.future
File “/usr/local/lib/python3.9/site-packages/middlewared/job.py”, line 403, in _run_body
rv = await self.method(*([self] + args))
File “/usr/local/lib/python3.9/site-packages/middlewared/schema.py”, line 973, in nf
return await f(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/pool/expand.py", line 78, in expand
part_data = all_partitions.get(child[‘device’])
KeyError: ‘device’

I cant find any resolution online. Any ideas what is going on and how I can add the new drive to the pool?

Another question I have is how do I put the drive da20p2 or da22p2 back as spares so I have again 2 spares the pool can fall back to when needed? (da20p2 is shown as SPARE in list with the remark UNAVAIL, please see earlier post for details)

Have a nice evening! Thanks again.

[Jorsher]

You don’t choose ‘expand,’ you go to the removed drive and select ‘replace.’ It will replace the old disk with the new one.

Go to “Storage,” click “Manage Devices” on the pool, find the removed disk, click it and select “Replace.” You should have the new disk in the dropdown.