Fixing a degraded array - not sure how I reached this state

TrueNAS General
1

One of my 2 Z2 vdevs that form my main storage pool is shown as being degraded. I believe this is because I accidentally restarted the server with the drive disconnected from when I was performing some maintenance.
My question is what do I need to do to fix the issue.
When I expand the array I see

unavail
unavail810×487 25.4 KB

replacing
replacing810×381 17.8 KB

zpool status yields

config:

        NAME                        STATE     READ WRITE CKSUM
        tank                        DEGRADED     0     0     0
          raidz2-0                  ONLINE       0     0     0
            sdk2                    ONLINE       0     0     0
            sda2                    ONLINE       0     0     0
            sdi2                    ONLINE       0     0     0
            sdh2                    ONLINE       0     0     0
            sdg2                    ONLINE       0     0     0
            sde2                    ONLINE       0     0     0
          raidz2-1                  DEGRADED     0     0     0
            sdj2                    ONLINE       0     0     0
            replacing-1             DEGRADED     0     0     0
              sdb2                  ONLINE       0     0     0
              14853449693688436450  UNAVAIL      0     0     0  was /dev/disk/by-partuuid/0678952a-bc37-40f4-a8dc-49ff3fd1b4bc
            sdn2                    ONLINE       0     0     0
            sdd2                    ONLINE       0     0     0
            sdc2                    ONLINE       0     0     0
            sdf2                    ONLINE       0     0     0

errors: No known data errors

Disk sdb shows options Extend, Detach and Offline

Does this mean that some point I may have swapped a fresh drive in before offlining the faulted one or done something to convince the array there should be something there that isn’t - I see 12 drives in total (2 x 6) as expected?

The numbered drive shows options Detach in the ZFS Info section or Replace in the Disk Info section of the UI for Manage Devices. All the other devices seem fine.

Can someone advise the correct course of action here?

2

Is the zpool resilvering / scrubbing? If so, then do nothing until it completes and then examine the state of the zpool.

zpool status is showing 12 drives in the zpool. Based on what you said, did you have a spare device? Did you connect the disconnected drive after realizing it was not connected?

3

The resilvering is complete. I believe it may have happened after pulling a drive and replacing with a new one without off lining it first (like a noob). I’ve then realised my mistake, checked it isn’t resilvering and put the original back - the error had come from a loose cable rather than a fault I believe (drive sdd). Whilst doing maintenance (fan controller connect) I’ve started the unit with sdb disconnected. I’ve then connected the drive. Any resilvering that has occurred in this time has been allowed to run its course.
The UI for each of the drives is thus…

truenas_numbered
truenas_numbered2261×421 39.6 KB

truenas_sdb
truenas_sdb1423×614 55.6 KB

I believe the array should be fine - all the original drives are present and unfaulted - but the system seems to think a drive is being replaced. Can I just detach the numbered drive? There are only 12 HDD drives connected and only ever were as that’s all the connections I have available.

4

I would:

  1. run a scrub
  2. if the scrub shows no errors, then
  3. detach via the GUI the missing drive
  4. run another scrub
  5. check scrub results, if clean you are done and good

Yes, I tend to run a lot of scrubs :slight_smile:

5

Thanks for that. The second scrub is running but the status of the array went from degraded to healthy when I detached the non-attached disk.

6

Glad to help you recover.

ZFS is amazingly resilient if you let it do it’s thing and don’t over think it :slight_smile: