Unexpected Power failure during resilvering

TrueNAS General
,
1

Unexpected Power failure during resilvering.

Hello hoping someone can help me with this. I had to replace a disk yesterday day (removed disk from computer and replaced with new) but during the initial resilvering a power outage occurred. I had a little time before my battery backup ran out to google and try to figure out what I should do. What I found in the time I had was to take the new disk offline (hoping to stop or pause the resilver process) and power off the server. Which I did.

Once the power outage was over I checked my volume and it was ok, still degraded but accessible.

Next I went into the pool status and went to replaced the old disk with the new disk again. I did the replace using the force command but got an error, shortly after the pool started doing scanning for a resilver. I let that run overnight.

This is the status of the pool now in the morning. I am running a RAIDZ1 with four disks.

root@freenas:/dev # zpool status MyVolume1
  pool: MyVolume1
 state: DEGRADED
  scan: resilvered 0B in 14:24:08 with 0 errors on Wed Nov 27 06:59:20 2024
config:

        NAME                                              STATE     READ WRITE CKSUM
        MyVolume1                                         DEGRADED     0     0     0
          raidz1-0                                        DEGRADED     0     0     0
            gptid/debd3c93-3f49-11ec-8e9a-ac1f6b251a04    ONLINE       0     0     0
            gptid/90ab8ae0-c388-11ee-acd1-ac1f6b251a04    ONLINE       0     0     0
            gptid/23ab438d-af55-11ea-b961-ac1f6b251a04    ONLINE       0     0     0
            replacing-3                                   OFFLINE      0     0     0  all children offline
              gptid/59a7a304-dba9-11e7-88fe-ac1f6b251a04  OFFLINE      0     0     0
              gptid/50f41041-ac3e-11ef-8dd6-ac1f6b251a04  OFFLINE      0     0     0

errors: No known data errors

I am not sure what the next steps are at this point. Im guessing I should have brought the new disk back online after the power failure, not sure if doing that now is a good idea or not.

2

Uh, ZFS was specifically designed to recover from power outages. If you did nothing, at boot, the server would have continued the disk replacement from where it was at shutdown or power loss. WITHOUT ANY DATA LOSS.

ZFS was purposefully designed and implemented to deal with both unexpected power losses and crashes. As well resuming scrubs or replacements after graceful shutdowns and then boots. Again without any data loss.

Well, except for any data in flight. Which in your case might mean an extra few seconds of re-doing some blocks that had not yet been fully re-synced before the power loss. But, that is the purpose of Mirroring or RAID-Zx, being able to recover data.

Now, as to where you go from here, try this;

zpool online MyVolume1 
gptid/50f41041-ac3e-11ef-8dd6-ac1f6b251a04

Assuming that disk is the new disk you put in.

Let us know the zpool status MyVolume1 after you online the disk.

2 Likes
3

If possible put back both the old disk and the new disk.

2 Likes
4

That worked, thank you for your help!

Disk went online and resilver started. completed overnight. pool is back to normal online state.

3 Likes