Replaced drive shows in both vdev and spares after resilver in both web UI and CLI

TrueNAS General
, ,
1

Hi folks. We’re running Scale 23.10 (can’t update at this stage because of an SMB extended-attributes bug in 24.10+ that breaks our backups) on two machines with 84 HDDs and 6 SSDs. Our layout is 11 vdevs of 7 drives in a RAID-Z2 with 7 spares, 3 log SSDs and 3 metadata SSDs.

3 months ago, we had a drive failure in one machine and it was replaced. While doing some maintenance, I just noticed something strange - the pool spec shows 8 spares, not 7. zpool status from CLI shows the exact same. I checked the serial numbers of the drives, and sure enough, it’s the replacement drive - it’s showing in both the vdev, where it should be, and in the Spares list. All of the vdevs show 7 drives.

I don’t know if this is harmful but it’s certainly confusing. What’s weirder is that these two machines are multi-site replication partners; the one with the replaced drive is the target, but I noticed the same thing on the source, it’s also showing 8 spare drives. We had a similar disk failure and replacement 5 months ago, but the extra spare drive in this instance is not the replacement.

Anyone have any suggestions for whether we need to do anything here? Like I said, we cannot currently upgrade these machines from 23.10; we are replacing our backup software and will eventually be able to upgrade, but not right now.

Cheers,

Rob

2

Minor correction, not extended attributes, alternate data streams. There’s a topic from my colleague on here but I’m unable to link to it.

3

i think the replace left that disk in the spare list too. zfs sometimes keeps the spare role after the new drive is already in the vdev.

check:
zpool status POOLNAME

then yank it from spares only (doesnt touch the vdev copy):
sudo zpool remove POOLNAME DEVICE

use the spare path from status (gptid/… or /dev/disk/by-id/…). if remove says its still attached as a spare:
sudo zpool detach POOLNAME DEVICE

after that status should show 7 spares again. same on the other machine if its showing 8 too.

not harmful while its listed twice, just confusing. paste zpool status if remove errors.