I have noticed this too.
I do recall it working in parallel in the past, but its certainly serial these days.
I suspect the behaviour may have changed when “Sequential Re-silvering” was added to ZFS.
And yes…
And good news, is apparently there’s a solution
You’re hitting the
resilver_deferfeature, which attempts to prevent restarting if another disk faults in the middle of a resilver, but mostly surprises people who try to resilver multiple things at once. See #14505 for a discussion.
So, the quick summary is this is a feature of resilver_defer feature which defers future pended resilvers until the current finishes, as you would have to restart the resilver that has already been running… and may have been running for days… thus future resilvers are deferred until current ones run…
But this means that in the situation where you start a bunch and you want them all to run simultaneusly, they won’t.
You can force it to do them all immediately with
zpool resilver.
So, if you instead force a resilver, it will start resilvering again… with ALL the pending resilvers being done in parallel.
A neat trick. And yes, it would probably be better if OpenZFS had implicit magic to make this decision.
BTW, this probably means you can replace 2 drives at a time per vdev… or perhaps even more?
Let me know if it works 