My new NAS does not yet have redundancy on it’s main pool. It is a single 10TB disk at present. I have 3 backup disks, (well, 4, but the 4th does not include the huge Media pool). So I am running with some protection. (The AI craze has reduced me to a Stripe pool for the moment.)
Now when this server was planned a long while back, I bought a 240GB SATA SSD to use a Metadata only L2ARC / Cache device. Just now it has been added into the single disk main pool. I ran multiple tasks to try and populate the L2ARC device. Unfortunately I have too much memory, (32GB), for the pool size. I guess most of the Metadata is simply in ARC.
But, that brings up a question.
Can the L2ARC act as a source of redundancy?
When scrubbing a pool, if a Data block is bad, (& copies=1), normally an un-correctable error is made. However, if that Data block is in L2ARC, (or even ARC!), can the scrub refresh the Data block on the Stripe vDev, making it all better?
For Metadata blocks it is more complicated. First, using normal defaults of “copies=1” and “redundant_metadata=all”, we already have 2 copies of regular Metadata blocks. And Critical Metadata blocks have 3 copies. All on single disk pools, (or Striped pools across several single disk vDevs). This seems to work, as I think I saw it recover such once on my Media server’s non-redundant media pool.
If the L2ARC can act as a source of redundancy, then perhaps I should go back to Caching Data blocks, in addition to Metadata blocks.
What do you think?
Is this a question for the ZFS Gurus on the OpenZFS mailing list?
I think it is, but the question and solution is only theoretical, and my guess is that the devs wouldn’t see it as worth the effort. As you noted, there are other safeguards in place, including multiple copies of metadata, even if you have copies=1 on a single-disk vdev. They might also argue that if data integrity is important to you, you might as well create a simple mirror that will protect everything, not just metadata. They might also argue that there are no assurances that all the metadata is found in the LARC and/or ARC. If a corrupt block is only on the storage vdev, but not in ARC or L2ARC, then it wouldn’t be caught or repaired by an arbitrary scrub or read. They might also argue that if corruption of metadata already occurred on the storage vdev, then the ARC and L2ARC cannot be used as a “redundant” copy to later repair it.
Hmmm, a lot of “you got a striped pool, so live without the redundancy”.
In the case of dual Metadata corruption on a striped pool, if the checksum is how they were caught as bad, and the checksum of the L2ARC still has a good checksum, then why wouldn’t it be suitable?
ZFS would not populate a L2ARC Metadata entry with a copy that failed it’s source checksum.
Yes, current ZFS scrubs might not use ARC for redundancy. But, a L2ARC device is actually part of the storage pool. So, why not scrub it too? And, if needed, use it for redundancy.
To be clear, my new NAS’ striped pool is intended to have a mirror. It is just that the cost is a bit high now… though I might just bite AI’s *ss, and pay it. Or I could use one of the backup disks temporarily as a copy.
To sum up, yes I can probably ask the ZFS gurus questions. But, I need to word the questions carefully. Like:
