Guys, come on! This thing is a 13-year-old consumer motherboard and CPU 
I doubt that there is an OpenZFS bug.
I enjoy watching this show as a troubleshooting puzzle. From a realistic perspective, itās not worth even looking into it.
Yes, it is! I would like to know the origin of the problem! As I say before, The H81 is a good candidate, but I donĀ“t know if is there a way to test it aloneā¦.
About the results of scrubā¦
</>pool: data-pool
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: Message ID: ZFS-8000-8A ā OpenZFS documentation
scan: scrub repaired 0B in 03:59:27 with 6 errors on Fri Feb 20 03:26:28 2026
config:
NAME STATE READ WRITE CKSUM
data-pool ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
4ef5eff0-f52b-4058-bc4c-dbbdc4416f4b ONLINE 0 0 40
f94d9511-92d3-433c-a9b1-92c2bbe6f6c8 ONLINE 0 0 40
2300bd5f-3068-4791-a5a4-3b8421c14229 ONLINE 0 0 40
errors: Permanent errors have been detected in the following files:
</>
Scrub seems to not repair any file. It shows that there are permanent errors in files, but no list showed.
About checksum errors, no way to stop it. It continue growing as same rate as beforeā¦.
Most important about the timming of the problem. IĀ“m checking when the problem starts andā¦one week ago, system alerted me, because pool was market as āDEGRADEDā. But as I can check today, system donĀ“t alert me about checksum errorsā¦ So, It could be possible that the checksum errors started after Truenas mayor update two months ago??? Is there a way or log for tracert then??? Is it possible that a fail partial update, stale zfs???
Keep the LSI9300, swap the mobo, see if the errors persist.
If not, you ruled out PSU, cables, HBA, drives.
I already order a new mobo+processor+ECC RAM
I hope, once of then will solve the problem!!!
While I donĀ“t receive the items, do I must disconnect all 3 affected drives for precaution???
Thanks for help!
Probably not a bad idea. I guess you donāt have any other hardware atm you could connect these three drives to and import the pool to see how things look?
1 Like
I have another two Truenas Server but only for enterprise useā¦. I could make another basic desktop hardware and test to import it!
Probably a good idea. That way you can get the pool healthy again.
1 Like
Youād be surprised at the bugs in OpenZFS that made it into stable and point releases.
Iām not saying this is the case here, but if all other options are exhausted, it doesnāt hurt to consider it.
IIRC, there was a rare bug that would not decompress LZ4 compressed blocks, which would load invalid data. The checksum (of the raw block pulled from disk) would match, but the application would see the file as corrupt / unusable.
Then thereās the bug of sending (without -w) an encrypted dataset that would corrupt files on the destination.
Then thereās the bug that would crash a system during import if block-cloning had been used before a vdev removal.
OpenZFS is not infallible.
If no other solution can be found, itās possible that:
- this is a reporting issue
- the wrong checksum algorithm is run in RAM (i.e, Fletcher4 saved on disk, Blake3 runs in RAM), maybe from a rare combination of factors
Heās had this hardware with no issues until suddenly he gets over a million errors per day, consistently?
How come it only affects this one particular pool, but not the others?
How come his system is running fine? If the hardware was so shoddy, youād think he would be experiencing constant instability and crashes.
Whatās really interesting is that it appears every single block on this very particular pool is failing the checksum, if weāre to believe that over a million blocks are consistently read every day. @luiscaloto showed that the checksum errors climb at a predictable rate. This is why I suggests he manually inspects files to confirm if they are being read correctly (text files, JPEG images).
I agree that itās unlikely this is a rare bug in OpenZFS, but itās worth considering if changing all the hardware does not resolves this.
3 Likes
@luiscaloto What were the permanent errors? I donāt actually need those files, Iām only asking to ensure that you did actually delete all original 19 files. 6 files is not bad.
If you find out that you missed deleting 6 files, then you can delete them, make sure they no longer exist, clear the zpool alarms, and run a scrub again.
If the files are different than the original 19, then, since you have ordered a new system, and if any data on this system is important, I would Export the pool and wait until you have your new hardware. Ensure you have tested the new hardware, MemTest86+ for at least 5 full passes and a CPU Stress Test for 4 hours. This should rule out a faulty component before you start all over again.
If you have all your data off your data-pool, you could destroy the pool, run a smart long test on each drive, and then recreate the data-pool VDEV while you wait for the new components. This would test the drives and maybe the system starts to act normally again. But I cannot say this will solve anything. And again, only if you can afford to destroy this pool.
If you have already run a smart long test on each drive in the past few days and they passed, there is not need to do it again.
Good luck. I hope you are able to find the issue.
Could be possible that this unkhow error only affect to RAIDZ1??? no Mirrors???
OK! Perfect!
IĀ“ll do test before move the server
I gonna move the pool to a new clean installation of Truenas 25.10.2 to check if checksum errors still be there!!!
If no solution, I gonna recreate data pool as you say!
IĀ“ll send news! Thanks to all 
It doesnāt hurt to confirm some things.
Have you manually inspected JPEG photos and text files yourself? It can be a random sampling of 50 or 100 files.
You can view them or copy them over an SMB share. If you have āoriginalsā, they can be compared with SHA512 hashes.
If a random sample of your files on this āheavily corrupted poolā are not corrupt, then I think it means thereās something else going on, and maybe ZFS is improperly marking your data as corrupted.
Iād like to know what is happening as well but we need to think about the user in trouble, what benefits that person the best.
Some of the things being suggested in this thread might be beyond a personās current knowledge level. I know I was there for a while, a long time ago, but I was still there. And Iām not saying anyone is wrong at all, I might do what you have suggested if I had to have the data.
I think heās taking a smart way out (for him) if all the data is backed up or not a big loss. It is fast and will tell pretty fast if there is a hardware issue. If I were messing around with this problem for a weekā¦ Iād want to pull my hair out, oop, God already took it. 
Actually, I would only take a few days before I nuked the pool and started all over again, but my important data fits on a 1TB NVMe. The rest are just backups of other systems, which if they validate, I could copy to another hard drive for recovery later, again, if they validate.
And you know me, if we disagree, Iām okay with that. Everyone has an opinion and should be able to say it. But of course, if you ever see me provide plain wrong advice, I hope you, or anyone else chimes in and corrects me. I know Iām not perfect, if I were perfect then my wife would kill me for always correcting her. It is āYes, you are right, or Yes, I can see it from your perspectiveā, never āYou are wrong and I am always right, get onboard with it.ā She watched those shows about how killers get caught and I saw her taking notes, humā¦
Cheers
I checeked then yesterday tonight. I check about 40 jpg an it seems ok. I will check tonight hashes to confirm it!
By the other hand, all important data is backed up!
IĀ“m fighting with this problem because I would want to find the guilty!
Thanks to all again!
If the JPEG photos appear without corruption and their hashes match the originals, but ZFS insists that youāre getting over 1 million checksum errors every day, then something fishy is going on.
Honestly, as much as I feel for the OPās situation, Iām really curious to see if changing the motherboard, CPU, and RAM actually makes a difference.
On the topic of drive temperatures: I remember reading a Google study a while back conducted on a large sample of hard drives. At the end they claimed that drives constantly running above 45ā50Ā°C were more prone to premature failureā¦ but interestingly, the same applied to drives that were running too cold. For me Itās not that hard to believe, if you make a broad comparison with engines, the same principle can be applied.
I have to admit, Iām not sure if that study still perfectly applies to āmodernā drives, but Iād definitely rather stay on the safe side and properly cool my disks 
1 Like
Hello!!! I have news, not good ones!!!
I decide to back up de pool before to recreate it, and I find (for the moment) two folders with problemsā¦
One of then is Immich App folder. ls -l show this:
</>
ls: cannot access ā/mnt/data-pool/FotoVideo_Hogar/Originales_Hogar/Immichā: Input/output error
total 1379
d??? ? ? ? ? ? Immich
</>
As can I know, the error disk appeared! So, I can scrub the pool, but error persistā¦ I already read, that, pool canĀ“t run fsckā¦ In this case, How can I know which is the problematic disk??? Is there a way to attemp to repair it, into the pool?? How smartctl donĀ“t get this info???
Thanks again!
I already find the problemā¦..!!!
No hardware issue except one disk. I replaced the affected disk, but checksum rate didnĀ“t stop! I found a subfolder used by Photoprism with Input/output error. The app was always deploying, causing the rate of Million checksum error per day!!!
Thanks to all for the help!
1 Like
IMHO you found a symptom causing the errors, not the reason behind these errors.
The reason was a faulty disks, the cause of error rate was the app always trying to start, as I understand.
Now, time to rebuild the pool!
As I can find, my ups battery was not healhy, so its power not enough to keep server alive before It can shutdown. I suspect that was the reason to kill metadata
As far I know, ZFS keeps multiple copies of metadata and was designed specifically to not corrupt data on powerloss.
I would not bet exclusively on an UPS to be the solution.