Endless checksum errors between multiple drives

TrueNAS General
21

Okay, this is getting weird.
After running sudo zpool clear HDDs, the whole .system directory doesn’t exist, and the error on it persist.

truenas_admin@HexOS[~]$ sudo ls /mnt/HDDs/.system/ -a
ls: cannot access '/mnt/HDDs/.system/': No such file or directory
truenas_admin@HexOS[~]$ sudo zpool status -v   
  pool: HDDs
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub repaired 0B in 06:48:33 with 2 errors on Tue Nov  4 22:47:34 2025
config:

        NAME                                      STATE     READ WRITE CKSUM
        HDDs                                      ONLINE       0     0     0
          raidz1-0                                ONLINE       0     0     0
            00f9e94d-2c50-4903-b984-501286530311  ONLINE       0     0     0
            974f1184-6c2d-49a4-b19f-f675387f7bbb  ONLINE       0     0     0
            98f8ae81-f736-414d-8afa-ada57c274e28  ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

        HDDs/.system/netdata-ae32c386e13840b2bf9c0083275e7941:<0xc>
        HDDs/.system/cores:<0x12d>

  pool: boot-pool
 state: ONLINE
  scan: scrub repaired 0B in 00:00:21 with 0 errors on Mon Nov  3 03:45:23 2025
config:

        NAME        STATE     READ WRITE CKSUM
        boot-pool   ONLINE       0     0     0
          sdb3      ONLINE       0     0     0

errors: No known data errors

BUT THE CHECKSUM ERRORS ARE GONE NOW. OK???
In any case, I’m running SMART on all the drives now. If it still reports no issue…I’ll be re-installing the OS.

22

This cleared the errors. I’m also pretty sure you’d have to mount the pool before you can check the directory.

23

sudo ls /mnt/HDDs/ works, it’s just listing personal stuff so I didn’t include them

1 Like
24

Okay, so there’s another guy facing the same issue, using his zfs list shows that it is also my legacy mounting point…Sadly no solution in the thread

Annnnnnd another guy having the same issue, seems like also because of the power splitter, cool

THIS IS WHAT I’M LOOKING FOR, ISN’T IT
…No, I don’t have a snapshot…Still, it’s progress…?

25

Well, the problem is with boot-pool, so actually you have snapshots (probably). Although, I don’t know whether they are useful nor whether they are not corrupted.

You can list them with something like sudo zfs list -t snapshot -r boot-pool/.system. Again, I don’t know what to do with them. I would already just back up the config and start with a new boot drive.

I’ve re-read your sudo zpool status -v output. I was wrong about boot-pool. You should ignore my message. Or adjust the command for your pool.

26

Yeah that’s the plan. I’m just waiting for SMART to finish.

27

Well, after making sure HexOS will still be functional, made a re-install without porting the config…

truenas_admin@truenas[~]$ sudo zpool status -v
[sudo] password for truenas_admin: 
  pool: HDDs
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub repaired 0B in 06:48:33 with 2 errors on Tue Nov  4 06:47:34 2025
config:

        NAME                                      STATE     READ WRITE CKSUM
        HDDs                                      ONLINE       0     0     0
          raidz1-0                                ONLINE       0     0     0
            00f9e94d-2c50-4903-b984-501286530311  ONLINE       0     0    20
            974f1184-6c2d-49a4-b19f-f675387f7bbb  ONLINE       0     0     0
            98f8ae81-f736-414d-8afa-ada57c274e28  ONLINE       0     0    20

errors: Permanent errors have been detected in the following files:

        HDDs/.system/netdata-ae32c386e13840b2bf9c0083275e7941:<0xc>
        HDDs/.system/cores:<0x12d>

Well, the problem WAS on the drive. In a path that ls shows nothing. Checksum errors still on 2 drives out of 3.
I guess…I’ll just have to dive into the error. One last resort would be copy everything into another drive, reformat and copy them back…I really hope not.