Truenas Scale can't boot - ZFS error, kernel panic

TrueNAS General
1

Hi,
my truenas scale version Dragonfish-24.04.2, won’t boot.
I’ve checked the logs and found this kernel panic:
kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module 545.23.08

kernel: VERIFY3(range_tree_space(smla->smla_rt) + sme->sme_run <= smla->smla_sm->sm_size) failed (17181896704 <= 17179869184)

kernel: PANIC at space_map.c:405:space_map_load_callback()

kernel: Kernel panic - not syncing: VERIFY3(range_tree_space(smla->smla_rt) + sme->sme_run <= smla->smla_sm->sm_size) failed (17181896704 <= 17179869184)

Unfortunately i can’t understand what’s wrong with my system and how to repair it.

I’ve already checked the pool disks, the memory (ECC), change the power supply and even the raid card with an identitcal one but without success.

Would you please help me recover my system?
Thanks.

2

Sounds like a failed or failing boot device. Do a clean install on another boot device, upload your backup of your config file, and you should be set.

3

Was the first thing i’ve thinking about, but removing pool disks resolve the issue. Without loading the defecting pool the the system works fine. So i’ve check power supply, raid card and disks.
Everything seems fine, even the boot device I think, otherwise the system would not boot even with the bad pool removed.

4

You analysis seems sound, except for one detail. Without the data pool to import, certain code segments of the boot pool won’t be executed.

So, with the data disconnected and the TrueNAS server booted, run a scrub on the boot pool. If you have any un-correctable errors, use this to see what they are;
zpool status -v boot-pool
Even if you get no errors, if you have a spare boot device available, it is pretty harmless to try one.

Note: Even with a ZFS pool that has no redundant vDevs, (like Mirrors or RAID-Zx), by default all Metadata is redundant. Thus, you can see correctable errors even on an un-Mirrored boot pool. (I saw this on my un-redundant media pool once.)

5

your pool has an issue with it’s free space map… did your server crash or shutdown ungracefully while freeing/deleting something? is this a data-pool or a boot-pool?

if it’s a data-pool… boot the machine with enough drives not connected such that it can’t import. at this point connect them back and see if it imports read only. and try a few more options on this page. these sorts of issues tend to be an issue with the last few transactions, and can usually be recovered from.

6

First thing first thanks everyone for the answers. As you suggest i’ve scrubbed the boot pool but can’t find any error, next step i’ll try to swap the boot device.
As much as i know the server didn’t crash, and it’s powered by an online ups with auxiliary battery.
Yes it’s a data pool and if i remove enough drives the system boot, so when i reconnect the drives it allows me to import the pool read only, showing each dataset etc. but couldn’t go any further.

7

Hi,
sorry for the absence of updates but i’m dealing with too much work at the moment and I am not able to focus on it. I hope to have a few minutes this weekend.

8

Hi,
some small updates, i’ve settle a brand new installation on a completely different machine, whenever i try to import the data pool the system reboot itself after the usual kernel panic.