Hi, I’ve been using Truenas for a while now and its been rock solid. Sadly today we had a power outage and on startup I’m getting a kernel panic. Before I start digging around I’d like to make sure I’m not going to hose up my data if possible.
The server is an oldish HP Elite desktop with 16GB, two 6TB WD drives configured as RAID 1 and a solid state primary disk.
It looks like the OS boots up, I see ‘Finished zfs-mount-service - Mount ZFS filesystems’.
The panic message starts with:
PANIC: zfs removing non-existant from range tree (offset=xxxx…)
Any suggestions on where I should go with this?
Pete
Do you have a recent saved configuration file and do you have a datta backup of your mirrored pool with two HDs?
I would suggest browsing the forum and reading a few threads and posts. We need to get your Trust Level up so we can see screenshots and have you post more.
Hi, thanks for replying. I’ve had a browse but can’t find anything with this particular problem.
In answer to your questions:
No, I don’t have a copy of the latest config file - its still on my todo list which isn’t very helpful.
Most of the data is in other places so I won’t have lost that, but its also got my daily backups from other systems, not a big deal if I lose that.
I have booted into single user mode this morning and pulled off the saved journal file for the last failed boot yesterday. I’ve attached this, sorry its in reverse time order (journalctl -ra)
journalctl.txt (65.4 KB)
The disks all look ok using lsblk (attached).
lsblk.txt (455 Bytes)
and the boot disk has plenty of space:
df-h.txt (1.5 KB)
As single user root what are the next steps to try and repair the pool? Sorry I’m not at all familiar with zfs and truenas and struggling to find a step by step guide to fault finding.
Finally this is a photo of the panic at boot:
OP’s file
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 5.5T 0 disk
├─sda1 8:1 0 2G 0 part
└─sda2 8:2 0 5.5T 0 part
sdb 8:16 0 223.6G 0 disk
├─sdb1 8:17 0 512K 0 part
├─sdb2 8:18 0 207.6G 0 part
└─sdb3 8:19 0 16G 0 part
sdc 8:32 0 5.5T 0 disk
├─sdc1 8:33 0 2G 0 part
└─sdc2 8:34 0 5.5T 0 part
sdd 8:48 1 2G 0 disk /mnt/usb
You can post text like above using Preformatted Text (ctrl +e) in the reply. Button is like </> right above where you type out your replies.
I’m going to wait to see if anyone else posts and comments on your kernel panic. I’m not sure if just a fresh install of TrueNAS on the boot drive will fix this.
You would have to set up all shares and apps all over again but the main pool should just get picked up or you would import pool
Thanks for the pointer about formatting the data.
This morning I managed to get the pool imported read-only using some zfs suggestions at https://github.com/openzfs/zfs/issues/13483
specifically on the linux command line I added zfs.zfs_recover=1 zfs.zil_replay_disable=1 and ran the zpool import -f -F -X pool1 command. That glagged up erros but imported the pool as far as I can tell.
That allowed me to get the UI up and take a backup of the configuration.
This didn’t fix the panic on the next boot so I’ve decided to re-install and as you say see if a fresh install works. If not I’ll nuke the entire box and start afresh, I don’t have much spare time to start digging further as this box also provides shares for the rest of the family and my business.
Thanks
Pete
