On truenas scale fangtooth 25.04.2.1. Every has been stable for awhile now, was working fine yesterday up until late last night. The last time I know it was working was I was using mealie to do some cooking.
Then I noticed the pihole wasn’t working late last night, and then when I had time to look at it today I noticed it wasn’t even showing up on the network and then I hooked it up to a monitor and noticed that it’s boot looping.
No errors as far as I can see. Tried booting it up in debug mode. Tried booting up older versions in debug mode.
it gets to 1/2 job ix-zfs.service/start running (38s / 15min 27s)
seen it sometimes get to 2/2.
Then it just black screens and goes back to bios.
The only thing I can think of that happened recently is we had a power outage, but that was friday.
Not really sure of how to even debug it since it isn’t spitting out errors and I’m not even getting to a shell.
Recommendations? Next steps?
If you have different boot environments (left from the previous versions), you can try to boot from one of those. Just select it during the boot.
I have tried going all the way back to 23.10.1, no luck. All are doing the exact same thing.
Agh, you already mentioned that you have tried to boot from older versions. My bad…
Seems like a hardware issue then. But I have nothing to advise. Mb you should try to boot from some live usb… Or mb install truenas from scratch (on another drive). Just to see whether your hardware is functional.
A few updates.
live usb of popos, worked.
A boot option for truenas I skipped over was the initial-install version. That runs.
I got an error to appear on boot. PANIC: zfs: removing nonexistent segment from range tree (offset=94d4d65000 size=2000) Kernal panic - not syncing: zfs: removing nonexistent segment from range tree (offset=
Well, after adding a new drive and putting a fresh install on it, if i try ti import one of my pools it completely crashes it
For anyone that runs into similar issues.
Best guess is that the pool got corrupted when the power outage happened and it took a day + for the effects to happen. My best guess off of some of the errors I ran into and other things I read is that it thought it had no freespace and was in a read only mode.
I don’t remember all the commands I ended up running exactly, but I was able to follow the below to restore a snapshot.
What I ended up doing is importing the pool as read only which was the only way it would work, gziping the snapshots recursively for the pool, formatting the drive and wiping it, using it to create a new pool, restoring the snapshot onto that new pool, and then going back to my original truenas install and importing the pool in. Everything so far seems to be working. Minor dataloss, but I went back 1 day on the snapshot to ensure the snapshot I restored wasn’t also corrupted.
https://blog.jeanbruenn.info/2024/06/13/restore-zfs-pool-from-snapshot-turn-single-disk-vdev-into-mirror/
1 Like
