root@truenas[~]# zdb -l /dev/ada2p2 -C /data/zfs/zpool.cache
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
root@truenas[~]#
You should be using SSH. The Shell in Core has been broken since 1500 A.D.
So then this?
zdb -l /dev/ada2p2 -C /data/zfs/zpool.cache
EDIT: Just saw you post it.
couldnt SSH cause WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
presumeably becuz of the same IP as SCALE
I had that wrong, it should be -U, not -C.
zdb -l /dev/ada2p2 -U /data/zfs/zpool.cache
still failed to unpack lable 0-3
I’m out of ideas.
I wonder if you had chosen “RAID” for SATA mode, rather than AHCI before the BIOS settings reset?
If anyone knows a way to do a deep search for the ZFS uberlock, please share.
This feels like it should be smacking us in the face…
im almost certain that i never used the RAID option -
You can try Klennet ZFS Recovery software to see if if finds anything.
Updated link to ZFS recovery section
This will be what I try soon I think as it seems like we’re running out of ideas and options
I do want to ask if you guys think professional data recovery services will be any step above what we’ve tried. Im not really sure how far up the tree we’ve climbed in terms of data recovery techniques and where the ceiling is in relation to what we’ve done
You can try that software and see what it reports back. You only pay if you want to recover what it finds. I would expect the scan and everything to take a while. Maybe search the forums for the name. I think at least one other person has used the scan. I don’t know if they used the paid services
yea im familiar with the fact that the evaluation copy is free to use. Im inquiring about taking harddrives to “data recovery specialists” businesses or labs
I really don’t know on that. I have no experience with using those services nor, from what I have seen in this thread, do we know what happened with your system.
I would try the Klennet Recovery just to see if there is any data to recover on those disks.
I edited the link for Klennet. I didn’t have it pointing to the ZFS section
Something about this really bugs me.
I don’t like the idea that an unexpected reboot can make you lose your entire pool, when there are no VMs involved, no HBAs, and you’re using “standard” SATA drives plugged into ports on the motherboard.
Something seems off about this.
If I was considering ZFS, I would avoid it if this was my first introduction to it. (“You can lose everything? Just like that?”) However, I know that ZFS is not so unreliable.
“Something” happened.
Never would I want to boot my server after a sudden power loss or unexpected reboot, only to find out “Oops! Looks like your entire pool is gone!” That’s simply unacceptable.
Okay, let’s get ugly. sudo needed if you’re on SCALE, CORE run without it:
sudo hexdump /dev/ada2p2 | head -n 40
Assuming ada2p2 is still the ZFS partition here.
1 Like
Pardon if this has already been mentioned (thread is looong), but maybe the motherboard is shot?
Has there been an attempt to connect the drives to different hardware running an OS that understands ZFS?
Already tried here.
root@truenas[~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 10.9T 0 disk
├─sda1 8:1 0 2G 0 part
└─sda2 8:2 0 10.9T 0 part
sdb 8:16 0 119.2G 0 disk
├─sdb1 8:17 0 260M 0 part
├─sdb2 8:18 0 103G 0 part
└─sdb3 8:19 0 16G 0 part
└─sdb3 253:0 0 16G 0 crypt [SWAP]
sdc 8:32 0 10.9T 0 disk
├─sdc1 8:33 0 2G 0 part
└─sdc2 8:34 0 10.9T 0 part
root@truenas[~]# sudo hexdump /dev/sda2 | head -n 40
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
0003fd0 0000 0000 0000 0000 7a11 b10c da7a 0210
0003fe0 2a3f 7f6e 8f80 97f4 cefc 58aa 9f16 af90
0003ff0 b48b ff6d ea57 cbd1 5fab 0d46 92db 6ec6
0004000 ffff ffff ffff ffff ffff ffff ffff ffff
*
0040000 0000 0000 0000 0000 0000 0000 0000 0000
*
0042000 ffff ffff ffff ffff ffff ffff ffff ffff
*
0080000 0000 0000 0000 0000 0000 0000 0000 0000
*
0400000 0000 0000 0000 0000 0008 0000 0000 0000
0400010 0008 0000 0000 0000 0000 0000 0000 0000
0400020 0000 0000 0000 0000 0000 0000 0000 0000
0400030 0000 0000 0000 0000 0007 0007 0902 8009
0400040 0000 0000 0000 0000 0000 0000 0000 0000
0400050 0000 0000 0000 0000 001c 0000 0000 0000
0400060 0000 0000 0000 0000 99e4 b3f4 6d61 a8e9
0400070 8f64 924a 60b5 2ab8 008a 0000 0000 0000
0400080 0002 0000 0000 0000 00b8 0000 0000 0000
0400090 7a11 b10c da7a 0210 4e4b 66ef 0005 0000
04000a0 7026 f691 14ef 0000 62ed c6fc 9e20 0028
04000b0 5dd0 21b6 53a0 3496 0000 0000 0000 0000
04000c0 0000 0000 0000 0000 0000 0000 0000 0000
*
0402000 ba4e b770 a81b ac59 0a76 0aae a654 adae
0402010 7cc4 cd8a 7dd7 a03d b111 5d79 2fcc 9daa
0402020 2d48 1412 a7ff 720c 7a10 9003 2062 c36f
0402030 1fc2 0b1d 793d ddc1 a3ab dabb 2ef1 126e
0402040 dffa 27b0 f07a 3424 08f0 0174 9cdf a4b3
0402050 f737 ba4e 268e 6060 75b8 e776 f0c1 56ff
0402060 3c86 952c 9426 e682 c13b ca24 0aff a2a3
0402070 0993 546d 1025 1e6c feec 6100 08b0 20d7
0402080 5e4e aeb6 8e32 2a95 cf0b 78e7 2278 905f
0402090 d37a 00b5 8677 cdef 7ed3 bdb8 120c 4dbe
04020a0 01e2 70de 0b59 d154 5095 a49c cb23 64b0
04020b0 a8ea 293f c35b 64cb 5df6 26a6 fe4b abee
04020c0 c4b1 d483 7b8f e16f 13cc 102d 91fe 14f3
“something”
I guess at the end of the day, tracking down the culprit even if i dont find my data can help prevent this in the future
How do you activate the SED encryption on the drives?
Maybe it is connected to the BIOS somehow, so the changes in the BIOS triggered something?
Sorry, can you redo with the -C option:
sudo hexdump -C /dev/ada2p2 | head -n 40