Boot drive AND backup gone with encryption key

TrueNAS General
1

Using 24.01.04 on a 4 bay Truenas. I ALWAYS back up my encryption keys to the desktop I use to set up the system from the get go. We recently had flooding and the PC that had the backup was running and, since water and electricity don’t mix- it fried it and everything in it. Even HD is unrecoverable. Then, Wednesday, the Truenas went down with all our data on it. Turns out the boot SSD is corrupt (I can see some files, but not all) so I removed it and put in a new one with TrueNAS on it but…I no longer have the encryption keys. I can see and import the pool, but can’t decrypt it. And, the pool still shows as intact and good from the last scrub.

Am I toast? This will very likely cost me my job and I have been awake for 2 days straight trying everything I can think of but I am at my wits end.

Do you love your family? Your family holds the key!
Cheapest and power efficient TrueNAS build with ECC
Do you love your family? Your family holds the key!
2

You said you backed up the keys. Use them. :point_down:

UPDATE:

When you say backup, do you mean a backup of your pool or the encryption key?

If the key, was it the only copy you had?

3

This might be your only shot at retrieving the key, and even then it’s unlikely, since you said the SSD is corrupt.

As soon as possible, I would make an image of the entire SSD using dd or ddrescue. If you have a large enough USB drive, you can save the image in there. It’s possible to pipe it through zstd to reduce the size.

4

The only copy of the keys was on the PC that was flooded. I will try to make an image of the boot drive but where are the keys stored? I know I can’t access all the files in the structure and I am hoping that I CAN access where the encryption keys are stored.

5

Make a full image first. If the SSD dies before you can locate and extract the needed files (or use file recovery software), you at least have an image to work with later.

6

The way to image the entire boot SSD looks something like this:

dd if=/dev/sda bs=1M status=progress of=/path/to/usb/sda.img

If space is limited, you can use ZSTD compression:

dd if=/dev/sda bs=1M status=progress | zstd > /path/to/usb/sda.img.zst

You shouldn’t be using the drive. A Linux live ISO is the best environment to do this.

If you have no choice but to do this in TrueNAS, then make sure nothing is trying to use or access the SSD.

EDIT: If you really want to try to grab the needed files without first making an image, they are under /data/ named freenas-v1.db and pwenc_secret. This is not obvious with your running TrueNAS system, since its current /data/ folder has nothing to do with the old SSD.

1 Like
7

Thanks Winnie, making an image with DDrescue right now. IT is erroring out ALOT because of the drive going bad but I hope I can access the /data/ directory

8

The DD failed. I am trying to mount the SSD on a Ubuntu distribution to see if I can make the drive browsable.

9

You’ll need to import the ZFS pool as readonly and not automount the datasets.

zpool import -o readonly=on -N boot-pool

This will allow you to choose which dataset to mount. You want the most recent boot environment.

1 Like
10

This SSD is failing with a different SATA cable and different SATA port? You’ve ruled this out?

11

yeah, I have moved the sata port and cable 3 times. I did try to mount it but it just hangs after I issue that command. The only thing I can think of is to try to get the drive recovered.

12

I’m sorry to say but I don’t believe you’ll be able to recover the needed files.

If the drive is in such a state of failure that the pool cannot be imported as readonly, then not even the best recovery software will be able to use any filesystem metadata. This means that the only possible recovery is based on file signatures. Unfortunately, the pwenc_secret file is random data that has no defining signature. No recovery software will be able to find it or retrieve it or even know that it exists or what its name is.

The file recovery tools that are able to recover files without proper filesystem metadata are designed for images, videos, documents, music, archives, and files with distinctive headers. They must also exist in a contiguous manner.

EDIT: Are you sure there are no other copies of the key? Maybe uploaded to a cloud drive? Maybe on a USB?

13

I know this doesn’t help you now, but I highly recommend you use this approach from now on.

The last thing you need to is lock yourself out from your own data without any last resort. This method lets you relax with how to save the “secret”, since no one knows about it. A friend or family member could have a photo album or music library, which allows you one last chance to unlock your encrypted dataset.