I’m running a stand alone TrueNAS system on a Dell Optiplex 7050. There is a 500 MB M.2 SSD on the motherboard and 2 matched 4 TB SSD running on SATA ports. Has been running flawlessly for a couple of years.
The other day we had a lightning storm. The power went out for a couple of minutes then came back on for 10 or 15 seconds and went out again for a couple of hours. When the power came back on for good, the TrueNAS system did not boot up.
I took the system off-line, set up monitor, mouse, etc. In an attempt to boot it presents the TrueNAS banner, then automatically tries to boot the system. The boot fails with the error:
“Mounting from zfs:boot-pool/default failed with error code 5”
It tries 10 or 15 more times, but continues to fail.
I attempted to read the 4 TB SSD on an Ubuntu machine. It sees the drives and shows my primary pool name on each one, but cannot read the zfs format files.
I’ve poked around the boot loader without much success. But I’m a real rookie in boot physiology. Lots of information available through the boot loader…I just don’t know what it means or where the breakdown has occurred.
Now the question…
I want to revive the stored files. Can I re-install TrueNAS on the 500 MB M.2 SSD on the motherboard and then remount the two 4 TB SSDs? Can the kernel for TrueNAS be recreated on the boot SSD without breaking all the links to the existing SSDs? Do I need to put in two blank SSD drives to load TrueNAS, then reattach the two 4 TB SSDs? (Unfortunately I only have 2 SATA ports on this motherboard…another problem.)
In theory it’s easy to recover from a lost boot-pool.
The idea that you can reinstall the OS and not lose any data or even settings relies on you having saved an up-to-date configuration file, including any encryption keys, beforehand.
You are expressly prompted to do so every time you update and there’s always the option to do so manually using the System-menu on the left.
If you didn’t save your configuration file, and you elected to use key encryption when setting your pool or datasets up, things get a lot more complicated. Hopefully you chose to save your key files when the encrypted pool/datasets were created. Using passphrase encryption should also let you side-step this issue since you would presumably still know your encryption passphrase.
If you didn’t encrypt your pool and you didn’t save your configuration file the pool will still be possible to import and mount, but you will need to recreate most settings, like users, shares, replication tasks and so on.
Be careful not to select your data drives when installing TrueNAS anew, in the installer interface. Doing so would be potentially catastrophic.
Yes. But the incident raises the question whether the boot device could be physically damaged, so you may want to grab a cheap new drive ($10) to install onto.
The pool could be mounted on any system… where ZFS is installed. Your Ubuntu machine probably lacks the ZFS package.
With ZFS installed on a new machine (or TrueNAS on a new drive), it may be possible to recover a configuration file from the pool which holds the system dataset.
neofusion: Thank you for then info and the suggestions. I believe that I’ve located the original configuration file on a separate computer. Since I haven’t updated the TrueNAS system from the original install I hope that this configuration file will be useful. I didn’t encrypt the data (I don’t believe) so I’m hoping the encryption complications may be avoided.
dtorix: Thank you for the suggestions.
The new M.2 SSD is ordered and will be here tomorrow. I’m researching how to install the ZFS kernel into my Ubuntu box to see if I can directly mount the data drives from my failed TrueNAS box.
I do have a question, though. Since my data is on two SSD’s, would there have been a default storage scheme with the original install? It could be mirror. In that case both SSD’s would have identical data, no problem. However, if the default is one of the RAID arrangements, it might be more difficult to mount and understand the data that comes off of the SSD’s (within an Ubuntu box). Comments?
ZFS is ZFS no matter where…
If the drives are a mirror (“RAID 1”), as they should, one is enough to access data, although mounting both is highly advisable. If they are striped (“RAID 0”), you need both… and any hiccup will lose the whole pool.
In any case ZFS will read the layout and the identity of pool members from the labels; no user input needed.
sudo zpool import
Solved! 
It took a lot of trial and error and about a thousand reboots, but I got the data back intact. Thank you etorix and neofusion for giving me the hints and links that got it done.
For someone who is faced with this problem, I will give an outline of what I did to get the data.
-
Replaced the M.2 corrupted drive on the TrueNAS machine with an new M.2 SSD. I was able to reload it with an OS without any problems. I haven’t checked the M.2 SSD for hardware damage…another time.
-
I have an Ubuntu box so I attempted to load the ZFS utilities onto that. Failed. The flavor of Ubuntu was version 25. I’m not sure why, but ZFS is not supported by the Linux kernel on that version of Ubuntu. I received an error message that indicated that I might have a kernel version that is more recent than the supported versions. Version 24 has Linux kernel 6.x which is supported.
-
I built another box with Ubuntu version 24 (Ubuntu 24.04 LTS) which is installed with Linux kernel 6.11.0-17 generic. This version is compatible with ZFS. Successfully loaded the ZFS utilities. (Instructions from etorix’s suggestion…I found several install sequences that did not work. Thanks etorix!)
sudo apt update
sudo apt upgrade
sudo apt install zfsutils-linux
- Created a storage pool on a spare SSD. Just to make sure the install was working.
sudo zpool create mypool /dev/sdc
Replace mypool with the name that you wish to use for the storage pool and replace the /dev/sdc with the location of the disk you wish to place the storage pool on.
- I then attached my two SSD’s with the data I wanted to rescue onto the Ubuntu machine I had just created. I probed the drives with:
sudo zpool status
This gave me a lot of useful information:
The name of the storage pool
The fact that the pool was Mirror-0
- I then imported the storage pool into the ZFS framework with:
sudo zpool import mypool -f
Replace mypool with the name of the storage pool that is to be rescued. The name was provided by the zpool status command. The -f option was required to force the import.
Then, the magic happened and the imported ZFS storage pool appeared in the Ubuntu file system. Of note, the graphical display of storage locations didn’t show the storage pool, but it was in the root directory by looking within the terminal. The report through zpool was that the data was not degraded. WooHoo!
Reboots found that the ZFS storage pool was always accessible without any action on my part.
Once I had the data I could move it to a neutral location in preparation to reload it into a rebuilt TrueNAS box.