I had a boot disk go bad in a TrueNAS server so I installed truenas on another device. Problem is, I am not entirely sure if the disks were encrypted (installed back with Freenas, later upgraded to truenas). When I view the disks, pool names are shown up as N/A. When I click Import Pool, nothing shows. Not sure if this indicates the disks are in fact encrypted.
Additionally, I am trying to recover the boot disk and attempt to get the old configuration file. Is there a specific file I should be looking for? Thanks
One thing to note, during the boot process of the old boot drive, it did load up with the truenas selection screen but spit out “No module named encodings” for python, as well as a middleware failed to start thereafter with the python errors. Not sure if this indicates issues with the boot drive or maybe something else is to blame?
Thanks
If the pool is encrypted it will see the pool but ask for a key, If it does not see a pool at all then start by checking that those drives are showing connected.
truenas_admin@truenas[~]$ sudo zpool import -a -d /dev/sda
no pools available to import
truenas_admin@truenas[~]$ sudo zpool import -a -d /dev/sda2
no pools available to import
truenas_admin@truenas[~]$ sudo zpool import -a -d /dev/sda1
no pools available to import
It’s possible that the pool was imported already, but is using a different mount point that the earlier FreeNAS or TrueNAS used. This is somewhat easy to fix.
Next, if there really is no pool to import, that implies TrueNAS Core, (aka FreeNAS / FreeBSD), encryption. Or that the disks were corrupted somehow. Either would bad.
truenas_admin@truenas[~]$ sudo zpool list
[sudo] password for truenas_admin:
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
boot-pool 28G 3.00G 25.0G - - 1% 10% 1.00x ONLINE -
truenas_admin@truenas[~]$ sudo zpool status
pool: boot-pool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
sdd3 ONLINE 0 0 0
errors: No known data errors
Not sure if this is good or bad
Additionally, on fdisk, one of the two drives is reporting The primary GPT table is corrupt, but the backup appears OK, so that will be used. Both disks were in raid 1.
truenas_admin@truenas[~]$ sudo fdisk -l
Disk /dev/sda: 5.46 TiB, 6001175126016 bytes, 11721045168 sectors
Disk model: ST6000VN001-2BB1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 16773120 bytes
Disklabel type: gpt
Disk identifier: DE347421-D10C-11EA-8EFA-0026B93B057E
Device Start End Sectors Size Type
/dev/sda1 128 4194431 4194304 2G FreeBSD swap
/dev/sda2 4194432 11721045127 11716850696 5.5T FreeBSD ZFS
Disk /dev/sdb: 5.46 TiB, 6001175126016 bytes, 11721045168 sectors
Disk model: ST6000VN001-2BB1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 16773120 bytes
Disklabel type: gpt
Disk identifier: DE63A44F-D10C-11EA-8EFA-0026B93B057E
Device Start End Sectors Size Type
/dev/sdb1 128 4194431 4194304 2G FreeBSD swap
/dev/sdb2 4194432 11721045127 11716850696 5.5T FreeBSD ZFS
The primary GPT table is corrupt, but the backup appears OK, so that will be used.
I was able to recover the db and secret file from the USB which was shown as corrupted. I restored on a new TrueNAS installation and was able to login with the old IP/password.
Unfortunately, the pools were not recognized and I am unable to import them. I am not sure how they could be corrupted when it was only the USB itself being corrupted. I did try scanning the disks in Windows in between but did not make changes to them.
Storage dashboard shows 2 unassigned disks and vault offline
root@freenas[~]# zpool list
zpool status
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
boot-pool 28G 3.07G 24.9G - - 1% 10% 1.00x ONLINE -
pool: boot-pool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
sdc3 ONLINE 0 0 0
errors: No known data errors
root@freenas[~]# sudo zdb -l /dev/sda2
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
root@freenas[~]# sudo zdb -l /dev/sdb2
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
Could it now be the different mount point you mentioned? I have some hope considering I have the backup files restored, just not the pools which I thought would be picked up. Additionally, these are two 6TB ironwolfs. The day prior when this issue occured, I tried uploading a file to a new folder which didn’t work and caused the folder to constantly load on an SMB mount on a Linux OS. I also deleted remote directory mount point, that shouldn’t have caused this issue right?
We have seen MS-Windows corrupt disks before, even though the user thought it was just R/O examination.
No. The pool would still import, just not mount properly.
This is BAD. Either you used FreeBSD’s based GEOM disk encryption, which is not available in Linux / TrueNAS CE / SCALE. Or their is something really bad with your disks.
Before OpenZFS supported Dataset & zVol encryption, the only way FreeNAS could get disk encryption was to layer ZFS on top of a GEOM encrypted partition. I have only limited knowledge of that software. Further, besides GEOM encryption not being available with TrueNAS CE / SCALE, later versions of TrueNAS Core, (based on FreeBSD), were dropping most of the support for GEOM. (Because OpenZFS would be more reliable.)
The only thing I can recommend at this point, is to first verify what OS was used on the prior install: TrueNAS Core, (based on FreeBSD), or TrueNAS SCALE / CE, (based on Linux). If it was Core, then you need to make a Core install and check the disks to see if they are GEOM encrypted. How that is done, I don’t know. Perhaps someone else will know.
Don’t overwrite your prior USB install. That disk(s) might have critical encryption keys you may need.
Thanks for this information. When I recovered the db file and keys from the original USB device, I noticed I had to recover the USB’s img in freebsd and it was identified as a free bsd boot device/pool plus I remember it being Core, 12 something. Additionally, I noticed geli key in there too. Hope restoring the config with older version of Core works, this sounds a little promising but will try tomorrow and report on it. The new install is on a fresh USB, I assume I don’t need it’s configuration as I already have the original’s data/ files. I will back up the config of the new just in case and try again. Many thanks
That was it! I booted up 12.0 and it recognized the pool mentioning it was encrypted. I did an export/disconnect then imported the disks, used the geli encryption key file to decrypt, and my pool went online and everything is there! Thank you so much!
For anyone wondering, I was able to create an .img file from my (somewhat) corrupted USB boot disk (using dd in Linux), copy that over to a FreeBSD system and ran a series of commands (using AI) to import the pool with some offset, mount it and pulled the latest database file, secret, and GELI key (most important). Lessons learned, back up configs (and keys if applicable) and don’t use a USB for boot drive, aside for temporary recovery in this case.
Kind of impressed this Kingston from 7+ years ago held. This was my first server/NAS setup back then and I didn’t know better, some guide I followed suggested using USB for boot and I ended up relying on it until now.
Glad you were both able to figure out how to do it, and able to get your Geli encryption key back.
In the early days of FreeNAS, it was fine to use USB flash drives. Even cheap ones worked okay. Part of this was due to FreeNAS before 9.3, (if I recall correctly), was a mostly read only image. Starting with FreeNAS 9.3, it became a ZFS boot pool, with active OS. That causes cheap USB flash drives to wear out quicker.
There is a way to remove FreeBSD Geli encryption from a TrueNAS Core pool. Basically you need to re-sync a disk, (either a new one, or existing one), but without the partition’s Geli encryption. Whence the first disk is done, you can do the same with the second disk.
Because your data pool is a 2 way Mirror, it is safer to add a third disk into the mix, except without the Geli encryption. That way you never loose redundancy during the Geli encryption removal process.
