SOLVED: Unable to unlock pool after Zpool Rename

TrueNAS General
1

Hi TrueNAS community,

I carried out the upgrade to Fangtooth and all was well, but noticed that because my pools (SSD Storage and Storage Pool) both had spaces, they were not being seen when trying to set a Instance pool.
So I went through the process of renaming the pool as kindly described by @Stux and noticed that the Storage-Pool (newly renamed from Storage Pool) (which is fully encrypted) is locked and one dataset (the only one encrypted on this pool) on my SSD-Storage (newly renamed from SSD Storage) was also locked.

image
image614×409 23.9 KB

Trying to import the key via the GUI gives me the following error


Error: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 515, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 562, in __run_body
    rv = await self.middleware.run_in_thread(self.method, *args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 599, in run_in_thread
    return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 596, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 178, in nf
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 52, in nf
    res = f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/pool_/dataset_encryption_info.py", line 134, in encryption_summary
    keys_supplied = {k: {'key': v, 'force': False} for k, v in retrieve_keys_from_file(job).items()}
                                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/pool_/utils.py", line 127, in retrieve_keys_from_file
    data = json.loads(job.pipes.input.r.read(10 * MB))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/__init__.py", line 341, in loads
    s = s.decode(detect_encoding(s), 'surrogatepass')
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc7 in position 1567: invalid continuation byte

Any help or advice on how to unlock them?

2

Open the keyfiles (.json) with a text editor. Copy the long hex string and manually paste it when you unlock the datasets. (There’s an option to manually enter the keystring.)

After it succeeds, you can export the keyfiles again, which will now have the proper names.

You can also rename the references to the datasets in the keyfiles with a text editor. Whatever you prefer.

1 Like
3

Thank you @winnielinnie, I was able to find the .json file for Storage-Pool and I have successfully unlocked it using your suggestion, however, I seem to have very safely and securely stashed away the .json file for the Cloud dataset in SSD-Storage to the point where even I can not find it!.

It seems that the only path left for me to retrieve the key for the Cloud Dataset is to decrypt the configuration saves (.tar) which contains the keys.
I am trying the following method as shown in this blogpost Robert Milkowski's blog: TrueNAS Scale & ZFS Wrapping Key, and I can successfully see my encrypted keys

root@truenas[/mnt/SSD-Storage]# sqlite3 /mnt/SSD-Storage/freenas-v1.db
SQLite version 3.40.1 2022-12-28 14:03:47
Enter “.help” for usage hints.
sqlite> select * from storage_encrypteddataset;
1|Storage Pool|encrypted_storage-pool_key=|
5|SSD Storage/Cloud|encrypted_ssd-storage_key=|
sqlite>

I created the decode_key.py file in /mnt/SSD-Storage and pasted the code as below

#!/usr/bin/python3

import sys
import base64
from Cryptodome.Cipher import AES
import sqlite3

PWENC_BLOCK_SIZE = 32
PWENC_FILE_SECRET = ‘/mnt/SSD-Storage/pwenc_secret’
PWENC_PADDING = b’{’

def pwenc_get_secret():
with open(PWENC_FILE_SECRET, ‘rb’) as f:
secret = f.read()
return secret

def pwenc_decrypt(encrypted=None):
if not encrypted:
return “”
from Cryptodome.Util import Counter
encrypted = base64.b64decode(encrypted)
nonce = encrypted[:8]
encrypted = encrypted[8:]
cipher = AES.new(
pwenc_get_secret(),
AES.MODE_CTR,
counter=Counter.new(64, prefix=nonce),
)
return cipher.decrypt(encrypted).rstrip(PWENC_PADDING).decode(‘utf8’)

if len(sys.argv) == 2:
print(pwenc_decrypt(sys.argv[1]))
exit(0)

dbcon = sqlite3.connect(‘/mnt/SSD-Storage/freenas-v1.db’)
dbcur = dbcon.cursor()
for row in dbcur.execute(‘select * from storage_encrypteddataset’):
ds_id, ds_name, ds_enc_key, kmip_enc_key = row
#print(ds_id, ds_name, ds_enc_key, pwenc_decrypt(ds_enc_key))
print(f’dataset: {ds_name}\n key: {pwenc_decrypt(ds_enc_key)}\n’)

But I get the following error when I try and run it with ./decode_key.py

root@truenas[/mnt/SSD-Storage]# ./decode_key.py
sudo: process 118926 unexpected status 0x57f
zsh: killed ./decode_key.py
root@truenas[/mnt/SSD-Storage]#

Any idea what I am missing?

4

That must be noexec doing its duty.

Try running “/usr/bin/python3 <scriptfilepath>”

1 Like
5

ah @smione, that is brilliant and it did the trick, but now the next obstacle reveals itself, the script is decoding the first key it has found, and unfortunately, the Cloud dataset key is the 2nd in line…

root@truenas[/]# /usr/bin/python3 /mnt/SSD-Storage/decode_key.py
dataset: Storage-Pool
key: Storage-Pool_supersecret_key

root@truenas[/]#

I will attempt at modifying the saved .db file to remove the Storage-Pool key and see if the script bypasses it.

1 Like
6

the freenas-v1.db file looks very intimidating, and I am struggling to identify which sections I should remove/edit out so that the script decodes the key for the SSD Storage/Cloud dataset.
I have searched for Storage Pool with a text editor but not certain what sections to truncate as it is not clear which part is for the encryption key and which ones are not.

7

@winnielinnie
You are a star, thank you so much for the udpated .py script you have posted here Pool encryption dataset - #6 by winnielinnie.
This was able to extract both encryption keys from the .db file.

I will post the script here for completion purposes and for anyone else (hopefully not myself again) that may run into the same problem in the future.

Note to anyone in the future,
Do not panic (actually you should take this opportunity to save the .json keys now, don’t delay), extract the truenas backup file .db and the pwenc_secret in the same location in your truenas, create a extractkeys.py file and edit it using your preferred text editor (such as nano) and paste the following command.

 #!/usr/bin/python3
 
 # based on /usr/lib/migrate113/freenasUI/system/migrations/0022_cloud_sync.py
 
 import sys
 import base64
 from Cryptodome.Cipher import AES
 import sqlite3
 
 
 PWENC_BLOCK_SIZE = 32
 PWENC_FILE_SECRET = 'pwenc_secret'
 PWENC_PADDING = b'{'
 
 
 def pwenc_get_secret():
     with open(PWENC_FILE_SECRET, 'rb') as f:
         secret = f.read()
     return secret
 
 
 def pwenc_decrypt(encrypted=None):
     if not encrypted:
         return ""
     from Cryptodome.Util import Counter
     encrypted = base64.b64decode(encrypted)
     nonce = encrypted[:8]
     encrypted = encrypted[8:]
     cipher = AES.new(
         pwenc_get_secret(),
         AES.MODE_CTR,
         counter=Counter.new(64, prefix=nonce),
     )
      return cipher.decrypt(encrypted).rstrip(PWENC_PADDING).decode('utf8')
 
 
 if len(sys.argv) == 2:
     print(pwenc_decrypt(sys.argv[1]))
     exit(0)
 
 dbcon = sqlite3.connect('freenas-v1.db')
 dbcur = dbcon.cursor()
 for row in dbcur.execute('select * from storage_encrypteddataset'):
     ds_id, ds_name, ds_enc_key, kmip_enc_key = row
     #print(ds_id, ds_name, ds_enc_key, pwenc_decrypt(ds_enc_key))
     print(f'dataset: {ds_name}\n  key: {pwenc_decrypt(ds_enc_key)}\n')

wipe your eyebrows sweat and save the .json files in multiple locations from now on. :slight_smile:

1 Like