Apps migration from Dragonfish to Electric Eel fails when ix-applications is on native encryption pool

Apps and Virtualization
1

Please advise how I might proceed with migrating most, if not all, of my Apps given that ‘ix-applications’ and host path datasets are all on encrypted pools and datasets. The following is the response I get for all of my Apps when I run midclt call -job k8s_to_docker.migrate <POOL_NAME_REDACTED>.

{"name": "plex", "error": "App is using encrypted ix-volumes which are not supported for migration", "successfully_migrated": false}
2

You will have to relocate your ix-applications dataset to an unencrypted dataset.

Encrypted Root Dataset:
Applications do not migrate to 24.10 if the ix-applications dataset is configured on a pool with an encrypted root dataset (see NAS-131561). Relocate installed applications to an unencrypted pool on 24.04 before attempting to upgrade to 24.10.

2 Likes
3

Thank you for the information. I clearly should’ve started with the version notes before proceeding.

Aside from finding some other HDD to temporarily attach to TrueNAS to create a non-encrypted pool, what other options do I have to create a dataset (without encryption) so that I can complete the migration process?

4

I don’t know that there is one, unfortunately - because the creation of the ix-applications dataset is programmatic, it will automatically inherit the root encryption setting from the pool.

6

I believe I have a solution for migrating Apps from an encrypted dataset. Feel free to offer any suggestions. Ensure you have enough space on your pool to hold two copies of ‘ix-applications’ before following this process.

  1. Rename the current ‘ix-applications’ dataset to something else.
  2. Drop to a shell and run the following code.
  • Be sure to change variables to reflect your environment
  • ‘dstDataset’ needs to be ‘ix-applications’ for the migration process to work
  • If Apps is already setup on a pool, ‘dstPool’ needs to be the same pool or unset the pool before starting this process
srcPool=tank
dstPool=tank
srcDataset=ix-applications-bak
dstDataset=ix-applications

for dataset in $(zfs list -Hro name -s name ${srcPool}/${srcDataset}); do
    echo "Working on '$dataset'..."
    dest=${dataset##${srcPool}/${srcDataset}}
    snapshots=$(zfs list -Ho name -s creation -t snapshot $dataset)
    [ -z $snapshots ] && { zfs create -o encryption=off ${dstPool}/${dstDataset}${dest}; continue; }
    
    snapshot=${snapshots%%$'\n'*}
    zfs send ${snapshot} | zfs recv -o encryption=off -o canmount=off ${dstPool}/${dstDataset}${dest}@${snapshot##*@}
    if [ $(wc -l <<< "$snapshots") -gt 1 ]; then
        stream=''
        while IFS= read -r snapshot; do
            ! [ -z "${stream}" ] && stream="${stream#-I }" && stream="${stream%% *}" && stream="-I @${stream##*@} "
            stream+="${snapshot}"
        done <<< "$snapshots"

        eval zfs send ${stream} | zfs recv ${dstPool}/${dstDataset}${dest}
    fi
    zfs set canmount=on ${dstPool}/${dstDataset}${dest}
done

rsync -acX /mnt/${srcPool}/${srcDataset}/ /mnt/${dstPool}/${dstDataset}/

midclt call -job k8s_to_docker.migrate ${dstPool}
2 Likes
7

Hi @whiskerz007 , what do you mean by rename? I don’t have any settings to rename the pool. Could you explain a bit more?

My situation is that I have a pool called app, and it’s encrypted with ix-applications inside.

I hope you can help me.

8

You do not need to rename the pool, only the dataset. You can rename a dataset from the shell. man zfs-rename would be a useful place to start in order to understanding how to accomplish the task.

9

tysm!

10

Thanks, I too confirm this works.

Two suggestions on that though:

First, in line 10 better use parentheses [ -z "$snapshots" ] (gave me an error without).

Second, for those wondering how to restore the previous, encrypted state of the ix-applications dataset, I adjusted the above code to be run (again) after successful migration:

srcPool=tank
dstPool=tank
srcDataset=ix-applications
dstDataset=ix-applications-enc

for dataset in $(zfs list -Hro name -s name ${srcPool}/${srcDataset}); do
    echo "Working on '$dataset'..."
    dest=${dataset##${srcPool}/${srcDataset}}
    snapshots=$(zfs list -Ho name -s creation -t snapshot $dataset)
    [ -z "$snapshots" ] && { zfs create ${dstPool}/${dstDataset}${dest}; continue; }
    
    snapshot=${snapshots%%$'\n'*}
    zfs send ${snapshot} | zfs recv -o canmount=off ${dstPool}/${dstDataset}${dest}@${snapshot##*@}
    if [ $(wc -l <<< "$snapshots") -gt 1 ]; then
        stream=''
        while IFS= read -r snapshot; do
            ! [ -z "${stream}" ] && stream="${stream#-I }" && stream="${stream%% *}" && stream="-I @${stream##*@} "
            stream+="${snapshot}"
        done <<< "$snapshots"

        eval zfs send ${stream} | zfs recv ${dstPool}/${dstDataset}${dest}
    fi
    zfs set canmount=on ${dstPool}/${dstDataset}${dest}
done

rsync -acX /mnt/${srcPool}/${srcDataset}/ /mnt/${dstPool}/${dstDataset}/

Afterwards delete the (unencrypted) ix-applications dataset and run zfs rename tank/ix-applications-enc tank/ix-applications.

Note that this method assumes the parent dataset/pool (in this case pool tank) to be encrypted.

11

note that your applications in 24.10 are not being run from ix-applications but a hidden dataset mounted to /mnt/.ix-apps. See App Dataset Types in TrueNAS Apps | TrueNAS Documentation Hub.

You can actually delete the unencrypted ix-applications dataset once you’re sure you won’t be rolling back to 24.04.

That said, trying to encrypt ix-apps could potentially lead to unintended consequences so I wouldn’t necessarily recommend it.

1 Like
12

Got it, I thought that ix-applications is still required in 24.10. nvm.

On that note, @DjP-iX can you confirm that the /mnt/.ix-apps dataset is on the selected pool for apps and will inherit this pool’s encryption (if previously set up)?

I’m bringing this up because my (sensitive) app-configs are mostly stored inside ixVolume datasets which were, pre 24.10, stored in the encrypted ix-applications dataset. I want to retain app-config encryption with 24.10 onwards.

13

I would have to look into that and get back to you.

That said if you are concerned about the data stored in ixVolumes, my suggestion would be to recreate these as datasets on an encrypted pool and mount them as host paths (though keep in mind you may need to redeploy the app to change configuration from ixVolumes to host paths). We’ve moved to explicitly recommending ixVolumes be used for test deployments only (also covered in the docs linked above).

1 Like
14

Much appreciated!

15

The ix-apps dataset is purposely set up to not inherit encryption, so as I said if you want your config storage, db, etc. encrypted you’d need to do that using mounted host paths.

1 Like
16

Moved here as I believe this deserves it’s own issue:

1 Like
17

I was in the process of migrating to Electric Eel and discovered this painful and annoying issue with migration of apps from encrypted pool when I read the release notes. @M43 is raising a good point about nesting unencrypted datasets in an encrypted pool and I wonder what has led to this limitation. Is there a plan to fix this somehow with future releases or am I stuck on Dragonfish? I do not fancy the idea of having ix-apps unencrypted and I don’t think that people should be worrying about whether apps in the future will break because they have to decide whether to encrypt a pool or not at install time.

1 Like