How do you backup app data in ElectricEel?

Yes a more recommended approach for production apps is to let .ix-apps contain the base app installation and then use host paths to mount storage paths on separate datasets.

As for recovering from your current situation, it seems likely that if you reverted to a 24.04 boot environment, deleted ix-apps completely so it doesn’t cause conflicts with the migration script, and then upgraded to 24.10 again it should properly migrate your apps–but I haven’t tested anything like that scenario, so no guarantees.

official/community apps have mounts for config/data/etc that each app uses. Should be able to just copy, from shell, those to another pool. Then decide if you want to mount those as hostpath/binds to your apps. That way you always know where they live!

Too bad Truecharts apps didn’t do it that way, would have made migration away much easier!

I missed this in the docs.

Note: It was not in 24.10 docs, but was in the TrueNAS Apps docs.

1 Like

FWIW, I think hiding the pool “ix-apps” dataset was perhaps not ideal. I understand it’s mounted in a strange place, but the issue is that there is now no visibility on its size in the GUI, or ability to delete it or replicate it.

1 Like

To conclude this thread, you’re not supposed to use the hidden ix-apps dataset for permanent storage, and so you’re not supposed to back it up.

If you have important app data you want to keep, you’re supposed to configure your apps to use your own datasets, and then you can back them up as normal.

For my situation where I had to wipe and recreate my only data pool, it left me in a bad state where I couldn’t even install apps anymore (installing an app requires the docker service, but no UI to start it). So I ended up just installing the latest OS from scratch and reconfiguring everything from scratch. After starting a newly installed app, configured to use my own dataset(s), I just stopped the app and then moved my old backed up app data to the new dataset(s), and starting the app again brought me back to my expected state.

Mostly incorrect statement:

“you’re not supposed to use the hidden ix-apps dataset for permanent storage, and so you’re not supposed to back it up.”

IF you do select to install Apps under the “IxVolume” (instead of Host Path), the app DATA would be saved under /mnt/.ix-apps/ (which is really under the selected Apps Pool), but the GUI does not give you the choice to snapshot it or replicate it.

Yes, you can snapshot/replicate recursively the Apps Pool root dataset, but the current state will bite people that don’t really know.

(I do believe IX Systems was supposed to add a specific option to snapshot/replicate the apps data).

It isn’t. Read the docs that others have posted above.

ixVolumes are not recommended for permanent storage volumes, they are intended for use as rapid storage for a test deployment of the container. We recommend adding datasets and configuring the container storage volumes with the host path option.

But I think you know this already, because you then go on to agree with me that you can’t back it up easily.

Why would you believe that?

The ix-apps dataset is for internal use only.

BTW, turns out that if you enter the path manually for the ix-apps dataset, ie <poolname>/ix-apps, you can add it to the list of datasets that get replicated as part of a replication task.

Restoring would be another thing.

Have you validated this? When I just tried it didn’t error when I added the path, but when I hit save the task creation seemed to get stuck and never completed.

It worked for me, but i did do it in a multi dataset replication.

I confirmed the dataset (and all nested) replicated. The mount point was cleared on the destination.

Here’s the ix-apps dataset appended to the multi-selection

Screenshot 2024-11-14 at 3.50.07 PM

Here’s the ix-apps dataset on the source

root@titan[~]# zfs list | grep ix-apps 
tank/ix-apps                                                       1.63G  33.2T   120K  /mnt/.ix-apps
tank/ix-apps/app_configs                                           1.41M  33.2T   484K  /mnt/.ix-apps/app_configs
tank/ix-apps/app_mounts                                             944K  33.2T   104K  /mnt/.ix-apps/app_mounts
tank/ix-apps/app_mounts/scrutiny                                    776K  33.2T   104K  /mnt/.ix-apps/app_mounts/scrutiny
tank/ix-apps/app_mounts/scrutiny/config                             100K  33.2T   100K  /mnt/.ix-apps/app_mounts/scrutiny/config
tank/ix-apps/app_mounts/scrutiny/influxdb                           572K  33.2T   456K  /mnt/.ix-apps/app_mounts/scrutiny/influxdb
tank/ix-apps/docker                                                1.57G  33.2T  1.13G  /mnt/.ix-apps/docker
tank/ix-apps/truenas_catalog                                       54.4M  33.2T  49.8M  /mnt/.ix-apps/truenas_catalog

And here is the destination

root@rhea[~]# zfs list | grep ix-apps
rhea/ix-apps                                                                  1.57G  7.31T   174K  /mnt/.ix-apps
rhea/ix-apps/app_configs                                                      1.06M  7.31T  1.06M  /mnt/.ix-apps/app_configs
rhea/ix-apps/app_mounts                                                        140K  7.31T   140K  /mnt/.ix-apps/app_mounts
rhea/ix-apps/docker                                                           1.46G  7.31T  1.46G  /mnt/.ix-apps/docker
rhea/ix-apps/truenas_catalog                                                   110M  7.31T   110M  /mnt/.ix-apps/truenas_catalog
rhea/replicas/titan/ix-apps                                                   1.53G  7.31T   174K  /mnt/rhea/replicas/titan/ix-apps
rhea/replicas/titan/ix-apps/app_configs                                       1.29M  7.31T  1.07M  /mnt/rhea/replicas/titan/ix-apps/app_configs
rhea/replicas/titan/ix-apps/app_mounts                                        1.33M  7.31T   151K  /mnt/rhea/replicas/titan/ix-apps/app_mounts
rhea/replicas/titan/ix-apps/app_mounts/scrutiny                               1.10M  7.31T   151K  /mnt/rhea/replicas/titan/ix-apps/app_mounts/scrutiny
rhea/replicas/titan/ix-apps/app_mounts/scrutiny/config                         145K  7.31T   145K  /mnt/rhea/replicas/titan/ix-apps/app_mounts/scrutiny/config
rhea/replicas/titan/ix-apps/app_mounts/scrutiny/influxdb                       825K  7.31T   657K  /mnt/rhea/replicas/titan/ix-apps/app_mounts/scrutiny/influxdb
rhea/replicas/titan/ix-apps/docker                                            1.46G  7.31T  1.23G  /mnt/rhea/replicas/titan/ix-apps/docker
rhea/replicas/titan/ix-apps/truenas_catalog                                   70.8M  7.31T  70.2M  /mnt/rhea/replicas/titan/ix-apps/truenas_catalog

I added iXVolumes for Scrutiny.

1 Like

I tested a local migration from one pool to another a few times, and documented it here: