I made some progress by fixing the mountpoints of /mnt/.ix-apps/ to the correct pool and stuff and that made my apps show up and the application service is running.
However there is still a broken link or bad mapping I’m not sure how to fix.
I connected to the server with sftp to see everything easier and noticed that what its looking for in /mnt/.ix-apps/app_mounts/adguard-home/
There is a conf and work folder there with the missing data that the app needs to start.
If I copy paste those folders into /mnt/.ix-apps/app_mounts/adguard-home/ then my adguard app starts up and works normally.
However, I’m sure this is the wrong way to go about it and the ix_volumes folder needs to be symlinked or mapped somehow to the /mnt/.ix-apps/app_mounts/ folder.
So my question is do i just symlink it and call it a day or is there a mapping that is missing in zfs mountpoint that i need to set to link the ix-volumes to the app_mounts?
output of: zfs get mountpoint | grep ix-
Also here is a image of the links that seems to be missing:
That would be highly unusual, since ix-applications was part of 24.04 and earlier and only remains for those who want to revert back to the previous version for some reason. It’s not used at all by 24.10 or 25.04.
That is, unless you did something when you “fixed the mountpoints”…
Reposting your mountpoint output
root@truenas[~]# zfs get mountpoint | grep ix-
DupPool/.ix-virt mountpoint legacy local
DupPool/.ix-virt@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/buckets mountpoint legacy local
DupPool/.ix-virt/buckets@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/containers mountpoint legacy local
DupPool/.ix-virt/containers@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/custom mountpoint legacy local
DupPool/.ix-virt/custom@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/custom/default_test-vp1qhj mountpoint - -
DupPool/.ix-virt/custom/default_test-vp1qhj@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/deleted mountpoint legacy local
DupPool/.ix-virt/deleted@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/deleted/buckets mountpoint legacy local
DupPool/.ix-virt/deleted/buckets@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/deleted/containers mountpoint legacy local
DupPool/.ix-virt/deleted/containers@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/deleted/custom mountpoint legacy local
DupPool/.ix-virt/deleted/custom@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/deleted/images mountpoint legacy local
DupPool/.ix-virt/deleted/images@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/deleted/virtual-machines mountpoint legacy local
DupPool/.ix-virt/deleted/virtual-machines@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/images mountpoint legacy local
DupPool/.ix-virt/images@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/virtual-machines mountpoint legacy local
DupPool/.ix-virt/virtual-machines@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/virtual-machines/Pterodactyl mountpoint legacy local
DupPool/.ix-virt/virtual-machines/Pterodactyl@auto-2025-06-19_14-39 mountpoint - -
DupPool/.ix-virt/virtual-machines/Pterodactyl.block mountpoint - -
DupPool/.ix-virt/virtual-machines/Pterodactyl.block@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications mountpoint /mnt/DupPool/ix-applications default
DupPool/ix-applications@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/catalogs mountpoint /mnt/DupPool/ix-applications/catalogs default
DupPool/ix-applications/catalogs@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/default_volumes mountpoint /mnt/DupPool/ix-applications/default_volumes default
DupPool/ix-applications/default_volumes@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/k3s mountpoint /mnt/DupPool/ix-applications/k3s default
DupPool/ix-applications/k3s@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/k3s/kubelet mountpoint legacy local
DupPool/ix-applications/k3s/kubelet@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases mountpoint /mnt/DupPool/ix-applications/releases default
DupPool/ix-applications/releases@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/adguard-home mountpoint /mnt/DupPool/ix-applications/releases/adguard-home default
DupPool/ix-applications/releases/adguard-home@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/adguard-home/charts mountpoint /mnt/DupPool/ix-applications/releases/adguard-home/charts default
DupPool/ix-applications/releases/adguard-home/charts@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/adguard-home/volumes mountpoint /mnt/DupPool/ix-applications/releases/adguard-home/volumes default
DupPool/ix-applications/releases/adguard-home/volumes@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/adguard-home/volumes/ix_volumes mountpoint /mnt/DupPool/ix-applications/releases/adguard-home/volumes/ix_volumes default
DupPool/ix-applications/releases/adguard-home/volumes/ix_volumes@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/adguard-home/volumes/ix_volumes/conf mountpoint /mnt/DupPool/ix-applications/releases/adguard-home/volumes/ix_volumes/conf default
DupPool/ix-applications/releases/adguard-home/volumes/ix_volumes/conf@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/adguard-home/volumes/ix_volumes/work mountpoint /mnt/DupPool/ix-applications/releases/adguard-home/volumes/ix_volumes/work default
DupPool/ix-applications/releases/adguard-home/volumes/ix_volumes/work@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/immich mountpoint /mnt/DupPool/ix-applications/releases/immich default
DupPool/ix-applications/releases/immich@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/immich/charts mountpoint /mnt/DupPool/ix-applications/releases/immich/charts default
DupPool/ix-applications/releases/immich/charts@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/immich/volumes mountpoint /mnt/DupPool/ix-applications/releases/immich/volumes default
DupPool/ix-applications/releases/immich/volumes@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/immich/volumes/ix_volumes mountpoint /mnt/DupPool/ix-applications/releases/immich/volumes/ix_volumes default
DupPool/ix-applications/releases/immich/volumes/ix_volumes@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/immich/volumes/ix_volumes/library mountpoint /mnt/DupPool/ix-applications/releases/immich/volumes/ix_volumes/library default
DupPool/ix-applications/releases/immich/volumes/ix_volumes/library@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/immich/volumes/ix_volumes/pgBackup mountpoint /mnt/DupPool/ix-applications/releases/immich/volumes/ix_volumes/pgBackup default
DupPool/ix-applications/releases/immich/volumes/ix_volumes/pgBackup@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/immich/volumes/ix_volumes/pgData mountpoint /mnt/DupPool/ix-applications/releases/immich/volumes/ix_volumes/pgData default
DupPool/ix-applications/releases/immich/volumes/ix_volumes/pgData@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/immich/volumes/ix_volumes/profile mountpoint /mnt/DupPool/ix-applications/releases/immich/volumes/ix_volumes/profile default
DupPool/ix-applications/releases/immich/volumes/ix_volumes/profile@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/immich/volumes/ix_volumes/thumbs mountpoint /mnt/DupPool/ix-applications/releases/immich/volumes/ix_volumes/thumbs default
DupPool/ix-applications/releases/immich/volumes/ix_volumes/thumbs@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/immich/volumes/ix_volumes/uploads mountpoint /mnt/DupPool/ix-applications/releases/immich/volumes/ix_volumes/uploads default
DupPool/ix-applications/releases/immich/volumes/ix_volumes/uploads@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/immich/volumes/ix_volumes/video mountpoint /mnt/DupPool/ix-applications/releases/immich/volumes/ix_volumes/video default
DupPool/ix-applications/releases/immich/volumes/ix_volumes/video@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/sfserver mountpoint /mnt/DupPool/ix-applications/releases/sfserver default
DupPool/ix-applications/releases/sfserver@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/sfserver/charts mountpoint /mnt/DupPool/ix-applications/releases/sfserver/charts default
DupPool/ix-applications/releases/sfserver/charts@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/sfserver/volumes mountpoint /mnt/DupPool/ix-applications/releases/sfserver/volumes default
DupPool/ix-applications/releases/sfserver/volumes@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/sfserver/volumes/ix_volumes mountpoint /mnt/DupPool/ix-applications/releases/sfserver/volumes/ix_volumes default
DupPool/ix-applications/releases/sfserver/volumes/ix_volumes@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-applications/releases/sfserver/volumes/pvc-98a199c3-d916-428f-8316-a90b9fad89fa mountpoint /mnt/DupPool/ix-applications/releases/sfserver/volumes/pvc-98a199c3-d916-428f-8316-a90b9fad89fa default
DupPool/ix-applications/releases/sfserver/volumes/pvc-98a199c3-d916-428f-8316-a90b9fad89fa@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps mountpoint /mnt/.ix-apps local
DupPool/ix-apps@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_configs mountpoint /mnt/.ix-apps/app_configs inherited from DupPool/ix-apps
DupPool/ix-apps/app_configs@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts mountpoint /mnt/.ix-apps/app_mounts inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/adguard-home mountpoint /mnt/.ix-apps/app_mounts/adguard-home inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/adguard-home@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/adguard-home/conf mountpoint /mnt/.ix-apps/app_mounts/adguard-home/conf inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/adguard-home/conf@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/adguard-home/work mountpoint /mnt/.ix-apps/app_mounts/adguard-home/work inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/adguard-home/work@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/immich mountpoint /mnt/.ix-apps/app_mounts/immich inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/immich@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/immich/backups mountpoint /mnt/.ix-apps/app_mounts/immich/backups inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/immich/backups@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/immich/library mountpoint /mnt/.ix-apps/app_mounts/immich/library inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/immich/library@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/immich/pgBackup mountpoint /mnt/.ix-apps/app_mounts/immich/pgBackup inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/immich/pgBackup@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/immich/pgData mountpoint /mnt/.ix-apps/app_mounts/immich/pgData inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/immich/pgData@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/immich/profile mountpoint /mnt/.ix-apps/app_mounts/immich/profile inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/immich/profile@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/immich/thumbs mountpoint /mnt/.ix-apps/app_mounts/immich/thumbs inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/immich/thumbs@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/immich/uploads mountpoint /mnt/.ix-apps/app_mounts/immich/uploads inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/immich/uploads@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/app_mounts/immich/video mountpoint /mnt/.ix-apps/app_mounts/immich/video inherited from DupPool/ix-apps
DupPool/ix-apps/app_mounts/immich/video@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/docker mountpoint /mnt/.ix-apps/docker inherited from DupPool/ix-apps
DupPool/ix-apps/docker@auto-2025-06-19_14-39 mountpoint - -
DupPool/ix-apps/truenas_catalog mountpoint /mnt/.ix-apps/truenas_catalog inherited from DupPool/ix-apps
DupPool/ix-apps/truenas_catalog@auto-2025-06-19_14-39 mountpoint - -
root@truenas[~]#
Your ix-apps dataset, containing stuff related to apps in the current TN versions, is mounted at DupPool/ix-apps.
What was the actual contents of /var/log/app_lifecycle.log ?
Edit: Nothing should be symlinked here. If you “fix” this by doing that you are living precariously.
Before you do anything else I recommend ensuring you have a good backup of anyhing you don’t want to lose.
I upgraded to 25.04 a while back and when i upgraded to 25.04.01 a few days ago TrueNas suddenly figured out that one of the SSD’s was completely dead.
I connected a external usb drive and followed this guide:
I just moved from my main pool having 2 disks to 4 disks.
I documented every single step and verified it works flawlessly by doing it a second time. This could save you hours if you are running the latest release because it talks about all the “gotchas” caused by bugs in the latest version of Scale and what the workarounds are.
Be sure to follow all the steps especially in moving the system dataset before you export the main pool!
add nvme card to slot near 4 eth ports and chelsio 10G eth c…
I went to apps screen, did unset pool option.
Then i restarted, went back to select pool, chose my pool DupPool and it worked and my apps showed up again, but wouldn’t start with the complaint.
The contents of app_lifecycle.log was:
So the part that tipped me off there was the /work folder at the end, I browsed around and saw the work folder existed in
I don’t know why the data is still in the old ix-applications and not in the new ix-apps.
I have the full backup still on the usb drive but that does me no good if I don’t know how to put things back in a working state.
The frustrating part is I’m SO close, since everything works when i manually connect the missing link but I realize this is not the right way to go about it.
Actually now that I look closer I think i see what went wrong.
The new ix-apps never went to the backup.
I think i saw a guide on doing the upgrade script manually, hopefully i can figure it out.
