All apps missing after pool hit 100%

Apps and Virtualization
1

My apps pool is on a pair of old 200gb enterprise SSDs I had lying around. I didn’t realize I would use the Apps as much as I did. It has slowly been filling, and I needed to move to bigger drives. I pulled the trigger the other day and got a couple 2TB SSDs to migrate to…Here’s the rub: Today I decided to make the move; however, now, apparently, my Apps drive is 100% full, and now all my apps are missing in the GUI. They are still running, as my Homer dashboard can get to all of them. I have SSH’d in and deleted some files to get some free space, but that didn’t help the GUI. I have rebooted many times, tried moving the apps pool to my large array…moved it back…nothing. Apps are running but not showing in the GUI.

I am on Scale 25.04.2.4. I have hundreds of TB free on my large array, but I don’t have any more drive slots, so the old APPs drives need to be pulled to add the new, bigger drives.

How do I move my apps to bigger drives?
Google searches all turn up the fact that changing the pool in the GUI will prompt to move data to the new pool…it does not. It just changes pool and then there are no apps available. Switching back seems to make all the apps work again.

How do I get apps to show back up in the GUI?

2

As far as moving your apps pool to another set of drives, I have used this: HOWTO: Copy the hidden ix-apps dataset from one pool to another

You have to do this in addition to changing the pool in the gui.

I have had my apps disappear from the gui but still run in the background, but I think that unsetting the apps pool in the gui and then choosing it again made them appear. Seems to be a different issue here if that didn’t work.

As far as physically connecting the new drives, you could use the link above and copy all the app pool data to your large data pool. Then export the app pool within the gui. Then make a new pool with the new drives and copy the data back to the new app pool. Someone might have a better solution, but that has worked for me in the past.

3

Or update to 25.10-RC and use the now build in move function for apps

4

I was afraid to upgrade while things were broken. Did the upgrade and all apps are working, but still not showing. I can’t do the migrations as it says ix-apps already exists on the other pool, which is weird because it’s never had apps on it.

I don’t have time or the mental capacity to deal with this tonight, but I will try destroying the ix-apps on my bulk pool tomorrow afternoon.

5

No joy updating to 25.10 and migrating. It fails every time, and after a failure, I must destroy the ix-apps on my bulk storage.

followed the link Manna provided, and it seems like it worked, but I still don’t have any apps showing. They seem to be running, but not in the GUI.

I also don’t know how I would confirm it’s tuning on my Bulk storage and not the old SSDs.

6

Is there no way to get apps to show in the GUI again? I assume that during some upgrades, the apps database must be rebuilt. Is there no way to run whatever command is run that looks through the app’s storage and determines what is there?

So far as I can tell all my apps are now in my bulk pool. Everything seems fine, but I don’t want apps on spinning rust. It’s pretty performant as it has 120 drives but I would feel better if apps were on their own SSD and showing in the GUI again.

What if I took a list of what I have installed… installed the apps in the GUI, then overwrote the folders with my configs? It sounds tedious, but I am desperate to get my GUI access back.

7

Since no one seems to know the answer, I set out to fix it or break it more, since even the forum answers were usually just start over from scratch… that’s not an option. Just my ARRs stack is 12 apps with fairly complicated configs. Tdarr alone is months of custom code for my AV1 setup.

OK. Enough bitching. Let’s get to the fix.

Need to open CLI. I SSH’d in and typed CLI to get there. From there, we need the apps namespace. type apps. (You can also do this in the GUI by going to System>shell or directly on the computer with option 6)

Some of these commands may need Sudo prefixed. I logged in as root, so I didn’t need it.

Your prompt should now look like this
[truenas] apps>

You can get a list of commands by typing LS, but the sytem will autofill for you if you just follow along.

[truenas] apps> used_host_ips

This will give you a list of the app names along with their associated IPs. we don’t care about the IPs, but the names are needed.

Now pick the app that means the least to you, just in case something bad happens. For me, it filebrowser as it really doesn’t have much config if I had to rebuild it.

Now let’s redeploy

[truenas] app> redeploy filebrowser

this will take a minute depending on the speed of oyout system.

Once done, head back to the GUI and magically all your apps should be waiting there for you. If any were missed, just follow this again and redeploy that app.

2 Likes