People's approach to app backup/rollback when using dockge/portainer

I apologize if this has been discussed previously, but I couldn’t come up with terms that let me find much.

A popular approach to maximize flexibility, ease of potential future migration, and deal with the shortcomings of the current Apps library/configuration shortcomings since EE is to install Portainer, Dockge, or soon Komodo, provide Stacks/Data datasets and then create and manage all other Compose containers through one of those.

At the same time, one of the advantages of TN’s Apps and using the Custom App feature is that app data is more distinct and there is a backup/rollback system that works out of the box. Perhaps I’m mistaken, but it seems to me that by using the above approach you essentially lose this advantage of more isolated data, since now presumably ALL of your containers and stacks sit under the same datasets. Of course you can snapshot the datasets, but if you ever want to manually alter/access container data you have to take down ALL of your containers, and if you ever one to revert to a snapshot after a bad update, mistake, or other form of data loss, ALL of your containers would be reverted to that snapshot in the process.

I’ll be able to migrate to EE soon and like the approach of not using the built-in apps system since I want to setup a reverse proxy and add multiple apps that are not in the catalogue, which is seems might be more challenging using the Custom Apps feature and so I’d rather just keep it all separate and use something like Dockge or Portainer. Plus, this would make it easier to migrate my apps off SCALE if I ever needed to; however, at the same time I’d also like to have my apps be as hard to “permanently” break as possible, where if somehow they really become screwed up I can always simple revert them to their previous state assuming I have an appropriate snapshot.

Of course I can put my stacks on Git or something (and even use them directly with Komodo for example), but that leaves the concern of the app data itself.

Essentially, my question is, does anyone somehow have a setup on SCALE that lets them use an “external” orchestrator, while keeping separate datasets for each app? Possibly even for the Compose files too? I doubt it’s possible to do this below the dataset level since you can’t snapshot folders obviously.

I guess what I’m saying is, the dream for me would be using a Container management app, while also being able to completely and utterly screw up an apps data or compose file and then just down the stack, restore the apps snapshot, up the stack and have it back and running like nothing ever happened, all the while my other stacks kept chugging along. That’s more-or-less how it was with the old K8s.

EDIT: Idk if EE still has the “Additional mounts” for apps where you can add arbitrary host paths, but I imagine that would let me do what I want in part, though idk if Dockge, for example, would be able to handle that separation, especially in regards to the stacks.

It’s been so long since I’ve really touched my TN machine so I forgot, but was reminded by a Stux video; when mounting a dataset with children, any child datasets are also mounted as sub-paths with the container, right? I know it works this way with SMB shares too.

I guess that solves most of my issue in regards to the data since I can just have one child dataset per app and mount the parent to the orchestrator… though not the stacks with dockge specifically since it doesn’t support compose files in sub-folders, though of course that’s a dockge issue and not a TN SCALE issue. Perhaps this is another reason to consider Portainer or Komodo, though having to manually restore the compose file is a little less of a hassle. The data was the main concern.

Though, seamless, container specific rollback of both the compose file and data all at once would still be nice. Seems like with the Custom App option you get more obfuscated data that’s annoying to snapshot and the limitations of app configuration, but nice updates and rollbacks, while with an other orchestrator you get more flexibility and easier snapshots/data organization, but then you have to handle updates and backups entirely on your own.

Maybe eventually the best of both worlds will be possible.

EDIT: This is an interesting in-between: Electric Eel - How I am using Dockerfile, .env files, compose files - #7 by sfatula

I have always advocated different datasets for different host mounts.

Combining this with tiered snapshots means I can restore any compose version from any snapshots. Ditto with the datasets.

And if you really want rollback, don’t use the “latest” tag on your images and instead fix the versions.

I prefer to let it ride until I’m bitten… then I begin fixing versions.

So, using Dockge, you can simply start/stop individual stacks and rollback, clone etc their datasets.

1 Like

Nested snapshots are tiered backup are what I already use and it was actually the first video you linked that I had watched which reminded me that nested datasets are automatically brought into a container when mounting a parent one :slight_smile:

So it seems the situation already wasn’t too far off from what I was hopping for and I had just forgotten that aspect of ZFS… woops.

Fixing the tags definitely seems like the way to go, and then either using “partial” tags (i.e. 1.1 instead of 1.1.0) to have automatic (or notifications of) patch or minor updates with something like Watchtower, or a more sophisticated update check with WUD.

Then let it fly as is with the auto updates on less critical containers and fix them when they break since semver practices are never perfect, and then probably always manually update the super important ones, if any.

Thanks for helping me twice, directly and indirectly.

1 Like