TrueNas Scale needs a first party application backup and restore system

[skittlebrau]

@sfatula

That’s not always trivial. Backing up a Nextcloud container for example, correctly, involves setting maintenance mode, running a mysqldump or pgdump, etc. You wouldn’t get a good backup just doing a server level backup of some sort. It’s not the only app with special backup procedures either. And with going to docker, it can get even worse as someone may run a separate postgres database “server” container for many purposes, making it even more difficult. I suppose they can make a best effort but it won’t be a substitute for me for a container based backup of the special apps.

If someone was using Nextcloud AIO (for standardisation sake - so everything is set up in a predictable manner) then I imagine if the backup/restore script had the option for running a script before/after backups, you could just set it to issue the php occ command to enable maintenance mode before the backup snapshot is taken.

[sfatula]

@skittlebrau

you could just set it to issue the php occ command to enable maintenance mode before the backup snapshot is taken.

My statement was just an example, there are tons of examples. For Nextcloud, if using mariadb or postgres, you can not get a consistent backup of the database simply by setting maintenance mode. You may need to stop the database from running, or issue a lock, or any number of other things. You cannot simply copy files and database files. For this reason, I backup nextcloud from inside the container so it can easily do all the steps.

I do agree it could be possible to include backup pre/post or even the script and backup each application if they are given the intelligence to do so. And that would allow single application restore thusly. There are many other apps with special backup procedures. Nextcloud was just one.

[Trudge]

Why can’t the containers simply be stopped before backing up? Seems like the most straightforward universal method…

[skittlebrau]

I was going to suggest the same thing. Stopping the containers before backup would be simplest. That would cover everything.

Too difficult to do live backups with full consistency.

[sfatula]

You can, unless… Some of us have containers referenced by other containers. So, I have a mariadb instance that is used by around 10 different things for me now, some are apps on Scale, some are external to scale, some are in a VM (HomeAssiatant), etc. My nextcloud uses the one mariadb container as it’s wasteful and slower in many cases to have 10 copies of mariadb, one for each “app”, and I despise sqlite. Some will say that’s not how you should do it, and that’s fine. There are cases for both.

For the standard user and likely most users, stopping the container will indeed enable a consistent backup and is not a bad idea. Some of my stuff, I would never ever want stopped so I would hope that it could be configured if that is the way it ends up going. I would undoubtedly keep using in app backups for most things like Emby, nextcloud, homeassistant, etc.

[Trudge]

Yeah that’s a valid point but it’s good to know there might be a simple solution that technically simple. That being said, ideally, a backup system would be more robust offering a live backup.

So this really comes down to how the ix app ecosystem is curated. Ix systems needs to provide the template or templates in order to ensure best practice. Any exception needs a visual warning. Shared resources through databases or other infrastructure should be encapsulated in best practice to maintain optimum compatibility for backup and restore, which shouldn’t be a problem if it’s a standard.

Once that’s published from ix the community needs to leverage those templates to build the ecosystem out.

To be clear I’m not talking about custom containers but IX apps. Backup/restore of custom containers might be possible with caveats but that seems a bit unlikely without some sort of template as mentioned above container adaptation or custom scripts per application.

All that being said it’s an IX systems responsibility to lay out the design goals and implementation before the community can contribute.

My greatest concern is not having a granular backup system that can restore individual apps. No one’s really addressed that yet.

[IDtheTarget]

I came across this post while trying to figure out how to backup my frigate app before changing from “Normal image” to “TensorRT image” (trying to figure out how to get my Coral TPU to work, but that’s an entirely different hair-pulling adventure).

In VMWare, I could simply take a snapshot and try whatever experiment I was going to try. If it didn’t work, I could revert back to the snapshot. I don’t see anything like that functionality for either apps or VM’s in the UI. I’ve got a lot of time invested in my frigate config (cameras, etc) and I really don’t want to lose it if this latest attempt to get a freaking USB dongle to be seen in an app. (sorry, been fighting this thing for a week).

I guess I should have used separate datasets for the configs and video instead of using the default ix-applications settings, at least then I could use the “periodic snapshot” or “replication task”. But this is a lot more cumbersome than what we have to do at work with VMWare

[E_B]

I’m in this same position but this thread might help:

Frigate on Dragonfish - upgrade but then roll back?

Hello I’m staying on Dragonfish for now: the version of Frigate which is running, and which was present when I first installed the app, is 0.13. Now there is an upgrade available to 0.14. Can someone confirm that, should 0.14 not work well, there will be a “roll back” button and hence I will be able to revert to 0.13 by using it please? I don’t know if all apps have roll back options - I know that in my case I see it for jellyfin and for adguard-home which I have updated frequently, but havi…

You ought to be able to fiddle around with things, mess them up and then roll back. (I haven’t had time to try it just yet).

[Peter_Atkin]

I brought dedicated hardware (A+ Server 2114S-WN24RT) for TrueNas Scale but when ever I go through theses forums, I get the feeling that I should walk well away and put a raid controller in, I the hardware for fast storage and backup for our ESXI vm servers, specs below.

• Supermicro A+ Server 1014S-WTRT - 1U - 4x SATA/SAS - 2x M.2 - Dual 10-Gigabit LAN - 500W 1+1 Redundant
• AMD EPYC 7203P Processor 8-core 2.80GHz 64MB Cache (120W)
• 8 x 16GB PC4-25600 3200MHz DDR4 ECC RDIMM
• 2 x 480GB Micron 7450 PRO Series M.2 PCIe 4.0 x4 NVMe Solid State Drive (80mm)
• 4 x 960GB Micron 5400 PRO Series 2.5" SATA 6.0Gb/s Solid State Drive
• 12 x 7.68TB Solidigm SSD D7-P5520 Series U.2 PCIe 4.0 x4 NVMe Solid
  State Drive
• Supermicro Update Manager (SUM) (OOB Management Package), included

From what I’m seeing TrueNAS scale is not mature at all, and should not be used for anything critical, as it just does not have the data redundancy features we come to expect from such devices, am i wrong?

[Peter_Atkin]

Sorry 8 x 64GB PC4-25600 3200MHz DDR4 ECC RDIMM (512GB)

[dan]

@Peter_Atkin

it just does not have the data redundancy features we come to expect from such devices

What exactly are the “data redundancy features” you “come to expect,” and what makes you think SCALE doesn’t have them?

[Trudge]

Welcome, Peter!

I would say it’s very mature, strictly for ZFS and traditional NAS functionality (except for ZFS Expansion)

I have no complaints of the reliability in those aspects. There are two aspects that I find troubling.

1. Hosting through TrueNas through their catalog and infrastructure is completely unusable to me without reliable backup and restore capability in any setting, including enterprise. I’m hopeful this will change.

All that being said, it can be apps very reliable if you’re willing to build your own infrastructure with setup independent of TrueNas rolling your own backup and restore system.

I am not one of those people willing to roll my own infrastructure. This is a functional appliance, not a hobby to which I will entrust very important aspects of my digital life.

2. Equally as important TrueNas needs to improve its usability. ACL permissions are a nightmare. There could be a vastly simpler user interface with a basic view and the enterprise advanced view. It’s my opinion TrueNas will never adopt that use case.

HexOS is in beta by another company built on top of TrueNas which focuses on user interface, simplicity, and still gives access the original interface of TrueNAS. In beta, it’ll cost $199.

HexOS financially invested in by ixsystems along with a technical partnership. Therefore… TrueNAS will never focus on accessible user-friendly ui with enterprise features behind advanced menus as described above. ixsystems has created market segmentation between HexOS and TrueNAS. HexOS will be targeting Synology, NAS-like experience and TrueNAS will increasingly focus on enterprise.

ixsystems is missing the opportunity to serve both the enterprise and the average user, creating a better experience for all involved in TrueNAS.

All that is to say, what is a person to do?

1. I’ve chosen to switch to HexOS because ixsystems no longer has incentives to make TrueNAS user friendly and vertically integrated by default without a wealth of technical knowledge. However, they do have motivation to create features that will be utilized by HexOS. Being in beta, currently HexOS is not ready for prime time.

2. The second option is to separate out NAS and hosting to different use cases through virtualization or independent hardware. This will require at least two different platforms and also a wealth of technical knowledge to make robust and reliable specifically for applications/hosting.

Anyway, just wanted to share my thoughts and wish you the best!

[Stux]

@Trudge

HexOS financially invested in by ixsystems

News to me.

Got a reference?

[dan]

@Trudge

ixsystems no longer has incentives to make TrueNAS user friendly and reliable/integrated

Surely iX has no incentives to make their own commercial product user friendly.

[Trudge]

References that HexOS financially is invested by ixsystems

Revealing my NEW Investment!

and

TrueNAS Launches “Powered by TrueNAS” with First Partner HexOS

[Trudge]

Sarcasm or not remember my earlier statement:

@Trudge

ixsystems is missing the opportunity to serve both the enterprise and the average user, creating a better experience for all involved

[dan]

@Trudge

Sarcasm or not

It was sarcastic, because your comment that I quoted was both nonsensical and indefensible. And the other comment of yours that you quote here doesn’t change that.

[Trudge]

It’s Internet you can believe whatever you want is “nonsensical and indefensible”. Frankly it’s a misnomer that enterprise focus product cannot be made user-friendly. User-friendly has nothing to do with a product not having enterprise features. I edited the quoted post to better reflect my thoughts. Beyond that I owe you nothing. So I’m not going to respond anymore to this particular subject matter regarding what you think. It’s not the purpose of this thread.

[dan]

@Trudge

It’s not the purpose of this thread.

Indeed not; nothing you’ve posted to this thread has anything to do with its purpose.

[pmh]

Weird. I have perfectly fine working backup for all of my apps. And I already had to use the restore case for a single app multiple times. Went without a hitch. I must be doing something wrong.

What I do:

• Set all volumes for all apps to a dedicated host path dataset under a common parent dataset.
• Recursively snapshot and replicate that to another TrueNAS (could be any system running ZFS) on site.
• Recursively replicate to a third off site TrueNAS.

In case an app breaks, either

• shut down, roll back, start
• delete, re-deploy, shutdown, copy the data from backup to the volumes, start again

BTW that’s how I treat FreeBSD jails on TN CORE, VMs, everything. All of this can be set up using the UI and TrueNAS standard functions.

The restore case needs the command line to copy the data back.

Kind regards,
Patrick