It looks like you said, you have a sVDEV that consists of a mirror pair and a single device.
Because it involves sVDEVs and a Raid-Z2 pool, destroying the pool and rebuilding may be the only option. What version of TrueNAS? I don’t know if you attempt to remove the single sVDEV if it will kill your pool
Unfortunately that’s not what happened. You added another disk as a stripe to your special vdev unknowingly. This currently puts your pool at great risk as if that single device fails you will lose your entire pool. As @SmallBarky said I think you’ll need to backup and destroy the pool and recreate it.
Not intending to add insult to injury but a friendly reminder that one can create a pool checkpoint before any topology changing operation. Just in case (like this one).
You’re not going to like it…
Backup. Destroy. Restore.
If you have a fourth slot, you may evolve to a less unsafe geometry by extending nvme2 to make it a 2-way mirror, so you have at least redundancy on all metadata. But this is not the setup you intended.
Did I lose all the data?
Just in case, I made three replicas of the dataset on three different systems.
Deleted the dataset, recreated it without sVDEVs, and restored one of the replicas.
Now, when choosing the Apps pool, I get the following error:
[Errno 2] No such file or directory: '/mnt/.ix-apps/app_configs'
Does this mean that TrueNAS stores the application configuration data in a hidden dataset that cannot be replicated or backed up?
You have remote replicas of all your datasets, you have copies of your configuration and keys, then something happens, I don’t know, your home burns to the ground and you lose all your hardware.
What are you supposed to do when you buy a new server?
Ideally you should use hostpath for the both the config storage and data storage.
I’ve created a single Dataset called “app-config” and each app has their own folder inside that dataset. That way if i screw up i only have to point a new install to the older folder and the app is back up in a few minutes.
But then again i only use a single app from the app catalogue, which is scrutiny. Every other app i’ve deployed is via my own compose-file and portainer.
That’s another reason (apart from the Kubernetes → Docker migration and the inability to update Apps on TrueNAS <24.10.2.2) why the only TrueNAS app I am using is Portainer… which then actually manages my Apps.
I know pointing that out won’t bring your data back of course so…sorry for your loss there.
I understand that, I don’t think that is the issue.
All my applications already use host path for everything, data, DB, configuration, etc.
The problem is that I cannot restore the Apps pool, essentially the Docker configuration space, as it seems to be a hidden place that nobody can access, copy, back up, or replicate. It does not make any sense!
