Help needed to clear an [EFAULT] when deleting an official app

TrueNAS General
1

Please can you show me How to clean out everything to do with an unwanted official app (which I “stopped” but then the “delete” process failed) but do no damage to existing official apps?

(These are not installed via docker/dockge/docker compose, rather they are installed from the app “catalogue”).

Frigate developed a fault so I stopped it and installed a second instance (Frigate2).

Frigate2 is working perfectly and hence I deleted the original Frigate via the TN webUI but at the end of the process I was presented with

FAILED
[EFAULT] Failed to delete dataset: cannot destroy 'mainraid/ix-apps/app_mounts/frigate': filesystem has dependent clones use '-R' to destroy the following datasets: mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config@ix-applications-backup-system-update--2024-12-21_17:48:48 mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config@ix-applications-backup-system-update--2024-12-22_14:20:39 mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config@ix-applications-backup-system-update--2024-12-14_16:54:57 mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config

and “more info” tells me

concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs_/dataset.py", line 210, in do_delete
    subprocess.run(
  File "/usr/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['zfs', 'destroy', '-r', 'mainraid/ix-apps/app_mounts/frigate']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.11/concurrent/futures/process.py", line 261, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 112, in main_worker
    res = MIDDLEWARE._run(*call_args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 46, in _run
    return self._call(name, serviceobj, methodobj, args, job=job)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 34, in _call
    with Client(f'ws+unix://{MIDDLEWARE_RUN_DIR}/middlewared-internal.sock', py_exceptions=True) as c:
  File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 40, in _call
    return methodobj(*params)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 55, in nf
    res = f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 183, in nf
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs_/dataset.py", line 221, in do_delete
    raise CallError(f'Failed to delete dataset: {error}', errno_)
middlewared.service_exception.CallError: [EFAULT] Failed to delete dataset: cannot destroy 'mainraid/ix-apps/app_mounts/frigate': filesystem has dependent clones
use '-R' to destroy the following datasets:
mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config@ix-applications-backup-system-update--2024-12-21_17:48:48
mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config@ix-applications-backup-system-update--2024-12-22_14:20:39
mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config@ix-applications-backup-system-update--2024-12-14_16:54:57
mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 509, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 556, in __run_body
    rv = await self.middleware.run_in_thread(self.method, *args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1367, in run_in_thread
    return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1364, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 268, in nf
    rv = func(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 55, in nf
    res = f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 183, in nf
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 351, in do_delete
    return self.delete_internal(job, app_name, app_config, options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 366, in delete_internal
    self.middleware.call_sync(
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1654, in call_sync
    return self.run_coroutine(methodobj(*prepared_call.args))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1694, in run_coroutine
    return fut.result()
           ^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 250, in delete
    return await self.middleware._call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1468, in _call
    return await self._call_worker(name, *prepared_call.args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1474, in _call_worker
    return await self.run_in_proc(main_worker, name, args, job)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1380, in run_in_proc
    return await self.run_in_executor(self.__procpool, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1364, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
middlewared.service_exception.CallError: [EFAULT] Failed to delete dataset: cannot destroy 'mainraid/ix-apps/app_mounts/frigate': filesystem has dependent clones
use '-R' to destroy the following datasets:
mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config@ix-applications-backup-system-update--2024-12-21_17:48:48
mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config@ix-applications-backup-system-update--2024-12-22_14:20:39
mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config@ix-applications-backup-system-update--2024-12-14_16:54:57
mainraid/ix-applications/releases/frigate/volumes/ix_volumes/config

I’m happy to ssh into TN and carry out commands as advised by experts here.

(This problem is similar to one I had several days ago for a different problem but I didn’t get to the bottom of it that time).

2

They’re working on a fix for this in 24.10.2, but in the meantime the easiest way to solve it is to first delete everything relating to frigate in ix-applications (which is the old app data) or just delete the whole ix-applications dataset altogether. That will clear out those dependent clones and then it should happily delete the docker app (which is saved in ix-apps).

3

It’s good to hear that there’s a fix due at some point. Nice!

I haven’t been able to bring myself to delete the directory at

/mnt/mainraid/ix-applications

because I am worried it could affect those apps which I have installed (from the catalogue) .

However, I did get rid of

/mnt/mainraid/ix-applications/backups/system-update--2024-12-21_17:48:48
/mnt/mainraid/ix-applications/backups/system-update--2024-12-22_14:20:39
/mnt/mainraid/ix-applications/backups/system-update--2024-12-14_16:54:57

which deleted recursively and I hope that has fixed the problem.

I do note that other dir you referred to, located at at /mnt/.ix-apps, which I understand is the new location for Electric Eel, hence the reason why the old location at /mnt/mainraid/ix-applications could be deleted seeing as it ought not to be in use any more.

But I was too scared!

Thanks for your guidance.

4

Exactly! ix-applications is essentially just legacy data that only would be used if you were to roll back to 24.04 and need to load the Kubernetes data. Nothing currently running in 24.10 relies on it.

Take a look at https://www.truenas.com/docs/truenasapps/#understanding-app-storage-volumes

5

Great! I remember that another iX member of staff kindly explained this to me a few weeks ago when I was struggling with something similar (but I failed to take the plunge then due to being wrapped up with solving other matters).

i’ll have a good read and then I shall delete it - no need for me to roll back. Thanks for this additional help and further guidance.