TrueNAS Scale Fangtooth 24.04.2.6
Added two new NVME drives in Mirror to a new pool ‘App_Pool’ and created a dataset
All media/apps/etc in pool called ‘Stuff’ which is a RaidZ2 of 4x8Tb HDDs
Was running Emby/Qbit/Tailscale/Syncthing/Portainer (Which had a dozen apps or so)
Kept getting and error when trying to switch or unset the app pool:
[EFAULT] Failed to umount dataset: cannot unmount ‘/mnt/.ix-apps/docker’: pool or dataset is busy
(Full error from log below)
Went in shell and as root ‘zfs unmount -f -a’, no change
Deleted all my apps and their images and restarted, no change
Restarted Docker service, no change
I have a snapshot and some app data and other important data in a 3-2-1 backup, but I will struggle to backup all 14Tb and rebuild
Please advise on how to fix this issue
Full error below:
concurrent.futures.process.RemoteTraceback:“”“Traceback (most recent call last):File “/usr/lib/python3/dist-packages/middlewared/plugins/zfs/dataset_actions.py“, line 70, in umountwith libzfs.ZFS() as zfs:File “libzfs.pyx”, line 534, in libzfs.ZFS.exitFile “/usr/lib/python3/dist-packages/middlewared/plugins/zfs\_/dataset_actions.py”, line 72, in umountdataset.umount(force=options\[‘force’\])File “libzfs.pyx”, line 4321, in libzfs.ZFSDataset.umountlibzfs.ZFSException: cannot unmount ‘/mnt/.ix-apps/docker’: pool or dataset is busy
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/usr/lib/python3.11/concurrent/futures/process.py”, line 261, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/worker.py”, line 116, in main_worker
res = MIDDLEWARE._run(*call_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/worker.py”, line 47, in _run
return self._call(name, serviceobj, methodobj, args, job=job)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/worker.py”, line 41, in call
return methodobj(*params)
^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/schema/processor.py”, line 178, in nf
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs/dataset_actions.py", line 76, in umount
raise CallError(f’Failed to umount dataset: {e}')
middlewared.service_exception.CallError: [EFAULT] Failed to umount dataset: cannot unmount ‘/mnt/.ix-apps/docker’: pool or dataset is busy
“”"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/middlewared/job.py”, line 515, in run
await self.future
File “/usr/lib/python3/dist-packages/middlewared/job.py”, line 560, in __run_body
rv = await self.method(*args)
^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/api/base/decorator.py”, line 93, in wrapped
result = await func(*args)
^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/docker/update.py”, line 117, in do_update
catalog_sync_job = await self.middleware.call(‘docker.fs_manage.umount’)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 1005, in call
return await self._call(
^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 720, in _call
return await methodobj(*prepared_call.args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/docker/fs_manage.py”, line 33, in umount
return await self.common_func(False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/docker/fs_manage.py”, line 20, in common_func
await self.middleware.call(‘zfs.dataset.umount’, docker_ds, {‘force’: True})
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 1005, in call
return await self._call(
^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 728, in _call
return await self._call_worker(name, *prepared_call.args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 734, in _call_worker
return await self.run_in_proc(main_worker, name, args, job)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 640, in run_in_proc
return await self.run_in_executor(self.__procpool, method, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 624, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
middlewared.service_exception.CallError: [EFAULT] Failed to umount dataset: cannot unmount ‘/mnt/.ix-apps/docker’: pool or dataset is busy