After a recent reboot, I encountered an issue where my datasets are not showing the files, but the storage space is still occupied. I did not mess with ACL or any other permissions, but I suspect this might be related to user perimissions somehow.
I use NFS to access my data and I mount as root. Advanced NFS settings are set to Mapall User: root and Mapall Group: wheel. I suspect this is not the best way to do it, but I`m not sure this is the cause of the issue. This is how I mount on my PC:
sudo mount -t nfs -v $available_ip:mnt/MrBig/Big $mount_location
Specifically, I see the following behavior:
- The main datasets storing data show used space in accordance to the data stored, but the data is not accessible (the folders are empty when mounting)
- There are no files when mounting with NFS, butI still have full permissions. I can create new files that seem to show up directly to parent pool quota (MrBig) instead of counting as files stored to that children (MrBig/Big).
- Clicking on the pools in the GUI results in an error related to space quotas.
- Error message:
[EFAULT] Failed retreiving USER quotas for MrBig/Big
*Sometimes I get a similar error message: 'Failed retreiving GROUP quotas for MrBig/Big`
Full error message: [EFAULT] Failed retreiving USER quotas for MrBig/Big
Error: concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 760, in get_quota
quotas = resource.userspace(quota_props)
File "libzfs.pyx", line 465, in libzfs.ZFS.__exit__
File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 760, in get_quota
quotas = resource.userspace(quota_props)
File "libzfs.pyx", line 3532, in libzfs.ZFSResource.userspace
libzfs.ZFSException: cannot get used/quota for MrBig/Big: dataset is busy
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.9/concurrent/futures/process.py", line 243, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 115, in main_worker
res = MIDDLEWARE._run(*call_args)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 46, in _run
return self._call(name, serviceobj, methodobj, args, job=job)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 40, in _call
return methodobj(*params)
File "/usr/lib/python3/dist-packages/middlewared/worker.py", line 40, in _call
return methodobj(*params)
File "/usr/lib/python3/dist-packages/middlewared/plugins/zfs.py", line 762, in get_quota
raise CallError(f'Failed retreiving {quota_type} quotas for {ds}')
middlewared.service_exception.CallError: [EFAULT] Failed retreiving USER quotas for MrBig/Big
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 184, in call_method
result = await self.middleware._call(message['method'], serviceobj, methodobj, params, app=self)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1317, in _call
return await methodobj(*prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1379, in nf
return await func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/pool.py", line 4116, in get_quota
quota_list = await self.middleware.call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1368, in call
return await self._call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1325, in _call
return await self._call_worker(name, *prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1331, in _call_worker
return await self.run_in_proc(main_worker, name, args, job)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1246, in run_in_proc
return await self.run_in_executor(self.__procpool, method, *args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1231, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
middlewared.service_exception.CallError: [EFAULT] Failed retreiving USER quotas for MrBig/Big
Here is more info about the pool
Type: FILESYSTEM
Sync: STANDARD
Compression Level: LZ4
Enable Atime: OFF
ZFS Deduplication: OFF
Case Sensitivity: ON
Path: MrBig/Big
**Permissions**
Owner: root
Group: root
Unix Permissions
root: Read | Write | Execute
root: Read | Write | Execute
Other: Read | Write | Execute
**User quotas**
Name ID Data Quota DQ Used DQ % Used Object Quota OQ Used OQ % Used
root 0 — 47.93 GiB — — 35859 —
— 1000 — 2.18 TiB — — 984939 —
**Group quotas**
Name ID Data Quota DQ Used DQ % Used Object Quota OQ Used OQ % Used
root 0 — 15.89 MiB — — 13 —
— 1000 — 1.85 TiB — — 775977 —
— 1001 — 392.80 GiB — — 244808 —
For context, I got this issue weeks ago and after some failed attemps at fixing it, I deleted and recreated my pools. This had solved the issue for the moment. I was able to upload all my data to the server and access it properly. But now,after some time and a reboot the issue is back again and I`m not sure how to properly debug it.
I hope I can fix any config mistakes so I can acces the files again. I have a backup so I can set it up again properly if necesary, but I am afraid that simply reseting will produce the issue again.
I am quite new and inexperienced with this, so I apologise for any rookie mistakes and for potenlitally not following the forum guidelines. Let me know if any information is necesary and thank you for taking the time to help me!