Help ! , Persistent 1.53 TB ghost USEDCHILD space on dataset with no child datasets visible

TrueNAS General
1

Hi Guys ,

I’m experiencing a persistent ghost space issue with one of my datasets.

The dataset Tank/NAS/Media shows 4.25 TB used, but the actual data inside (USEDDS) is only 2.72 TB. ZFS is reporting 1.53 TB under USEDCHILD, even though zfs list -r shows no child datasets. I have already cleaned up all old automatic snapshots (they are now at 0B), tried creating/destroying a temporary child dataset, and rebooted the system. The pool itself is healthy (ONLINE, no errors after scrub).

This 1.53 TB of ghost space has remained for a while and does not appear to be reclaiming. Has anyone seen this kind of stubborn USEDCHILD accounting before, and is there a reliable way to force ZFS to release this phantom space without having to move all the data to a new dataset?

Any help would be greatly appreciated.

Cheers

2

Did you at any point in time create a manual snaphot with unlimited retention time?

3

Hi There , Yes i have done snapshots of this dataset , as a regular snapshot task . However, I have since deleted all visible snapshots (both automatic and manual) on this dataset. Currently zfs list -t snapshot -r Tank/NAS/Media shows almost nothing (only one very recent small snapshot that I just deleted).Is it possible that an old manual snapshot with long retention is still holding the 1.53 TB even after being destroyed?

4

I wonder if you somehow created a bookmark of a snapshot that no longer exists, or checkpointed your pool? What do zfs list -t bookmark -r Tank and zpool status -v Tank show?

1 Like
5

Thanks for the suggestion.
I ran both commands:

zfs list -t bookmark -r Tank
→ no datasets available

zpool status -v Tank
→ Pool is ONLINE with no errors

No bookmarks exist either. The pool looks clean.

6

Run fifty complete memtests overnight.

We can’t keep guessing without the full output.

zfs list -r -t fs,vol -o space Tank/NAS/Media

Preferably the entire pool.

zfs list -r -t fs,vol -o space Tank

Checkpoint, dedup, and block-cloning.

zpool list -o size,alloc,free,freeing,ckpoint,dedup,bclone_saved Tank
2 Likes
7

As @winnielinnie asked, the output of the commands will show where the space is used. To be clear, the parameter USEDCHILD is for child Datasets. So if you have ANY nested datasets, like this, it will show the space:

> zfs list -r -t fs,vol -o space rpool/src  
NAME                        AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
rpool/src                   1.00G  4.00G        0B   1.92M             0B      4.00G
rpool/src/6.18.18-gentoo    1.00G   764M        8K    764M             0B         0B
rpool/src/6.18.18-gentoo.1  1.00G   375M        0B    375M             0B         0B
rpool/src/6.18.26-gentoo    1.00G   764M        8K    764M             0B         0B
rpool/src/6.18.26-gentoo.1  1.00G   375M        0B    375M             0B         0B
rpool/src/6.6.30-gentoo     1.00G   672M        0B    672M             0B         0B
rpool/src/6.6.30-gentoo.1   1.00G   298M        8K    298M             0B         0B
rpool/src/6.6.30-gentoo.2   1.00G   231M        8K    231M             0B         0B
rpool/src/6.6.30-gentoo.3   1.00G   303M        8K    303M             0B         0B
rpool/src/6.6.30-gentoo.4   1.00G   310M        0B    310M             0B         0B
8

Hi Mate thanks for the info . Here is the output .

zfs list -r -t fs,vol -o space Tank/NAS/Media
NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD
Tank/NAS/Media 6.84T 4.25T 0B 2.72T 0B 1.53T

zpool list -o size,alloc,free,freeing,ckpoint,dedup,bclone_saved Tank
SIZE ALLOC FREE FREEING CKPOINT DEDUP BCLONE_SAVED
58.6T 48.4T 10.2T 0 - 1.00x 5.75G

9

Are you omitting anything? I understand if you want to censor or redact sensitive names. You can change the names of datasets in your output.

USEDCHILD for Tank/NAS/Media is 1.53T, but there are no rows for any children in your output, even though you used the -r flag.

You’re not using deduplication, there is no pool checkpoint, and the savings from block-cloning is only 5.75G. We can confidently say the discrepancy is not for those reasons.

10

Did you ever “clone” a snapshot into a new dataset?

We might have to use zpool history to determine if there was a zfs or zpool action that happened in the past that might explain this.

Do you remember ever creating a child dataset underneath Tank/NAS/Media? Maybe it was somehow created with a non-printing character? I would assume that this wouldn’t prevent its listing with the zfs command, but you never know.

11

Thanks for the help ,I ran the command and here is the output of zpool history | tail -100:
I have pasted that under the message I never created any child datasets under Tank/NAS/Media. The only datasets I’ve ever created under Tank/NAS were in other folders (Work , Data , Media, etc.).
I have however created and deleted manual snapshots on the Media dataset in the past. All snapshots have since been removed.
Is it still possible for old deleted snapshots or some other action to leave behind 1.53 TB of USEDCHILD space even after everything visible has been cleaned up?

zpool history | tail -100

2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-RC.1/conf with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-RC.1/audit with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.1/var/log/journal with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-BETA.1/var/lib/incus with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2.6/usr with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-RC.1/var/ca-certificates with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2/data with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.1/audit with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.1/var/lib with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2/conf with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-BETA.1/conf with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2/etc with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.0/var/log/journal with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2/var/lib with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10.3.1/conf with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2/root with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04-RC.1/etc with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-BETA.1/var with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.0/data with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-BETA.1/opt with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.1/var/ca-certificates with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.0 with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.1/var with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.0/root with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04-RC.1/var/lib with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04-RC.1/usr with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.1 with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2.6/root with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/.system with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-RC.1/var/log with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.0/conf with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.0/var/log with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2/usr with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.1/var/lib/incus with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.1/usr with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-RC.1/var/lib/incus with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.0/var/lib/incus with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-RC.1/opt with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2/var/log/journal with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.1/home with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-RC.1 with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-BETA.1/var/log/journal with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-BETA.1/home with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10.0/var/lib/incus with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2.6/var/log with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04-RC.1/data with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.0/var/lib with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04-RC.1/var/ca-certificates with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.1/root with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2 with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04-RC.1 with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-RC.1/usr with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/grub with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10.0/root with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2.6/var/lib with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10.0/var/lib with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-RC.1/var/lib with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2.6/var/ca-certificates with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/.system/configs-ae32c386e13840b2bf9c0083275e7941 with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10.3.1/var/log/journal with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10.0/etc with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-BETA.1/etc with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.2.6/var/lib/incus with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10.3.1/home with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04-RC.1/audit with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.10-RC.1/etc with properties: {“refreservation”:“none”}
2026-05-11.12:45:40 tn-mw: zfs update boot-pool/ROOT/25.04.0/mnt with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/25.10.0/var/log/journal with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/25.10.0/usr with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/25.10-BETA.1/data with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/25.04.0/var with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/25.10.3.1/var/lib/incus with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/25.10.0/var/log with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/25.10.0/mnt with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/25.10-BETA.1/mnt with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/25.10-BETA.1/var/log with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/25.04.1/var/log with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/25.10.3.1/etc with properties: {“refreservation”:“none”}
2026-05-11.12:45:41 tn-mw: zfs update boot-pool/ROOT/26.0.0-BETA.1/audit with user properties: {“org.freenas:refquota_warning”:“75”,“org.freenas:refquota_critical”:“95”}
2026-05-12.12:38:11 zpool import -N -f boot-pool
2026-05-13.19:04:57 zpool import -N -f boot-pool
2026-05-14.10:19:39 zpool import -N -f boot-pool
2026-05-14.10:29:58 zpool import -N -f boot-pool
2026-05-15.12:06:10 zpool import -N -f boot-pool
2026-05-16.08:19:45 zpool import -N -f boot-pool
2026-05-16.08:57:32 zpool import -N -f boot-pool
2026-05-16.12:31:22 zpool import -N -f boot-pool
2026-05-17.12:40:32 zpool import -N -f boot-pool
2026-05-18.09:31:03 zpool import -N -f boot-pool
2026-05-19.16:33:24 zpool import -N -f boot-pool
2026-05-20.09:50:35 zpool import -N -f boot-pool
2026-05-21.09:44:37 zpool import -N -f boot-pool
2026-05-22.10:02:05 zpool import -N -f boot-pool
2026-05-23.08:44:16 zpool import -N -f boot-pool
2026-05-23.13:05:03 zpool import -N -f boot-pool
2026-05-24.08:17:02 zpool import -N -f boot-pool
2026-05-24.12:58:23 zpool import -N -f boot-pool

12

I recommend not skipping ahead because it makes it hard to troubleshoot or help.

Not only did no one say how to run the command, but you’re not even using preformatted text or hiding it with [details][/details]. I trust you understand why this makes this thread hard to follow?

List all created datasets in the pool Tank.

zpool history Tank | grep create | grep -v @

List all destroyed datasets in the pool Tank.

zpool history Tank | grep destroy | grep -v @

List all renamed datasets in the pool Tank.

zpool history Tank | grep rename | grep -v @

Please make sure to use preformatted text and put [details] above the text and [/details] below the text, so that it will collapse everything.

List all cloned datasets.

zfs list -r -t fs,vol -o name,origin Tank | awk '$2 ~ /@/'

If this doesn’t lead us to what the issue is, I have one more idea. If my last idea doesn’t help, then we’ll need to call in the big guns because this is not making any sense.

1 Like
13

Hi @winnielinnie ,I really appreciate the effort you’ve put into helping with this issue, but I think it’s getting too complicated. I’m just going to create a new dataset, transfer everything over, and then delete the old one.You guys rock! I mean, all of you are full of zest for ZFS, and it’s great to know I’m part of this community. Thanks again

@winnielinnie I really appreciate the help. Cheers,
Emmany