Data VDEV is bigger than drive size?

[Haldi]

Yesterday i created some child Datasets because i want to backup certain folders and thought snapshot would make that easier…
So from Dataset /SSD i created a /SSD/Fotos and /SSD/Fotos/FerienFotos and /SSD/Musik.
All those were normal folders before. I simplay renamed the original folders, created a Dataset, and moved all content inside over.

And Today i got greeted by a “80% Used” warning.

You can see the 1.86TB Capacity of the 2TB drive?
How comes that is now seen as "usable Capacity 2.64TB and 2.13TB Used?

Is it because the /SSD is shown as 1.84TB ?

Shouldn’t have anything to do with Snapshots right?

Because i set “Recursive” Child Datasets should be included.

.

Anyone has an idea what i borked up, and a easy way to fix it?

Edit: here in CMD:

root@NAS[~]# zpool list -v
NAME                                       SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
HDD                                       21.8T  9.90T  11.9T        -         -     0%    45%  1.00x    ONLINE  /mnt
  raidz1-0                                21.8T  9.90T  11.9T        -         -     0%  45.4%      -    ONLINE
    feea6654-9a6e-443f-acc0-b11b462a9624  5.46T      -      -        -         -      -      -      -    ONLINE
    72fe2c50-bbc5-4e5d-9670-f0d165fdd940  5.46T      -      -        -         -      -      -      -    ONLINE
    2cec057d-a57c-4285-bc57-32156e1e6ae0  5.46T      -      -        -         -      -      -      -    ONLINE
    de0f3b0f-36aa-4754-a883-f806565c35a2  5.46T      -      -        -         -      -      -      -    ONLINE
NVME                                      1.86T  1.20T   671G        -         -    12%    64%  1.00x    ONLINE  /mnt
  df0be91b-3660-4ed9-8f4c-3c1ae6303df4    1.86T  1.20T   671G        -         -    12%  64.8%      -    ONLINE
boot-pool                                  220G  13.0G   207G        -         -     5%     5%  1.00x    ONLINE  -
  sdd3                                     222G  13.0G   207G        -         -     5%  5.90%      -    ONLINE
root@NAS[~]#
root@NAS[~]#
root@NAS[~]# zfs list
NAME                                                                                            USED  AVAIL  REFER  MOUNTPOINT
HDD                                                                                            7.23T  8.51T   151K  /mnt/HDD
HDD/BackupSSD                                                                                  2.85T  8.51T   209K  /mnt/HDD/BackupSSD
HDD/BackupSSD/SSD                                                                              1.84T  8.51T  25.6G  /mnt/HDD/BackupSSD/SSD
HDD/BackupSSD/SSD/Fotos                                                                         621G  8.51T   541G  /mnt/HDD/BackupSSD/SSD/Fotos
HDD/BackupSSD/SSD/Fotos/FerienFotos                                                            79.7G  8.51T  79.7G  /mnt/HDD/BackupSSD/SSD/Fotos/FerienFotos
HDD/BackupSSD/SSD/Musik                                                                         242G  8.51T   242G  /mnt/HDD/BackupSSD/SSD/Musik
HDD/BackupSSD/VM                                                                               55.5G  8.51T   140K  /mnt/HDD/BackupSSD/VM
HDD/BackupSSD/VM/HAOS                                                                          55.5G  8.54T  14.5G  -
HDD/BackupSSD/docker                                                                           22.1G  8.51T  8.13G  /mnt/HDD/BackupSSD/docker
HDD/BackupSSD/web                                                                              20.2M  8.51T  6.68M  /mnt/HDD/BackupSSD/web
HDD/PUBLIC                                                                                     4.38T  8.51T  4.38T  /mnt/HDD/PUBLIC
NVME                                                                                           2.13T   525G   120K  /mnt/NVME
NVME/.system                                                                                   1.55G   525G  1.23G  legacy
NVME/.system/configs-ae32c386e13840b2bf9c0083275e7941                                          6.12M   525G  6.12M  legacy
NVME/.system/cores                                                                              120K  1024M   120K  legacy
NVME/.system/ctdb_shared_vol                                                                     96K   525G    96K  legacy
NVME/.system/glusterd                                                                           104K   525G   104K  legacy
NVME/.system/netdata-ae32c386e13840b2bf9c0083275e7941                                           322M   525G   322M  legacy
NVME/.system/rrd-ae32c386e13840b2bf9c0083275e7941                                                96K   525G    96K  legacy
NVME/.system/samba4                                                                            1.14M   525G   360K  legacy
NVME/.system/services                                                                            96K   525G    96K  legacy
NVME/.system/webui                                                                               96K   525G    96K  legacy
NVME/Home                                                                                       148K   525G   148K  /mnt/NVME/Home
NVME/SSD                                                                                       1.84T   525G  19.6G  /mnt/NVME/SSD
NVME/SSD/Fotos                                                                                  621G   525G   542G  /mnt/NVME/SSD/Fotos
NVME/SSD/Fotos/FerienFotos                                                                     79.7G   525G  79.7G  /mnt/NVME/SSD/Fotos/FerienFotos
NVME/SSD/Musik                                                                                  243G   525G   243G  /mnt/NVME/SSD/Musik
NVME/VM                                                                                         124G   525G    96K  /mnt/NVME/VM
NVME/VM/HAOS                                                                                   52.9G   559G  12.9G  -
NVME/VM/Proxmox-wsvy5n                                                                         10.2G   534G  1.64G  -
NVME/VM/Windows-whrh7b                                                                         60.9G   569G  16.9G  -
NVME/docker                                                                                    22.1G   525G  8.24G  /mnt/NVME/docker
NVME/ix-applications                                                                            143G   525G  11.2M  /mnt/NVME/ix-applications
NVME/ix-applications/catalogs                                                                  1.09G   525G  1.09G  /mnt/NVME/ix-applications/catalogs
NVME/ix-applications/default_volumes                                                            154M   525G    96K  /mnt/NVME/ix-applications/default_volumes
NVME/ix-applications/default_volumes/pvc-9c79978a-26f8-4cc9-9d6a-667cc4e95400                   154M   256G  83.8M  legacy
NVME/ix-applications/k3s                                                                        116G   525G   115G  /mnt/NVME/ix-applications/k3s
NVME/ix-applications/k3s/kubelet                                                               9.77M   525G  9.77M  legacy
NVME/ix-applications/releases                                                                  26.3G   525G   152K  /mnt/NVME/ix-applications/releases
NVME/ix-applications/releases/audiobookshelf                                                   15.3M   525G    96K  /mnt/NVME/ix-applications/releases/audiobookshelf
NVME/ix-applications/releases/audiobookshelf/charts                                            2.41M   525G  2.26M  /mnt/NVME/ix-applications/releases/audiobookshelf/charts
NVME/ix-applications/releases/audiobookshelf/volumes                                           12.8M   525G    96K  /mnt/NVME/ix-applications/releases/audiobookshelf/volumes
NVME/ix-applications/releases/audiobookshelf/volumes/ix_volumes                                  96K   525G    96K  /mnt/NVME/ix-applications/releases/audiobookshelf/volumes/ix_volumes
NVME/ix-applications/releases/audiobookshelf/volumes/pvc-67af6621-47ac-4551-9577-2ad2df01fdf1  1.28M   256G   568K  legacy
NVME/ix-applications/releases/audiobookshelf/volumes/pvc-a484690c-6d7a-42f8-a656-601711d21591  11.4M   256G  1.85M  legacy
NVME/ix-applications/releases/batteryhistorian                                                  576K   525G   104K  /mnt/NVME/ix-applications/releases/batteryhistorian
NVME/ix-applications/releases/batteryhistorian/charts                                           280K   525G   280K  /mnt/NVME/ix-applications/releases/batteryhistorian/charts
NVME/ix-applications/releases/batteryhistorian/volumes                                          192K   525G    96K  /mnt/NVME/ix-applications/releases/batteryhistorian/volumes
NVME/ix-applications/releases/batteryhistorian/volumes/ix_volumes                                96K   525G    96K  /mnt/NVME/ix-applications/releases/batteryhistorian/volumes/ix_volumes
NVME/ix-applications/releases/filebrowser                                                      1.40M   525G   104K  /mnt/NVME/ix-applications/releases/filebrowser
NVME/ix-applications/releases/filebrowser/charts                                                728K   525G   600K  /mnt/NVME/ix-applications/releases/filebrowser/charts
NVME/ix-applications/releases/filebrowser/volumes                                               600K   525G    96K  /mnt/NVME/ix-applications/releases/filebrowser/volumes
NVME/ix-applications/releases/filebrowser/volumes/ix_volumes                                    504K   525G    96K  /mnt/NVME/ix-applications/releases/filebrowser/volumes/ix_volumes
NVME/ix-applications/releases/filebrowser/volumes/ix_volumes/config                             408K   525G   108K  /mnt/NVME/ix-applications/releases/filebrowser/volumes/ix_volumes/config
NVME/ix-applications/releases/homarr                                                           1.13M   525G    96K  /mnt/NVME/ix-applications/releases/homarr
NVME/ix-applications/releases/homarr/charts                                                     504K   525G   440K  /mnt/NVME/ix-applications/releases/homarr/charts
NVME/ix-applications/releases/homarr/volumes                                                    560K   525G    96K  /mnt/NVME/ix-applications/releases/homarr/volumes
NVME/ix-applications/releases/homarr/volumes/ix_volumes                                         464K   525G   112K  /mnt/NVME/ix-applications/releases/homarr/volumes/ix_volumes
NVME/ix-applications/releases/homarr/volumes/ix_volumes/configs                                  96K   525G    96K  /mnt/NVME/ix-applications/releases/homarr/volumes/ix_volumes/configs
NVME/ix-applications/releases/homarr/volumes/ix_volumes/data                                     96K   525G    96K  /mnt/NVME/ix-applications/releases/homarr/volumes/ix_volumes/data
NVME/ix-applications/releases/homarr/volumes/ix_volumes/icons                                    96K   525G    96K  /mnt/NVME/ix-applications/releases/homarr/volumes/ix_volumes/icons
NVME/ix-applications/releases/immich                                                           24.3G   525G    96K  /mnt/NVME/ix-applications/releases/immich
NVME/ix-applications/releases/immich/charts                                                    1.92M   525G  1.76M  /mnt/NVME/ix-applications/releases/immich/charts
NVME/ix-applications/releases/immich/volumes                                                   24.3G   525G    96K  /mnt/NVME/ix-applications/releases/immich/volumes
NVME/ix-applications/releases/immich/volumes/ix_volumes                                        24.3G   525G   120K  /mnt/NVME/ix-applications/releases/immich/volumes/ix_volumes
NVME/ix-applications/releases/immich/volumes/ix_volumes/library                                  96K   525G    96K  /mnt/NVME/ix-applications/releases/immich/volumes/ix_volumes/library
NVME/ix-applications/releases/immich/volumes/ix_volumes/pgBackup                               1.02G   525G  1.02G  /mnt/NVME/ix-applications/releases/immich/volumes/ix_volumes/pgBackup
NVME/ix-applications/releases/immich/volumes/ix_volumes/pgData                                 2.89G   525G   705M  /mnt/NVME/ix-applications/releases/immich/volumes/ix_volumes/pgData
NVME/ix-applications/releases/immich/volumes/ix_volumes/profile                                  96K   525G    96K  /mnt/NVME/ix-applications/releases/immich/volumes/ix_volumes/profile
NVME/ix-applications/releases/immich/volumes/ix_volumes/thumbs                                 20.4G   525G  20.4G  /mnt/NVME/ix-applications/releases/immich/volumes/ix_volumes/thumbs
NVME/ix-applications/releases/immich/volumes/ix_volumes/uploads                                  96K   525G    96K  /mnt/NVME/ix-applications/releases/immich/volumes/ix_volumes/uploads
NVME/ix-applications/releases/immich/volumes/ix_volumes/video                                   432K   525G   192K  /mnt/NVME/ix-applications/releases/immich/volumes/ix_volumes/video
NVME/ix-applications/releases/jackett                                                          2.96M   525G    96K  /mnt/NVME/ix-applications/releases/jackett
NVME/ix-applications/releases/jackett/charts                                                   2.68M   525G  2.50M  /mnt/NVME/ix-applications/releases/jackett/charts
NVME/ix-applications/releases/jackett/volumes                                                   192K   525G    96K  /mnt/NVME/ix-applications/releases/jackett/volumes
NVME/ix-applications/releases/jackett/volumes/ix_volumes                                         96K   525G    96K  /mnt/NVME/ix-applications/releases/jackett/volumes/ix_volumes
NVME/ix-applications/releases/jdownloader2                                                     1.38M   525G    96K  /mnt/NVME/ix-applications/releases/jdownloader2
NVME/ix-applications/releases/jdownloader2/charts                                              1.09M   525G  1.02M  /mnt/NVME/ix-applications/releases/jdownloader2/charts
NVME/ix-applications/releases/jdownloader2/volumes                                              192K   525G    96K  /mnt/NVME/ix-applications/releases/jdownloader2/volumes
NVME/ix-applications/releases/jdownloader2/volumes/ix_volumes                                    96K   525G    96K  /mnt/NVME/ix-applications/releases/jdownloader2/volumes/ix_volumes
NVME/ix-applications/releases/jellyfin                                                          255M   525G   104K  /mnt/NVME/ix-applications/releases/jellyfin
NVME/ix-applications/releases/jellyfin/charts                                                   432K   525G   368K  /mnt/NVME/ix-applications/releases/jellyfin/charts
NVME/ix-applications/releases/jellyfin/volumes                                                  254M   525G    96K  /mnt/NVME/ix-applications/releases/jellyfin/volumes
NVME/ix-applications/releases/jellyfin/volumes/ix_volumes                                       254M   525G   104K  /mnt/NVME/ix-applications/releases/jellyfin/volumes/ix_volumes
NVME/ix-applications/releases/jellyfin/volumes/ix_volumes/cache                                 253M   525G  28.8M  /mnt/NVME/ix-applications/releases/jellyfin/volumes/ix_volumes/cache
NVME/ix-applications/releases/jellyfin/volumes/ix_volumes/config                                 96K   525G    96K  /mnt/NVME/ix-applications/releases/jellyfin/volumes/ix_volumes/config
NVME/ix-applications/releases/jellyfin/volumes/ix_volumes/transcodes                            640K   525G   368K  /mnt/NVME/ix-applications/releases/jellyfin/volumes/ix_volumes/transcodes
NVME/ix-applications/releases/logitech-media-server                                            3.18M   525G   104K  /mnt/NVME/ix-applications/releases/logitech-media-server
NVME/ix-applications/releases/logitech-media-server/charts                                     2.89M   525G  2.69M  /mnt/NVME/ix-applications/releases/logitech-media-server/charts
NVME/ix-applications/releases/logitech-media-server/volumes                                     192K   525G    96K  /mnt/NVME/ix-applications/releases/logitech-media-server/volumes
NVME/ix-applications/releases/logitech-media-server/volumes/ix_volumes                           96K   525G    96K  /mnt/NVME/ix-applications/releases/logitech-media-server/volumes/ix_volumes
NVME/ix-applications/releases/netdata                                                          1.59G   525G   104K  /mnt/NVME/ix-applications/releases/netdata
NVME/ix-applications/releases/netdata/charts                                                    628K   525G   556K  /mnt/NVME/ix-applications/releases/netdata/charts
NVME/ix-applications/releases/netdata/volumes                                                  1.59G   525G    96K  /mnt/NVME/ix-applications/releases/netdata/volumes
NVME/ix-applications/releases/netdata/volumes/ix_volumes                                       1.59G   525G   104K  /mnt/NVME/ix-applications/releases/netdata/volumes/ix_volumes
NVME/ix-applications/releases/netdata/volumes/ix_volumes/cache                                 1.58G   525G   316M  /mnt/NVME/ix-applications/releases/netdata/volumes/ix_volumes/cache
NVME/ix-applications/releases/netdata/volumes/ix_volumes/config                                 456K   525G   136K  /mnt/NVME/ix-applications/releases/netdata/volumes/ix_volumes/config
NVME/ix-applications/releases/netdata/volumes/ix_volumes/lib                                    592K   525G   120K  /mnt/NVME/ix-applications/releases/netdata/volumes/ix_volumes/lib
NVME/ix-applications/releases/nextcloud                                                        29.7M   525G   104K  /mnt/NVME/ix-applications/releases/nextcloud
NVME/ix-applications/releases/nextcloud/charts                                                  944K   525G   808K  /mnt/NVME/ix-applications/releases/nextcloud/charts
NVME/ix-applications/releases/nextcloud/volumes                                                28.6M   525G    96K  /mnt/NVME/ix-applications/releases/nextcloud/volumes
NVME/ix-applications/releases/nextcloud/volumes/ix_volumes                                     28.5M   525G   104K  /mnt/NVME/ix-applications/releases/nextcloud/volumes/ix_volumes
NVME/ix-applications/releases/nextcloud/volumes/ix_volumes/ix-postgres_backups                  944K   525G   688K  /mnt/NVME/ix-applications/releases/nextcloud/volumes/ix_volumes/ix-postgres_backups
NVME/ix-applications/releases/nextcloud/volumes/ix_volumes/ix-postgres_data                    27.5M   525G  19.7M  /mnt/NVME/ix-applications/releases/nextcloud/volumes/ix_volumes/ix-postgres_data
NVME/ix-applications/releases/openspeedtest                                                    3.00M   525G   104K  /mnt/NVME/ix-applications/releases/openspeedtest
NVME/ix-applications/releases/openspeedtest/charts                                             2.71M   525G  2.53M  /mnt/NVME/ix-applications/releases/openspeedtest/charts
NVME/ix-applications/releases/openspeedtest/volumes                                             192K   525G    96K  /mnt/NVME/ix-applications/releases/openspeedtest/volumes
NVME/ix-applications/releases/openspeedtest/volumes/ix_volumes                                   96K   525G    96K  /mnt/NVME/ix-applications/releases/openspeedtest/volumes/ix_volumes
NVME/ix-applications/releases/paperless-ngx                                                    30.2M   525G   104K  /mnt/NVME/ix-applications/releases/paperless-ngx
NVME/ix-applications/releases/paperless-ngx/charts                                              256K   525G   256K  /mnt/NVME/ix-applications/releases/paperless-ngx/charts
NVME/ix-applications/releases/paperless-ngx/volumes                                            29.8M   525G    96K  /mnt/NVME/ix-applications/releases/paperless-ngx/volumes
NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes                                 29.7M   525G   112K  /mnt/NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes
NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes/pgBackup                          96K   525G    96K  /mnt/NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes/pgBackup
NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes/pgData                          29.4M   525G  19.6M  /mnt/NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes/pgData
NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes/trash                            160K   525G    96K  /mnt/NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes/trash
NVME/ix-applications/releases/pdf                                                               576K   525G   104K  /mnt/NVME/ix-applications/releases/pdf
NVME/ix-applications/releases/pdf/charts                                                        280K   525G   280K  /mnt/NVME/ix-applications/releases/pdf/charts
NVME/ix-applications/releases/pdf/volumes                                                       192K   525G    96K  /mnt/NVME/ix-applications/releases/pdf/volumes
NVME/ix-applications/releases/pdf/volumes/ix_volumes                                             96K   525G    96K  /mnt/NVME/ix-applications/releases/pdf/volumes/ix_volumes
NVME/ix-applications/releases/qbittorrent                                                      1008K   525G   104K  /mnt/NVME/ix-applications/releases/qbittorrent
NVME/ix-applications/releases/qbittorrent/charts                                                448K   525G   384K  /mnt/NVME/ix-applications/releases/qbittorrent/charts
NVME/ix-applications/releases/qbittorrent/volumes                                               456K   525G    96K  /mnt/NVME/ix-applications/releases/qbittorrent/volumes
NVME/ix-applications/releases/qbittorrent/volumes/ix_volumes                                    360K   525G   104K  /mnt/NVME/ix-applications/releases/qbittorrent/volumes/ix_volumes
NVME/ix-applications/releases/qbittorrent/volumes/ix_volumes/config                              96K   525G    96K  /mnt/NVME/ix-applications/releases/qbittorrent/volumes/ix_volumes/config
NVME/ix-applications/releases/qbittorrent/volumes/ix_volumes/downloads                           96K   525G    96K  /mnt/NVME/ix-applications/releases/qbittorrent/volumes/ix_volumes/downloads
NVME/ix-applications/releases/teamspeak3                                                       37.3M   525G   104K  /mnt/NVME/ix-applications/releases/teamspeak3
NVME/ix-applications/releases/teamspeak3/charts                                                2.64M   525G  2.46M  /mnt/NVME/ix-applications/releases/teamspeak3/charts
NVME/ix-applications/releases/teamspeak3/volumes                                               34.5M   525G    96K  /mnt/NVME/ix-applications/releases/teamspeak3/volumes
NVME/ix-applications/releases/teamspeak3/volumes/ix_volumes                                      96K   525G    96K  /mnt/NVME/ix-applications/releases/teamspeak3/volumes/ix_volumes
NVME/ix-applications/releases/teamspeak3/volumes/pvc-64c39469-90dd-4998-89da-e61434d1cc59       940K   256G   100K  legacy
NVME/ix-applications/releases/teamspeak3/volumes/pvc-9dcd21a4-c415-4c6e-95f2-b6afd0f775a2      33.4M   256G  2.86M  legacy
NVME/ix-applications/releases/webdav                                                           1.14M   525G    96K  /mnt/NVME/ix-applications/releases/webdav
NVME/ix-applications/releases/webdav/charts                                                     876K   525G   804K  /mnt/NVME/ix-applications/releases/webdav/charts
NVME/ix-applications/releases/webdav/volumes                                                    192K   525G    96K  /mnt/NVME/ix-applications/releases/webdav/volumes
NVME/ix-applications/releases/webdav/volumes/ix_volumes                                          96K   525G    96K  /mnt/NVME/ix-applications/releases/webdav/volumes/ix_volumes
NVME/jailmaker                                                                                 6.81G   525G  84.2M  /mnt/NVME/jailmaker
NVME/jailmaker/jails                                                                           6.73G   525G   112K  /mnt/NVME/jailmaker/jails
NVME/jailmaker/jails/HAOS                                                                      5.33G   525G  5.33G  /mnt/NVME/jailmaker/jails/HAOS
NVME/jailmaker/jails/aroz                                                                      1.40G   525G  1.40G  /mnt/NVME/jailmaker/jails/aroz
NVME/web                                                                                       19.9M   525G  6.63M  /mnt/NVME/web

Edit:
Found this command:

root@NAS[~]# zfs list -o space
NAME                                                                                           AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
HDD                                                                                            8.51T  7.23T        0B    151K             0B      7.23T
HDD/BackupSSD                                                                                  8.51T  2.85T      957G    209K             0B      1.91T
HDD/BackupSSD/SSD                                                                              8.51T  1.84T      990G   25.6G             0B       863G
HDD/BackupSSD/SSD/Fotos                                                                        8.51T   621G        0B    541G             0B      79.7G
HDD/BackupSSD/SSD/Fotos/FerienFotos                                                            8.51T  79.7G        0B   79.7G             0B         0B
HDD/BackupSSD/SSD/Musik                                                                        8.51T   242G        0B    242G             0B         0B
HDD/BackupSSD/VM                                                                               8.51T  55.5G        0B    140K             0B      55.5G
HDD/BackupSSD/VM/HAOS                                                                          8.54T  55.5G     5.51G   14.5G          35.5G         0B
HDD/BackupSSD/docker                                                                           8.51T  22.1G     13.9G   8.13G             0B         0B
HDD/BackupSSD/web                                                                              8.51T  20.2M     13.5M   6.68M             0B         0B
HDD/PUBLIC                                                                                     8.51T  4.38T        0B   4.38T             0B         0B
NVME                                                                                            525G  2.13T        0B    120K             0B      2.13T
NVME/.system                                                                                    525G  1.55G        0B   1.23G             0B       330M
NVME/.system/configs-ae32c386e13840b2bf9c0083275e7941                                           525G  6.12M        0B   6.12M             0B         0B
NVME/.system/cores                                                                             1024M   120K        0B    120K             0B         0B
NVME/.system/ctdb_shared_vol                                                                    525G    96K        0B     96K             0B         0B
NVME/.system/glusterd                                                                           525G   104K        0B    104K             0B         0B
NVME/.system/netdata-ae32c386e13840b2bf9c0083275e7941                                           525G   322M        0B    322M             0B         0B
NVME/.system/rrd-ae32c386e13840b2bf9c0083275e7941                                               525G    96K        0B     96K             0B         0B
NVME/.system/samba4                                                                             525G  1.14M      804K    360K             0B         0B
NVME/.system/services                                                                           525G    96K        0B     96K             0B         0B
NVME/.system/webui                                                                              525G    96K        0B     96K             0B         0B
NVME/Home                                                                                       525G   148K        0B    148K             0B         0B
NVME/SSD                                                                                        525G  1.84T      997G   19.6G             0B       864G
NVME/SSD/Fotos                                                                                  525G   621G      128K    542G             0B      79.7G
NVME/SSD/Fotos/FerienFotos                                                                      525G  79.7G     7.83M   79.7G             0B         0B
NVME/SSD/Musik                                                                                  525G   243G        0B    243G             0B         0B
NVME/VM                                                                                         525G   124G        0B     96K             0B       124G
NVME/VM/HAOS                                                                                    559G  52.9G     5.84G   12.9G          34.2G         0B
NVME/VM/Proxmox-wsvy5n                                                                          534G  10.2G        0B   1.64G          8.52G         0B
NVME/VM/Windows-whrh7b                                                                          569G  60.9G        0B   16.9G          44.1G         0B
NVME/docker                                                                                     525G  22.1G     13.8G   8.24G             0B         0B
NVME/ix-applications                                                                            525G   143G      296K   11.2M             0B       143G
NVME/ix-applications/catalogs                                                                   525G  1.09G        0B   1.09G             0B         0B
NVME/ix-applications/default_volumes                                                            525G   154M        0B     96K             0B       154M
NVME/ix-applications/default_volumes/pvc-9c79978a-26f8-4cc9-9d6a-667cc4e95400                   256G   154M     70.3M   83.8M             0B         0B
NVME/ix-applications/k3s                                                                        525G   116G      855M    115G             0B      9.78M
NVME/ix-applications/k3s/kubelet                                                                525G  9.78M        0B   9.78M             0B         0B
NVME/ix-applications/releases                                                                   525G  26.3G      160K    152K             0B      26.3G
NVME/ix-applications/releases/audiobookshelf                                                    525G  15.3M        0B     96K             0B      15.2M
NVME/ix-applications/releases/audiobookshelf/charts                                             525G  2.41M      160K   2.26M             0B         0B
NVME/ix-applications/releases/audiobookshelf/volumes                                            525G  12.8M        0B     96K             0B      12.7M
NVME/ix-applications/releases/audiobookshelf/volumes/ix_volumes                                 525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/audiobookshelf/volumes/pvc-67af6621-47ac-4551-9577-2ad2df01fdf1   256G  1.28M      740K    568K             0B         0B
NVME/ix-applications/releases/audiobookshelf/volumes/pvc-a484690c-6d7a-42f8-a656-601711d21591   256G  11.4M     9.52M   1.85M             0B         0B
NVME/ix-applications/releases/batteryhistorian                                                  525G   576K        0B    104K             0B       472K
NVME/ix-applications/releases/batteryhistorian/charts                                           525G   280K        0B    280K             0B         0B
NVME/ix-applications/releases/batteryhistorian/volumes                                          525G   192K        0B     96K             0B        96K
NVME/ix-applications/releases/batteryhistorian/volumes/ix_volumes                               525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/filebrowser                                                       525G  1.40M        0B    104K             0B      1.30M
NVME/ix-applications/releases/filebrowser/charts                                                525G   728K      128K    600K             0B         0B
NVME/ix-applications/releases/filebrowser/volumes                                               525G   600K        0B     96K             0B       504K
NVME/ix-applications/releases/filebrowser/volumes/ix_volumes                                    525G   504K        0B     96K             0B       408K
NVME/ix-applications/releases/filebrowser/volumes/ix_volumes/config                             525G   408K      300K    108K             0B         0B
NVME/ix-applications/releases/homarr                                                            525G  1.13M        0B     96K             0B      1.04M
NVME/ix-applications/releases/homarr/charts                                                     525G   504K       64K    440K             0B         0B
NVME/ix-applications/releases/homarr/volumes                                                    525G   560K        0B     96K             0B       464K
NVME/ix-applications/releases/homarr/volumes/ix_volumes                                         525G   464K       64K    112K             0B       288K
NVME/ix-applications/releases/homarr/volumes/ix_volumes/configs                                 525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/homarr/volumes/ix_volumes/data                                    525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/homarr/volumes/ix_volumes/icons                                   525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/immich                                                            525G  24.3G        0B     96K             0B      24.3G
NVME/ix-applications/releases/immich/charts                                                     525G  1.92M      168K   1.76M             0B         0B
NVME/ix-applications/releases/immich/volumes                                                    525G  24.3G        0B     96K             0B      24.3G
NVME/ix-applications/releases/immich/volumes/ix_volumes                                         525G  24.3G        0B    120K             0B      24.3G
NVME/ix-applications/releases/immich/volumes/ix_volumes/library                                 525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/immich/volumes/ix_volumes/pgBackup                                525G  1.02G      480K   1.02G             0B         0B
NVME/ix-applications/releases/immich/volumes/ix_volumes/pgData                                  525G  2.89G     2.20G    705M             0B         0B
NVME/ix-applications/releases/immich/volumes/ix_volumes/profile                                 525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/immich/volumes/ix_volumes/thumbs                                  525G  20.4G     9.23M   20.4G             0B         0B
NVME/ix-applications/releases/immich/volumes/ix_volumes/uploads                                 525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/immich/volumes/ix_volumes/video                                   525G   432K      240K    192K             0B         0B
NVME/ix-applications/releases/jackett                                                           525G  2.96M        0B     96K             0B      2.87M
NVME/ix-applications/releases/jackett/charts                                                    525G  2.68M      184K   2.50M             0B         0B
NVME/ix-applications/releases/jackett/volumes                                                   525G   192K        0B     96K             0B        96K
NVME/ix-applications/releases/jackett/volumes/ix_volumes                                        525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/jdownloader2                                                      525G  1.38M        0B     96K             0B      1.28M
NVME/ix-applications/releases/jdownloader2/charts                                               525G  1.09M       80K   1.02M             0B         0B
NVME/ix-applications/releases/jdownloader2/volumes                                              525G   192K        0B     96K             0B        96K
NVME/ix-applications/releases/jdownloader2/volumes/ix_volumes                                   525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/jellyfin                                                          525G   255M        0B    104K             0B       255M
NVME/ix-applications/releases/jellyfin/charts                                                   525G   432K       64K    368K             0B         0B
NVME/ix-applications/releases/jellyfin/volumes                                                  525G   254M        0B     96K             0B       254M
NVME/ix-applications/releases/jellyfin/volumes/ix_volumes                                       525G   254M       64K    104K             0B       254M
NVME/ix-applications/releases/jellyfin/volumes/ix_volumes/cache                                 525G   253M      224M   28.8M             0B         0B
NVME/ix-applications/releases/jellyfin/volumes/ix_volumes/config                                525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/jellyfin/volumes/ix_volumes/transcodes                            525G   640K      272K    368K             0B         0B
NVME/ix-applications/releases/logitech-media-server                                             525G  3.18M        0B    104K             0B      3.07M
NVME/ix-applications/releases/logitech-media-server/charts                                      525G  2.89M      200K   2.69M             0B         0B
NVME/ix-applications/releases/logitech-media-server/volumes                                     525G   192K        0B     96K             0B        96K
NVME/ix-applications/releases/logitech-media-server/volumes/ix_volumes                          525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/netdata                                                           525G  1.58G        0B    104K             0B      1.58G
NVME/ix-applications/releases/netdata/charts                                                    525G   628K       72K    556K             0B         0B
NVME/ix-applications/releases/netdata/volumes                                                   525G  1.58G        0B     96K             0B      1.58G
NVME/ix-applications/releases/netdata/volumes/ix_volumes                                        525G  1.58G        0B    104K             0B      1.58G
NVME/ix-applications/releases/netdata/volumes/ix_volumes/cache                                  525G  1.58G     1.28G    312M             0B         0B
NVME/ix-applications/releases/netdata/volumes/ix_volumes/config                                 525G   456K      320K    136K             0B         0B
NVME/ix-applications/releases/netdata/volumes/ix_volumes/lib                                    525G   592K      472K    120K             0B         0B
NVME/ix-applications/releases/nextcloud                                                         525G  29.6M        0B    104K             0B      29.5M
NVME/ix-applications/releases/nextcloud/charts                                                  525G   944K      136K    808K             0B         0B
NVME/ix-applications/releases/nextcloud/volumes                                                 525G  28.6M        0B     96K             0B      28.5M
NVME/ix-applications/releases/nextcloud/volumes/ix_volumes                                      525G  28.5M        0B    104K             0B      28.4M
NVME/ix-applications/releases/nextcloud/volumes/ix_volumes/ix-postgres_backups                  525G   944K      256K    688K             0B         0B
NVME/ix-applications/releases/nextcloud/volumes/ix_volumes/ix-postgres_data                     525G  27.5M     7.80M   19.7M             0B         0B
NVME/ix-applications/releases/openspeedtest                                                     525G  3.00M        0B    104K             0B      2.89M
NVME/ix-applications/releases/openspeedtest/charts                                              525G  2.71M      184K   2.53M             0B         0B
NVME/ix-applications/releases/openspeedtest/volumes                                             525G   192K        0B     96K             0B        96K
NVME/ix-applications/releases/openspeedtest/volumes/ix_volumes                                  525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/paperless-ngx                                                     525G  30.2M        0B    104K             0B      30.1M
NVME/ix-applications/releases/paperless-ngx/charts                                              525G   256K        0B    256K             0B         0B
NVME/ix-applications/releases/paperless-ngx/volumes                                             525G  29.8M        0B     96K             0B      29.7M
NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes                                  525G  29.7M        0B    112K             0B      29.6M
NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes/pgBackup                         525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes/pgData                           525G  29.3M     9.80M   19.6M             0B         0B
NVME/ix-applications/releases/paperless-ngx/volumes/ix_volumes/trash                            525G   160K       64K     96K             0B         0B
NVME/ix-applications/releases/pdf                                                               525G   576K        0B    104K             0B       472K
NVME/ix-applications/releases/pdf/charts                                                        525G   280K        0B    280K             0B         0B
NVME/ix-applications/releases/pdf/volumes                                                       525G   192K        0B     96K             0B        96K
NVME/ix-applications/releases/pdf/volumes/ix_volumes                                            525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/qbittorrent                                                       525G  1008K        0B    104K             0B       904K
NVME/ix-applications/releases/qbittorrent/charts                                                525G   448K       64K    384K             0B         0B
NVME/ix-applications/releases/qbittorrent/volumes                                               525G   456K        0B     96K             0B       360K
NVME/ix-applications/releases/qbittorrent/volumes/ix_volumes                                    525G   360K       64K    104K             0B       192K
NVME/ix-applications/releases/qbittorrent/volumes/ix_volumes/config                             525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/qbittorrent/volumes/ix_volumes/downloads                          525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/teamspeak3                                                        525G  37.3M        0B    104K             0B      37.2M
NVME/ix-applications/releases/teamspeak3/charts                                                 525G  2.64M      184K   2.46M             0B         0B
NVME/ix-applications/releases/teamspeak3/volumes                                                525G  34.5M        0B     96K             0B      34.5M
NVME/ix-applications/releases/teamspeak3/volumes/ix_volumes                                     525G    96K        0B     96K             0B         0B
NVME/ix-applications/releases/teamspeak3/volumes/pvc-64c39469-90dd-4998-89da-e61434d1cc59       256G   940K      840K    100K             0B         0B
NVME/ix-applications/releases/teamspeak3/volumes/pvc-9dcd21a4-c415-4c6e-95f2-b6afd0f775a2       256G  33.4M     30.6M   2.86M             0B         0B
NVME/ix-applications/releases/webdav                                                            525G  1.14M        0B     96K             0B      1.04M
NVME/ix-applications/releases/webdav/charts                                                     525G   876K       72K    804K             0B         0B
NVME/ix-applications/releases/webdav/volumes                                                    525G   192K        0B     96K             0B        96K
NVME/ix-applications/releases/webdav/volumes/ix_volumes                                         525G    96K        0B     96K             0B         0B
NVME/jailmaker                                                                                  525G  6.81G        0B   84.2M             0B      6.73G
NVME/jailmaker/jails                                                                            525G  6.73G        0B    112K             0B      6.73G
NVME/jailmaker/jails/HAOS                                                                       525G  5.33G        0B   5.33G             0B         0B
NVME/jailmaker/jails/aroz                                                                       525G  1.40G        0B   1.40G             0B         0B
NVME/web                                                                                        525G  19.9M     13.3M   6.63M             0B         0B

Does this line mean 997gb is snapshots and 864gb is child dataset and thats why it expects 1.84TB used Data?

NAME                                                                                           AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
NVME/SSD                                                                                        525G  1.84T      997G   19.6G             0B       864G
NVME/SSD/Fotos                                                                                  525G   621G      128K    542G             0B      79.7G
NVME/SSD/Fotos/FerienFotos                                                                      525G  79.7G     7.83M   79.7G             0B         0B
NVME/SSD/Musik                                                                                  525G   243G        0B    243G             0B         0B

If i would delete all Snapshots or if the change times out after 2 weeks it will automatically go back to correct values?

And if it is Snapshot related… why does it not show up in the web interface?

[winnielinnie]

@Haldi

And if it is Snapshot related… why does it not show up in the web interface?

How much space a snapshot “uses” is not straight forward.

The totality of your snapshots under NVME/SSD adds up to 1 TiB. This means that all snapshots combined are “holding onto” previously deleted data that adds up to 1 TiB.

Because blocks (of previously deleted data) can be referenced by multiple snapshots, it means that no single snapshot is consuming this space.

For example, if two snapshots are referencing a previously deleted 4 GiB .iso file, then checking how much each snapshot consumes will show you “0 bytes”. However, if you destroy one of these snapshots, then check again? It will reveal that the last remaining snapshot indeed consumes 4 GiB.

---

In order to truly know how much space will be liberated, you need to use the command-line, since no such feature exists in the TrueNAS GUI.

The syntax is this:

zfs destroy -nv pool/dataset@someoldsnap%somenewsnap

This will reveal how much space you will free up if you are to destroy all snapshots within this range.

DO NOT RUN IT AS ROOT OR WITH “SUDO”. Even though the -n flag is a “dry-run” that does not actually do anything, there’s the risk you might accidentally forget to use it. So if you run this as root or with “sudo”? You will destroy everything.

Here’s a more specific example:

zfs destroy -nv NVME/SSD@auto-2024-03-01_00-00%auto-2024-05-04_00-00

[Haldi]

@winnielinnie

How much space a snapshot “uses” is not straight forward.

Yeah… I was aware that ZFS does a lot of awesome things that go way over my head.
But then again I thought I understood it a bit better…

@winnielinnie

For example, if two snapshots are referencing a previously deleted 4 GiB .iso file, then checking how much each snapshot consumes will show you “0 bytes”. However, if you destroy one of these snapshots, then check again? It will reveal that the last remaining snapshot indeed consumes 4 GiB.

Uhmmm okay… Because reasons?

@winnielinnie

DO NOT RUN IT AS ROOT OR WITH “SUDO”.

Oh.
Is there a checkbox or command that I have to unlock to use the zfs command on a normal account?
Because I get the zsh: command not found: zfs error when I try without sudo.

Edit: AHH yes I see.

PATH to ZFS utilities broken in TrueNAS SCALE?

Because I use the account created during setup, which is not root anymore this is broken.

Edit2:

admin@NAS[~]$ /usr/sbin/zfs destroy -nv NVME/SSD@auto-2024-04-25_00-00%auto-2024-05-04_00-00
would destroy NVME/SSD@auto-2024-04-25_00-00
would destroy NVME/SSD@auto-2024-04-26_00-00
would destroy NVME/SSD@auto-2024-04-27_00-00
would destroy NVME/SSD@auto-2024-04-28_00-00
would destroy NVME/SSD@auto-2024-04-29_00-00
would destroy NVME/SSD@auto-2024-04-30_00-00
would destroy NVME/SSD@auto-2024-05-01_00-00
would destroy NVME/SSD@auto-2024-05-02_00-00
would destroy NVME/SSD@auto-2024-05-03_00-00
would destroy NVME/SSD@auto-2024-05-04_00-00
would reclaim 14.3G

Uff…
Is that because of the child dataset that only 14.3gb can be reclaimed?

[winnielinnie]

@Haldi

Is that because of the child dataset that only 14.3gb can be reclaimed?

Your only specified from April 25 to May 5 as your contiguous range of snapshots to “destroy”. Hence, it is telling you this will only liberate 14 GB.

If you try a larger range (that goes back even further), you might find it liberates much, much more space.

---

@Haldi

Uhmmm okay… Because reasons?

Imagine a truck with boxes. The load is getting full and you need to “free space” by removing some boxes.

Newly added boxes are always tagged with a white sticker. In order to physically remove a box from the truck, you must first remove its white sticker.

However, in the past, someone wanted to “protect” some boxes, so they slapped blue stickers on many boxes. Another person did the same thing, but with red stickers. And another person did the same thing, but with green stickers.

If a box has even a single sticker on it (of any color), it cannot be physically removed from the truck.

You come along and remove the white stickers from all the boxes you want taken out of the truck. But you cannot yet physically remove the boxes, since they still have at least one sticker on them.

Someone comes along and asks “How many boxes can we free up if we remove all the blue stickers?”

You answer: “Not much. Almost none.”

Someone else comes along and asks “How many boxes can we free up if we remove all the red stickers?”

You answer: “Not much. Almost none.”

Someone else comes along and asks “How many boxes can we free up if we remove all the green stickers?”

You answer: “Not much. Almost none.”

Finally, someone different comes along and asks “How many boxes can we free up if we remove all the blue, red, and green stickers?”

You answer: “SOOO MANY BOXES CAN NOW BE PHYSICALLY REMOVED!”

See how that works? Only looking at one sticker color at a time doesn’t really tell you much, since there are multiple colored stickers involved.

Now replace “boxes” with “data blocks” and “colored stickers” with “snapshots”.

EDIT: See this post that has pretty pictures for a better explanation.

[Haldi]

@winnielinnie

Your only specified from April 25 to May 5 as your contiguous range of snapshots to “destroy”. Hence, it is telling you this will only liberate 14 GB.

Ahhh yes, i picked that date at random because i didn’t realize i had to pick the first to get all data, and when i used an earlier date it gave the “could not find any snapshots” error.

admin@NAS[~]$ /usr/sbin/zfs destroy -nv NVME/SSD@auto-2024-03-06_14-22%auto-2024-05-05_00-00
would destroy NVME/SSD@auto-2024-03-06_14-22
would destroy NVME/SSD@auto-2024-04-22_00-00
would destroy NVME/SSD@auto-2024-04-23_00-00
would destroy NVME/SSD@auto-2024-04-24_00-00
would destroy NVME/SSD@auto-2024-04-25_00-00
would destroy NVME/SSD@auto-2024-04-26_00-00
would destroy NVME/SSD@auto-2024-04-27_00-00
would destroy NVME/SSD@auto-2024-04-28_00-00
would destroy NVME/SSD@auto-2024-04-29_00-00
would destroy NVME/SSD@auto-2024-04-30_00-00
would destroy NVME/SSD@auto-2024-05-01_00-00
would destroy NVME/SSD@auto-2024-05-02_00-00
would destroy NVME/SSD@auto-2024-05-03_00-00
would destroy NVME/SSD@auto-2024-05-04_00-00
would destroy NVME/SSD@auto-2024-05-05_00-00
would reclaim 997G

I still think the reclaimable space should somehow be shown in the Web interface… i.E beside the “Batch Operations: Delete” Button, that wen you select 5 Snapshots it would show much space that could give back. But i guess that could be more complex to implement than it sounds.

But still… great explanation sir, you truly deserve the MVP title.

After manually getting rid of the old unneeded snapshots the usage has gone down to 64% from 1.8TB

[ericloewe]

Colorful analogy, no pun intended. It should really help visualize things and I’m totally going to steal it use it as inspiration next time someone asks.

[winnielinnie]

@ericloewe, feel free to steal this as well, which goes the extra mile with pretty pictures!

Snapshots defy math and logic. "THEY DON'T MAKE SENSE!"

Why ZFS “snapshots” don’t make sense A children’s book for dummies, by a dummy “I don’t really understand how snapshots work.” “Why doesn’t destroying my snapshot free up more space?” “How come these numbers don’t add up?” Does this sound like you? Stop thinking about ZFS, pools, datasets, blocks, and snapshots. You want to demystify snapshots? Think about trucks, boxes, and colored stickers. What this illustrative walkthrough is not: It’s not a technical explanat…