Hello folks,
I’m testing Truenas in a virtual environment to take a look before deploying in a real ambient. During my tests I set up it with a 3 virtual disks of 3Gib in raidz1 for my storage. According with zfs my whole pool it says that is 5,48GiB
Now, after I set up immich and plex, I put via SMB some pics and two videos for testing apps. The problem is that truenas says me that mystorage is almost full ! and I used more than 4 GiB on it. But if I check via shell the space utilized from apps and storage with a simple ‘du -h’ in that pool, it says that I used more or less 911 MB ! is there something that I have to check or something that I’m doing wrong ?
thanks in advance
Alex
I am assuming that ZFS uses some space for base metadata (i.e. pool definitions, probably some space reserved for ZIL) though I am surprised that it is as much as 10%. But of course, ZFS is primarily intended for TB pools not GB pools, so the overhead on a tiny pool may be much more significant as a proportion of the entire pool size. I would be interested to know what the result is if you used 3 virtual disks of 30GB each.
(If the virtual drives are 3GB rather than 3GiB then you might certainly get this sort of reduction in reported space.)
I have no idea why du -h is not listing all the sub-directories - it seems to work just fine on my box. If you use the -x switch then it won’t work because each dataset is a separate mount.
Given that this is a test instance, would you be willing to share the detailed output of du -h here?
In your next test, use volumes 10x the size and see how that changes the relative numbers.
if the problem is only the tiny virtual disk, I’m able to do another test with more space(more or less 10GB for each). But It’s weird that both ‘zpool list’ show that I’m using a lot of space when ‘du -h’ output is about 900MB. Below the output about both commands:
truenas_admin@sharednas[/mnt/tank]$ sudo zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
boot-pool 15G 2.77G 12.2G - - 0% 18% 1.00x ONLINE -
tank 8.50G 6.26G 2.24G - - 9% 73% 1.00x ONLINE /mnt
truenas_admin@sharednas[/mnt/tank]$ sudo du -h
22K ./storage/mago/test
41K ./storage/mago
19K ./storage/asya
60K ./storage
708M ./Movies
8.4M ./oldPics/ASYA - LAUREA 2024
60M ./oldPics/FotoPodere
68M ./oldPics
875K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/b6/ef
876K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/b6
247K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/86/77
247K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/86
849K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/62/18
850K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/62
545K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/7f/82
546K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/7f
316K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/01/0c
317K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/01
870K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/cf/da
871K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/cf
535K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/88/a1
535K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/88
780K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/b8/85
780K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/b8
268K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/b0/e9
269K ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/b0
1.2M ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/6c/11
1.2M ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c/6c
6.4M ./apps/immich/thumbs/7de47a65-8afa-426a-bb71-0bae4e26252c
6.4M ./apps/immich/thumbs
1.0K ./apps/immich/backup
512 ./apps/immich/db/pg_logical/mappings
512 ./apps/immich/db/pg_logical/snapshots
2.0K ./apps/immich/db/pg_logical
1.0K ./apps/immich/db/pg_xact
49M ./apps/immich/db/base/16384
512 ./apps/immich/db/base/pgsql_tmp
2.4M ./apps/immich/db/base/4
2.5M ./apps/immich/db/base/1
2.5M ./apps/immich/db/base/5
56M ./apps/immich/db/base
512 ./apps/immich/db/pg_snapshots
1.0K ./apps/immich/db/pg_vectors/indexes/17347/startup
28K ./apps/immich/db/pg_vectors/indexes/17347/segments/f0482d3b-39a1-4691-9596-6587b4e85002/indexing/raw
41K ./apps/immich/db/pg_vectors/indexes/17347/segments/f0482d3b-39a1-4691-9596-6587b4e85002/indexing
41K ./apps/immich/db/pg_vectors/indexes/17347/segments/f0482d3b-39a1-4691-9596-6587b4e85002
42K ./apps/immich/db/pg_vectors/indexes/17347/segments
61K ./apps/immich/db/pg_vectors/indexes/17347
34K ./apps/immich/db/pg_vectors/indexes/17591/segments/579878f7-f6c1-490d-a0ce-f5772364fa47/indexing/raw
41K ./apps/immich/db/pg_vectors/indexes/17591/segments/579878f7-f6c1-490d-a0ce-f5772364fa47/indexing
42K ./apps/immich/db/pg_vectors/indexes/17591/segments/579878f7-f6c1-490d-a0ce-f5772364fa47
42K ./apps/immich/db/pg_vectors/indexes/17591/segments
1.0K ./apps/immich/db/pg_vectors/indexes/17591/startup
61K ./apps/immich/db/pg_vectors/indexes/17591
122K ./apps/immich/db/pg_vectors/indexes
1.0K ./apps/immich/db/pg_vectors/startup
124K ./apps/immich/db/pg_vectors
1.0K ./apps/immich/db/pg_subtrans
512 ./apps/immich/db/pg_stat
512 ./apps/immich/db/pg_stat_tmp
512 ./apps/immich/db/pg_wal/archive_status
5.4M ./apps/immich/db/pg_wal
512 ./apps/immich/db/pg_tblspc
512 ./apps/immich/db/pg_dynshmem
512 ./apps/immich/db/pg_notify
512 ./apps/immich/db/pg_replslot
512 ./apps/immich/db/pg_serial
1.0K ./apps/immich/db/pg_multixact/offsets
1.0K ./apps/immich/db/pg_multixact/members
2.5K ./apps/immich/db/pg_multixact
512 ./apps/immich/db/pg_twophase
512 ./apps/immich/db/pg_commit_ts
179K ./apps/immich/db/global
62M ./apps/immich/db
1.0K ./apps/immich/library
1.0K ./apps/immich/video
1.0K ./apps/immich/profile
9.7M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/cb/53
9.7M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/cb
13M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/ea/20
13M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/ea
1.7M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/d1/04
1.7M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/d1
1.1M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/94/aa
1.1M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/94
1.4M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/f5/7d
1.4M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/f5
4.4M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/1c/8a
4.4M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/1c
10M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/04/03
10M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/04
9.7M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/f1/08
9.7M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/f1
10M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/d0/1f
10M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/d0
7.6M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/2b/06
7.6M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c/2b
68M ./apps/immich/upload/7de47a65-8afa-426a-bb71-0bae4e26252c
68M ./apps/immich/upload
135M ./apps/immich
512 ./apps/plex
135M ./apps
911M .
truenas_admin@sharednas[/mnt/tank]$
I will try with 10Gb for each Virtual HD
Thanks
I tried this on my server:
sudo zpool list hdd-pool
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
hdd-pool 18.2T 8.03T 10.2T - - 2% 44% 1.00x ONLINE /mnt
du -c -h | tail
...
1.2T ./SystemBackups
3.5M ./unifi-backups
73T .
73T total
So according to du -h I am using 73TB on a pool that is c. 15TB of physical space.
I don’t understand what you mean…using 75TB on 15TB pool ?
Anyway…I made a fresh install to see if the problem comes back. After few changes the storage was at 2,5% so I installed immich. I saw that there is in storage a hide folder called .ix-apps. After installation my pool is used for 41,9% (2.3 GiB) but again it has no sense. According with ‘sudo dh -h’ that folder is more then 4.5G
.....CUT....
4.4G ./.ix-apps/docker
4.5G ./.ix-apps
512 ./tank/MoviesLibrary
19K ./tank/shared/mago
19K ./tank/shared/asya
38K ./tank/shared
512 ./tank/PicsLibrary
1.0K ./tank/configApps/immich/backups
512 ./tank/configApps/immich/database/pg_commit_ts
512 ./tank/configApps/immich/database/pg_replslot
1.0K ./tank/configApps/immich/database/pg_xact
512 ./tank/configApps/immich/database/pg_dynshmem
1.0K ./tank/configApps/immich/database/pg_vectors/startup
1.0K ./tank/configApps/immich/database/pg_vectors/indexes/17347/startup
512 ./tank/configApps/immich/database/pg_vectors/indexes/17347/segments
20K ./tank/configApps/immich/database/pg_vectors/indexes/17347
1.0K ./tank/configApps/immich/database/pg_vectors/indexes/17591/startup
512 ./tank/configApps/immich/database/pg_vectors/indexes/17591/segments
20K ./tank/configApps/immich/database/pg_vectors/indexes/17591
40K ./tank/configApps/immich/database/pg_vectors/indexes
42K ./tank/configApps/immich/database/pg_vectors
512 ./tank/configApps/immich/database/pg_wal/archive_status
5.3M ./tank/configApps/immich/database/pg_wal
512 ./tank/configApps/immich/database/pg_logical/snapshots
512 ./tank/configApps/immich/database/pg_logical/mappings
2.0K ./tank/configApps/immich/database/pg_logical
1.0K ./tank/configApps/immich/database/pg_multixact/members
1.0K ./tank/configApps/immich/database/pg_multixact/offsets
2.5K ./tank/configApps/immich/database/pg_multixact
512 ./tank/configApps/immich/database/pg_twophase
512 ./tank/configApps/immich/database/pg_stat
512 ./tank/configApps/immich/database/pg_tblspc
512 ./tank/configApps/immich/database/pg_notify
2.4M ./tank/configApps/immich/database/base/4
512 ./tank/configApps/immich/database/base/pgsql_tmp
2.5M ./tank/configApps/immich/database/base/1
4.9M ./tank/configApps/immich/database/base/16384
2.5M ./tank/configApps/immich/database/base/5
13M ./tank/configApps/immich/database/base
512 ./tank/configApps/immich/database/pg_serial
512 ./tank/configApps/immich/database/pg_snapshots
179K ./tank/configApps/immich/database/global
512 ./tank/configApps/immich/database/pg_stat_tmp
1.0K ./tank/configApps/immich/database/pg_subtrans
18M ./tank/configApps/immich/database
1.0K ./tank/configApps/immich/library
1.0K ./tank/configApps/immich/profile
1.0K ./tank/configApps/immich/video
1.0K ./tank/configApps/immich/uploads
1.0K ./tank/configApps/immich/thumbs
18M ./tank/configApps/immich
18M ./tank/configApps
18M ./tank
4.5G .
truenas_admin@sharednas[/mnt]$
while ‘zpool list’ says 5.0G free
truenas_admin@sharednas[/mnt]$ sudo zpool list tank
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
tank 8.50G 3.45G 5.05G - - 3% 40% 1.00x ONLINE /mnt
truenas_admin@sharednas[/mnt]$
and ‘Usage widget’ report 2.3GiB
I’m very confused !
Alex
Not running dh as sudo means that it doesn’t include any data you don’t have access to. Running it with sudo has given you reasonable numbers.
The discrepancy between du reporting 4.5GB of actual data (the size in bytes of your data) and zpool list reporting 3.45GB (the disk space actually used) can be accounted for by the default ZFS compression.
2 Likes
