Hello, everyone.
I have a nicely deduplicated SSD pool at 2.6x dedupe.
However, wherever I check the space consumed/available, I get the non-deduplicated value.
Whats the catch here?
Is this just wrong reporting? Does TrueNAS always report non-deduplicated consumption? What happens if the pool goes “full” in reported consumption but in reality is deduplicated many x times?
Anyone? Any ideas?
What does this report? You can’t go of what Windows reports.
zfs list -o space
Paste results back using preformatted text (</>) on toolbar
I recently stumbled across block cloning doing the same thing so wouldn’t be surprised at all if dedup is the same.
The “space” command reports the same wrong value.
NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD
HDD1DS 1.76T 348G 0B 294K 0B 348G
HDD1DS/HDD1 1.76T 348G 0B 348G 0B 0B
SSD1POOL 2.50T 2.46T 0B 294K 0B 2.46T
SSD1POOL/.system 2.50T 2.20G 0B 1.47G 0B 755M
SSD1POOL/.system/configs-ae32c386e13840b2bf9c0083275e7941 2.50T 3.67M 0B 3.67M 0B 0B
SSD1POOL/.system/cores 1024M 281K 0B 281K 0B 0B
SSD1POOL/.system/netdata-ae32c386e13840b2bf9c0083275e7941 2.50T 749M 0B 749M 0B 0B
SSD1POOL/.system/nfs 2.50T 415K 0B 415K 0B 0B
SSD1POOL/.system/samba4 2.50T 1.21M 0B 1.21M 0B 0B
SSD1POOL/SSD1DS 2.50T 2.38T 0B 2.38T 0B 0B
SSD1POOL/ix-apps 2.50T 347M 0B 166K 0B 347M
SSD1POOL/ix-apps/app_configs 2.50T 153K 0B 153K 0B 0B
SSD1POOL/ix-apps/app_mounts 2.50T 153K 0B 153K 0B 0B
SSD1POOL/ix-apps/docker 2.50T 1.16M 0B 1.16M 0B 0B
SSD1POOL/ix-apps/truenas_catalog 2.50T 345M 0B 345M 0B 0B
However the “zpool list” command show the deduplicated value.
Please note its not only Windows. Mac shows the same wrong available space, so do NFS shares through Veeam…
Are we just looking at a space reporting bug here?
I think so. Bug might suggest it once worked but I’m not sure that’s true so could perhaps be a feature request if you have the time and energy. All I’d ask is if you do please include block cloning in the request also.
I can gladly assist. What would be the best avenue to take to report? Just straight from TruneNAS GUI or some other place?
No space reporting bug. I think you are give the full size intentionally. It makes more sense if you are moving or replicating data to know the full size.
I think the question is to figure out how to see the smaller size that the deduped data is taking up. I was hoping it would have been with the space command. I’m not sure how to pull it up as Dedupe has always been more of an Enterprise feature.
@HoneyBadger Can you or someone else explain how to figure this out?
I’d suggest via the forums under the Feature Request category as that allows users to up vote and hopefully build traction for the change.
I see your point but that’s not currently how compression works. The post compressed values are currently reported in the UI.
Compression is not de duplication though. The space savings only exists while all copies of the data is the same on the pool with deduplication on. Back that pool up to a pool without dedup and it takes up the full space.
Agreed but also backup your data with compression to a different system without compression be it ZFS or not and you don’t have compression anymore.
The UI should probably report both values RAW and data reduction be it compression, dedup or block clone.
Most storage appliances crow about the space savings they’ve given to you. TrueNAS seems to be more modest 
My question here is if the non-dedupilicated available value is expressed to clients on purpuse, how do clinets know they could go past 100% usage?
If not it completely defeats the purpouse of deduplication.
In all dedupe appliances I’ve seen (StoreOnce for example), space values are represented, for example 20TB pool / 15TB used / 17TB available, implying deduplication. So this seems buggy to me.
We’re working on changes to the space accounting and reporting. Part of the challenge is being able to do this in a lightweight way - especially when dedup and bclone are involved, because we don’t want to have to walk metadata or tables every time the page is loaded.
I don’t know if there’s a “feature request” post on this yet, but creating one with these examples would be a good idea. 
Thanks, i was hoping for @HoneyBadger to sanity check the findings before I went “feature request”.
I will write up the request and post findings tomorrow.
Is this changes to the CLI and OpenZFS or it this just TrueNAS GUI on space accounting and reporting?
I would expect ZFS would have accurate reporting related to space and dedup since that is a stable feature that has been around a while.
I can’t imaging everyone not voting for accurate reporting for space. Does this even need a FR? Maybe TrueNAS the company should create Feature Request posts on items being worked on or for the future, make company comments and then lock it as upcoming or being worked on.
That’s actually very easy to imagine: We’re out of votes because the few we have are held forever on old FR which have not been reviewed and decided upon. Tactical de-voting only goes so far, and it’s hard to fight the feeling that this exercise is going nowhere…