Do not forget the fine prints:
Unfortunately I don’t have much time left to spend on this project so I would have to leave this for a follow-on project. I think that it shouldn’t be much more work to do this as an extension, where we have RAIDZ Expansion for a while and then we add “RAIDZ Expansion improved space accounting”, and blocks written after that point would have the improved accounting (that is, assuming it’s possible at all).
Assuming that it is possible at all…
Goes to show how often I actually delete things that I didn’t even consider the case of deleting old data.
Interesting read through this thread, and another reason I’ll stick to my large drive mirrored vdevs.
Pickle-flavored ice cream for sale. 
I just realized. Wait a minute.
Why doesn’t the TrueNAS GUI just use zpool (against the pool itself), rather than zfs (against the root dataset) when presenting capacity info in the dashboard?
In order to account for freed space, the deflate ratio of a freed block needs to be known.
This is initially the original parity ratio. And in theory should change for newer blocks.
Rather than have different deflate ratios for pre/post expansion blocks, the deflate ratio is simply never changed.
And it all works.
The only issue is that your bytes are now made out of fatter bits.
Zpool doesn’t include parity/redundancy cost
You could derive the capacity by working out the capacity from each VDev based on its structure, but the allocated space is still wrong, so you can’t tell the free that way.
It gets complicated with multiple vdevs.
If you can find out the “free deflated space” for a given vdev then you can convert that to the new parity ratio in the gui, then sum up for each VDev.
Poor @joeschmuck. He probably thinks I’m a psychopath with the way I retract/edit my posts. Eating apple-flavored glue sticks and selling pickle-flavored ice cream.
LOL, pulling me back into this thread I see. I do like Apples and I like Ice Cream. But wait, I like Tootsie Pops, Banana Pudding, and Kielbasa. Not all together. Now I want an Apple Pie. Damn you @winnielinnie !
Thankfully the wife has a Pumpkin Pie and a can of Whip Cream. Do you know why they use N2O for this stuff? I thought it was so we could suck on it but the real reason is it makes the whipped cream, fluffier and smoother. A full body I guess. But it is too early in the morning to eat that pie. I may finish it all if I start now.
I did just purchase a rear brake kit for my wife’s car. She said it was making squealing noises, she was right, metal on metal on both sides. When those arrive tomorrow, I will have another item on the honey do list to complete. Maybe Apple Pie after that.
And how about that RAIDZ Expansion, isn’t it great? (felt I had to mention it in this thread at least once)
Who took on this subproject in the meantime, if not Matt?
If I understand you correctly, that is exactly what I did with my tests a few posts up. I first expanded an empty pool and then filled it up with dd up to the brim. The “estimated” numbers for the pool size did not change whether it was empty or full.
If you accidentally use a CO2 cartridge you’ll find out.
PS: carbonated cream is not nice
I really hope that comment wasn’t from personal experience, Ha Ha
So, you could store more data than the estimated pool size?
There must be a better algorithm for estimation that could be used…
root@truenas[/home/truenas_admin]# zpool list -v test_pool
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
test_pool 9.98T 6.43M 9.98T - - 0% 0% 1.00x ONLINE /mnt
raidz1-0 9.98T 6.43M 9.98T - - 0% 0.00% - ONLINE
115cd1b7-a793-414e-a129-45e2b957b1bd 1024G - - - - - - - ONLINE
3e4a8cf3-9b3e-4e62-86e8-130ef583a51c 1024G - - - - - - - ONLINE
c68390ef-3625-42ba-a584-aa9d6f6b3516 1024G - - - - - - - ONLINE
d3cc02b6-de80-4c5e-a53a-1ff18ddb6e58 1024G - - - - - - - ONLINE
e224f009-c449-4ca9-852c-c792f6e86a0e 1024G - - - - - - - ONLINE
f4c85545-4b94-40c0-ac20-05e701a6f846 1024G - - - - - - - ONLINE
d94d899b-fc1b-4b7d-b664-12c0c087cb68 1024G - - - - - - - ONLINE
cf74cdfc-e80b-4524-93fc-f49bc7d93fa8 1024G - - - - - - - ONLINE
c537eec4-9522-4a6b-87b9-bc4f0d4b23de 1024G - - - - - - - ONLINE
82ecf9b3-0a04-41f3-a506-3b16ca9525a9 1024G - - - - - - - ONLINE
For each vdev, multiple the FREE space by the storage efficiency ratio, and accumulate.
eg:
raidz1-0 has 9.98T free, multiply by the number of data members divided by the number of members
Thus…
9.98T * 9 / 10 = 8.982T
Repeat and accumulate for each vdev.
Working out capacity is similar, but with the SIZE, rather than FREE column.
A mirror/stripe has 1 data member. RaidZx has members-x data members
DRAID is left as an excercise 
Yes. I was able to store as much data as was physically possible. The system did not limit me to the amount that was calculated wrong. So at least we can use all the space we pay for, “only” the calculations are wrong
I like the FREE space model… assuming that’s always reported accurately for RAID-Z expansion. It would provide the amount of space for new data.
Using SIZE wouldn’t work???.. because we don’t know the efficiency ratio for the data that is still stored.
If people like this, it could be a “suggestion” for a new feature. Provide a better estimate of space available on an expanded vdev.