Performance when low capacity?

Hi all,

So here’s the thing…

When I originally configured my pool years ago I read somewhere that if you allow your capacity % used on ZFS to go above 80% then if causes a serious degradation in performance.

So at the time I made what I thought was a smart decision, and I created the pool while leaving 20% of my available disk capacity unused and just used the remaining 80% for the pool, because I thought to myself that although I would have less available capacity, even if I hit 100% capacity used, I am still not going over the 80%.

Hope that makes sense…

But now years down the line, I have very little space left and although I have cleared what I can it’s still low and I am getting regular warnings about having over 90% space used in my pool.

Which has made me wonder, did I interpret the rules incorrectly… is it the total % file system utilisation and not the % disk utilisation that I need to be concerned about?

Because if so, I may as well just add in that extra 20% space I haven’t been using all this time (if it’s possible to do that) and that solves my space problem and utilisation problem all at once because I cannot afford to buy bigger replacement disks at the moment.

Typing this out now I am starting to feel like this may indeed have been an error in logic on my part when I set it up, but I was new to TrueNAS at the time and there was a LOT of new things to learn and consider, so it was quite overwhelming.

Anyway regardless I would really appreciate some confirmation from somebody who knows on this issue.

Thanks in advance :slight_smile:

I wonder how you achieved that. Did you partition your drives with 20% empty partitions?

AFAIK, no, it’s not about file system (aka dataset) utilisation; it’s about pool utilisation. With growing utilisation, it is harder and harder for zfs to find sequential free blocks. So even sequential operations act more like random operations, thus making performance worse.

Lets look at your pool layout and space. Please run the following in the Shell or Console and post the results back using Preformatted Text </> on toolbar or Ctrl+e
Replace [POOL_NAME] with the name of your pool, please

sudo ZPOOL_SCRIPTS_AS_ROOT=1 zpool status -vLtsc lsblk,serial,smartx,smart

sudo zfs list -o space [POOL_NAME]

Ok I have investigated my NAS web portal to check how I did it.

So basically I did it at the dataset level. Under the default dataset (pool?) I created a dataset that is the one I actually use.

On this dataset I set the “Quota for this dataset and all children” to 80% of the total space available to the default dataset (pool?)

Since I don’t have another dataset under this default dataset (pool?) that means that in addition to the 9% free capacity on the dataset I am using, there is still 20% of my total capacity free on top of that.

In practical terms there is 7.2TB in the pool and the one and only dataset I am using has about 5.7TB assigned to it.

So in this situation… how does that leave me?

…Making this more confusing to me is that I can’t seem to find the word “pool” anywhere on the web portal… (I am on TrueNAS Scale now but I initially started on TrueNAS Core). I hope I’m describing this right.

Thanks for your help.

 root@truenas[~]# sudo zfs list -o space General
NAME     AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
General  1.78T  5.35T        0B    368K             0B      5.35T

You missed running the other command.
Post screenshots of the windows where you have made these settings. Try to include the entire GUI window so we can see what menu sections you are in. I use the Snipping Tool in Windows to capture and trim.

Guessing you are under Datasets, Dataset Space Management, hitting Edit button and then setting Capacity Settings. Be verbose if you have to describe what menus you bring up.

My apologies

 pool: General
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 10:05:21 with 0 errors on Tue Jul 15 10:05:23 2025
config:

        NAME        STATE     READ WRITE CKSUM  SLOW    size  vendor                      model           serial  hours_on  pwr_cyc  health  realloc  rep_ucor  cmd_to  temp  pend_sec  off_ucor  ata_err
        General     ONLINE       0     0     0     -
          raidz2-0  ONLINE       0     0     0     -
            sdb2    ONLINE       0     0     0     0    1.8T     ATA         ST2000DM006-2DM164         Z4ZBYMS7     42622      125  PASSED        0         0       0    44         0         0        -  (trim unsupported)
            sdd2    ONLINE       0     0     0     0    1.8T     ATA         ST2000DM006-2DM164         Z4ZBGTQ8     41846      130  PASSED        0         0       0    44         0         0        -  (trim unsupported)
            sda2    ONLINE       0     0     0     0    1.8T     ATA         ST2000DM006-2DM164         Z4Z87YED     50167      238  PASSED        0         0       0    37         0         0        -  (trim unsupported)
            sdf2    ONLINE       0     0     0     0    1.8T     ATA       WDC WD20EURX-63T0FY0  WD-WCC4M3412790     17336       23  PASSED        0         -       -    36         0         0        -  (trim unsupported)
            sdg2    ONLINE       0     0     0     0    1.8T     ATA       WDC WD20EZRX-00DC0B0  WD-WCC1T0959005     43269     3776  PASSED        0         -       -    40         0         0        -  (trim unsupported)
            sdc2    ONLINE       0     0     0     0    1.8T     ATA       WDC WD20EURX-63T0FY0  WD-WCC4MPP404ZV     19754       76  PASSED        0         -       -    41         0         0        -  (trim unsupported)
        spares
          sde2      AVAIL       1.8T     ATA       WDC WD20EURX-63T0FY0  WD-WCC4MEFT3RHK      5289       22  PASSED        0         -       -    38         0         0        -  (trim unsupported)

errors: No known data errors

Ok so I go to Datasets->Dataset Space Management->Press Edit, like you said.

Then I have 5.7TiB entered under “Quota for this dataset and all children”.

I tried to give you a snipping tool of it but the forum won’t let me, it gives the error “An error occurred: Sorry, you can’t embed media items in a post.”

There is also no edit button for my post so I also can’t even edit my own post to tell you that the forum has trimmed the text from my console output so you can’t see the right hand columns, but they are there if you need to you select all the text and copy into notepad to see it as intended.

Your status shows ‘new user’.
Browse some other threads and do the Tutorial by the Bot to get your forum trust level up then you can post images and edit posts.

TrueNAS-Bot
Type this in a new reply and send to bring up the tutorial, if you haven’t done it already.

@TrueNAS-Bot start tutorial

Are you posting the command results using Preformatted text mode? It more looks like it is bold or something. If you hit Ctrl+e you should get a hint at posting. You can also hit the (</>) button on the tool tips above where you post replies. Triple backticks, “```”, on my keyboard with ~ by the #1 leading and after the text you want formatted also do Preformatted Text mode. It shouldn’t cut of text then. You should be able to expand the section of Preformatted Text or copy and paste it. It acts like another window.

```
type or paste code here
```

Edited so you could see the triple backticks. I had to put a backlash before them to get them to show, otherwise the forum just used them

After reading your post and looking at the commands outputs, I’m not sure I got your config correctly. Nevertheless, (AFAIK), dataset performance is not dependent on (or better said correlated with) its utilisation. Performance (degradation) correlated with the entire pool occupancy.

So, removing the quota from your dataset won’t affect performance, but it affects your ability to write to this dataset above a certain limit (and it affects notifications for sure). If your pool has 1.78T free, you can definitely use some part of it (with overall performance getting worse and worse).

It’s not very different from other devices you have. With 95% occupancy on your smartphone, you would not have a great time… At least it’s true for iPhones.

Speaking of performance and free space. You can (should) also tune your pool by creating separate datasets for the different types of data. And set different recordsize and compression values for them.

For example, setting 1M (or even 4M) as a recordsize for a media dataset will slightly improve performance and reduce (not by much) overall metadata size.

Compression requires some CPU power, but changing it from the default lz4 to something like zstd-3 can save you up to a few hundred GB per TB of compressible data. FYI – media is usually not compressible.

Thanks for your posts that makes sense now and it looks like I was thinking along the right lines originally regards leaving some space unused. I think I’ll continue to do that because I don’t want to start losing performance.

The compression thing is kind of shocking if it can save that much data… Would it not slow down how fast files can be read if they are compressed more though? For context I am working with files directly on the NAS instead of on my computer for as many tasks as it makes sense for. It’s not just for long term data storage.

Also I think the reason I did it all in one dataset to begin with was because I wanted to be able to map a network drive in Windows to access my data, and I didn’t want to be forced to have 3-4 different mapped drives when I could just have one that contains everything, if that makes sense. It’s way too much hassle to not just have it all accessible in one place like that.

1 Like

You started losing performance the moment you started to fill the pool :slightly_smiling_face:. Aiui, 80% and 95% are just empiric values for “you shouldn’t go above” and “you better never go above” or something like that.

And performance loss level probably depends on the data usage scenario. I don’t think 95% occupancy will degrade performance much if data was only written once and read many. For one, this can be the case when the entire pool is a media library (seems not to be your case).


Welp, typical “large” data files are not compressible (much). Because, you know, they are usually already compressed. So, you won’t save much in the most cases. It works well with documents, system images, and… perhaps there are other types of (highly) compressible data.

Depends on your NAS CPU. Decompression (during the read) is usually fast. Compression during writes is another matter – you should make some tests and look at your CPU utilisation.


Understandable. Now you are aware of some caveats.

Ok lovely that’s some stuff for me to think about then. Thank you so much for all your help it is much appreciated.

And thanks to SmallBarky too of course :slight_smile: