Arc Size "Decays" to nothing and repeats

TrueNAS General
1

nothing seems to be operating wrong with my system after upgrading to 25.10.1 but i noticed something strange with my arc cache size

Untitled
Untitled1035×262 24.1 KB

Previously my Arc size has always been hovering around the 50% of my available 128GB of RAM which can be seen to the left before upgrading, but now it seems to increase to where I expect and then slowly “decay” down to the minimum arc size and repeats

looking here

https://utcc.utoronto.ca/~cks/space/blog/linux/ZFSOnLinuxARCMemoryStatistics

“If the ‘available’ number goes negative, the ARC shrinks; if it’s (enough) positive, the ARC can grow.”

In my arc_summary (Link to drop box), Available memory size was reporting -3124645888 Bytes. I find this weird as the truenas web gui showed 90GB RAM free at this time, so not sure what is occurring here and why the available memory size is negative.

I have restarted my system to see if there is any change in behavior and it did not make a difference

this morning, my Truenas GUI is reporting this for memory where one can clearly see i have 42.9GB of free memory

Untitled2
Untitled2429×313 11.7 KB

However the arc_summary (dropbox link) from this morning says

Free memory size: 16.6 GiB
Available memory size: 569.1 MiB

i do have zfs_arc_sys_free set to exactly 16GB so it makes sense that 16.6 - 16 = 569.1MB. what does NOT seem to make sense is why is ZFS not seeing that i have 42.9GB of free RAM? it thinks i have only 569.1MB available to it.

is this a bug? never had this issue until moving to goldeye.

2

I’d see no reason not to toss a bug report, though after the reboot not sure if any relevant logging got cleared.

Only time I see any mildly weird arc results (like ignoring my enforced arc limits) are when I start/stop vms. Any chance you were doing either at the time?

3

i am using no VMs or containers, using only apps

for the time being i have set

zfs_arc_max = 50%

and

zfs_arc_min = 40%

of my available RAM and i will see if that does anything.

so far i do not see any apparent negative impact on my system with this behavior.

i have created a bug report NAS-139074 and we shall see what comes of it

1 Like