My kubernetes pods are Evicted repetitively

General Discussion
1

I have a k8s environment, where I am running 3 masters and 7 worker nodes. Daily my pods are in evicted states due to disk pressure.

I am getting the below error on my worker node.

Message: The node was low on resource: ephemeral-storage.
Status: Failed
Reason: Evicted
Message: Pod The node had condition: [DiskPressure].

But my worker node has enough resources to schedule pods.

Please help me resolve this issue

2

TrueNAS doesn’t use k8s so this sounds like a custom install. Come 24.10 TrueNAS won’t even use k3s. Either elaborate how you think it relates to TrueNAS or look at how you set your kubernetes install up.

Reading about Eviction Thresholds here looks like a good start:

3

It seems like your Kubernetes pods are being evicted due to disk pressure, even though your worker nodes appear to have enough resources. This can happen if the system detects low ephemeral storage and enforces eviction policies. To resolve this, you might need to adjust the eviction thresholds, clear up unused storage, or optimize your pod resource requests.

Here’s a detailed guide to help fix eviction-related issues: Fix Evict Pod Issue