Heyo, I would like to continue the discussion of k3s-server uses 10% CPU for no reason | Page 3 | TrueNAS Community. I found the blog Resolved: High Idle CPU Usage Bug in k3s - A Milestone for Stability that was monitoring this issue for a long time and could finally report that this has been resolved in version 1.28 of k3s. Any chance we could upgrade in dragon fish? it still has 1.26 
1 Like
With the next major update later this year k3s will be depreciated and replaced by native docker. So i dont think iX will invest time in something that will be removed later this year.
2 Likes
LarsR is correct. Upgrading k3s version in a dot release is very risky so we have no reason to do it unless there is another very critical bug (vulnerability) that cannot be backported.
An interim solution, if you use docker apps, would be to use jailmaker to host them instead, until the official rework of the app subsystem is complete.
More work to set up compared to official TN apps but once up it uses less resources.
1 Like
âŚalthough this may be another reason TrueCharts are suggesting that a VM-based replacement would require less system overhead than the system as it currently exists. They expect to have their beta out early next month.
1 Like
But even with less CPU overhead I donât want my files to live inside the VM zvol (which is a filesystem, e.g. ext4 on top of the zfs filesystem). Nor do I want to create a dedicated zvol for files inside the VM and I donât want to make the files available to the VM via NFS or SMB either for performance, compatibility and security reasons. Even if virtiofs could be used to mount host directories inside a VM on SCALE (which currently isnât possible) that would still cause a noticeable performance hit. Iâm curious what they come up with of course but I donât see how this could become great.
The biggest problem I have with NFS is how it handles nested datasetsâif I have datasets of video and video/tv, and expose video via SMB, everything in video/tv is available as well (as long as permissions allow it). No so with NFS; I need to separately mount both video and video/tv.
As to performance, I expect my pool is going to be the limiting factor rather than either network protocol (and if the VM is running on my NAS, the network itself wonât be a limit at all). If I were running a big, all-NVMe pool, I might worry about performance, but mine is all spinners; I donât expect it to be a problem.
But everyoneâs needs are different. You may find that k3s in a sandbox is a better way for you to go. That isnât what TC say theyâre doing; I expect at least part of their reason is to move as far away as possible from anything iX controls, and wouldnât blame them in the least if that were the case. But itâs a possibility. Or use their migration solution, which theyâre suggesting will be VM-based. Or set up your own Kubernetes-compatible cluster and manage it however you like. Or ditch k3s and apps entirely, and run Docker in a sandbox. Or use whatever iX comes up as an apps system in 24.10. Or no doubt other possibilities Iâm not even thinking of.
Just for reference, this is also true of mounting datasets inside BSD jails.
With jailmaker you have the systemd-nspawn --inaccessible flag which could be used to prevent this I think. systemd-nspawn
Also, I think specifying norbind as a bind option could restore the nfs style behavior, as opposed to the default recursive bind rbind
1 Like
What helps for me is to reboot the TrueNAS server.
The CPU usage is still there by the k3s-server process, but itâs less than before the reboot.
Server 1:
Server 2:
Well, Electric Eel will âsolveâ this issueâŚ
1 Like
In OctoberâŚ
You can âunsetâ the apps pool to solve the problem right now
(Of course, the apps will stop working, not a problem if youâre running docker in a sandbox instead)
Or, you know, just deal. 10% (of one core, which is often omitted from the discussion) used for k3s[1]? You still have 90% of that core, and 100% of the others, to use for whatever else you need (like running the NAS, for example). I have to suggest that if 10% of one core is that big of a problem, your hardware really isnât suitable for TrueNAS (or at least for running apps on TrueNAS).
Over the past 24 hours, my UGREEN NAS running SCALE 24.04.2 reports an average CPU idle time of 94% on its i5-1235U. Thatâs with 13 TrueCharts apps installed and running, as well as a TalOS VM. So even with a pretty low-end CPU, itâs not taking 10% of the whole CPU. âŠď¸
1 Like