Excessive syslog-ng disk writes

TrueNAS General
1

Hi all,

I’m running TrueNAS SCALE 25.10.1 as a VM on Proxmox, and I’ve noticed what seems to be an unusually high amount of disk writes to the OS drive.

After digging into it, I found that syslog-ng and auditd are generating a massive amount of log writes. For example, over just a 10-minute window:

PID    COMMAND       WR_OPS    WR_BYTES   CMDLINE
---    -------       ------    --------   -------
6374   syslog-ng     1072633   3.25GB     /usr/sbin/syslog-ng -F
3439   auditd        921253    234.4MB    /sbin/auditd

This means roughly 430GB/day for syslog and 35GB/day for auditd being written to the OS SSD, this seems excessive and potentially harmful for SSD longevity. I confirmed these numbers with iotop from the proxmox host and tracking total disk writes to the SSD the VM disk sits on.

  • Is this level of logging expected behavior on SCALE 25.10.1?
  • Are there recommended ways to reduce logging verbosity for syslog-ng or auditd?

Does anyone have advice on…

  • Moving logs to a tmpfs to reduce disk wear?
  • Redirecting logs to the storage pool (spinning disks) instead of the OS drive?
  • Specific config tweaks that others are using to mitigate this?

Any guidance, best practices, or similar experiences would be greatly appreciated. I want to reduce unnecessary wear on the OS drive without breaking anything important.

Thanks in advance!

2

I tried mounting /audit, /var/log, & /var/lib/syslog-ng to folders on the storage array

mount --bind /mnt/data/local/audit /audit
mount --bind /mnt/data/local/logs /var/log
mount --bind /mnt/data/local/syslog-lib /var/lib/syslog-ng

Disk writes to the OS drive over 10 minutes dropped to less than 8MB :tada:

I noticed that the data section of the pool widget and also all temperature information initially broke, but after about 30 minutes everything came back and appears to be working normally.

I would still like to find a solution to reducing the amount of data syslog and auditd are writing though. Even though it isn’t impacting the SSDs any more, it still seems like an excessive amount of writing for logs.

Edit: I did rsync the contents of the directories over to the new mount points first.

3

Did you look around in the System, Advanced Settings menus?

4

I did, but the only setting that I found which looked reasonable was the syslog level, but even setting it to Emergency didnt seem to impact the write volume.

Edit: I also saw System Dataset Pool: data, but switching it to boot-pool and back also didn’t seem to make any impact.

5

Opened a bug report just in case… Jira

6

Hi , i don’t know if my problem is related to yours , i’m not enough good in linux to search for but my post is there, if it can be in same spirit.