24.10 crash for first time in over a decade

TrueNAS General
1

Yesterday I upgraded from 24.04 to 24.10. Everything went pretty smooth. Had to manually rework one TrueCharts container to native docker (mosquitto) but that also worked like a charm. Everything fine until I woke up this morning to find that none of my containers were running. Turns out that restart was disabled by default (so enabled that) and all containers had status crashed.
Turns out that Truenas crashed just after 2am, came back up (some 12 minutes later? Clock may be the issue here). Nothing in the logs other than the restart.

Moving from freenas to truenas core to scale, this has never happened to me. I have seen all sorts of things over the years (decade) but never a complete restart with no indications as to why :-(.

Bugger…

Herman

2

Do you have a UPS?

(Blah blah)

3

So close! Only two characters away.

4

Yup: 2: a directly connected UPS and a home battery.
Why?

5

Because a power blip could explain the behavior too.

7

Same again, now 1.10am.

Last messages in the log:
Nov 4 00:53:55 freenas smbd[6785]: [2024/11/04 00:53:55.560804, 2] …/…/source3/smbd/server.c:822(remove_child_pid)
Nov 4 00:53:55 freenas smbd[6785]: Could not find child 441833 – ignoring
Nov 4 01:00:07 freenas systemd[1]: Started ipa-epn.service - Execute IPA Expiring Password Notification (EPN).
Nov 4 01:00:07 freenas systemd[1]: Starting sysstat-collect.service - system activity accounting tool…
Nov 4 01:00:07 freenas systemd[1]: sysstat-collect.service: Deactivated successfully.
Nov 4 01:00:07 freenas systemd[1]: Finished sysstat-collect.service - system activity accounting tool.
Nov 4 01:00:08 freenas ipa-epn[458304]: IPA client is not configured on this system.
Nov 4 01:00:08 freenas ipa-epn[458304]: The IPA-EPN command failed. See /var/log/ipaepn.log for more information
Nov 4 01:00:08 freenas systemd[1]: ipa-epn.service: Main process exited, code=exited, status=1/FAILURE
Nov 4 01:00:08 freenas systemd[1]: ipa-epn.service: Failed with result ‘exit-code’.
Nov 4 01:08:55 freenas smbd[6785]: [2024/11/04 01:08:55.671568, 2] …/…/source3/smbd/server.c:822(remove_child_pid)
Nov 4 01:08:55 freenas smbd[6785]: Could not find child 482454 – ignoring
Nov 4 01:10:02 freenas systemd[1]: Starting sysstat-collect.service - system activity accounting tool…
Nov 4 01:10:02 freenas systemd[1]: sysstat-collect.service: Deactivated successfully.
Nov 4 01:10:02 freenas systemd[1]: Finished sysstat-collect.service - system activity accounting tool.

8

This may be a bug in the Asus motherboard whereby the voltage drops too much when the system is idle. Since Kubernetes has been removed, especially at night, the system load is really low.
Changed the power supply idle control in the BIOS (C6 option). Let’s see tonight…

2 Likes