How to debug Truenas Scale as it becomes unresponsive from time to time

Hi there,

I am using Truenas Scale for the last months and after some time I got used to it. Normally all runs smoothly and I have no problems.

Lately and to be honest maybe since I started using Truenas Scale gets unresponsive from time to time. I can’t connect to the WebGUI anymore but also the wired ethernet connection is not “active” connected to the Router anymore and even when I connect a monitor/keyboard it doesn’t show anything (doesn’t get a signal).

This all happens from time to time and sometimes I have a uptime of 3 days but somtimes even 14 days… my longest uptime streak was around 2 weeks.

I first thought it’s something with my Router but as I don’t get any response from the monitor/keyboard as well I am thinking it has todo with the System/Hardware itself.

My question now, how can I debug this and see some “older” log files before the system freezes? Are they saved somewhere? Because I found some commands but they show only the most recent log?

I am really a bit out of thoughts what todo and only a power circle helps in the end to reboot the computer. It’s all new hardware and it runs all smoothly till it freezes. Temps are all okay I check them frequently.

Thanks a lot if you have any idea how to debug a problem like that.

What version are you running?

TrueNAS-SCALE-23.10.2

Have you checked the logs in the debug file, Settings > Advanced >Save debug

Thanks a lot that is what I was searching.

I opened the Error log and it shows error when the system froze.

Now the question is what happened

May 29 08:22:14 truenas kernel: ixgbe 0000:04:00.0: Adapter removed
May 29 08:22:15 truenas kernel: ixgbe 0000:04:00.1: Adapter removed
May 29 09:10:00 truenas kernel: hid-generic 0003:1532:028D.0004: No inputs registered, leaving
May 29 09:10:01 truenas kernel: Error: Driver ‘pcspkr’ is already registered, aborting…
May 29 09:10:01 truenas kernel:
May 29 09:10:01 truenas kernel: NVRM: The NVIDIA GeForce GT 710 GPU installed in this system is
NVRM: supported through the NVIDIA 470.xx Legacy drivers. Please
NVRM: visit Unix Drivers | NVIDIA for more
NVRM: information. The 535.54.03 NVIDIA driver will ignore
NVRM: this GPU. Continuing probe…
May 29 09:10:29 truenas blkmapd[2602]: open pipe file /run/rpc_pipefs/nfs/blocklayout failed: No such file or directory
May 29 09:10:29 truenas systemd[1]: Failed to start nslcd.service - LSB: LDAP connection daemon.
May 29 09:10:32 truenas libvirtd[2934]: invalid argument: cannot find architecture arm
May 29 09:10:32 truenas haproxy[4946]: backend be_20 has no server available!
May 29 09:10:33 truenas haproxy[4946]: backend be_32 has no server available!
May 29 09:10:33 truenas haproxy[5267]: backend be_20 has no server available!
May 29 09:11:00 truenas kernel: NVRM: The NVIDIA GeForce GT 710 GPU installed in this system is
NVRM: supported through the NVIDIA 470.xx Legacy drivers. Please
NVRM: visit Unix Drivers | NVIDIA for more
NVRM: information. The 535.54.03 NVIDIA driver will ignore
NVRM: this GPU. Continuing probe…
May 29 09:11:22 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:22 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:22 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:22 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:22 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:22 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:22 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:22 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:24 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:24 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:27 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:27 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:27 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:27 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:27 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available
May 29 09:11:27 truenas kernel: IPVS: rr: UDP 172.17.0.10:53 - no destination available

I think it’s actually this one as the rest was my attempt to connect my display to the server.

May 29 08:22:14 - truenas kernel: ixgbe 0000:04:00.0: Adapter removed

So it seems this is connected to my NIC which I installed. Guess I have to start searching there or use the internal LAN again which worked without problems. I got me an Intel x520 Dual NIC but guess it has problems or is the cheap chinese variant everyone warning about.

@ABain again thanks a lot I was exactly searching that error log to debug and find some clues :slight_smile:

1 Like