I am new to TrueNAS and got a number of apps set up a couple days ago.
I previously was on TrueNAS core and upgraded to scale, then imported my pool. I then initialized the apps and configured multiple applications. This ran for a few days but then the app service crashed.
I couldn’t get it working so I reinstalled TrueNAS and imported the pool. Apps service would still not start.
I read a post that said to delete the ix-applications dataset. I did this and rebuilt the apps. That is when everything worked for about 12 hours and crashed/stopped and just displays “initializing apps service”.
I saw another post mention to unset the pool, reboot, and then set the pool again. I tried this and it is still stuck. Any assistance would be greatly appreciated.
I am running Dragonfish-24.04.2.1.
admin@truenas[~]$ sudo watch -d k3s kubectl get pod -A
Every 2.0s: k3s kubectl get pod -A truenas: Mon Sep 16 10:03:12 2024
NAMESPACE NAME READY STATUS RESTARTS AGE
ix-sonarr sonarr-bd5cd44d-jd688 0/1 Unknown 0 33h
ix-radarr radarr-9bd647f98-qlbkf 0/1 Unknown 0 33h
kube-system csi-nfs-node-4vqgp 3/3 Running 12 (32h ago) 47h
kube-system csi-smb-node-d6pwm 3/3 Running 12 (32h ago) 47h
ix-sabnzbd sabnzbd-65d5bc6d-gbdv4 0/1 Unknown 0 33h
kube-system snapshot-controller-546868dfb4-v6gpq 0/1 Unknown 0 33h
kube-system snapshot-controller-546868dfb4-jd82c 0/1 Unknown 0 33h
kube-system snapshot-controller-546868dfb4-mwjrh 1/1 Running 0 32h
ix-handbrake handbrake-6884fb65f4-ldhv9 1/1 Running 0 32h
kube-system coredns-59b4f5bbd5-qphr6 1/1 Running 0 32h
ix-sonarr sonarr-bd5cd44d-wjkxh 1/1 Running 0 32h
ix-radarr radarr-9bd647f98-b8hg5 1/1 Running 0 32h
ix-plex plex-6c897645b8-jml26 1/1 Running 0 32h
ix-lidarr lidarr-b667789b9-99cjb 1/1 Running 0 32h
ix-sabnzbd sabnzbd-65d5bc6d-wt986 1/1 Running 0 32h
kube-system nvidia-device-plugin-daemonset-slkbk 0/1 CrashLoopBackOff 13 (32h ago) 32h
kube-system snapshot-controller-546868dfb4-hp8rg 0/1 Error 0 32h
kube-system csi-nfs-controller-7b74694749-bbp8j 3/4 Error 0 32h
kube-system csi-smb-controller-7fbbb8fb6f-ml6gh 3/3 Running 1 (32h ago) 32h
admin@truenas[~]$ journalctl -xeu k3s.service
Hint: You are currently not seeing messages from other users and the system.
Users in groups ‘adm’, ‘systemd-journal’ can see all messages.
Pass -q to turn off this notice.
No journal files were opened due to insufficient permissions.
admin@truenas[~]$ sudo journalctl -xeu k3s.service
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Txn call completed” 1ms (08:33:47.868)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Retry value restored” 0ms (08:33:47.868)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“About to Encode” 0ms (08:33:47.868)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Encode succeeded” len:566 0ms (08:33:47.868)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Txn call completed” 1ms (08:33:47.869)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Retry value restored” 0ms (08:33:47.869)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“About to Encode” 0ms (08:33:47.870)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Encode succeeded” len:566 0ms (08:33:47.870)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Txn call completed” 1ms (08:33:47.871)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Retry value restored” 0ms (08:33:47.871)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“About to Encode” 0ms (08:33:47.872)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Encode succeeded” len:566 0ms (08:33:47.872)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Txn call completed” 1ms (08:33:47.873)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Retry value restored” 0ms (08:33:47.873)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“About to Encode” 0ms (08:33:47.873)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Encode succeeded” len:566 0ms (08:33:47.873)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Txn call completed” 1ms (08:33:47.875)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Retry value restored” 0ms (08:33:47.875)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“About to Encode” 0ms (08:33:47.875)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Encode succeeded” len:566 0ms (08:33:47.875)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Txn call completed” 1ms (08:33:47.877)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“Retry value restored” 0ms (08:33:47.877)
Sep 16 08:34:17 truenas k3s[18193]: Trace[1413816014]: —“About to Encode” 0ms (08:33:47.877)
Sep 16 08:34:26 truenas systemd-journald[736]: [🡕] Suppressed 64438 messages from k3s.service
â–‘â–‘ Subject: Messages from a service have been suppressed
â–‘â–‘ Defined-By: systemd
â–‘â–‘ Support: Debian -- User Support
â–‘â–‘ Documentation: man:journald.conf(5)
â–‘â–‘
â–‘â–‘ A service has logged too many messages within a time period. Messages
â–‘â–‘ from the service have been dropped.
â–‘â–‘
â–‘â–‘ Note that only messages from the service in question have been
â–‘â–‘ dropped, other services’ messages are unaffected.
â–‘â–‘
â–‘â–‘ The limits controlling when messages are dropped may be configured
â–‘â–‘ with RateLimitIntervalSec= and RateLimitBurst= in
â–‘â–‘ /etc/systemd/journald.conf or LogRateLimitIntervalSec= and LogRateLimitBurst=
â–‘â–‘ in the unit file. See journald.conf(5) and systemd.exec(5) for details.
Sep 16 08:34:26 truenas k3s[18193]: time=“2024-09-16T08:34:26-07:00” level=info msg=“Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:6443/v1-k3s/readyz: 500 Internal Server Error”
Sep 16 08:34:30 truenas k3s[18193]: time=“2024-09-16T08:34:30-07:00” level=info msg=“Waiting for API server to become available”
Sep 16 08:34:30 truenas k3s[18193]: time=“2024-09-16T08:34:30-07:00” level=info msg=“Waiting for API server to become available”
admin@truenas[~]$