I messed up, or, how i lost access to my webgui

TrueNAS General
1

Just going to put it all out ther. I usd chat gpt to help me with my server and now ive lost acess to the web gui and all my apps are dead

Heres a rough timeline

Afew weeks ago:
I was trying to remove a dataset that wasnt wanting to be deleted and had to do it from the shell. I used gpt to help me. One of the things i did was turn off docker and turn it on again

A week ago:
Ive been trying to install afew new apps and none of them has been installing. I thought it might be related to the week before turning docker on and off and during this process i discovered Kubernetes was not installed. I again used gpt and it asked me to unmount and mount something. I managed to get it installed and i thought it had returned the system back to how it was but now i have no gui and all my apps are dead.

Today:
If i turn the machine on and off i can acess gui for around 40 seconds.Ive had the machine turned off while i worked up the courage to post here asking about what i might have done. I dont have ssh enabled but i will be able to access the command line by plunging a monitor to the machine.

I deserve any mockery i receive for this

problem with my truenas

I had to reinstall k3s
Did a bunch of mounting and remounting
Was in readonly mode
I lost access to the gui on the url via 404
restarted and have gained access
now all my apps are stopped

Concerns
Apps dont work
Im not sure if ive damaged my system and want it to be running properly

Alright — let’s go through this step by step to help you get your TrueNAS system healthy again.

:gear: What likely happened

Because:

  • you reinstalled k3s (the Kubernetes cluster TrueNAS SCALE uses to run Apps),
  • did mounts/remounts (especially if the pool was readonly),
  • and had filesystem issues,

your TrueNAS apps stopped. TrueNAS SCALE apps depend heavily on:

  1. ix-applications dataset on your pool (stores k3s config, charts, data).
  2. k3s cluster running properly.
  3. Correct mount points for your apps.

If any of these were disrupted, apps won’t start.

:white_check_mark: Let’s do a basic health check

1. Check the pool status

Run from shell or SSH:
zpool status

Look for errors or DEGRADED states.
If it says ONLINE, good.

2. Check ix-applications dataset

Run:
zfs list | grep ix-applications

You should see something like:
tank/ix-applications

If this dataset is missing or corrupted, apps will not work.

3. Check k3s status

Run:
systemctl status k3s

It should be active (running).
If not, try:
systemctl restart k3s
and then check again.

4. Check TrueNAS Apps service

Run:
midclt call chart.release.query

This will list all app releases and their status.

5. Try to restart apps

Go to the Apps → Installed Applications page in the GUI and try clicking Start All.
Or from shell:
midclt call chart.release.start

for each app.

:wrench: If it still doesn’t work

  • Run:
    midclt call app.get_status

to see global status.

  • Check logs:
    journalctl -u k3s -f

while trying to start an app.

:stethoscope: Check if your system is otherwise healthy

You can also run:
midclt call alert.list

This will show any system alerts TrueNAS knows about.

:bulb: What if the apps are still broken?

Depending on the logs & dataset health, you may need to:

  1. Reconfigure the Apps pool in the GUI (Apps → Settings → Choose Pool).
  2. As a last resort:
    • Backup your ix-applications dataset (or your app configs if you can).
    • Delete & recreate the ix-applications dataset.
    • Reinstall apps.

:mag: Not sure what damage you’ve done?

Run:
zpool status
zfs list
df -h
systemctl status k3s
midclt call app.get_status

and copy the output here.
I can help you analyze if your system is fundamentally damaged or just needs the k3s/apps stack rebuilt.

:white_check_mark: Want me to write a summary “full health check” script for your TrueNAS so you can paste it in and get a quick report?
Say so and I’ll prepare one for you.

aaand its gone again

Alright — let’s not panic.
We’ll step through a systematic triage, so we can figure out why it’s going away again.

:ambulance: Quick emergency checks

Run these from your TrueNAS shell (via SSH or the console):

:fire: Check your pool

zpool status

:fire: Check your datasets

zfs list

:fire: Check if k3s is still running

systemctl status k3s
systemctl restart k3s

:fire: Check for TrueNAS UI & middleware health

midclt call system.info
midclt call app.get_status

:memo: Can you copy-paste the output of these for me?

zpool status
zfs list
systemctl status k3s
midclt call app.get_status

With that, I can give you a targeted plan.

:zap: If it’s completely gone (no GUI at all)

systemctl restart middlewared

or
reboot

:white_check_mark: Want a full health dump script?

If you want, I can give you a one-liner like:
echo “…”; zpool status; zfs list; …

cant acess gui and done have ssh

:white_check_mark: Got it — that’s super important context.
If you can’t access the GUI and have no SSH, we’re down to direct console access.

:rocket: Use the local console

Go to your TrueNAS machine physically (or via IPMI / console-over-IP if it’s a server).
When you’re at the TrueNAS console screen (the blue menu), use option 9) Shell.

:hammer_and_wrench: From the shell, check these

zpool status
zfs list
systemctl status middlewared
systemctl restart middlewared
systemctl status k3s
systemctl restart k3s
systemctl restart nginx

:stethoscope: Next steps

Run & copy outputs:
zpool status
zfs list
systemctl status middlewared
systemctl status k3s
df -h

IPMI?

:computer: What is IPMI?

It’s a remote management system on many servers — lets you:
:white_check_mark: Power on/off
:white_check_mark: View sensors
:white_check_mark: Open a remote console like you’re physically there.

:desktop_computer: How do I use it?

Go to your server’s IPMI IP address in a browser, login, click Remote Console.

:house: What if I’m on a home PC?

You probably don’t have IPMI.
Then plug a monitor & keyboard into the TrueNAS box.

:white_check_mark: If you want, I can also output this chat as a text file (like this).

2

Better title would IMHO be, “I messed up, because I trusted ChatGPT”

ChatGPT is an LLM. It isn’t intelligent, it isn’t AI (or at least what we used to call AI. Since we destroyed the meaning of that word, we now have to use AGI for real intelligence). It just babbles what seems context appropriate.

It is great for things that don’t have to be true. Something like: “Hey, tell me a bedtime story for my small kid. I want it to be about a rabbit and his best friend a crocodile”.

I would never use it for anything TrueNAS or ZFS related.
I would also not use it for anything that requires context. “AI” does not understand context.

Since you did not ask for any help, my guess is that you just wanted to share your story and are not asking for advice?

2 Likes
3

Which version of TrueNAS are you running? Kubernetes has been removed and just installing it might break your system as you found out.

Your data is probably all there - I’d recommend a clean reinstall and importing the pool, then manually recreating the configuration.

2 Likes
4

Yeah, this is such a mess that I recommend you reinstall TrueNAS.

But before you do:

  • Verify what version of TrueNAS you’re running ie. CORE version X or SCALE 2#.##.##.
  • Doublecheck if you encrypted your storage pool(s). If you did and you used key encryption, ensure you have copies of the keys saved somewhere safe (not on the server).
  • When you eventually reinstall, make real sure you’re not selecting your storage pool drives when choosing device(s) to install TrueNAS on. That’s an easy way to inadvertently wipe data.

Do not use Chat GPT or similar chat systems for advice again, it’s ill suited for it no matter how much certain big companies try to push it down your throat.

2 Likes
5

It never ceases to amaze me that people trust AIs (Artificial Idiots) who literally hallucinate answers.

They are, in essence, just an much more advanced version of the 1964 chatbot Eliza - they gobble up information from the internet without any intelligence to weigh up and discriminate between those sources of information that are sensible/correct/true and those that are stupid/incorrect/false. Then they regurgitate the nonsense they accumulated based on a pattern recognition match (which can be spot on, close, or just a vague match), and sometimes they use vague pattern matching to combine two sources of information (each of which may both either correct or false) and come up with something new (so called hallucinations, which are more often than not likely to be false).

In this case they didn’t know that they needed to ask which version of TrueNAS you are running (to know whether you are using Kubernetes or Docker), they didn’t advise you to backup key information (like your configuration file), etc. etc. etc. because its not their data, they have never actually run TrueNAS, they have no conceptual understanding of risk and risk mitigation etc. etc. etc.

And that is why I call them Artificial Idiots.

A genuine non-Artificial Intelligence (aka experienced TrueNAS Community Member) would know to ask these things, and give risk-aware advice.

2 Likes
6

Forgive me, I definitely welcome guidance from anyone willing to help. I don’t know what to do next

7

I’m on the latest version.

I think your right about reinstall. Is there a way I can get the data from it before reinstalling?

Maybe from plugging a monitor to the machine and scess the shell?

8

I’m in the latest version of scale.

I have a raid arrangement. If I disconnect my drives will I be able reimport the pool or do I need to move the files elsewhere first?

9

I certainly have learnt my lesson. I am an idiot for thinking it was an appropriate way to learn about the machine

10

You can import the pool with all data intact, if you did not setup encryption.

Only make sure you do not accidentally perform the fresh installation to one or more of the data drives. So it might be a good idea to temporarily disconnect them.

But that’s exactly why TN mandates a dedicated OS/firmware drive. Your data is not at (much :slightly_smiling_face:) risk.

1 Like
11

Hello again everyone

I just wanted to ask a question before doing a reinstall.

Is there anyway to save an image before reinstall? Is there a quick temporary patch I could look into to get more time with the GUI

If I do a rollback to a older version, will that help? Or upgrade to a newer one?

Failing all that I’ll be doing a standard reinstall. Obviously I’ve got no one to blame but myself but I’ve spend hour upon hours getting my apps perfect and now ive shot myself in the foot

12

Thank you, pmh

I’ll disconnect them and insure the SSD is the boot drive.

Not encrypted Thankfully

13

Apps should just “reappear” once you import your pool and set the app pool in the app settings.

For everything like users, shares, etc. you can export the configuration and store it somewhere safe.

14

You say in your original post that you can gain access for a minute or so - would this be enough to download a copy of your configuration file?

(Once you are back up and running, implement one of @joeschmuck’s scripts so that you get sent a backup copy of your configuration file by email once a week.)