Truenas boot up hangs on importing pool after hostname change

TrueNAS General
1

Hi there. I’m new to truenas, but I do believe that I understand basic concepts.

Assembling my new NAS took some time. I’ve installed truenas before I finally received the drives for my main pool. And I’ve managed to update truenas a couple of times. So, before I’d created my first non-boot pool, I’d updated truenas from 24.10.2 to 24.10.2.1.

I’ve created a pool with mirrored vdevs, connected to AD, set up smb shares and transferred some backup files from my old (non-zfs) NAS. I’ve managed to back up my macbook to the TimeMachine share on the truenas. Then I’ve changed the hostname under the Network → Global Configuration. MacOS time machine wasn’t able to backup to the “new” disk (“disk already in use” or something like that), which is kinda ok. I’ve decided to reboot, but… truenas never came back online.

I’ve connected the monitor and saw that it hangs on importing pools. Ctrl+C did nothing. Neither did a single power button press. Only barbaric power button long press “solved” the issue. Did try to boot up a couple of times with the same result.

I’ve booted into the previous version (24.10.2) and managed to import my pool via GUI, which looks safe and sound (at least I didn’t see any warnings). As expected, no AD or smb shares were configured.

So, the question is – what should I do now? I mean, of course I can set up it all once more, but perhaps the problem can occur again. I’m not sure if this is somehow related to the hostname change, but I didn’t make anything “unusual” except this one.

2

Is there a reason you want / need 24.10.2.1? You can try going to 25 and see if that works.

3

No particular reason. I just think that I may have made something wrong, because deathlike-hang is definitely not a good sign. That’s why I am asking for help before I encounter it again.

As my initial question is kinda vague, I will try to be more specific:

  1. Should I consider my 24.10.2.1 boot env dead and just delete it?
  2. If so. How do I prevent this in future (as losing all the recent config doesn’t sound fun)? Should I make a snapshot of the boot dataset before/after every config change?
  3. Is changing the hostname a bad idea? Did you guys ever face major issues after that?
4

I am not an expert, but

  1. You can upgrade to 25 and still switch back to 24, so I would delete it after trying 25. Maybe someone else will have an idea what to do.
  2. I don’t think you are losing a config. Before upgrading, you should ALWAYS back up your configuration to something outside of the NAS. That way, worst case you can reinstall and reload the configuration, and are running again.
  3. Changing the hostname should be no issue, I would guess it’s unrelated.
1 Like
5

Thanks for your suggestions.

  1. Well, 25 doesn’t show up in the upgrade list, and TBH, I don’t want to be an early adopter (especially when I’m not experienced in truenas). Truenas offers to upgrade back to 24.10.2.1 (as I am on 24.10.2 now).
  2. I’ve already lost a config of AD and smb shares (as I’ve set them up after the upgrade). And I’ve lost it not during an upgrade but after a simple reboot. However, after your suggestion, I found this doc. It implies that “TrueNAS automatically backs up the configuration database to the system dataset every morning at 3:45 (relative to system time settings).” I didn’t find its Scale counterpart, but perhaps I still can restore my config. Will investigate it.
  3. Nice to hear.
6

I’ve managed to restore my config files. It looked like:

sudo mkdir /mnt/sys-conf-dump                                                                                                                              
sudo mount -t zfs boot-pool/.system/configs-<hexadecimal-abracadabra> /mnt/sys-conf-dump
# create sys-conf-ds dataset with share
cd /mnt/sys-conf-dump
sudo cp -r * /mnt/<my-main-pool>/sys-conf-ds/
# download content of sys-conf-ds share

Then I’ve updated to 24.10.2.1 (my previous 24.10.2.1 boot-env has been destroyed, btw). Then I’ve applied the restored config to my “fresh” 24.10.2.1 . Truenas went for a reboot… and then went for the second reboot… The connected monitor showed nothing, so I’ve powered off the machine with a barbaric long press.

After another power on with 24.10.2.1 I’ve seen that the boot goes to the “Applying kernel parameters” (or something like that) and reboots. Then it goes to the “Applying kernel parameters” again and hangs. Apparently that was the same 2-reboot behaviour from the initial config applying.

Conclussions:

  1. While it’s very cool that truenas backups the config daily, accessing these backups could be tedious. There is definitely room for improvement (like accessing them from the GUI).
  2. Apparently, I have config that can brick the 24.10.2.1 boot-env. Should I somehow provide it to the truenas dev team?
  3. I will try to restore truenas with configs from the previous days.
7

I’ve applied the-day-before-disastah config to fresh 24.10.2.1. That restored AD and (almost) all shares.

Then I’d changed some settings (including changing the hostname), and TA-DAH! my truenas had bricked again. Except for the hostname change, I’ve also changed the network interface’s ip from DHCP to static and its MTU from 1500 to 9000. These settings I hadn’t considered “unusual” in my original post.

So, after some extensive testing including constant cloning of boot-envs, applying settings one by one, rebooting and testing shares after each change, I hereby declare:
Changing MTU from 1500 to 9000 bricks my 24.10.2.1 boot env in a way that it hangs during boot on “Finished systemd-sysctl.service - Apply Kernel Variables.” or on “(2 of 2) Job ix-netif.service/start running ”.

Perhaps I will create another topic about MTU soon.