Dragonfish 24.04.1.1 in a VM, hangs on boot, can stall on shutdown

TrueNAS General
,
21

So, I have made one piece of progress.

I was unable to get my pool to mount. /dev/vdb was… well… not formatted.

Rolling back did not help.

It appears somewhere along the line with all my trials, the sector size on the disk device got set to 512B instead of 4096.

That’s one thing.

22

Sigh.

Just cloned the boot volume zvol, then did a clean install from iso of 24.04.1.1

(was never asked to re-use pre-existing config on disk)

Same hang.

Screenshot 2024-06-04 at 11.08.13 AM
Screenshot 2024-06-04 at 11.08.13 AM1668×338 16.9 KB

23

So, I clean installed 24.04.0 and it booted right up. I then uploaded a config… and it restarted with out timing out…

…

but once I upload my config… then it times out at shutdown.

must be something that’s configured.

that’s interesting.

The config is fairly minimal.

24

I had a disabled ubuntu VM in the TrueNAS VM.

deleting the VM means TrueNAS doesn’t block at restart time.

I suspect there is a bug in the libvirt stuff…

…

after deleting the VM, I am able to do an online update without the update hanging on restart… and then timing out as above.

@John

25

Update still hangs on boot

…

when it hangs, 3 cores (of 4) are spinning at 100%

658604 libvirt+ 20 0 9900.8m 8.2g 30700 S 301.7 6.5 19:54.27 qemu-system-x86

26

Its the config I think. Something is tripping up the startup when the config is applied.

And it happens on two different VMs

And if I reset to defaults… then I can upgrade… and as soon as I apply the config it will hang

27

So, that’s interesting.

reset to defaults, create root account, create a bridge, upgrade… hang.

Screenshot 2024-06-04 at 6.31.10 PM
Screenshot 2024-06-04 at 6.31.10 PM1544×206 10.9 KB

reset to defaults, upgrade… no hang.

28

setting root only… no hang.

set root and static ip… hang.

guess its time to try rebooting the hypervisor.

29

That worked btw.

no it didn’t.

back to the random freezes

Screenshot 2024-06-04 at 7.18.10 PM
Screenshot 2024-06-04 at 7.18.10 PM2124×234 12.3 KB

30

That is interesting and it seems reproducible. Hopefully you find “The” solution. You really do this forum a huge service, thank you.

1 Like
31

Wonder why it works for you, but definitely not for me. Even a clean install.

What is your Nic device?

32

Dumb question and I’m not sure if I have an idea but what does “setting root” mean. Set password, set shell? …I really dont have much of an idea but I’ll look something up and see if I can pull on that thread a little bit to see what happens and get back to you.

You think your config has something to do with it. Have you tried viewing with the DB viewer?

33

I mean setting a “root user” account password in the installer.

I had been testing by minimizing the changes from default.

The problem is it doesn’t fail 100% of the time, leading to false positives, so I had begun to think that using a root account vs an admin acount, makes a difference.

The latest results are I can do a clean install, with a boot device and no pool. Set either an admin password or a root password and it will hang. Typically while generating some keys.

When set to 8GiB of ram

But not 100% of the time.

It works 100% of the time with 24.4.0.

34

Okay, my possible dumb idea was ‘run0’ and Systemd 256. I’m going on the concept of my previous thought about “read only”.

This could be a time suck so I’d verify the versions first before going down this path.

Also, typing on my phone so not at a laptop.

1 Like
35

Well, you don’t really need dracut in the initrd at all anymore to have a working initrd. Except for certain kinds of exotic storage

Quoting pottering…

Wonder if ZFS counts as exotic :wink:

36

I shall be ordering more memory so I can embark on this journey also! See if I have the same issue if not resolved by the time I get it and install it, probably a couple weeks. I’ll probably use 16GB ram though for the VM.

I won’t be updating like you though, so it will be different than yours. I guess I could briefly try the same version as you before continuing on to Electric Eel once the docker stuff makes it to the Beta.

1 Like
37

Here’s another example of something that sounds very similar, but with Proxmox as the Hypervisor.

38

Here is a reproduction video I made.

It would be good if anyone else could reproduce…

All you need to do is create an 8GB VM with a 32GB boot zvol…

A clean install of TruenNAS will hang when running in a VM with 8GB of RAM. It works fine on all previous versions of TrueNAS (Scale, Core and FreeNAS)), including 24.04.0

In the video on a TrueNAS 23.10.2 system with 128GB of RAM and a Xeon E5-2699A v4 CPU, I setup a VM with 8GB of RAM, 4 cores, a VirtIO NIC on a bridged interface, and a VirtIO 32GB boot zvol.

I then clean install 24.04.1.1 from verified ISO, and reboot it 6 times. It hangs on startup 3 out 6 times.

I then clean install 24.04.0, and reboot it 5 times, it starts correctly 5x in a row.

The ISOs are shasum OK. I have also confirmed this behaviour on a Dragonfish 24.04.1.1 system running on a Xeon D-1541 with 32GB of RAM.

39

Bug Report:

NAS-129406 : Clean Install of Dragonfish 24.04.1.1 in a VM fails to boot 50% of the time, works with 24.04.0

40

I may have missed the real important details, among everything else, but need to ask:

Which specific vendor/version Hypervisor and any other details, not to be assumed?