Hi all, I’m struggling with an upgrade issue where I moved from TrueNAS-13.0-U6.7 to 13.3 and my VMs stopped booting up and I could not manually start them. I have reverted back to 13.0 U6.7 and everything is running again.
The reason I went with the update to 13.3 is that there is no mouse functionality through Bhyve and I had read another post that mentioned a fix for that was earmarked for 13.3 if that makes any sense.
Not sure what is broken between the two versions or if there is some fix it work to be done after the upgrade? Any pointers would be great!
I upgraded again today to see if there was anything else I could provide. One thing I noticed was that each time I attempted to start either of the previous VMs and also the new one I just created, there is a quick circle dialogue like when it’s trying to start and then it goes away. At the same time, I get the following lines in log file as it happens.
The vnet name ticks up one number each time regardless of which VM I try to start.
Feb 20 16:45:50 kernel: tap0: changing name to ‘vnet18’
Feb 20 16:45:50 kernel: vnet18: promiscuous mode enabled
Feb 20 16:45:50 kernel: vnet18: promiscuous mode disabled
Feb 20 16:45:51 kernel: tap0: Ethernet address: XX:ff:bc
Feb 20 16:45:51 kernel: tap0: changing name to ‘vnet19’
Feb 20 16:45:51 kernel: vnet19: promiscuous mode enabled
Feb 20 16:45:51 kernel: vnet19: promiscuous mode disabled
Feb 20 16:45:51 kernel: tap0: Ethernet address: XX:ff:bc
Feb 20 16:45:51 kernel: tap0: changing name to ‘vnet20’
Feb 20 16:45:51 kernel: vnet20: promiscuous mode enabled
Feb 20 16:45:51 kernel: vnet20: promiscuous mode disabled
Feb 20 16:45:51 kernel: tap0: Ethernet address: XX:ff:bc
Feb 20 16:45:51 kernel: tap0: changing name to ‘vnet21’
Feb 20 16:45:51 kernel: vnet21: promiscuous mode enabled
Feb 20 16:45:51 kernel: vnet21: promiscuous mode disabled
Feb 20 16:45:52 kernel: tap0: Ethernet address: XX:ff:bc
Feb 20 16:45:52 kernel: tap0: changing name to ‘vnet22’
Feb 20 16:45:52 kernel: vnet22: promiscuous mode enabled
Feb 20 16:45:52 kernel: vnet22: promiscuous mode disabled
@imcintyr Since I do not have this problem anywhere on my 13.3 systems, I guess we should compare VM settings. Please post everything - settings, devices, their settings … for one of the VMs.
I’m having similar issues. Everything worked in 13.0-U6.7, but VMs are not starting in the 13.3 build. I have tried 13.3-RELEASE and 13.3-U1.1
Here is the log entries I get starting the VM
Feb 22 17:40:56 freenas kernel: tap0: Ethernet address: 58:9c:fc:10:ff:f5
Feb 22 17:40:56 freenas kernel: tap0: changing name to 'vnet2'
Feb 22 17:40:56 freenas kernel: vnet2: promiscuous mode enabled
Feb 22 17:40:56 freenas kernel: vnet2: link state changed to UP
Feb 22 17:40:57 freenas kernel: pid 5417 (bhyve), jid 0, uid 0: exited on signal 6
Feb 22 17:40:57 freenas kernel: vnet2: link state changed to DOWN
Look into /var/log/libvirt/bhyve for a corresponding log of non-booting VM. Mine stopped booting after upgrading to 13.3 since I was using iGPU passthrough. I had to rebuild TN 13.3 from source with specific bhyve patches to regain this functionality.
Thanks for the reply @pmh, your posts have been very helpful over the years. And thank you as well @betelgeuse, I am not sure I can rebuild TN as you suggest but it helps to know others have experienced issues.
Before I send the settings, and I am not sure if this matters, but do you want me to send the settings as seen from 13.3 U1.1 or 13.0-U6.7? I have both on the boot pool so I can easily switch just let me know.
Thanks, below are the settings of the HomeAssistant VM I am mainly concerned about, ‘haos’. Hope this is the right format for the settings. Wasn’t sure about how to export them.
Blockquote
haos
RUNNING
Virtual CPUs:1
Cores:2
Threads:4
Memory Size:2.00 GiB
Boot Loader Type:UEFI
System Clock:LOCAL
VNC Port:35790
Com Port:/dev/nmdm1B
Description:Home Assistant
Shutdown Timeout:60 seconds
Edit NIC 19
Adapter Type: * Intel e82585 (e1000)
MAC Address 0d:a3:60
Nic to attach: * em0
Device Order 1002
Edit DISK 20
Zvol */home_assistant
Mode AHCI
Default Disk sector size
Device Order 1001
Edit VNC 21
Port * 35790
Delay VM Boot Until VNC Connects [No]
Resolution 1024x768
Bind 0.0.0.0
Password
Web Interface
1002
Edit PCI 28
PCI Passthru Device * 1/0/0
@betelgeuse Thanks and yes, VMs and not Jails. Not sure where the jails came from but this has been about VMs only.
I see you pass through PCI device. Is that a iGPU? If so, remove it, it is not required to run HAOS. TN 13.3 is missing crucial patches to run bhyve iGPU pass through. That is why I have build it myself.
The passthrough is a PCI USB controller so that HAOS can see the Z-wave stick/hub. That’s a requirement for this whole thing so if it tanks the use of 13.3 I will just need to wait where I am until it gets a proper update.
When I strip out both the VNC and PCI devices from the hoas VM it still won’t start. Oddly, I clicked on the ‘Shell’ in the lefthand nav of TrueNAS (13.3), it blinked like it was going to open but then didn’t, and the button on the left hand nav for ‘Shell’ disappeared. SSH works just fine though, and I have attached the logs here.
‘MintyFresh’ is a new Linux VM of Mint that I set up in 13.3 to see if that would work and if so, I would just rebuild all my VMs if needed. But the new one won’t even start either. Still has the image in the CDROM waiting to install.
Let me know what other info I can provide and again, I appreciate the attention to this from you.
Looking at the logs … are you sure you thread per core setting (CPU topology) is correct? bhyve is not starting complaining about vm_set_topology in 2_linuxvm.txt:
