Well you can’t really test the mouse if the screen doesn’t update can you? 
1 Like
Are you sure that is a file we provide? I checked our typical location for those overrides and don’t see any libvirt/bhyve files we ship with that default.
Again, patches welcome if you think you’ve pinpointed that somewhere our build is the one causing the problem here.
I can prove it … 
On your TN 13.3 beta edit /usr/local/lib/python3.9/site-packages/middlewared/plugins/vm.py, go to line 922 that looks like this:
f'vncserver={attrs["vnc_bind"]}:{attrs["vnc_port"]}',
Duplicate the line, place a comment sign in front of the copy, edit the other one so now the two lines look like this:
f'tcp={attrs["vnc_bind"]}:{attrs["vnc_port"]}',
# f'vncserver={attrs["vnc_bind"]}:{attrs["vnc_port"]}',
Clear the python cache (not sure this is strictly necessary):
rm /usr/local/lib/python3.9/site-packages/middlewared/plugins/__pycache__/vm.cpython*
Restart middlewared:
service middlewared restart
Restart your VM. Look up the VNC port from the UI. Use an external VNC viewer to connect to <IP of your NAS>:<VNC port>
Voila! Plain bhyve VNC without extras just works.
This procedure breaks the builtin web VNC viewer but it clearly shows that the problem is with some addition(s) by iX and not with bhyve.
I’d rather (similar to the shell) have no web VNC than no VNC at all.
Kind regards,
Patrick
2 Likes
Type man bhyve on a stock FreeBSD 13.3 - the vncserver parameter simply is not there.
I have e.g. FreeBSD 13.3 installed in all my jails. So:
truenas# iocage console cloud
cloud# man bhyve
[...]
lpc LPC PCI-ISA bridge with COM1, COM2, COM3, and
COM4 16550 serial ports, a boot ROM, and,
optionally, a fwcfg type and the debug/test
device. The LPC bridge emulation can only be
configured on bus 0.
fbuf Raw framebuffer device attached to VNC
server.
xhci eXtensible Host Controller Interface (xHCI)
USB controller.
[...]
Agree? The feature does not exist in FreeBSD. And removing it from the configuration breaks WebVNC but heals plain VNC. So I think it is legitimate to assume that iX added it at some time specifically for WebVNC.
Although I do not see the necessity to change bhyve. One could ship a VNC client that runs in the browser and call it a day. For some reason you decided differently - which is perfectly valid - but that is the state we have now.
Kind regards,
Patrick
1 Like
So you have investigated it, great! Sucks that the solution is a choose between WebVNC or direct VNC and that makes sense since when we originally added this VM capability that was the way to expose it to a web client. Again patches welcome. We don’t support bhyve / VMs on CORE at all, so we won’t really be validating it ourselves, we’d need community members to confirm it fixes the desired behaviors. And then be prepared for any complaining and fallout that results
Alternative is to work on upstream bhyve to maintain some improved backwards compat with that feature/setting which I presume worked on prior versions of FreeBSD back when TrueNAS was 13.0 or earlier based.
Could you bring anyone on board who worked on that WebVNC feature and might provide some pointers? For example why a bhyve modification was chosen over a plain Javascript VNC viewer?
I’m working on an Apache Guacamole install recipe for jails/TrueNAS, anyway. We could add a pointer to that documentation so people can use Guacamole if they do not want to install a local VNC client on their desktop.
The original authors are no longer at iX, so that may be difficult. If there is some big desire for this functionality to remain in CORE, it will be on community folks to investigate and submit patches if they want to change its functionality. I don’t recall why they did it that way originally, but usually there’s always a reason when an engineer does more work than shipping something that already exists. Usually because some feature or other functionality was critically lacking at the time.
1 Like
In TrueNAS CORE 13.0-U6.1 a VM with a VNC device translates to bhyve command that contains a device defined,for example, as:
-s 29,fbuf,vncserver,tcp=192.168.0.99:18966,w=800,h=600
If the framebuffer in FreeBSD 13.3 no longer works with/requires the vncserver keyword, does that mean changing the middleware vm,py code to generate a byhve command minus the vncserver keyword can still work with the novnc proxy?
Changing the middleware - as outlined above by me - to use tcp instead of vncserver makes direct VNC connections from a desktop VNC viewer to bhyve work again in 13.3 BETA, but drops support for the novnc thingy entirely. With the vncserver in use, both are available, but none of them works (no screen updates).
The framebuffer in FreeBSD never required the vncserver keyword as FreeBSD only ever supported the tcp keyword and direct VNC connections. novnc seems to be a TrueNAS addition.
1 Like
Apologies @pmh, I didn’t read the thread thoroughly. You’re absolutely right about the vncserver keyword use, in plain FreeBSD it is just the keyword “tcp” and your workaround works.
My knowledge of FreeBSD is limited, but I have “novnc” running in FreeBSD 14.1 connecting to a VM created with vm-bhyve where connecting from both the web via “novnc” or via a vnc client work.
So what has Ix got wrong here with novnc?
1 Like
That’s exactly what I set out to find out 
1 Like
@pmh After a bit more testing, the workaround does allow “novnc” to work but the colour order appears swapped from RGB to BGR. Not surprisingly, this is true for both TrueNAS CORE and FreeBSD. Perhaps this was something iX had fixed by introducing their vncserver bhyve option.
I see iX has added a related post in the BETA.2 thread TrueNAS CORE 13.3-BETA2 is Now Available - #73 by Captain_Morgan
It may be that was part of the reason we had to do the vncserver option back in the day. Again, I think we ourselves don’t recall the “why”, but there was a pressing good reason(s) to do so back then.
From our perspective though this is really barking up the wrong tree. If you want to seriously run VM’s on TrueNAS, then version 24.04 or later is your best (and supported) bet. 13.3 we aren’t actively investing time and resource into VM issues. There are nearly 2x the number of people running VM’s on SCALE than on CORE, so that’s where we’re going to focus time and effort on fixes.
Appreciate the answer, but Patrick’s workaround allows me to stick with TrueNAS CORE as I only need normal VNC connections to work on TN CORE 13.3 to ensure I can install/use certain VM’s.
I don’t do anything serious with VM’s, but I’d prefer a Linux platform that used standard networking with the option to use vlan aware bridges, openvswitch, with full access to virsh, virt-install and associated tools.
There may be twice as many SCALE VM deployments than on CORE, but I wonder how many of those VMs are being used just to run docker?
Yesterday i have installed the beta. Today i just have received this alert:
TrueNAS @ truenas.local
New alerts:
The following system core files were found: smbd.core. Please create a ticket
at https://ixsystems.atlassian.net/ and attach the relevant core files along
with a system debug. Once the core files have been archived and attached to
the ticket, they may be removed by running the following command in shell: ‘rm
/var/db/system/cores/*’.
Is something to care about? In case, what i need to attach on ticket?
Sorry if is a newbie question, but i didnt find anything on this post about this smdb.core file
Apparently it’s from an abandoned project by Marcelo Araujo:
https://reviews.freebsd.org/D11768
Alexander Motin imported it into TrueNAS and also commited the last change about 5 months ago:
Since the original project has been abandoned upstream I’ll concentrate my efforts on how to fix the colour switch with a plain fbuf,tcp configuration.
@kris any comments?
Edit: that was easier than I thought:
So it’s probably a bug in upstream bhyve, unless there are TrueNAS specific changes in the standard framebuffer implementation, too.
I’ll check if it can be reproduced with a plain FreeBSD bhyve installation and NoVNC. If yes, we need an upstream fix.
Anyway I suggest the general direction should be to apply my “hack” to the middleware and document the colour switch problem in the release notes so at least users get working VNC back, even if the colours are wrong. Whoever relies on VNC can then use a desktop client to connect.
I tried your hack and it works.
unfortunately no shell in the GUI, so I had to restart the system.
Color issue still there
You can use the “Report a Bug” link at the top of this page to submit a Jira ticket. After you submit it, you’ll get an automatically generated comment with a link to a private upload service. Use that to upload a debug file from the system and any files you find in in /var/db/system/cores/.
1 Like
@pmh I already mentioned yesterday plain FreeBSD 14.1 bhyve + novnc has the colour order problem. Surely it’s a bhyve problem?
Looks like it, thanks for confirming. So as a first measure I will propose changing the middleware to drop the TN specific option.
Can you file a report at bugs.freebsd.org?