That would be “In Goldeneye, we keep BOTH libvirt VMs and Incus VMs and eventually we pick one”.
However, if that was the case, there would be no need for this:
If Incus-based VMs are still a thing in Goldeneye, there would be nothing to migrate.
This announcement only addresses the shorter term.
Virtualization (libvirt) re-enabled in 25.04.2 and will continue through to Goldeye.
Goldeye and beyond plans are being worked separately. We’ll announce them when we know what we can and can’t do. There are lots of moving pieces.
We understand the need to enable smoother software updates.
2 Likes
This is welcome news, as I found the current Incus implementation to be problematic in my case.
2 Likes
Seems to me this should’ve been the plan since the beginning, but I am glad to hear the original system is returning alongside the Incus implementation. I had several problems with Ubuntu and Windows Server VM’s with Instances.
The API and feature set is not stable yet, so there is something to migrate… to the next evolutionary stage.
While not topical it is worth asking for me because it ties directly to the experimental status of instances. Any idea how long Jailmaker will be able to work in TrueNAS?
Which version of TrueNAS are you operating?
I don’t know of any testing/validation of jailmaker in 25.04
Still works on 25.04 without Problems
4 Likes
Well. It worked for me. Enough that i could follow my migration video to migrate out.
It seems to work well as long as you aren’t using GPU passthrough.
1 Like
I’m using gpu passthrough in my sandbox for jellyfin and it’s working perfectly fine.
2 Likes
This is a very good call.
Incus is very opinionated in how it handles VMs, such as quite-fiercely forcing users into specific VM storage layouts. Which often does not sync well with how TrueNAS is designed to handle VM storage.
This indeed affects both portability, performance and reliability.
I’ve tested this feature thoroughly and was not happy with the quality of the VMs with the new incus backend, not just because it was hastily implemented, but also because incus just isn’t that good.
Its also YET ANOTHER opinionated middleware in-between:
TrueNAS → Incus → KVM
In most failure cases I encountered, it was the double middleware that was throwing-up, not the VMs or KVM itself.
What would need to migrate for Incus VMs is any of the configuration parameters that change between 25.04 and 25.10. The backend is the same (and even the same as libvrt / 24.10, KVM/QEMU).
Agreed… KVM itself is very solid…
Well, double middleware is the case even with libvirt.
At least Incus has nice REST API.
Only way to not use “double middleware” would be to directly control QEMU (for example via QMP) but why bother if you have nice management layer like Incus.
And if you dont like something that Incus does? Incus exposes simple tools to directly configure QEMU however you want.
2 Likes
Just as a personal note here. I’m genuinely sharing a quick user story thing as an Instances user.
I’m actually travelling for work and I’m improving my personal DNS infrastructure so I can stop dealing with DNS problems in my homelab. TLDR; Parsec/Tailscale integration into everything. I will probably make a post at some point.
Disclaimer:
From my community involvement as well as some of my own testing that I have shared in the community, I know that there have been some bugs that have prevent some users from benefiting from the current state of Instances. My thoughts and opinions on this matter are my own.
I do have a history of contributing forum posts and resource guides on some of these topics, including how I migrated from ESXI–>SCALE back in Angelfish days and some best practices in the Bluefin days, plus some cool nested virtualization stuff.
Some of you may have noticed, I do also work for the people who make TrueNAS but this post is my own thoughts,
The current state of Instances is really cool in what it can do even in the out of box configuration right now.
Quick example, I’ve been running PiHole as a VM in Ubuntu for years now. One of the things I’ve always wanted was temperature reporting in a VM. IIRC, This used to be more complicated than it should have been. I’m fairly confident this didn’t work on my VMWare VMUG 6.x host but I could be wrong. In Instances it just worked.
I just logged into the shell of my PiHole from a hotel room 6 hours away and it’s reporting the temperature correctly. The amount of engineering from all sides of the software stack Ubuntu,TailScale,PiHole,TrueNAS,MacOS,Windows over the past few years is really impressive.
Oh man… 
I recently went from TrueNAS Core 13.3 to SCALE 25.04.1 and had a “fun” time getting everything working with Incus (aka Instances) VMs, which was new to me.
I like KVM/LibVirt/Qemu a lot more since it is mature, well documented and I am familiar with it. Incus is not bad but it (yet) misses some features/options.
Will “Virtualization” (KVM/Libvirt/Qemu) stay with SCALE?
Since this will be the recommendation, will you provide the required steps for users who wish to manually migrate their VM back? In other words, how does one get their zvol back out of the hidden location that Incus imports to so it can be used to recreate the VM in the Virtualization screens?
Yes… that is the essence of this announcement. It is being re-enabled in 25.04.2
2 Likes
Yes, but the focus on that process will be Goldeye. For Fangtooth the focus is on co-existence.
Not an official response, just a point of fact. It should be theoretically possible to just use standard ZFS commands to copy the zvol for this reason among many other “DO AT YOUR OWN RISK” type reasons/solutions. I believe @Stux had a thread about this already somewhere.
3 Likes