I doubt anyone, consumer or enterprise, asked for “lets nuke the current stable VMs and introduce something experimental with limited featureset and less freedom”.
LXC/LXD was very appreciated by a lot of people, ofcoarse. I think we can all agree on that.
But literally no one asked for this whole VM change debacle.
However:
I do understand, from a tech maintenance perspective, how great it would be to push everything through incus. Dont take me wrong on that, but that doesn’t make it part of the end-user Wishlist.
And part of the problems are not iX fault either, a lot of the issues with incus VMs are due to the close-minded design of incus. Not iX at all!
Things like incus deciding how storage layout and even zvol settings have to look. Basic things all how us prefer freedom in.
Could iX spend ages working against the incus upstream? yes!
But no-one would benefit for that.
Could iX spend even more ages on getting incus to play nice with people wanting freedom? Yes, but that wouldn’t be of direct interest to all of us: the end-users either.
Close minded? Incus has some defaults but it easily allows you to do anything you want.
You want to configure QEMU in some special way? Just use raw.qemu, raw.qemu.qmp or even raw.qemu.scriptlet. You can do anything you want with QEMU. Instance options - Incus documentation
Or if you dont want Incus managing your ZFS zvol you can just use path on host and point to your zvol and then manage it however you want. incus config device add <instance_name> <device_name> disk source=<path_on_host> [path=<path_in_instance>]
You can do anything you want with Incus if you take the time to actually understand it and read the docs.
So please dont say Incus is close-minded when its not true.
And the Incus dev is actually nice and adds things users request so working against the incus upstream is also false.
Next time when you tell someone they dont know their shit, ask questions first.
My point is the fact that if you want all that raw stuff, one (ix in this case) could just-as-well save themselves the effort of migrating.
and there is no “I”, when iX is managing incus. That was the point.
Close minded defaults, yes.
In my opinion.
Just because you cán override things, doesn’t mean it’s a good idea. thats what I meant with iX having to override all sorts of upstream stuff, to the point of it not being worthwhile.
But close minded was not the right word indeed, I meant “opinionated”, sorry I’m not native English and just responding on the side.
The aggression is completely uncalled for. Guess you’re an incus dev? Otherwise the aggression is even more completely bonkers.
There are many more things that Incus manages that make it worth it, no just the few raw stuff you may need.
If you want to support both LXCs and VMs Incus is the best option.
LXC is made by the same guy who develops Incus.
Yes, Incus is opinionated. Laravel is also opinionated and its the most popular PHP framework.
The point being opinionated is not bad thing.
It would be bad if it forced you to do something certain way. But it offers workarounds so if you need anything different you can do it.
The raw.qemu is workaround because it allows you to add certain QEMU devices that Incus doesnt support. And it doesnt support them because of security, so this opinion actually has some reason to it.
And if you need them, using few QEMU commands is actually not that hard.
In this specific way its kinda more flexible than libvirt by being able to easily directly configure QEMU while still being able to use Incus for other things.
And the path on host option in Incus isnt even workaround. Its just something Incus offers. So if you want to manage ZFS yourself you dont need any workarounds.
And no, I am not Incus dev and I am sorry if it came out as aggresive.
I just dont like people blaming a tool when its unwarranted.
The tool actually works well, you just need to understand it.
The Incus integration simply isnt finished. That why some things doesnt work. But this is not a fault of Incus. Incus itself is ready and works, it just needs some time to be properly integrated into Truenas.
Before that happens it makes sense to just keep using libvirt. But its wrong to say Incus is somehow fundamentally bad for Truenas.
Keeping in mind that the features of libvirt were also never fully implemented or finished in Scale. There is nothing wrong with libvirt, been around forever and works very very reliably. One could also use neither, and, simply use qmeu. I understand you have a strong preference to Incus. Nothing wrong with that.
Good point.
I would say my main preference for Incus in this situation stems from its dev being also the dev of LXC.
libvirt is management layer for:
LXC - Linux Containers
OpenVZ
QEMU/KVM/HVF
VirtualBox
VMware ESX
VMware Workstation/Player
Xen
Microsoft Hyper-V
Virtuozzo
Bhyve - The BSD Hypervisor
Cloud Hypervisor
Thats quite a few things.
On the other hand Incus is management layer for:
LXC
QEMU
And what are Truenas requirements? QEMU and LXC.
So they can either choose Incus which focuses only on the two things Truenas actually wants and its dev also being the dev of LXC which should make it great for managing LXCs.
Or libvirt which also focuses on ton of other things which is nice but not so focused.
And from this point of view Incus seems to me like the better choice because its dev time is actually spent on things Truenas wants.
But this is just the reason why I have that preference.
I get it, I am preferring libvirt myself for selfish reasons. In Truenas, you have a bunch of users, many of whom (maybe even mostly) are non technical. They get a version running, and, after immense effort and time and questions on reddit, these forums, chatgpt (gasp), etc. it works at last! They are so happy! Now, IX decides it wants Incus as like you they feel it is better than libvirt. So, they rip out the old interface and throw in a partial Incus interface. But even if it were a semi full Incus interface (seriously doubt it will ever be full as they have to limit some stuff), there would still be grief for those existing users I suspect.
For those non technical users, Scale has been a wild ride. So many technical changes.
I am not against Incus at all really. Like many here, just think both should have been in 25.04 as it soon will be. Rather than spinning up a test Scale in a VM, one could have then tried their existing VMs in Incus since both are on the system and play around as time permits until it works. Though many would still delay until it’s (if ever) removed. As a technical user myself, I will probably still spin up a Fangtooth VM once it’s ready for libvirt again. I will not be testing just VMs, but Fangtooth as a whole.
I understand. After all, the most important thing is the user experience no matter what is running in background.
And I agree that this switch was quite bad user experience.
I just want to say, that this bad experience isnt because Incus couldnt do all the things users need and want. Its just because its integration wasnt given enough time.
So I agree that for now reverting to libvirt is good idea.
Then Incus integration can continue in background without affecting users. And after its really done and ready everyone can try it.
Then it should be easier to tell if there really is something fundamentally wrong about Incus or if this bad experience was just incomplete implementation (like I believe).
There is the rub. With change comes the need to learn the “new” way - whatever that is. We all have limited time and for some (like me!) a lot of effort is needed to make things happen that the demigods here would maybe need 10 minutes for.
Step by step tutorials like the ones @stux, @yorick, or @dan have published go a long way in illustrating not only the general process of getting something done but some of the subtleties also.
Where I’d really like to see improvement for that matter is the currently usually-semi-useless inline help buttons in the TrueNAS webgui that say plainly obvious things like “this turns on local networking” without links to documentation that explains why a user might or might not want to use that feature.
Often, I find I need a very large screen or a collection of screens to help me navigate what I need to be setting up on the NAS in one open window while having multiple other windows open to reference help guides, TrueNAS documentation, and so on.
Libvirt is collection of software that provides a convenient way to manage virtual machines and other virtualization functionality, such as storage and network interface management. These software pieces include an API library, a daemon (libvirtd), and a command line utility (virsh).
An primary goal of libvirt is to provide a single way to manage multiple different virtualization providers/hypervisors. For example, the command ‘virsh list --all’ can be used to list the existing virtual machines for any supported hypervisor (KVM, Xen, VMWare ESX, etc.) No need to learn the hypervisor specific tools!
Sometimes, it feels like these (i) information placeholders were inserted with malicious compliance?
I wish the pop-up windows contained links to the relevant sections in the help documentation to better explain what each setting means. Ie much better integration by design. Presently, the WebGUI and the help documentation seem to evolve separately instead of being integrated from the start?
In my opinion, this topic has digressed from the subject at hand to become a B session with no real purpose. I am checking out of watching this thread. Have fun…
