Incus VM copy/rsync, import, export, custom zvol root

Incus VMs are basically black boxes at the moment with zero control over them. You cannot import VMs, you cannot export VMs, you cannot back them up to another system via rsync, you cannot use a custom root.

Please, add the above functionality. At least for me, personally, Incus VMs are completely useless as-is. I am not going to use VMs that I have so little control over or that I can’t even back up.

The vm part for incus is still a WIP state and my guess is that these features will be available, the question is just when.

Maybe they have already planned to include such features, maybe haven’t. I wouldn’t know. I can only go by what I see and make feature requests based on that.

I agree, a lot of the same problems also can be found in the container part of the current incus implementation. I really hope they make those features available in the full release (preferably for both VMs and containers).

There is reference to more VM enhancements coming in the Goldeye release (24.10) in T3 episode 10:

14:44 Fangtooth - New Incus VM Changes in Nightly

I dont want to create another post for this so I will just post here.

Another pain point can be current location of Incus in /var/lib/incus which can get wiped upon Truenas upgrade.
Because of this any directly created configuration (outside of Truenas UI) can be lost. Mentioned here: Fangtooth Unifies the TrueNAS Community Editions - #125 by Stux

This could be solved by:

  1. Making /var/lib/incus mountpoint/symlink to another location on ZFS. Kinda discussed here: IncusScripts - #7 by bketelsen
  2. Setting INCUS_DIR environment variable which sets location of Incus data dir. Environment variables - Incus documentation

I like how current Docker apps allow both configuration in Truenas UI but also manual configuration either directly by YAML or through separate apps like Dockge and Portainer.

I wonder if some similar balance could be achieved for Incus.

3 Likes

@Foxtrot314

Would a manually set up cronjob that sets custom config options with incus config set on boot be a workaround for this?

Maybe, basically using script to reapply anything that is lost during truenas upgrade. But seems like too much work.

Another solution could be truenas allowing custom configured instances in UI, similar to how you can use custom yaml for apps.

Yea I agree that would be very nice, but I’m pretty sure this won’t get added in 25.04 so I’m searching for a workaround until it’s added. Mainly to set core.metrics_address in the incus config.

Well, you can add custom configuration pretty easily into Incus. Just write it down so you can redo it if it gets wiped in upgrade.

But no need doing it after every restart.

I would just do it in a cronjob after every reset to make sure it’s always applied. Also I’m lazy and don’t want to manually change the config when it breaks. :sweat_smile:

I played with Incus briefly last night and while I can make Incus use a custom dataset for all of its data, I haven’t yet been able to figure out a way of using passphrase encryption with it. With the old VM mechanism, you could just use passphrase encrypted datasets without an issue, but with Incus it’d clearly need more messing around.

Alas, my VMs are on encrypted datasets and now I have this sinking feeling about this whole thing that Incus is going to be major headache for me…

There are options for boot.autostart etc

Seems to me you would need activate the dataset first, then the vm.

boot.autostart just makes the container/VM fire up when Incus is launched and isn’t relevant to what I said.

I can delete the default storage and make a new one using an encrypted dataset, yes, but after reboot even after I unlock the dataset and restart Incus, it’s lost all of its configuration and won’t find any of the VMs or containers.

My intention was that you’d attach your external encrypted dataset to the instance.

The instance only accesses the dataset when it is started.

If it is not auto started then you can unlock the dataset before starting the instance as part of a script, or manually.