Is there any way for me to make Incus use a Zvol of my choosing for the root drive that it installs (in my case) Windows Server on? At first I thought this was a simple yes and that to accomplish this I would simply use the GUI to import the premade (empty) Zvol for Incus to install the virtual machine onto, however it turned out not to be so simple. At this point I have tried numerous combinations of things but below I will detail my thought process and what I assumed would work.
As said above I imported the Zvol then quickly realized upon running the windows server installer that the default 10 GiB root drive was still there and my imported Zvol did not replace it. I expected this, however what I did not expect was that when I selected my imported Zvol of 350 GiB for windows to install to, it installed successfully however TrueNAS doesn’t display that any space is being used in the Zvol (windows displays over 200 GiB being used by the C drive), in fact, it’s just blank as seen in the attached screenshot.
Upon seeing this I asked in the Discord about this behavior and a user instructed me that Incus simply copies the data from your imported Zvol to a hidden dataset rather than using it where it is.
This is concerning to me because I recently needed to access data from my replication task that was supposed to be replicating the snapshot data from the Zvol that I had assumed the Virtual Machine was using as its root disk, however to my surprise the replicated Zvol only had 105 KiB of data contained in the snapshots. The data was backed up elsewhere, however I would like to understand if my expectations of being able to store the actual virtual machine data on a Zvol of my choosing is flawed or somehow not the way Incus is intended to operate. And if it is the case that it’s possible to make Incus use my chosen Zvol as its installation location for the Windows Server, how do I do that? And if not, where is the location of my Virtual Machine data so that I can create a replication task for that location so I don’t have any more data loss.
This has been confusing me for the last few days, any help is GREATLY appreciated.
Thank you for your time.
In Truenas Fangtooth, which uses Incus under the hood, before creating any instances, you are required to specify the pool in which instances will be stored, and then Incus stores all zvols for instances in that pool in a folder that is not visible in the Truenas UI. There is no way that I’m aware of, at least using the UI, to changes this, and all instances must be stored in the same pool.
This is my understanding based on testing Fangtooth on a spare computer, as I have not yet upgraded my production servers to Fangooth, but I’m pretty sure that’s how it works.
Thank you for your post, I performed the steps outlined and the virtual machine booted successfully but now when I go to check in the “Datasets” tab and I navigate down to my instances dataset in which the Zvol I created was stored, it is now missing… Is this expected behavior?
I understand by performing this command zfs rename Gamma/applications/instances/windowsServer2025Ark Gamma/.ix-virt/virtual-machines/windowsServer2025Ark.block
I am moving the location of the Zvol to where Incus expects the root devices to be, however my whole intention was to make Incus use the Zvol where I created it so that I can then create a replication task for my entire applications dataset, not copy the data from the Zvol and move it back into some Incus specific location.
If I am not mistaken this is practically the same issue I was running into before, the data is ultimately stored in a separate location managed by Incus that is independent from my Zvol, I never had any data from the VM in Gamma/applications/instances/windowsServer2025Ark to copy in the first place, I am creating a whole new VM.
Should I just create a replication task for Gamma/.ix-virt/virtual-machines/ for the time being if my goal is to have a replication task to backup snapshots of my virtual machines?
It partially depends on if you selected “Clone” or “Move” for the original zvol, but essentially “yes”. Stux’s steps essentially completely relocate your zvol to be where the original root zvol was, which is of course under the hidden Incus specific dataset and is to be managed entirely by Incus and not you any longer. If you have cloned your original zvol, it would still be there and just not used, with the implication being that you’d delete it once you’re sure it’s working in its new location.
It might be possible to modify the Incus config to use a zvol at an arbitrary path, but this would be unsupported and likely not persist reboots and/or upgrades. IX has stated than in later versions there will be proper UI features to enable snapshots for zvols managed by incus, though I’m not sure about replication.
It is a bit annoying that it’s now separate and not managed the same as all of your other datasets/block storage, but technically nothing stops you from snapshotting or replicating the zvol as you had before, but it would all need to be done via scripts or the shell as of course it won’t show up in the UI anymore for snapshots or replications tasks, so it isn’t quite the same.
I just snapshot my whole system pool using the UI currently, which does include that zvol. The same could be done for replication, but then it is you’re entire system pool…
Thank you for the response, it seems the desire for snapshots and replications is heard by the TrueNAS team and is being actively worked on for Goldeye. I suppose for now ill just take snapshots of the entire pool just in case anything happens until now and Goldeye being released. Patrick submitted a question for the podcast regarding snapshots and their answer suggested to me that the functionality of replication will also be included / worked on for Goldeye. Here’s hoping!
