So I’ve been using truenas since 22.10 and fall in love with the platform. It has all the necessary tools to do more than I need. My primary focus is data integrity. Most of the stuff is just personal photos, backups, projects that can live happily inside a dataset shared via smb. For computing stuff I use something like 3 VMs, each one with its own functions. 2 VMs are small, kinda self contained machines that relies just on its own zvol so no issue here. The third one is the media management vm.
That vm has docker inside with basically arr stack. all the Linux images takes a lot of space and can’t live on the same zvol as the os itself. The solution is to create a zvol on a hdd pool and use that storage as a disk from the vm. Here is the issue: zvol is a block device, not a file directory as a dataset. So it comes with a lot of inconveniences:
With a zvol you need to plan how much space to allocate. A dataset is dynamic in nature
A zvol is a virtual disk, so it too has to be formatted. It creates a sorta nested file system, wasting resources and unnecessary overhead.
It is hard to recover a single file from a zvol
It is not easy to know how much space is left inside a zvol from truenas gui.
So a truenas machine kinda gets 2 enviroments: datasets with file access and zvols, which are not so easy to work with
So my request is to add a well known feature called VirtioFS to enable direct access to dataset from the vm. ( there is a workaround with bridges but it is clunky at best, permissions are a complete nightmare and it still relies on smb or nfs which introduces permission issues, more overhead, so no, too fragile). With VirtioFS a vm could get direct access to files, removing all the unnecessary issues as permission, latency, overheads, stupid configurations and would set the truenas machine to a different beast: data management and compute instead of data management and workarounds to make a vm work. And not all:
No hard limit for space, you can set quotas or easily expand in the future, with way easier file migration
Higher performance or less overhead thus lower power consumption
Way easier vm management as it is indipendent from data
Probably some more useful features but it is late and I need to sleep.
Firstly a complete enviroment isolation. Any changes inside the vm stays in the vm. So I’m host indipendent.
Also, as I’m host indipendent, if I really needed, I can make a full image of the vm and run it on a different hardware. My vms are the result of years of trying and finding all the right software for my needs. I just want to be able to restore everything very easily. I did so many changes that a lot of stuff i just don’t remember, but they are there, working silently.
I’m a gui guy. Running, applying changes, or just file exploring is just so much faster with a desktop enviroment instead of cli. It saves so much time doing changes that I can see
I don’t think they plan to add it, but let’s hear the official answer on virtiofs support for TN…
Anybody?
Now, for your issue, not sure if raw files (as disks) would help your issue, but something to check.
I prefer to use raw files as they are easy to move, copy, grow, administer.
(and there is minimal/no performance hit vs zvols, as some that have never tried them, insist/repeat).
Guess I will end up running disk benchmarks on both, to prove it and shut mouths .
Of course you’re linked to the host even with virtiofs but that folder with files is way easier to manage than a zvol for example. You can share it to another machine via samba or nfs but this is beyond the point.
As for lxc, setting a gui or making a disk image is doable but it is more difficult than with a full vm. I see this as a workaround so more work.
I saw already the old post about libvirt and virtiofs, I hope for a future integration. It will make some lives much easier
.