Poll: Why do you use VMs on TrueNAS?

TrueNAS General
,
1

I’m genuinely curious why people use VMs on TrueNAS.

If you don’t use VMs on TrueNAS, please don’t vote. I don’t use VMs, personally.

If you use at least one VM, you can vote for multiple choices that apply to you.

“The reason I run at least one VM on my TrueNAS server is because…”

  • I need to run Windows-only software on my server
  • I need to run a graphical desktop on the server (SPICE)
  • I need to run software not available from the “Apps” catalogs
  • I need to run custom, patched, forked, or unofficial software
  • (Other reason not listed. Please share in your reply.)
0 voters

Don’t read these until after you voted.

Click here if you voted for choice 3.

Did you know that if an “App” is not in the official or community catalogs, you can still install custom Docker apps? This has a significant reduction in overhead and system requirements compared to virtualization. No need to allocate CPU cores or RAM. The software also gets direct access to the local storage. No need to use a network filesystem to access your ZFS datasets.

If you already knew about this, what is the reason you’re still using VMs for reason number 3?

Click here if you voted for choice 4.

Did you know that with an LXC container (“Linux jail”), you can install an entire Linux distro with a complete filesystem, network stack, and package management system? You can install and update anything in this container, the same as if you were using a Linux server. This has a significant reduction in overhead and system requirements compared to virtualization. No need to allocate CPU cores or RAM. The container also gets direct access to the local storage via “mountpoints”. No need to use a network filesystem to access your ZFS datasets.

If you already knew about this, what is the reason you’re still using VMs for reason number 4? Are you still waiting for this feature to be better integrated into SCALE beyond version 25.04?

2

Here is a great example of a user who saw immediate power-savings by ditching their VM. Yes, they moved from Core to SCALE, but I believe the VM was likely the main reason why power consumption was notably higher before.

3

I manage a TrueNAS CORE system for my parents, and use a VM on that for Pi-Hole, as it won’t run (AFAIK) under FreeBSD. I’ll probably upgrade that to Fangtooth in the not-too-distant future, and then run Pi-Hole under Docker instead. In my own home/lab, I use Proxmox for VMs; TrueNAS VM facility is far too basic.

4 Likes
4

I currently use ESXi to run VM’s however once 25.04 is released, I fully plan to use jails. I previously used jails in CORE and will be getting back to jails soon.

2 Likes
5

Why do you use VMs on TrueNAS? That’s a good question.
I’m currently testing Incus so I don’t die an idiot. I currently have 10 applications in Docker, and they all run very stably and with few resources.
For the moment, I don’t see any reason to use virtual machines. I haven’t voted.

1 Like
6

I am not running 25.04 as it’s not stable (25.04.2) so that leaves me with Eel. Docker Home Assistant has a lot of limitations and extra maintenance to install things into it (addons). HAOS takes care of everything for you. Therefore, I run HAOS.

I used to run a Win 10 VM, but, no longer do. I don’t have any Windows machines.

I used to run a MacOS VM, but, no longer do as that’s what I am typing on so not needed.

I WILL be running a Fangtooth VM once .1 comes out to prepare for the eventual update.

Couldn’t care less about power usage, cpu usage, memory, etc. Not even remotely an issue for me. I WILL be using HAOS on Fangtooth eventually.

1 Like
7

Haha I’m the weirdo running Windows 11 VM and Docker Desktop. Gotta say though, works like a charm. I smelt a rat early on with truecharts, so learnt docker and never looked back.

My NAS runs in a full tower using a I7-8700 and 128G ram, so I’m not one to worry about power much.

2 Likes
8

I’m running:

2 small Ubuntu server VM’s on my TrueNAS Core for docker, each on a different vlan. These docker vm’s are used for applications that dont exist as pkgs in jails.

1 VM for Home Assistant OS even though this is currently off as I have it hosted on proxmox at the moment whilst my TrueNAS Core server is off most of the time until I get its mini server room built.

I’m hoping to get my server running full time again someday. I’m also hoping to move over to scale and go for less VM’s and maybe even upgrade the hardware.

Only when I have the time and the cash :rofl:

Once I get better hardware later on, I will probably run a windows 11 vm for sandbox testing and perhaps even look into AD or something.

1 Like
9

I run Windows and Linux based workloads in VMs on TrueNAS CORE. I could use any hypervisor but I prefer TrueNAS by quite some margin, because of open source, ZFS, snapshots and replication.

2 Likes
10

I didn’t vote because I don’t run VMs on TrueNAS.

I run VMs on other machines for the same reason that I use VMs rather than piling everything onto one or two servers natively: I’ve found over the years that while parting functionality out to different VMs means that there are more things to track and maintain, it’s also MUCH simpler to manage through an upgrade cycle. Rather than having to plan and implement a migration of a bunch of services at once (which can result in procrastination and letting the problem build due to the time requirement), using VMs allows me to do it piecemeal and make incremental progress.

So coming back to TrueNAS, I use it for what it’s best at (a NAS), which simplifies diagnosis, upgrades, and lift-and-shifts there and for dependent services.

2 Likes
11

I use VMs because I run applications that benefit from running directly on TrueNAS rather than on a remote hypervisor and don’t have an App available (e.g. borgbackup). Other VMs run on XCP-ng.

In addition, I have more control of what I’m running on a VM. I can setup networking, containers, security, etc exactly how I want.

Also, bear in mind VM is the virtualisation feature more stable in TrueNAS Scale. Despite the change from libvirt to Incus it is still running QEMU underneath it. Export your zvol to an image, import that image into another QEMU based hypervisor and you are back in business.

This works since Angelfish. I can’t say the same about the other virtualisation solutions so far.

I guess everyone works with what is more familiar to them. Although TrueNAS VMs are not as feature rich as a dedicated hypervisor, so far they have been solid for me.

I may consider migrating some of these applications to LXC containers once this feature is more mature, but I see no compeling reason to do it if my server have enough resources to run them in VMs.

12

Actually i’m not running any VM on TN, but i was under Core (Alpine Linux). Simple reason, it grant me a better migration from jails to docker.
So i didn’t vote :smile:

I have experienced the same. Honestly, not so much as reported (if i well remember, just ~5w)… But also to mention that from Scale sidegrade

  • i don’t have an iGPU anymore (upgraded in the same time of the sidegrade from an i3 7100T to a 1260L v5)
  • my 2 NVME started literally to sit in a low power state that they never enter before with Core
1 Like
13

Cause I’m dumb. There are many better and more efficient ways. I just follow the blind.

2 Likes
14

For me personally my homelab has always been a lite version of what I did professionally. For several years I paid VMWare for VMUG so I had all of the fancy bells and wistles, and even ran a Horizon instance at home for a while. But then Steam Link became good and my wife never used her VDI so it was a waste of cycles.

I had always been using ESXI+ TrueNAS for iSCSI, e.g. I always had two or more servers. Then Angelfish dropped and I wrote out my vmks to zvols, turned off that extra ESXI server and bobs your uncle. I reduced power by removing half my compute load and I’ve been doing VMs on SCALE ever since. Some of those same VMs I’d been running in ESXi are pre covid and still running in fangtooth.

I hail originally from the K12/Municipality space. No one there used Docker or containers, and mostly everything is the Wonderful World of Windows. I still maintain an entire Windows domain with multiple domain controllers in my house, so thats part of why I use VMs.

I also host production services in VMs like DNS, my firewall, a syslog server, homebridge, and other critical infrastructure that I would never trust inside of a Docker container. DNS, DHCP, are all seperate VMs hosted on differant TrueNAS hosts thereby reducing the impact of any one TrueNAS going down for an upgrade. My wife, while she does complain about the electric bill, never compains that “the internet is down”.
:rofl:

Apps and Docker are great for something you need to spin up fast, but I am a VM guy through and through.

2 Likes
15

HomeAssistant, and using a VM for HAOS specifically, seems to be a recurring pattern. Is it really not feasible to run HomeAssistant in a “Linux jail” (LXC container)?

If LXC containers have first-class support in SCALE 25.10+, and can be easily created, reviewed, and managed in the GUI, you wouldn’t consider giving it a try to see if it can replace HAOS?

I don’t use HomeAssistant, but I can’t imagine it requires that much command-line tinkering after you install the packages and configure the services and settings.

Looking at their official documentation, someone could make an “installer script” for HomeAssistant on Debian, which should work in an LXC container (“Debian jail”) in TrueNAS SCALE.

Everything after installation seems to be done via their web UI.

If you ever get less dumb, I think you’ll be pleased with “jails”. What Core did for me, with its FreeBSD jails, only further pushed me away from virtual machines. “Jails” in SCALE means access to even more software, due to the underlying OS being Linux instead of FreeBSD.

I can understand those who need VMs for Windows software or to use a graphical remote desktop. However, I think it’s overkill to use VMs for software that is native to Linux.

With TrueNAS SCALE, if you want easy installation? The “Apps” catalogs or custom Docker Compose has you covered.

If you want a full operation system without the overhead, then “jails” and “LXC containers” have similar flexibility, but without the overhead or restrictions of VMs. Zvols of a specific size for a “virtual drive”? Don’t need to use them. NFS or SMB on the VM, just to access ZFS datasets on the TrueNAS host server itself? Not needed. Pure, native, direct access to your datasets with “jails”.

If you’re using a VM to “install and run an application”, but it’s available for Linux, then you probably need to rethink your approach.

Don’t worry. I’m not picking on you, @Chris_Roberts. I’m speaking about VM users generally, who might not realize that what they are doing can be accomplished with something leaner than VMs.

16

Voluntary Vendor Lock-In? :laughing:

17

“Enterprise License Agreements” :grimacing:

1 Like
18

Just painting with a broad brush here. Home Assistant really needs mDNS (multicast) to work properly and talk to all of your IOT devices. Historically, this has been far easier to accomplish by setting up HAOS in a VM where it inherently has its own network stack, including its own network interface with its own MAC.

Things like passing thru Bluetooth/Zigbee/Zwave dongle have also historically been easier to accomplish with a VM or a physical box. This issue kind of transcends TrueNAS :rofl: I know people who solve these types of problems by having random RasPis all over the place doing things.

I’m sure things a better now than they were previously and the old wisdom of just using HAOS isn’t as strictly necessary as it used to be. I’d expect it should work pretty easily in an incus container.

Still, I’d continue to advocate for running Home assistant in HAOS in a VM. The devs also make a lot of configurations and stuff easier by bundling in packages and scripts and things of that sort. I’ve always liked the idea of importing an entire preconfigured OS in an OVA file to configure a virtual appliance. Home assistant is just as much an appliance as TrueNAS is, and I feel like they should manage the OS for the same reasons.

I’ve actually P2V and V2P migrated HAOS a few times. Being able to move VMs around to metal or to different hypervisors is another big reason I advocate for VMs.

As an aside, Home Bridge is just easier for my wife to deal with since it’s all just native Apple-y. Home Assistant can do this now too, but seems like overkill

2 Likes
19

HAOS is their officially supported method of running HA. Everything else is discouraged.

Scroll down to the “Advanced installation methods” section.

20

Ok, so go here:

Look at the chart, notice, the HAOS column had everything checked, the other columns do not. While it is possible to run stuff using the other methods with more work, it’s much simpler and simply works better to use HAOS.

HA replied somewhere, that they do not support LXC. Here is one piece of it (note “supervisor” runs in HAOS also, part of it):

Yes, you can run the HAOS docker container, but again, you lose things.

HAOS is soooooo simple., you literally do nothing with the VM ever, it maintains itself. It’s the best way.

Here’s a direct quote from a case on github dealing with lxc:

“We don’t support LXC - please don’t use that or make no issue reports if you do that”

Yes, people have got it to run in lxc with effort and there is doc out there (unofficial of course), doesn’t always do everything right and it’s unsupported. Even if it did, doesn’t matter, unsupported = no go for me. There’s a lot of complexity to HA with dealing with smart devices.

So, given all that, why do you care if I run a VM, what’s wrong with that? Why do you think I am losing something, what’s the angle you are getting at? That I lose 5 watts? Ok, 1.5c/kwH, 5 watts = 3.6kwH in a month = almost a nickel, gee. I have solar, and, already saved what I paid to install it, so, 1.5cents/kwH is what it costs me average over the course of a year.

2 Likes