Setting up TrueNAS in Combination with a Hypervisor

TrueNAS General
1

Hi guys,

A bit of noob question, from a happy TrueNAS user. I have a hardware box on which I’m running Windows HyperV for the moment. I’m running a mix of Linux and Windows VM on this server but have no strong dependency on Windows. For some reason beyond the scope of this post, I ended up with a lot of disk capacity on the server. Concretely I have 6 2TB M.2 SSDs in the server: one of them is used for the basic installation of HyperV, one of them contains the virtual disks and 4 of them are available.

In an attempt to make good optimal use of the extra disks, I would like to run TrueNAS on this platform as a secondary NAS. My first attempt was to use the passthrough feature of HyperV to assign the disks directly to a HyperV VM and run TrueNAS on that. I don’t like that setup at all. The way HyperV handles the disks is cumbersome and HyperV itself has a large footprint making the whole setup rather wasteful. Also, backups are not so easy.

So I started thinking about introducing Proxmox in the equation and, also based on a recommendation of a friend, I came up with the following setup:

  • Add two SATA SSD disks in a (ZFS) mirrored config to the server
  • Install Proxmox on these
  • Create a Proxmox VM and pass all the 6 SSDs to this VM
  • Install TrueNAS on that VM using one of the ZFS RAID options
  • Make a big chunk of storage from that TrueNAS available (using SAN or NFS -not sure which is the best) to store the disks of the other Proxmox VMs

Is this a good approach? Are there any pitfalls with this?
The main question is whether I need Proxmox. I could do the same with TrueNAS Scale. Are Proxmox and TrueNAS Scale really equivalent as Hypervisors?
Any feedback would be appreciated.

2

It’s a generally good approach so long as you’re passing through the raw PCIe devices for the SSDs. Note that TrueNAS needs a dedicated boot device (which, in this application, could be a virtual drive under Proxmox).

Of course, there’s a bit of a chicken-and-egg problem in using a VM to provide storage to the hypervisor for other VMs.

They aren’t even in the same universe.

1 Like
3

Thanks that helps a lot.

In terms of “chicken-and-egg avoidence” could it be a good idea to use the SATA SSDs (where also Proxmox would be installed) for the TrueNAS boot device?

Another question about the interdependency: when the Proxmox host reboots, it will shutdown all VMs, so I could end up with a situation where the shutdown of the TrueNAS VM yanks away the storage from the other VMs. How big is that risk?

4

A virtual drive on them, sure.

Significant. You can specify startup order for VMs that start on boot, but I’m not sure about shutdown order.

5

I must admit I just switched from Proxmox with virtualised CE to using CE on the hardware. All the time I had only two VMs in Proxmox: CE and ElastiFlow. Now I have two in CE: ElastiFlow and Windows 11 (for testing).

Performance and management looks good so far and I only have one system to monitor, update, backup etc.

Spice is ok for installation. Using RDP for Windows and of course SSH for Linux.

6

Oh now give us the explanation of “different universes”.

If that was the case, I would be running Proxmox myself, but there is nothing that I’m missing from PM.

The GUI? Sure, TN has more work to do now that they decided to use virtio finally,.
Speed? They BOTH use Linux, ZFS and virtio, correct?

We want to know what’s on that “other” universe…

7

I would not virtualize truenas for a production system, it will work but has a some issues if the virtualization layer fails.

It is recommend to always put TN on the bare metal as that is what it is designed for.

8

Even releases that do not break VMs?

1 Like
9

“We”?

If you’re asking the question, the odds are you either wouldn’t understand the answer or you aren’t asking in good faith. But on the off chance that neither of these is true, here are some some ways that Proxmox VE is just in a different category from TrueNAS, as a hypervisor:

  • Most importantly, it’s a mature hypervisor. It’s been around as a hypervisor for a long time. It doesn’t have a history of “break all your VMs in a point release,” which iX has done repeatedly.
  • Supports clustering (you know, the thing that was going to make SCALE, SCALE)
  • Supports clustered storage
  • Supports network-attached storage
  • Supports backing up and restoring VMs and LXCs
  • They don’t engage in the nonsensical wordplay of “we’ve released this, but we don’t recommend anyone actually use it.” When they put out a release, it’s because they believe it’s stable for general use.
    • This doesn’t mean, of course, that their releases are bug-free or perfect. But they don’t pull the nonsense that iX does on their software status page, where they typically don’t recommend $RELEASE for general use until the eve of $RELEASE+1.

Now, if none of these things matter to you, that’s up to you. But I think they’re a good start to explaining why the two products are just in different worlds, as hypervisors.

2 Likes
10

I saw opinions on the proxmox forum that this is rather “they believe it’s stable for gamma testers (aka no-subscription users)”. Nevertheless, your point stands.

1 Like