VMs out of zpools, data in Zpools

TrueNAS General
1

Hi.
I am going to setup a new TrueNAS with 2x2Tb nvme disks + 4x20Tb hdd.
Data will go into the HDDs using RAIDZ2.

I would like to use the 2nvme for booting and host truenas but also take advantage of the free space to run some VMs on it to make them performant.

Is it possible? How? When creating a VM I see no option to use hostpath, I only see datasets.

2

Hi,

I am afraid thats not possible yet. Path for VM disks have to live in a data pool.

3

I’d suggest adding a 32-256GB boot SSD or two, and then use the nvme to make a regular data pool.

1 Like
4

Being a complete proxmox newbie maybe it is too dare.
I am wondering if this would be possible using Proxmox in the host:

  • install proxmox in a 100Gb raidz/mirror partition in the nvme disks
  • manually create two extra partitions with the rest of the disk /dev/nvme0n1p3 and /dev/nvme1n1p3
  • create a vm and passthrough the 4 hdd disks AND the two partitions (1)
  • install TrueNAS in the VM and within Truenas:
  • create a zpool raidz/mirror with the two nvme partitions
  • create a zpool raidz2 with the 4 hdd data disks

zpools could be created in the proxmox host and passedthrough but as far as I know TrueNAS create datasets/folder on them, so better let it do ti all by itself.

What do you think?

(1) kvm virtualization - KVM Pass through disk Partition - By Label or UUID - Server Fault

5

Do NOT passthrough HDDs or partitions: Passthrough SATA/SATA controllers and whole NVMe drives (i.e. the NVMe controller). ZFS assumes, and requires, full control of drives.
So your scenario is:

  • Get an additional (NVMe) drive to install Proxmox onto.
  • Install TrueNAS in a Proxmox volume on the boot drive.
  • Pass through the chipset SATA controller (Proxmox does not use any SATA drive) and the two 2 TB NVMe to True NAS.
6

Thanks for your reply @etorix
Since I don’t think there is room for extra drives, I think think the best approach will be:

  • Install proxmox in the two nvme disks mirrored, 100Gb partitions
  • Create zfs partitions for the rest of the nvme space (1.8Tb)
  • Install Truenas as a Proxmox VM (on its nvme volumes)
  • passthrough the SATA controller to the TrueNAS VM (so data and iscsi targets will be handled by truenas)
  • Create other proxmox VMs in the free space nvme disks where VM disk performance is important
  • Create other VMs on TrueNAS where the disk space is important

With this approach I think all is covered:

  • Backup proxmox partition snapshot to HDD (and offiste)
  • Backup of Proxmox VMs (truenas and others) to HDD (and offsite)
  • Backup of data disks only offsite since they are already in zraid

What do you think?

7

Still do not pass partitions rather than whole drives/controllers. Use one drive for Proxmox (or TrueNAS bare metal) and the other as single drive vdev.

Better: Check again whether you really have no room for extra drives—including a SSD on a USB adapter for use as boot drive.

8

That is what I meant; promox vm will use nvme disks themselves, so creating a TrueNAS vm passingtrhough the controller will do.

There is no space in this servers for more disks. Checked.