Hello I recently built my homelab and learning about virtualizing, to preface this I have a system that is on proxmox and I used a VM for Truenas Scale. I’ve been following the guides and managed to create a dataset that has two SMB shares and one iscsi share that I use. The issue that I ran into is after downloading some steam games and testing it out, I noticed that load times are high, sometimes it takes a while for audio and textures to load into the game. After a while the game crashes, I am wondering if this is normal behavior.
Not enough hardware details (motherboard? HBA? NIC?).
Not enough details about configuration (what’s passed through?).
Possibly not enough RAM for iSCSI.
Possibly some MX500 bug.
TrueNAS would never create ext4 partitions, so you’re passing virtual drives.
Backup, nuke the whole setup and start anew. Bare-metal would be best. If not, look for the virtualization resource here.
Yes, unfortunately, even running iSCSI you are adding a lot of latency by accessing the data over the network. Latency impacts different games in different ways. Games that load stuff into memory during loading scenes will play fine, after you finally load. Games that continually need timely streamed data all the time may perform poorly, or become unstable.
Here’s a person testing this and reporting some stats, if you want to compare to your numbers:
Also, if you’re running 1GbE on your home network, you are hamstringing your loading even further, since that’s not even giving you HDD read speeds.
Thank you for that resource, but I will need to delete my dataset for my current pool on Truenas Scale luckily I am just testing and nothing is on the drives. One user mentioned that I am passing virtual disks and that is hurting my performance. I am assuming the order would be delete the dataset on truenas and then go on proxmox and wipe the disk and reinitialize the disk with GPT?
Proxmox should not be touching these disks. You should forward the SATA controller to the TrueNAS VM. Keep in mind that any other disks on that controller will not be available to Proxmox. Alternatively, you could purchase a HBA and forward that to the TrueNAS VM…
Is this system primarily for file storage with a few VMs? You woul have fewer issues running TrueNAS bare metal, and you can run virtual machines under TrueNAS.
It’s not impossible to run TrueNAS well as a VM, but the things you’re doing are why users were advised not to for a long time
Yeah this is just a small home lab that is mainly used for myself and maybe my siblings. When I was doing research to work on a personal server/NAS I saw a lot of guides for using proxmox as bare metal. Yeah @etorix spotted that I am passing the disks virtually and I need to fix that. I am assuming I wipe the disk and then reinitialize it with GPT?
I’ve never used Proxmox, only ESXi. I’ve never used passthrough for a SATA controller, only HBAs. So…
I would guess you would pass the SATA controller to TrueNAS. I could be wrong, but I can’t see a reason you would initialize them from Proxmox. You want to pass the controller to TrueNAS, so that Proxmox isn’t doing anything with any disks connected to the controller, and create the VDEV by selecting the disks in TrueNAS. TrueNAS will initialize them…
Worse than that, this is setting you for total pool loss whenever Proxmox fails to timely record critical information for ZFS.
The order is either to go bare-metal or to pass-through the drive controller (HBA or chipset) to the TrueNAS VM. When using Proxmox you should also blacklist the controller to prevent Proxmox from ever trying to touch the drives. TrueNAS will take care of reinitialising the drives when creating a pool.
Yeah I will be going with that route, this discussion made me realize that I could have put truenas on bare metal, but before this when I googled about server/NAS and a friend recommendation that is when I found Proxmox. Luckily I found a guide to pass the physical hardware through the CLI on Proxmox. In hindsight I might have complicated this when I started this homelab, but I guess I will learn two things now. Thank yall for the advice.
Just an update on my post @etorix and @Jorsher I was not able to pass the onboard controller, but I had to pass each individual disk through the CLI. The speeds are significantly better compared to before, I transferred a large video file and it was done within minutes. Looks like this is my limited speed at 1 Gb, but it is still much faster than before thank yall for the help.
If IOMMU grouping does not allow passing the SATA controller alone, your consumer board is not suitable as a virtualisation platform. Mind that you’re at risk of irrecoverable corruption if the virtual driver misbehaves and rearrage some critical ZFS writes out of order.
