I was looking for replacement for my old home QNAP-TS509 (smb storage) and now i have bright new AOOSTAR R7. It has lots of CPU and RAM and i can move all my home lab VMs from my home VMware host.
The options to choose OS from were XPenology, Proxmox and Truenas. XPenology is creepy by design, Proxmox is very bad as a storage so the only option left for me was TrueNAS.
In my life i did lots of migration in any directions between VMware, Hyper-V and Proxmox (with ZFS as the main filesystem). So when i see how-to convert VMware VM to TrueNAS it makes me sick. Itâs 2024 now! Why do we need to manually determine exact disk size, create zvol and convert vmdk to raw? In Proxmox itâs one-liner: qm importdisk VMid VMDKfile zvolume --format raw. Even in Synology there are WebUI options for importing and exporting VMs.
TrueNAS virtualization is simple only if your are starting from scratch. But even in that case you have no simple way to backup and restore VMs. In Hyper-V there is Windows Backup, in Proxmox there is Backup feature, in VMware there are a lot of commercial and free tools. And the TrueNAS forum says âuse snapshotsâ. Snapshots are not a backup. Backup is a file. File which can be easily copied to the external drive, remote server, uploaded to S3 and so on.
So now i have a question what is a use case for virtualization on TrueNAS?
Which you can do with snapshots and ZFS replication, just saying.
How so? Iirc on CORE you could easily manipulate the zvol⌠I suppose itâs the same on SCALE.
Which is more important to you: virtualization, or storage? Proxmox is vastly better for the former, but surely not as strong on the latter. TrueNAS is pretty much the inverse.
Nonsense.
By what definition?
Itâs one line in TrueNAS too.
Would be nicer if it was zero lines.
âŚbut to answer your question, IMO itâs âas minimal as possibleââbut then, I have a four-node Proxmox cluster I mainly use for virtualization. Iâll likely be running a TalOS VM on TrueNAS to migrate my TrueCharts apps, but I may end up moving that to Proxmox as well.
I agree. Snapshots are absolutely a backup, albeit a backup with certain characteristics that make it primarily suitable for restores in-place and unsuitable for remote esp. off-site backup to non-ZFS boxes i.e. incapable of ZFS replication.
However ZFS Snapshots can be far a more efficient way of holding backups because the only disk space used is for blocks that have changed.
And ZFS replication to another server can be far more efficient than file backups because it uses snapshots to work out what block have changed and replicate only those blocks.
A backup does NOT have to be a file, however a backup to a file is definitely a backup, one with different characteristics to a snapshot.
I would imagine that you can probably use qemu-img (or even dd) to convert a Zvol to a VMDK as a means of backing up a VM Zvol to a file. And you can presumably pipe that into e.g. zip or similar to compress it still further if you wish.
(I suspect that qemu-img would be more intelligent about e.g. not backing up unused portions of the Zvol than dd.)
IMHO the choices that iX Systems made of what to expose to the GUI, and what not, are a bit random.
Which really goes back to the question of what constitutes a backupâsee, e.g., Is snapshot and replication a âtrueâ backup?. And there doesnât seem to be much agreement on that questionâthough saying that a backup must by definition be a file is just silly.
I think a more helpful way to look at the issue is to consider the threats/risks against which youâre trying to protect. Some possibilities:
- Accidental file deletion
- Drive failure
- Pool failure
- Catastrophic system failure
- Natural disaster
- Malicious data destruction
Snapshots on the same pool will protect against the first and possibly last (depending on whether the attackerâs able to destroy the snapshots). RAID will protect against the second (âRAID is not a backupâ is, at best, incomplete). Snapshots replicated to a second pool in the same machine will cover the first, second, and third, and possibly sixth. Once you replicate them to another systemâperhaps geographically-distantâyou potentially address all of these risks.
While this doesnât give a binary answer to âis X a backup?â, I think itâs a more helpful way of looking at the question.
I wouldnât consider snapshots alone backup[1], but snapshots + replication is totally a backup.
the one that saves the day when a drive (or worse) fails to collaborate. âŠď¸
I think @Dan has the right concept:
- What are the risks to your data that you are trying to protect against?
- Any protection that allows you to recover your data when a risk you are protecting against comes true is IMO âa backupâ.
And you donât need to have a hardware problem to need to recover, and (@Davvo) equally I would say that a redundant disk in a RAIDZ vDev is NOT a backup despite it saving the day when a drive fails.
So if you want to protect against ransom-encryption and are not worried about having a separate copy on another machine possible in a remote location, then a snapshot constitutes a backup.
What is YOUR usecase? Is this home use, SMB, Enterprise, etc? How Many VMs? Are you a VMWare customer looking for a âfreeâ option?
In my life i did lots of migration in any directions between VMware, Hyper-V and Proxmox (with ZFS as the main filesystem). So when i see how-to convert VMware VM to TrueNAS it makes me sick. Itâs 2024 now! Why do we need to manually determine exact disk size, create zvol and convert vmdk to raw? In Proxmox itâs one-liner: qm importdisk VMid VMDKfile zvolume --format raw. Even in Synology there are WebUI options for importing and exporting VMs.
Iâve toyed with creating a migration assistant tool with a simple Python script. I have parts of it built and it works, but havenât gotten it into a state worth sharing. It effectively just does exactly what you are describing manually, but with prompts at the terminal. The value here is limited, because itâs something you only have to do once.
Perhaps make a feature request for importing VMDKs in the WebUI. Latest Feature Requests topics - TrueNAS Community Forums
I wrote about using TrueNAS SCALE as a hypervisor, and have done some cool stuff with it.
Using TrueNAS Scale as a Hypervisor
Using TrueNAS SCALE to build a âDatacenter In a Boxâ
But its not VMWare or Proxmox. Itâs a storage device that can do virtualization. Iâm sure more love will come to the hypervisor, but it hasnât been a focus of development.
But even in that case you have no simple way to backup and restore VMs. In Hyper-V there is Windows Backup, in Proxmox there is Backup feature, in VMware there are a lot of commercial and free tools. And the TrueNAS forum says âuse snapshotsâ. Snapshots are not a backup. Backup is a file.
Couple of things here. You can replicate the ZVOL (and all of its snapshots) for your VM to another location, either on a differant pool on the same system, or on a remote system. This is rather trivial and easy to do.
What you canât do, however, is do a backup of the VM configuration, which may have some issues with licenses on certain software. The BIOS Serial Number and MAC addresses are generated when you create the VM, and if it changes (like creating a new VM and pointing it to the ZVOL), then evil software may break.
Again, if youâd like to see this feature Iâd happily vote for backing up the VM as an entity (including its configuration file and all ZVOLS/disks) you can make a feature suggestion. Iâd happily vote for this.
File which can be easily copied to the external drive, remote server, uploaded to S3 and so on.
You can. With dd, make a container for your ZVOL in a file. You can also back it up to tape by just dding the ZVOL to a tape.
Using LTO Tapes to backup your TrueNAS?! - Resources - TrueNAS Community Forums
You can zfs send a snapshot and pipe it to a file, possibly to a non-ZFS drive or host.
You skipped the part âdetermine exact size of vmdk disk and manually create zvolâ. Or qemu-img does this magic?
Funny. I donât do that part.
I just make a zvol big enough⌠not even sure how to determine âthe exact sizeâ of the vmdk.
I guess I just look at the vmdk size with ls and round it up a bit to give some free space.
Seems to work well. Must be magic involved.
Canât help the OP on their original question, but I am a fool who feels they need to say something on the snapshot/backup debate.
Snapshots alone are not a backup. If your snapshots rely on the original underlying filesystem/medium/pool/storage/whatever-word-you-want-to-use, it is distinctly not a backup, because it is inclusive of the original system.
A git repo alone is not a backup, it is version control. You must make an entirely separate and distinct copy of the git repo on another machine before you can in any way approach calling that repo âbacked upâ.
I think you could ask 10 IT professionals âWhat is a backupâ and get 20 distinct answers. Thereâs so many attributes to consider - is the âbackupâ portable? Is the âbackupâ OS independent? Is the âbackupâ on two or more separate data storage mediums/systems? Is the âbackupâ able to be taken offline? Does the âbackupâ have the ability to form chains of fulls and incrementals/differentials? Is the âbackupâ encrypted with separate keys than the original system? Can the âbackupâ be restored reliably to a foreign system? Can the âbackupâ be tested independent of the original system? Can administration of the âbackupâ be done without administrative rights to the original system?
Snapshots can be used as part of a backup process, but snapshots alone are not backups.
A few things.
Yes, you can easily backup VMs, itâs called replication in zfs. A snapshot is needed to send a copy, which gives you your files. There are also many free tools for backups, not only do I use replication, but I also use Kopia. a NAS is about data, and, data integrity is important. Thatâs why zfs is used, and zfs is much different than most filesystems.
I suspect your major problem is itâs simply different, you will have things to learn of course.
I have 4 virtual machines for various purposes, all run great, one of them is HomeAssistant which I use as a VM instead of an app as I can run HAOS. The use case is the same as any other VM I suppose.
You say you canât backup the VM config!? There are several ways to do this. One way is simply virsh dumpxml.
If itâs not for you, donât use it is my advice.
Proxmox (aka PVE) has the same ZFS abilities as any other modern Linux, maybe not as good in lower level drive management from web interface, but under the hood it has pretty much the same version of the same ZFS implementation, AFAIK
âŚwhich is pretty important for a storage server. But even more important is the availability and management of sharing protocols, which are completely absent in PVE. Can it be made to do them? Sure. Can you get a GUI for it? Yeah, you can install Webmin or Cockpit, or probably other tools. But at that point you really arenât running PVE any more.