I’m currently running a TrueNAS Scale virtualized in proxmox using ISCSI and i want to migrate proxmox away so it are 2 seperate instances.
I will be reinstalling TrueNAS Scale bare metal on the same hardware.
Question is, can i safely migrate my Storage Topology somehow without losing all of my data? information about my topology = below. (couldn’t embed pictures.)
RaidZ1
SDB ISCSI
SDD ISCSI
SDF ISCSI
Cache
SDE ISCSI
Log
SDC ISCSI
Thanks for the information. If you need any further information i’ll be happy to provide.
I am not a Proxmox expert, but providing you configured your TrueNAS VM correctly in Proxmox i.e. passed through the controller and disks as raw devices, then I would imagine that if you save your configuration file and install a new raw O/S and then restore your configuration, your server should be up and running almost immediately. This will be because the VM will have had the raw devices and so the pools it created will be as if it had been running raw.
However, if you didn’t define your VM correctly so that devices were virtualised, then all bets are off.
Raidz1 and iSCSI do not go well together. But if this means that your drives are virtual, there’s only one way out: Backup, destroy, and restore to a sane environment—the sooner, the better.
Not sure what good reason you could have to do sync writes on a raidz1…
Take some time to rethink your pool before recreating it.
Ah - yes - as @etorix says, it does seem that you haven’t followed the trueNAS Proxmox virtualisation guidelines, and have in fact done something that doesn’t follow even basic Proxmox guiidelines.
Since it appears that you have not passed through even your raw disks, then it seems highly unlikely (which is an understatement for almost certainly impossible) that you can import your pools when you start to run TrueNAS natively.
As @etorix says, you need to backup all your data elsewhere, and reformat your entire system, and then restore your data back again.
As @etorix also says, your use of L2ARC and SLOG are almost certainly also bad choices.
So when you decide to reformat you system, if you need any advice on what the best pool layout is for your use cases, do please ask here and we will be happy to help.
First of all, thanks alot for your time and advice guys @etorix@Protopia .
I run alot of home lab stuff here with multiple databases and vm’s. Had really slow data transfer speeds in the past, so i tried adding a SLOG to see what effect this had. It seemed to have solved the problems i had since afterwards my transfer speeds were capped the the bottleneck of my NIC.
Do you think the problem could have been something different?
Yes - almost certainly you were doing synchronous writes when you probably shouldn’t have been.
But your pool is VERY poorly constructed, and you will need to move it (somehow) to a new, properly constructed pool in order to fix this.
If you have more disks to construct a pool in parallel that would be the easiest.
Backing up the data to somewhere else and later restoring it is probably the 2nd easiest.
Or we can perhaps use a disk replacement strategy if you have 1 spare disk of the right size. You add it to Proxmox by defining it correctly, and we can resilver to it to replace one incorrectly defined disk. We then recycle that disk and define it correctly and resilver again. Once we have done this 3 times, you should have a pool which can be mounted. It will be time consuming, but it should be safe to do.
If you don’t have a spare disk we can do something similar but it will mean your data having no redundancy during this process, so it will be riskier.
Yes i was already doing that but it will take some time. I’m copying all my files via SMB over to a new 8TB HDD because i had no SATA slots left in my server.
Afterwards i will be reïnstalling TrueNAS Scale Bare metal and copy over all files manually due to the lack of SATA ports. It will take some time but its fine.
I didn’t see any serial numbers of passed through hard drives in TrueNAS so i made the same assumption as you guys verified.
What do you think the best storage pool layout would be with the hardware i have at hand?
What would you guys recommend for me to do in the future?
My goal is to use proxmox to virtualize everything. In the first place i’ll be using local storage of the proxmox server at one point i might run out of storage and then i want to have the ability to use storage of my TrueNAS Scale server. And ofcourse for backups.
Hardware of the TrueNAS server:
Intel(R) Core™ i5-7500 CPU
64Gb Ram
3x 8TB hdd
1x 250gb NVME
1x 1TB NVME
That depends on your use case. But with only 3 drives, options are limited. And if you only have 3 SATA ports, the hardware is not really suitable for a NAS—or you’re not disclosing everything, e.g. what you’re booting from.
I’m unsure how that fits with “TrueNAS baremetal”. TrueNAS serving storage for a distinct Proxmost server?
As @etorix has said we need to understand your use cases, but making a guess from what you have said it sounds like you have some iSCSI usage to support Proxmox and possibly some other stuff.
Here is how I would use your hardware:
250GB NVMe - boot drive - no redundancy
1TB NVMe - VM iSCSI virtual boot disks - no redundancy but replicated to HDD
3x8TB HDDs - RAIDZ1 for non-iSCSI data and replica of 1TB NVMe.
I would limit the data I put on iSCSI to the VM boot drives and VM database files (i.e. the data that actually needs synchronous writes), and move all other files accessed by VMs sequentially to the HDD RAIDZ1 pool and use NFS or SMB shares instead of iSCSI - and I would ensure that all datasets on the HDD pool have async=disabled.
If you have another NVMe slot I would add a 2nd 1TB as a mirror for the first. If you have a suitable PCIe slot, you might want to consider a PCIe board giving you an NVMe slot - and if it allows you to boot off it (which it may not) put the boot drive on this and both the 1TB cards on the MB.
