There is a workaround by using a sometimes buggy API. The ask here is to allow Proxmox to more natively be integrated into TrueNAS.
Impact
Big benefits to the vast majority of TrueNAS users if Proxmox integration was more “native” and “out of the box just works” a large amount of people have migrated to Proxmox recently after the vmware exodus.
User Story
Users who leverage Proxmox clusters as their hypervisor want to be able to use TrueNAS managed storage remotely and natively thru Proxmox’s web interface.
I’m also using the FreeNAS / TrueNAS API plugin for the moment. I just noticed that upgrading to 25.04 Fangtooth breaks the plugin completely, which forces me to stay on 24.10 until a better solution comes around. I am really hoping that this feature request gets implemented in the near future, since my setup is very dependent on ZFS-over-ISCSI.
I dont understand how proxmox does it, but if it requires it to have LIO or IET as the target implementation in TrueNAS instead of SCST this is not going to happen.
I could very well reverse the question, a lot of enterprise features are built on top of SCST and changing it to a completely different backend would be a lot of worth and risk regression, so what are the advantages of LIO or IET over SCST?
A request to implement a proxmox plugin is way more reasonable and easier than changing away from SCST.
I don’t disagree. This FR is about someone build a plugin themselves and the thing isn’t reliable, can TrueNAS just natively integrate with Proxmox?
As long as the solution allows a close integration with the hypervisor and its performant, I don’t think people would care how the problem is solved. I did file this request with the information I had available at the time which was proxmox expects XYZ protocols to be used for ZFS over iSCSI to work natively.
Apparently other companies have solved the integration in a different way, by rather than using the protocol solution via LIO or IET they just baked themselves into proxmox with some plugin which is fine solution.
If you feel renaming the title of the FR is appropriate, you have my permission to change it to Native Proxmox VE integration or w/e.
Hello,
I’m also intersted. I use ceph with proxmox as main storage in my small clusters but I still have some truenas servers that was used with vmware.
ZFS over iscsi is for now the best implementation.
Proxmox needs to use our APIs to create iSCSI LUNs and do snapshots. Doing manual config via SSH to set up luns without coordination with the TrueNAS backend is a non-starter that will result in broken systems. IIRC this happened in the past (proxmox “zfs over iscsi” causing subtle TrueNAS Core breakage). Forums old timers probably remember having to help people out with recovery after proxmox made a mess of things.
I have a shot in the dark on what might be happening w.r.t 25.04 making the plugin fail, so I’ll see if I can look into this in a few days.
As @awalkerix mentioned, it’s important that this is done through the API in a manner that TrueNAS expects. The “SSH as root” method results in Proxmox bypassing the middleware.
I cant speak for anyone else but I only used the old buggy ssh based plugin to get ZFS over iscsi for my vms. Official support, even via a plugin, is my dream come true.
That’s the issue for the plugin not working with Fangtooth. A few people are digging into the code there and tagging potential issues. I admit I don’t understand everything there, but looks like a combination of some hard-coded version strings (an easier problem to fix) and something having changed about how TrueNAS passes data between itself and the plugin (and Proxmox?).
Apparently, the plugin relies to a degree on using HTTP to pass commands/data around, and Fangtooth does that differently enough that the plugin broke.
The consensus also seems to be that converting the plugin to use the new websockets API would be less an update and more a fork/rewrite.
We have made progress resolving the issues with the plugin. I have a working patch that is promising but requires more testing. @TheGrandWazoo has gotten involved so I anticipate that the plugin will live on until the REST API is finally pulled.
Given that Trunas provides the API’s required to accomplish the mission the problem seems to lie in the proxmox implementation of zfs over iscsi and the use of iscsiadm for some of the operations.
In addition to my previous comment. We have been trying to develop a native plugin for Proxmox ZFS over ISCSI however the API in insufficient to accomplish all of the functions required for a fully functional plugin.
I’m opening a feature request to have the API extended to allow for some additional zfs dataset / zvol management.
Regarding the existing freenas-proxmox plugin, I’m confident the solution will continue to be maintained.
A fork was created and converted into pure ssh+root commands, bypassing the API entirely.
I’m working on a fork that will integrate the new websocket API. The problem is the API has gone through some changes that makes backwards compatibility a challenge.
The current recommendation, in my opinion, is to continue to use original REST API plugin created by @TheGrandWazoo up to version 25.04.0. A working patch is available to restore operation in 25.04.0 but the official package hasn’t been updated yet.
In order to honor the great work of the original developers I’m going to work with them before releasing a websocket compatible version.
I really appreciate y’all jumping down the rabbit hole to get this working. Watching the work here and on the GitHub has been really inspiring. And I’ve learned a lot about how all this works just from watching.
Could you please add a link here to the feature request for the API additions? I’m sure some of us would like to track that.
Just to clarify, the original REST API plugin’s main branch will eventually be updated with the working patch, correct?
If so, I think for those of us who haven’t set this up yet at all, waiting on the patch to go to the mainline release will be the easiest way to go.
I’m confident an update will be released at some point. I think @TheGrandWazoo is working on testing. I’m fairly new to participating in opensource so I’m being careful not to overstep.
I intend to collaborate with the original devs before releasing my websocket version of the plugin. I’ll post a link regarding my feature request once I post it.
We haven’t heard anything from @TheGrandWazoo so at this point I think we are considering the project abandoned. I have reworked the project to use websockets. I will be removing the alpha tag from my project next week.
