Hey all, I’m sure a few of you have seen the plugin we are developing over on Reddit, but I wanted to make a formal post on the forum here to hopefully get more testers and feedback.
The plugin, source code, and documentation is all available here on the Github:
Here is a quick rundown:
iSCSI Block Storage - Direct integration with TrueNAS SCALE via iSCSI targets
ZFS Snapshots - Instant, space-efficient snapshots via TrueNAS ZFS
Live Snapshots - Full VM state snapshots including RAM (vmstate)
Cluster Compatible - Full support for Proxmox VE clusters with shared storage
Configuration Validation - Pre-flight checks and validation prevent misconfigurations
Dual API Support - WebSocket (JSON-RPC) and REST API transports
Rate Limiting Protection - Automatic retry with exponential backoff for TrueNAS API limits
Storage Efficiency - Thin provisioning and ZFS compression support
Multi-path Support - Native support for iSCSI multipathing
CHAP Authentication - Optional CHAP security for iSCSI connections
Volume Resize - Grow-only resize with preflight space checks
Error Recovery - Comprehensive error handling with actionable error messages
Performance Optimization - Configurable block sizes and sparse volumes
In the future we plan on adding NVMeOF support, automatic snapshot management (similar to Pure Storage “SafeMode” snapshots), API key rotation, and a few other things that you can view in the wiki under “Ideas.md”
The aim of the project is to have TrueNAS be a seamless block storage available to PVE, without all the headaches of doing it manually. Plus to have well put together documentation and guides.
Currently there is a note worthy limitation in TrueNAS itself that prevents reads from reaching their full speed over a single interface. Noted in the known limitation wiki, under the multipath section.
I’m hoping with some discussion here that TrueNAS/iXSystems can implement more knobs for iSCSI tuning on the server side. NVME-oF should in theory not have the same problems, but this has not been implemented into the project yet, but is on the list of features to be added.
If you have any suggestions, feedback, or want to help out please leave comments here or in the discussions board on the Github repo. And for those willing to try it, please try to break it! The more testing that can be done, the more robust the plugin can be! We’ve tried creating tools to help us with development (as found in the tools folder) to make sure all features work as expected. If any part of the documentation is confusing or not clear enough, please let us know!
I haven’t done a comprehensive look between mine and theirs, but I have had a few others on the Proxmox forum mention it as well, here is my response:
The goal of the project is to be as robust and documented as possible to be “enterprise grade” or at the very least, have a really good foundation for others to build upon.
Welcome to the party. Your plugin is quite a bit more complicated than mine and has a few features that I haven’t implemented yet. Multipath and CHAP being the most notable. My goal was to keep things simple and build a strong foundation on TrueNAS 25.10. I think the recent changes to the TrueNAS API will open the flood gates to many new solutions.
When freenas-proxmox broke and the original developer was not around to fix it, I figured I’d give it a go. Since open-source has been so good to me I saw an opportunity to finally give something back.
That and I couldn’t start my VMs after upgrading my home lab
In a similar vein, I have a big powerful TrueNAS system with plenty of storage on it but my Proxmox cluster, even with LinStor, doesn’t have enough room or bandwidth to do some larger projects… And I wasn’t about to setup LUNs one by one lol. I could have used NFS, but where’s the fun in that.
Still doing the current testing on my 1GbE network, but did get some of the plugin refactored to support NVMe. It is a definite improvement in performance, even on the slower networking.
I have installed the plugin on my 3 node cluster 9.2.21 . It have being running for a week now, and works wery well. The only problem i have for now is proxmox back-up. If I run backub in snapshot mode on 1 LXC with root disk and mount point it runs perfectly , It makes a snapshot and zvol on truenas do the backup and clean up deleting the snashot and zvol. But when the normal scheduled backups run, the backup completes without error but leaves LXC’s locked and the truenas is not cleaned up. Something I can do about it ??.
sounds like the scheduled vzdump finished but cleanup of the plugin snapshot/zvol died halfway, so the lxc stays locked.
id try from a proxmox node:
pct unlock CTID
(or qm unlock VMID if its a qemu)
then on truenas look under the dataset the plugin uses for leftover snapshots/zvols named after that ct/vm and delete the orphan ones the job left behind.
if only scheduled jobs do it, check whether those jobs use a different mode/storage than ur manual snapshot test. manual working + schedule locking usually means the schedule hits a mountpoint the plugin doesnt clean the same way.
Oh yes thanks I know how to clean it up afterwards but it gets boring to do so every day ;-).
No special shares or mountpoints. But it seems to be random whether it finishes corectly or not. I was thinking it might be a timeout problem coursed by the load when severel backub’s are started. I do not care about how long it will take to run.
Previously the same LXC’s where stored locally on the node’s or Ceph or with NFS on a Qnap NAS, and gave no problem with backup. I use the Proxmox BU server that stores on an NFS Qnap.