I just did an upgrade on my proxmox system to v9 (trixie) and proxmox backup server v4 (trixie).
It turns out that the python3-zfslib doesn’t exist in trixie which breaks SSH+Netcat replication.
I use the PBS for proxmox backup as well as a replication target for my NAS.
Switching to ssh only works but is not ideal.
Any chance this will be resolved in the future?
I tried to install the bookworm .deb package but there are dependency errors.
root@pbs:~# apt install ./python3-libzfs_0.0+git20230207.c1bd4a0-1_amd64.deb
Note, selecting 'python3-libzfs' instead of './python3-libzfs_0.0+git20230207.c1bd4a0-1_amd64.deb'
Solving dependencies... Error!
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
Unsatisfied dependencies:
python3-libzfs : Depends: libzfs4linux (>= 2.0) but it is not installable
Depends: python3 (< 3.12) but 3.13.5-1 is to be installed
Error: Unable to correct problems, you have held broken packages.
Error: The following information from --solver 3.0 may provide additional context:
Unable to satisfy dependencies. Reached two conflicting decisions:
1. python3-libzfs:amd64=0.0+git20230207.c1bd4a0-1 is selected for install
2. python3-libzfs:amd64 Depends libzfs4linux (>= 2.0)
but none of the choices are installable:
[no choices]
What package are you using for replication engine? What libzfs python package are you talking about? Our internally-developed python-libzfs?
bookworm has a package called python3-libzfs that does not exist in trixie. It looks like TrueNAS is the maintainer.
apt install python3-libzfs worked in proxmox backup server v3 (bookworm). It does not in v4.
Is Truenas the maintainer of the python3-libzfs package? It look abandoned by debian. I’m guessing that TrueNAS won’t be updating the tool until it starts moving to trixie?
https://tracker.debian.org/pkg/py-libzfs
Yeah, it looks like someone else took our internal python-libzfs repository and made an upstream debian packages for it. We don’t consume that (whatever is in upstream debian) internally and we’re in the process of deprecating that old library. There are quite a few issues with it that IMO make it unfit for general consumption (of course that apparently doesn’t stop people from apparently trying to push it upstream for us) 
We’ll probably make the git repo readonly once we finish migrating to the new module that’s replacing it.
For what it’s worth its’ been replicating my data for over a year.
Ok so for anyone that sees this don’t upgrade to trixie just yet if you are using that library for replication. Can you ballpark a release date on the replacement?
I don’t understand how you’re using the python module. Is this part of proxmox? Some personal scripting?
No I’m just using proxmox backup server as a replication target. It’s debian / zfs under the hood.
I don’t see where py-libzfs fits into this though. Does the proxmox backup server rely on it?
No the TrueNAS replication task using SSH+Netcat uses the library.
Okay. That makes some sense. You’ll probably have to build it yourself then on trixie. Shouldn’t be that hard. TrueNAS isn’t quite there yet (trixie). The new library doesn’t have the replication-related code written yet I’ve been focusing on performance sensitive items like various iterators and thread-safety of course. GitHub - truenas/truenas_pylibzfs
Once we get closer to 26.04 it’ll have a proper readme and such.
Thank you. Any chance you have some instructions on how to build it. It was having a hard time finding zfs.h on my system. I need a list of dependencies and then try to build it on a vm.
You probably don’t need zfs.h. Typically you just need the libzfs and libnvlist development header packages.
Thank you both for sharing!
I’m currently also faced with this problem. Same situation as @BooMShanKerX with the hybrid PBS
Could any of you share some guideline on how to build this module? Never had to compile a Python module before and my Google search revealed a lot of references to setup.py files or init,py files, which i can’t see in the repository, so i’m not quite sure on how to approach this.
Thanks in advance!
Once I get over this darn flu I intend to give it another go. I want to do the compiling on a seperate VM so I don’t bork my TrueNAS install. I will post progress here once I get started.
At some point in the near-ish future we will rebase our master nightlies on trixie, and you can probably build the debian package then.
Just git clone our repo, and on debian trixie install the depenencies and debuild -ui -uc -b to create installable debian packages that you can install on a debian system via dpkg.
I tried to build the module but got some compile errors.
output.txt (60.3 KB)
This is getting a bit beyond me. I’m willing to keep trying but need a bit of guidance.
Error compiling Cython file:
------------------------------------------------------------
...
proptypes = []
c_type = <zfs.zfs_type_t>t
iter.type = c_type
iter.props = <void *>proptypes
with nogil:
libzfs.zprop_iter(self.__iterate_props, <void*>&iter, True, True, c_type)
^
------------------------------------------------------------
libzfs.pyx:520:38: Cannot assign type 'int (int, void *) except * nogil' to 'zprop_func' (alias of 'int (*)(int, void *) noexcept nogil'). Exception values are incompatible. Suggest adding 'noexcept' to the type of '_ZFS__iterate_props'.
GROK
This indicates that Cython-generated code uses except * nogil (allowing Python exceptions) in callback functions, but the ZFS headers expect noexcept nogil (no exceptions). This mismatch causes compilation failures for multiple functions, including _ZFS__iterate_props,
I applied that pull request however there are still many compile errors. I think migrating to trixie is going to be more complicated. We are probably going to have to wait until TrueNAS moves to trixie for it to be sorted out.
pull297.txt (165.9 KB)
Yeah, pylibzfs has a lot of legacy cruft and being cython raises a lot of ongoing maintenance overhead that isn’t present in pure cpython. I’ll maybe start plumbing the ZFS send/recv into a truenas-pylibzfs in a week or two and have a build option to provide a legacy wrapper for zettarepl use.
