SMB read troubles

Hello,

I developed a python file copyer in a Docker using smbclient library.

I’m trying to read a csv file existing on my TrueNas to hash a md5 from it :

with smbclient.open_file(fullname_csv, mode="rb") as remote_file:
                remote_content = remote_file.read()
                samba_md5 = utils.sign_md5_base64_from_file(remote_content)

Thing is, my read() gets stuck or has concurrent access issues :

ERROR:file_sender:Failed sending to Samba Server : \\my-server.org\D\poub\MV\Quids. Erreur : [Error 1] [NtStatus 0xc0000043] The process cannot access the file because it is being used by another process: '\\my-nas.org\nfs_smb\my-smb.org\D\poub\MV\Quids\test_NH.csv'
ERROR:main:Une erreur est survenue lors du traitement de la requête.
Traceback (most recent call last):
  File "/app/file_sender.py", line 61, in send_file
    with smbclient.open_file(fullname_csv, mode="rb") as remote_file:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/smbclient/_os.py", line 533, in open_file
    raw_fd.open()
  File "/usr/local/lib/python3.12/site-packages/smbclient/_io.py", line 463, in open
    transaction.commit()
  File "/usr/local/lib/python3.12/site-packages/smbclient/_io.py", line 349, in commit
    raise failures[0]
smbprotocol.exceptions.SMBOSError: [Error 1] [NtStatus 0xc0000043] The process cannot access the file because it is being used by another process: '\\my-nas.org\nfs_smb\my-samba-server\D\poub\MV\Quids\test_NH.csv'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/main.py", line 132, in download_file
    file_sender.send_file(value)
  File "/app/file_sender.py", line 99, in send_file
    raise RuntimeError(f"Erreur d'envoi Samba :  {str(e)}")
RuntimeError: Erreur d'envoi Samba :  [Error 1] [NtStatus 0xc0000043] The process cannot access the file because it is being used by another process: '\\my-nas-server.org\nfs_smb\my-samba-server\D\poub\MV\Quids\test_NH.csv'

My NAS has no issues receiving my write / stat / makedirs requests from smbclient.

I did many tests and am frying my brain over this.

Some causes i’m exploring :

  • trying to hash and read in the same with : tested without the hashing, as the example from the library github repo, doesn’t work
  • NAS server configuration is incorrectly sending SMB close requests : i don’t want to tcpdump smb server requests…please, mercy.
  • my smbclient is not using the right SMB protocol : i think the library is intelligent enough to do what’s right.
  • i’m trying to read a CSV file and it does not have an EOF character : untested - i would have to manually put EOF in a csv to see what happens, or copy a txt file

if you have any idea, you’re welcome to help !

Dedicated Stackoverflow topic : stackoverflow(dot)com/questions/79275465/reading-and-hashing-a-file-from-a-smb-with-python

What smbclient library is this? On the surface it looks like a client error in handling locks.

smbprotocol Python library.

pypi(dot)org/project/smbprotocol/
github(dot)com/jborean93/smbprotocol

Developer response for what it’s worth :

This is a bit weird, the read calls are sending SMB2 READ requests [1] with a newer offset. With the read call it is meant to continue to do this until the whole file is read. I have no idea why your example is hanging but you should look into the requests sent to see what offset/length it is on and whether it is exceeding the length of the file or not. This sounds more like a server implementation detail where it is not sending the READ response once the end of the file is reached.

It’s a common problem with pipes as a pipe won’t have an EOF until the server end is closed but the EOF for a file is simply once the file has no more data.

Have you tried this against a Windows server?

I personally use the python SMB client distributed by the samba project (which is a wrapper around libsmbclient the Samba userspace SMB client library used by many other projects).

yeah this is what i use :

import smbclient

smbclient.ClientConfig(username=variables.NFS_USER, password=variables.NFS_PASSWORD)

smbclient.stat(fullname_csv)

smbclient.makedirs(dossier_destination, exist_ok=True)

 with smbclient.open_file(fullname_csv, mode="rb") as remote_file:
                remote_content = remote_file.read()
                samba_md5 = utils.sign_md5_base64_from_file(remote_content)

And the open_file crashes due to permissions locked, or times out.

That’s not the samba smbclient. That’s a library someone wrote with a name collision with an existing C library IIRC.

I did not test it against a Windows Server, i will do on Monday.

This will help me determine if the issue is more server or client side :wink:

Well if you have something better, i’m happy to try it. For now i’m using jborean’s smbprotocol / smbclient wrappers

/*
Template code to use this library:

-------------------------
from samba.samba3 import libsmb_samba_internal as libsmb
from samba.samba3 import param as s3param
from samba import (credentials,NTSTATUSError)

lp = s3param.get_context()
lp.load("/etc/samba/smb.conf");

creds = credentials.Credentials()
creds.guess(lp)
creds.set_username("administrator")
creds.set_password("1234")

c = libsmb.Conn("127.0.0.1",
                "tmp",
                lp,
                creds,
                multi_threaded=True)
-------------------------
*/

For upstream Samba python SMB client.

Another trivial example:

from samba.samba3 import libsmb_samba_internal as libsmb
from samba.dcerpc import security
from samba.samba3 import param as s3param
from samba import credentials
from samba import NTSTATUSError


LP_CTX = s3param.get_context()
LP_CTX.load_default()

cred = credentials.Credentials()
cred.guess(LP_CTX)
cred.set_username('smbuser')
cred.set_password('Cats')

HOST = '127.0.0.1'
SHARE = 'SHARE'
FILE_NAME = 'canary'

conn = libsmb.Conn(HOST, SHARE, LP_CTX, cred, force_smb1=False)

fhdl = conn.create(
    FILE_NAME,
    CreateDisposition=1,
    DesiredAccess=security.SEC_GENERIC_READ,
)

print(conn.read(fhdl, 0, 2000))
root@truenas[/home/admin/audit_rules]# python3 /tmp/samba_smbclient_test.py
b'BOB\n'
1 Like

Hey,

We finally found out.

It was coming from the MTU size between Docker / LXC / Proxmox hypervisor

Docker daemon had a size of 1500 and the LXC / Proxmox was 1460.

I am pretty sure it resulted in the read requests being cut and the read cursor getting lost.