Smb4.conf additional configuration for max open files

MikeHawkinsEidurmam · April 7, 2025, 11:29am

Right after upgrading to SCALE I hit a max open files limit in Unreal Editor, causing crashes, and sometimes assets weren’t loading.
I had to enter the TrueNAS cli and set the following:

[TrueNAS SCALE]> service smb update smb_options="max open files = 1048576"

FYI the TrueNAS cli is a command line client to the sqlite database, open it by running cli in the shell.

The other thing I changed was that I created an override to the SMBD Systemd service file
systemctl edit smbd.service

[Service]
LimitNOFILE=1048576
LimitNOFILESoft=1048576

smbd now shows more than 16k max open files, and Unreal no longer crashes

cat /proc/$(pgrep -n smbd)/limits | grep 'open files'
Max open files            1048576              1048576              files

winnielinnie · April 7, 2025, 1:15pm

~~Seems like SCALE is using the SMBD defaults of 1024 files per CPU core.~~

EDIT: Maybe not. Maybe 16K files was picked since it’s the default for Windows 7, per @MikeHawkinsEidurmam in another post.

But then why is Core’s so much higher?

With TrueNAS Core 13.3-U1, this is the default on my system:
max open files = 938988

awalkerix · April 7, 2025, 2:01pm

https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html#MAXOPENFILES

The limit of the number of open files is usually set by the UNIX per-process file descriptor limit rather than this parameter so you should never need to touch this parameter.

and

github.com/truenas/samba

source3/param/loadparm.c

88613bd54


      
          
          static bool lp_set_cmdline_helper(const char *pszParmName, const char *pszParmValue);
          static void free_param_opts(struct parmlist_entry **popts);
          
          /**
           *  Function to return the default value for the maximum number of open
           *  file descriptors permitted.  This function tries to consult the
           *  kernel-level (sysctl) and ulimit (getrlimit()) values and goes
           *  the smaller of those.
           */
          static int max_open_files(void)
          {
          	int sysctl_max = MAX_OPEN_FILES;
          	int rlimit_max = MAX_OPEN_FILES;
          
          #ifdef HAVE_SYSCTLBYNAME
          	{
          		size_t size = sizeof(sysctl_max);
          		sysctlbyname("kern.maxfilesperproc", &sysctl_max, &size, NULL,
          			     0);
          	}

MikeHawkinsEidurmam · April 7, 2025, 3:51pm

On another machine I tried setting only the override in the service file, and this appears to be enough. But on this one I don’t use smb shares, it’s my backup, so smbd runs without users.
Where I actually use SMB shares, no matter what I did the limit remained 16k until I also changed the hidden smb_options parameter.

MikeHawkinsEidurmam · April 24, 2025, 9:45am

This hit again after upgrading to fangtooth. I was expecting it and recreated the service override file and now everything is OK.
The LimitNOFILESoft is not a thing by the way, only LimitNOFILE is needed.

GeeSwishy · July 12, 2025, 12:41am

I am not sure if you folks have noticed trying to actually utilize anywhere near to the upper bound on open file descriptors you’ve set (1048576) in the in the smbd.service or via the cli’s smb service update smb_options will not work…

once your SMB daemon starts to exceed the 59392 open file descriptors it’s limited not by the operating system configuration, but by a hard coded value in truenas samba repo which came from upstream based on the fact that samba mainly uses uint16_t pointers because of some old school Windows limitation from what I understand…

When you begin to exceed 59392 file handles you’ll notice smbd log the following message:

root@truenas:/var/log/samba4# grep “file_init_global” log.smbd
[2025/02/26 00:15:05.186764, 1] …/…/source3/smbd/files.c:1919(file_init_global)
file_init_global: Information only: requested 196608 open files, 59392 are available.
[2025/02/26 17:39:51.297349, 1] …/…/source3/smbd/files.c:1919(file_init_global)
file_init_global: Information only: requested 5898240 open files, 16344 are available.
[2025/02/26 18:07:41.898912, 1] …/…/source3/smbd/files.c:1919(file_init_global)
file_init_global: Information only: requested 5898240 open files, 59392 are available.
[2025/02/26 18:07:53.499161, 1] …/…/source3/smbd/files.c:1919(file_init_global)
file_init_global: Information only: requested 5898240 open files, 59392 are available.
[2025/02/26 18:24:09.726056, 1] …/…/source3/smbd/files.c:1919(file_init_global)
file_init_global: Information only: requested 5898240 open files, 59392 are available.
root@truenas:/var/log/samba4#

Sure enough if you read:

github.com/truenas/samba

source3/smbd/files.c

88613bd54


      
          		SMB_ASSERT(!state.fsp_left_behind);
          	}
          }
          
          /****************************************************************************
           Initialise file structures.
          ****************************************************************************/
          
          static int files_max_open_fds;
          
          bool file_init_global(void)
          {
          	int request_max = lp_max_open_files();
          	int real_lim;
          	int real_max;
          
          	if (files_max_open_fds != 0) {
          		return true;
          	}
          
          	/*

#define FILE_HANDLE_OFFSET 0x1000

real_max = real_lim - MAX_OPEN_FUDGEFACTOR;

if (real_max + FILE_HANDLE_OFFSET + MAX_OPEN_PIPES > 65536) {
	real_max = 65536 - FILE_HANDLE_OFFSET - MAX_OPEN_PIPES;
}

So FILE_HANDLE_OFFSET is 4096 (0x1000 in hex),

github.com/truenas/samba

source3/include/local.h

88613bd54


      
          #define MAX_DIRECTORY_HANDLES 2048
          
          /* maximum number of file caches per smbd */
          #define MAX_WRITE_CACHES 10
          
          /*
           * Fudgefactor required for open tdb's, etc.
           */
          
          #ifndef MAX_OPEN_FUDGEFACTOR
          #define MAX_OPEN_FUDGEFACTOR 40
          #endif
          
          /*
           * Minimum number of open files needed for Windows7 to
           * work correctly. A little conservative but better that
           * than run out of fd's.
           */
          
          #ifndef MIN_OPEN_FILES_WINDOWS
          #define MIN_OPEN_FILES_WINDOWS 16384

And MAX_OPEN_FUDGEFACTOR is 40.
and MAX_OPEN_PIPES is 2048.

And if the file descriptor on the system are greater than 65536 (16 bit max int) then we get to the imposed limit of 65536 - 4096 - 2048 = 59392.

This was clearly done to keep the pointer from rolling over and causing unexpected behavior, potential security issues, denial of service, etc.

Why they thought it a good idea to use uint16_t everywhere is beyond me.

MikeHawkinsEidurmam · July 24, 2025, 10:26am

Well that’s awkward! I hope Unreal won’t start opening that many files. Wouldn’t this lead to issues with other file servers though? So after all, this might be an considered an Unreal Editor bug.

MikeHawkinsEidurmam · September 15, 2025, 8:09am

Upgraded again, this time I added the systemd override step to Ansible. Works like a charm!

The “smb_options” hidden setting sticks around even after upgrading. This only needs to be set once.