Core files were found: smbd.core - SMD service down

This happens the second time now (on several TN Core machines):

New alerts:
* The following system core files were found: smbd.core. Please create a ticket
at httXX://ixsystems.atlassian.net/ and attach the relevant core files along
with a system debug. Once the core files have been archived and attached to
the ticket, they may be removed by running the following command in shell: ‘rm
/var/db/system/cores/*’.

I can see that the /var volume is small

Filesystem Size Used Avail Capacity Mounted on
boot-pool/ROOT/13.0-U6.8 5.5G 1.3G 4.2G 24% /
devfs 1.0K 1.0K 0B 100% /dev
tmpfs 32M 9.5M 22M 30% /etc
tmpfs 4.0M 8.0K 4.0M 0% /mnt
tmpfs 2.6G 2.6G 0B 100% /var
fdescfs 1.0K 1.0K 0B 100% /dev/fd

and full, although I cannot find where the data is that filled it up.

I have no idea why smbd has crashed. A reboot helped last time but the problem came back roughly after 1 week.

I am seeing a lot of No space left on device and NT_STATUS_DISK_FULL in log.smbd

The pool volumes are far from 100% full.

Any ideas?

So if you run 100% out of space on a device, then smbd (and potentially many other processes) will crash. I’ve seen this before when running out of storage entirely on data pool, but in your case it may be a process writing to one of your tmpfs filesystem. I vaguely recall there were some edge cases where cloud sync failure logs could build up in the rundir and go OOM.

But the logs are not on tmpfs, they are mounted to ZFS.

There are various samba state files that are on tmpfs.