Snapshots Appear Empty When Browsing on Linux

TrueNAS General
1

Hi,

I set up a new server with TrueNAS Scale v23.10.2 a few weeks ago and configured snapshots on my pool. I then shared my drive as read-only in SMB and read-write in NFS.

When I use “Previous Versions” in Windows I am able to browse the snapshots as expected:

image
image552×705 24.5 KB

When I cd into the .zfs/snapshots folder on Linux, I can see all of the snapshots, but there is nothing inside them.

[Fri Jun 21 03:08:34:/pool/.zfs/snapshot]$ ll | grep 2024-06-21
drwxrwxrwx   1 root root 0 Jun 21 00:00 auto-2024-06-21_00-00/
drwxrwxrwx   1 root root 0 Jun 21 01:00 auto-2024-06-21_01-00/
drwxrwxrwx   1 root root 0 Jun 21 02:00 auto-2024-06-21_02-00/
drwxrwxrwx   1 root root 0 Jun 21 03:00 auto-2024-06-21_03-00/
[Fri Jun 21 03:08:44:/pool/.zfs/snapshot]$ cd auto-2024-06-21_00-00/
[Fri Jun 21 03:09:07:/pool/.zfs/snapshot/auto-2024-06-21_00-00]$ ll
total 0
drwxrwxrwx   1 root root 0 Jun 21 00:00 ./
drwxrwxrwx 414 root root 2 Jun 21 03:00 ../

I had this working on TrueNAS Core without any special configuration that I can remember, is there a step I am missing here?

2 Likes
2

I am having the same issue. I have found on the archived TrueNAS forums two threads (1, 2) that seem to also have this problem. They say that the issue is with NFSv4 vs POSIX ACLs, but one iXsystems dev says that NFSv4 permissions with .zfs/snapshots were going to be fixed in Scale 22.02.2, but this doesn’t seem to be the case afaik (see the last reply in the second thread).

3

I updated to v24.04.1.1 and see the same issue. I also tried changing to snapshot at the dataset I wanted directly, but no change.

4

I’m having the same issue. I didn’t have any issues with TrueNAS core but all my TrueNAS Scale have this issue. From Linux host all the snapshots looks empty. I force the NFS version 3 but still didn’t help. Only way to get to snapshot is CIFS share and mount the CIFS or from storage controller /mnt///.zfs.

5

I still have no solution. Probably going to format the system back to TrueNAS Core until Scale works properly.

6

I just ran into this today. I just Linux for everything. I had to use the command line within my TrueNas system to retrieve the file I was looking for. When is this getting corrected?

1 Like
7

I agree with everyone above. Snapshots in Truenas Scale(Dragonfish) is very flaky over Windows SMB share. The Previous Versions tab is empty all the time although I do see the .zfs\snapshot folder over the smb share but even that is empty many times and I need to restart the smb services from system settings for it to show the details of the snapshot folders.

Will definitely give Truenas Core a try. I dont care about the future of Truenas Core. I prefer stability over everything else.

8

I’m running into the same issue, and I think I found a little more information:
Accessing the snapshot folder via the share doesn’t seem to trigger TrueNAS to mount the snapshot, resulting in an empty folder.
But if I first navigate to that snapshot folder (i.e dataset/.zfs/snapshot/snapshot_name) via the TrueNAS shell, this triggers the snapshot mount. Afterwards the snapshot IS accessible via the share… at least until it’s unmounted again.
It doesn’t really help as that would be wildly impractical to use, but maybe this means it’s a bug rather than permission issues?

9

@Suvu I’m unsure what you mean by “navigate to that snapshot folder”, but will give it a try myself later. What version are you running?

10

Hi Nicholas,
I meant if you open a command shell on the TrueNAS machine, then “cd” to the path of a snapshot (eg: ‘cd /mnt/pool/dataset/.zfs/snapshot/snapshot_name’), then that snapshot is mounted and then becomes accessible via the smb share.
Like I said, it’s not at all practical, but might provide a little more insight into the issue?
I believe this shows that it is possible, but for some reason the snapshot isn’t being automatically mounted when accessing the respective folder from an smb share.

11

Generally snapshots should be accessed via VSS modules (that’s how it’s designed for access via SMB protocol). The linux SMB client has ability to mount by snapshot name (see manpage).

Simply enabling the snapdir and browsing to it is generally not sufficient to trigger and maintain automounts in the snapshot directory since samba internally holds references through O_PATH opens. Our shadow copy module maintains the the automount by keeping a refcounted O_DIRECTORY open of the snapdir (which prevents auto unmount).

If there’s an explicit problem with the VSS module then a bug ticket should be raised with relevant collateral since it’s currently passing automated tests. That said nested datasets within same share can get hairy for VSS clients since they generally make assumption that there is only single filesystem within a share and will error out trying to access previous versions that don’t exist in parent dataset.

12

Thank you for your reply Awalkerix,
I’m very much a beginner in this topic so please excuse my inexperience, but the documentation suggests to use this method to manage snapshots :

A user with permission to access the dataset contents can view the list of snapshots by going to the dataset .zfs directory from a share, like SMB, NFS, and iSCSI, or in the TrueNAS SCALE CLI. Users can browse and search any files they have permission to access throughout the entire dataset snapshot collection.

To access snapshots using a share, configure the client system to view hidden files. For example, in a Windows SMB share, enable Show hidden files, folders, and drives in Folder Options. From to the dataset root folder, open the .zfs directory and navigate to the snapshot.

Are you saying that browsing snapshots via the .zfs/snapshots directory (of a share) is in fact not supported at all, or do you mean that the client needs to be re-configured?
Mounting snapshots directly is certainly helpful and I’ll read up on how to do that, but it sounds far less convenient than “simply” (from the user perspective) browsing a series of snapshots to find/restore as needed.
Also I have tried with both nested and un-nested datasets, with the same result, but again thank you for this information, I don’t recall reading this anywhere else and I will be re-ordering my datasets to avoid nested shares.

1 Like
13

@Suvu, I just confirmed I see the same behavior in ElectricEel-24.10.0.2

1 Like
15

Is there any advice on how to mount the snapshots in Linux? Maybe I am dumb, but I cannot for the life of me figure out how to read snapshot data of an SMB share from Linux. I’ve read man pages for samba vfs, searched stackexchange, reddit, and youtube. I’m still stuck.

I’m running ElectricEel-24.10.1 and trying to connect from debian bookworm.

I can see all the snapshot directories in .zfs/snapshot, but they are empty.

When I run ‘testparm -v’ on the server, I get:
…
[archive]
comment = Packrat Archive
ea support = No
path = /mnt/packrat/archive
posix locking = No
read only = No
smbd max xattr size = 2097152
vfs objects = streams_xattr shadow_copy_zfs acl_xattr zfs_core io_uring
tn:vuid =
fruit:time machine max size = 0
fruit:time machine = False
tn:home = False
tn:path_suffix =
tn:purpose = DEFAULT_SHARE

All of the documentation I can find refers to ‘vfs objects ’ shadow_copy2’ and explicitly specifies snapshot location and format. Do the ‘shadow_copy_zfs’ and ‘zfs_core’ options automatically interpret the zfs snapshots? If so, how does one read snapshots from a client?

I would greatly appreciate any help here. I have been completely unable to read snapshot data from Linux.

-Matt

16

EDIT: Never mind. I’m still on Core. I realize this is unique to SCALE users.