Restoring Deleted Files from Dataset Clone

A user within my organization deleted a bunch of large file folders from one of our shares and I have been tasked with restoring said files from FreeNas. I have a few noob questions that I’m hoping can be answered here.

  1. Should the cloned dataset be 8 KB even though it’s a 708 GB folder?
  2. When I change directory to the clone and run the “ls” command, the folders are missing. Why aren’t they visible?
  3. How, after finding these files, do I restore them so that users can access them again?

All help will be greatly appreciated.

Can you post which version you are using?

What exactly did you do? Did you work in the web interface or GUI? Do you mean that you cloned a snapshot to a new dataset?

If so and if it is 8KB you cloned the wrong snapshot.

If I understand you correctly you are going down the right path, clone the snapshot to a new dataset and copy the files from there.

Or if you can browse the snapshots from the share (if configured) you could for example browse to the root folder where the folders where deleted and right-click there and go to previous versions, find the snapshot where the files existed, open that snapshot and copy the files from there. This part I describe works for a SMB share accessed via a Windows machine where the snapshots are browsable.

I don’t have experience with freenas, I use truenas scale. I assume freenas works similarly.

Bottom line is, you need to give more details of your setup and what you did. What you don’t want to do is restoring the snapshot as all new files would be lost then.

Can you post which version you are using?

I’m using FreeNAS 11.3-U5.

I looked for today’s snapshot in the GUI and cloned to a new dataset. What I find curious is that the file shows “Used 0” and “Referenced 104K” which has led me to believe that you’re correct about cloning the wrong snapshot. My problem now is, how do find the correct snapshot of today? All the snapshots are basically saying 0.

Was there a configuration mishap on my part which has made these snapshots useless?

You need to look a snapshot that references more than 700 GB.

Do you know when the user deleted the files?

Go back further in time.

Again not a freenas expert, but when you have a snapshot that was created prior to the deletion of the files and the snapshot is of the parent dataset I wouldn’t know on top of my head why you shouldn’t be able to get the files from there / what you could have misconfigured.

Edit: for example a misconfiguration would be: the user deleted the file last week and you only have a snapshot retention policy of three days. Then the files are gone.

How large is the dataset itself? Does its size indicate the missing 700 GB are still part of it?

Do you have backups?

Do you know when the user deleted the files?

It was definitely today. I had to go to her computer to cancel the delete.

How large is the dataset itself? Does its size indicate the missing 700 GB are still part of it?

Do you have backups?

I basically adopted this entire network and everyday I figure something else new as I wasn’t left with a manual to run this operation, unfortunately. I had to hack my way in getting access to this NAS so I have no idea what the retention policy could be, nor would I know where to check. Gonna have to do some research on that.

As it relates to the backups, where would I find those?

Also, the dataset itself is 700+ GB. I just performed a manual snapshot and it’s confirmed, but I can’t find an auto snapshot that provides the same amount of data.

UPDATE: It seems like the retention period is 3 days cos I can only see snapshots every 2hrs from now up until 3 days ago.

You are not working in a commercial operation responsible for the IT infrastructure I assume? :face_with_peeking_eye:

I don’t know where you find backups or if you have some. Your department IT should be able to answer that, but as you are the one trying to restore the files I fear you have no one to ask.

Either you have cloud backups (if that was already a feature of freenas) or maybe you have replication tasks setup pointing to another server, or …

How random that I suggested three days.

If the data existed three days before today clone the oldest snapshot you can access and look for the files there.

Now you need to figure out if the data changed since then (i.e. work your way up to newer snapshots).

It seems like you know where to look and know the exact time. Just in case, can you provide information (screenshots?) on the snapshot task and your dataset structure? I hope there’s nothing wrong and you just looked at too recent snapshots.

This is no critique to you personally. If operations weren’t handed over properly to you that’s on the company.

Allocate time to familiarize yourself with everything and raise potential concerns with management. You need to get on top of the setup before you can claim responsibility for it.

Also, when everything is working well you may consider upgrading the OS.

I sure am haha, but I’ve basically mastered this environment as it relates to operations. When I asked for backups I was wondering if there was something in FreeNAS I was missing. I just didn’t realise these backups weren’t being done as I thought at the time I had glanced over them things were smooth sailing. If it weren’t for our lovely employee deleting almost everything today I wouldn’t have known that these snapshots weren’t being done properly.

I was able to find a snapshot with 600+ GB, but that was done all the way in October of last year so everything within that folder from that time till now will basically be lost.

How do I go about restoring this folder though? I’d just copy over the files from that folder to our current folder and hope the unrecoverable files aren’t that important lol

Basically this, just dont restore the entire dataset to that snapshot.

Per my logic this can’t be true. If nothing changed in the snapshot routine, and you have snapshots of yesterday, you must be able to restore the files from yesterdays snapshot.

Please provide info on the snapshot task and your dataset structure (blur out names we aren’t supposed to see if needed), I still have hopes to recover everything without dataloss (at least a max. data loss of 2 hours).

If there are neither cloud sync tasks nor replication tasks setup I would assume you have no backups. You just found a new task for yourself :wink:

What is your pool structure?

I don’t test my backups as regularly as I should, but especially in a commercial environment
If you have not successfully restored from backup and verified it’s integrity you do not have a backup.

Thoroughly investigate and test your snapshot workflow and do the same for future (offsite) backups. This will save you a lot of headache.

Is you want to get extra credit: Document the process in detail.

There are a few different ways to make use of snapshots to recover data like this

  1. you can use windows shadow copies to restore data
  2. you can access the hidden .zfs/snapshots directory on the dataset, then copy the data out
  3. you can shallow clone the most recent snapshot that has the data, and then copy the data out
  4. you can rollback to before the incident but that will lose what happened since.

You can copy the data on the server. Or by mounting the clone as a share and copying across.

Copying on the server is the most efficient. And you should probably use rsync to do that with best fidelity.