Nucked my Data trying to recover

[Beastyrabbit]

Dear TrueNAS Community,

I nucked my Dataset and of course i did not have any Snapshot or anything setup.
(Just stupid … Just stupid)

I’m trying getting my data back.
Similar issue to How to restore deleted dataset? - #4 by ranXerox

I was able to shutdown my system in a few minutes and there was not a lotgoing on in the first place (I hope)

I followed this guide. To rollback to before the dataset was deleted.

How to recover a destroyed dataset on a ZFS pool - Endless Puzzle

I got the TXG

I exported the Pool => no errors.

Then i ran into issues while trying to import.
The Status of the Pool is fine

root@truenas[/home/admin]# zpool import                                              
   pool: DATALAKE
     id: 13791121047710780761
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

        DATALAKE                                  ONLINE
          raidz2-0                                ONLINE
            da249872-7f98-4756-bf7b-1f27327d3f7a  ONLINE
            3e12fd8c-02f3-43ea-a0b1-1970483e82d0  ONLINE
            191c2526-fa1e-4f8c-b2b3-f1aa99312162  ONLINE
            5d078919-da24-486f-8639-29d54496f807  ONLINE
            205697b5-59a8-4fec-9ee1-2b124878950a  ONLINE
            5282208e-4098-4954-b44f-c5b61165a672  ONLINE
            ed55f523-542d-43db-b5c1-724f22d30371  ONLINE
            bc753077-6b0c-4156-8d83-8c120e4231cb  ONLINE
            64f4c0e7-1e9f-4df9-9729-f859d1c63020  ONLINE
            4c8f1824-a141-4c6c-b0a7-52617c2d6f8d  ONLINE
          raidz2-1                                ONLINE
            ca8cb18f-6b36-4d2d-80fb-28a0db49cb60  ONLINE
            9fb2f470-4f39-4a36-b522-a0f8b1e4cadf  ONLINE
            a1de2fce-df84-461c-960b-d487f03a266a  ONLINE
            6bb75100-036d-46e3-951f-4b716195aa49  ONLINE
            be3e8790-f5cf-435d-b2a2-b8dead2e3183  ONLINE
            7dacdc05-3e76-4e70-b1f7-0205458d6038  ONLINE
            10b335fa-7035-487a-beda-88363d7b6cc5  ONLINE
            8b6a825c-2e08-4828-b83c-93bd7f940486  ONLINE
            f50d7fdd-93de-4789-98a1-f81e2521b26f  ONLINE
            ffb34b93-cc8a-4dd8-83de-3325b3299639  ONLINE
          raidz2-2                                ONLINE
            442c7b09-53b4-4c1e-845a-40afe1ef082f  ONLINE
            91e36650-f4b4-43e8-88f5-acef8d31ce65  ONLINE
            0a814815-cf11-4e49-9a1f-453dd5e2f11c  ONLINE
            03e821c3-c7ee-43a6-a795-3beae6c2568f  ONLINE
            2c1a354c-70d4-415c-b047-2f2c40dee0c5  ONLINE
            370f94a8-826c-4b3a-80a1-bf1a2e24364a  ONLINE
            034bfae9-098b-47a0-b3ad-637f4efac675  ONLINE
            daa4408b-f266-4dee-99e6-780957dab9df  ONLINE
            14bf186c-0936-4735-bf7b-0044da17cb1a  ONLINE
            5a1e90c4-db87-4054-b70d-2e604411a5fd  ONLINE
          raidz2-3                                ONLINE
            ff0803eb-2ede-49aa-b8ec-cbff764d5436  ONLINE
            3ec5c493-1249-448e-97b3-2a9a268c524f  ONLINE
            bac67ce3-3f46-4054-92b9-921eb35ec0cc  ONLINE
            a1de910c-f760-492d-aa5d-d7c0b96151e3  ONLINE
            f1f562ee-15e2-40e4-a9ce-26a73d8ab9c0  ONLINE
            6565b915-f5c0-4795-8c7b-508851b4c7f0  ONLINE
            ac9a3aa3-344c-4c5b-8bab-3e2c94dcaf1c  ONLINE
            b44afb02-8b94-4a6f-a08f-14e97925fbd7  ONLINE
            6bbc8d4f-3d3e-429c-b189-548e7dc782a5  ONLINE
            e3a5d944-fc44-4f62-aef8-a8448f65e73e  ONLINE
        cache
          e0136e14-160b-4762-b8e3-f43400ffa6c4
        spares
          45142de8-c5ef-4e94-bdad-5224492813c0
          32df0e3f-65dc-46c5-9b82-f70c43c512cc
        dedup
          mirror-6                                ONLINE
            c8c27ece-4248-440e-8386-957db39aa12d  ONLINE
            3bfdf4e2-4655-4cab-852d-0a305d96fc2a  ONLINE
        special
          mirror-5                                ONLINE
            1d9b2468-7866-4bfa-88bf-40b474bedd8f  ONLINE
            4f9a4f37-cce1-4805-b795-bf9a08a71d42  ONLINE
        logs
          mirror-4                                ONLINE
            7c7d61ea-7754-44da-8d5f-20bec53af9f6  ONLINE
            9c1e4513-f370-4007-8b82-a88e343f75b5  ONLINE

I tried a lot of differnet option to rollback but everytime i get the same error.
zpool import -T 48342 -o readonly=on 13791121047710780761

root@truenas[/home/admin]# zpool import -T 48342 -o readonly=on  13791121047710780761 
cannot import 'DATALAKE': one or more devices is currently unavailable

Can somebody help me get this resolved?
I was also looking into Data Recovery. But i always have to send in the disk.
I there a service that is helping remotly?

Regards
Tobias

[winnielinnie]

@Beastyrabbit

I tried a lot of differnet option to rollback but everytime i get the same error.

Maybe because the “cache” vdev is not available? Hard to say. (You have quite the complex pool: 4 x RAIDZ2 data vdevs + a two-mirror SLOG + a two-mirror dedup vdev + a two-mirror special vdev + an L2ARC cache + hot spares)

Do you have a backup of your important data?

---

@Beastyrabbit

I nucked my Dataset and of course i did not have any Snapshot or anything setup.

Snapshots are a property of a dataset. If you destroy the dataset, you de facto lose all your snapshots. Only a “checkpoint” can restore a destroyed dataset. (Or a fancy recovery process, with a lower chance of success, however.)

---

Somewhat related, if the “checkpoint” feature was exposed to the GUI, it might nudge users into making it a habit to use it when doing migrations, destructive activities, major changes, or just “once in a while” safeguards.

[Beastyrabbit]

Hi thanks for the reply.
No of course i was stupid enough to remove the backup before makign sure the new one was running correctly.
You have no idea how frustrated I am with myself.
So no backup at all.

can I remove cache and spares before importing?
I can remove Logs with “-m” but could not find anything about Cache and spares.

[winnielinnie]

@Beastyrabbit

No of course i was stupid enough to remove the backup before makign sure the new one was running correctly.

That’s not how backups work.

[Apollo]

You need to provide hardware specs.
First of all, why trying import via CLI? Shouldn’t GUI more user friendly and less user error prone?
Let’s assume you were able to import the pool, what are you trying to rollback from?
If you are trying to rollback “winuser”, then I am afraid this is not possible because you deleted the dataset and everything it contained (ie no more dataset, no more snapshot).
Other forum user can correct me if I am wrong.

[winnielinnie]

How come you’re trying to import it readonly? Per the instructions in the other thread, you’re meant to import it R/W when invoking the TXG number:

@Arwen

The pool import with the specific ZFS write transaction ID has to be R/W. Otherwise when you export the pool, the destroyed dataset is still destroyed on any simple import of the pool.

[Beastyrabbit]

Hi I tried it with read only and without.
I tried with “-f” and without
I tried with “-m” and without
I tried with the “-d /dev/disks/by-id” and without

I tried a lot of options. And some reboots.
But always the same Error.

[Beastyrabbit]

I just reread some of your information.
You sad in a post that i need to have a snapshot to rollback?
How can i check if i have one?

[winnielinnie]

@Beastyrabbit

You sad in a post that i need to have a snapshot to rollback?

That’s not what I said.

@winnielinnie

Snapshots are a property of a dataset. If you destroy the dataset, you de facto lose all your snapshots.

[Beastyrabbit]

Yes checkpoints. I was talking about Checkpoints.
You sad:

@winnielinnie

Only a “checkpoint” can restore a destroyed dataset. (Or a fancy recovery process, with a lower chance of success, however.)

How can i check for a Checkpoint

[winnielinnie]

@Beastyrabbit

How can i check for a Checkpoint

You very likely do not have one, since it’s not exposed in the GUI, and you would have had to manually create one in the command-line.

To check for its existence requires the pool to be imported in the first place.

To import a pool (rolling its entire existence back to the checkpoint state), you would use the --rewind-to-checkpoint parameter in the zpool import command. But since you don’t really know if/when you created one, this can be a very dangerous operation itself. That’s why it’s recommended to only use such an option when importing the pool (back to its checkpoint) in readonly mode, so you can check its state before committing to anything.

[Beastyrabbit]

@winnielinnie

To import a pool (rolling its entire existence back to the checkpoint state), you would use the --rewind-to-checkpoint parameter in the zpool import command. But since you don’t really know if/when you created one, this can be a very dangerous operation itself. That’s why it’s recommended to only use such an option when importing the pool (back to its checkpoint) in readonly mode, so you can check its state before committing to anything.

root@truenas[/home/admin]# zpool import -f -T 48342 DATALAKE
cannot import 'DATALAKE': one or more devices is currently unavailable
root@truenas[/home/admin]# zpool import -T 48342 -o readonly=on --rewind-to-checkpoint 13791121047710780761 
cannot import 'DATALAKE': checkpoint does not exist
        Destroy and re-create the pool from
        a backup source.

Yep you were correct no checkpoints

[Beastyrabbit]

Hm shit.
I tried to import the Pool over the GUI.
Server restarted but nothing has happend.
This is not a good sign