Potential way to recover data after the dreaded "uncorrectable I/O failure - suspended." error

TrueNAS General
,
1

Hi all, first post here for me.

I have been using TrueNAS for a while now, I’m probably not really using it to it’s fullest capability - I have it set up for personal use with Plex and Nextcloud set up on it, as well as some windows shares. I know my way around both it and Linux/FreeBSD a fair amount, but am by no means an expert!

So…as well as many others I had the dreaded:

“Solaris: WARNING: Pool ‘whicheverpoolitis’ has encountered an uncorrectable I/O failure and has been suspended.”

error and was at my wits end. This wasn’t critical data, but would have been a MEGA pain to lose (the important stuff is on a separate RAID’d pool).

The following worked for me, there were indeed errors in the drive and some data was lost, but I managed to recover 98-99% of it! It may not work for other instances, but it’s definitely worth a shot before giving up!

To the solution:

  1. Start up TrueNAS without the faulty drive connected.

  2. Export/Disconnect the pool (I didn’t delete config - not sure if that’d make much difference)…

  3. Reconnect the drive, and import the pool back in again. TrueNAS webUI will probably freeze up here, just leave it and open another browser tab to it again and that will work fine.

At this stage, the pool should be back up again, but won’t work ‘properly’. You will be able to see the pool mounted in shell “/mnt/”

Next:

  1. Start the SSH service, with allow login as root enabled.

  2. Get WinSCP if you don’t already have it and connect to your TrueNAS IP with your root username/password.

  3. There, you’re in. Your pool ‘should’ now be in the /mnt/ directory in WinSCP also.

You should now be able to copy files or directories straight to the machine you’re using. I use the synchronize function, more out of habit I guess and that tells you when there’s a problem copying a particular file, but won’t cancel the transfer of the rest of the files. I don’t see why just dragging and dropping wouldn’t work, and give the same result.

I really hope that helps at least one person, because we all know it’s awful to lose data. It was sad to see so many people have lost all their data because of this error.

I am happy to help further if anyone runs into issues with this.

Cheers,