Raid expansion didn't go well with ElectricEel

So, I upgraded yesterday to the latest ElectricEel nightly. I decided to try out the extend feature so I could had another drive to my pool. I upgraded the pool and then went to manage devices where I was able to select extend for the VDEVI wanted to use. It started the process then it hit 25% where it said something like waiting for extension to start. I waited for over an hour+ and it just stayed at 25% with that message. So I rebooted knowing nothing was going to happen and I had no other option. The pool did mount after it rebooted but now it’s in a degraded state. When I go into manage devices it shows what looks like should have been the 5th drive I was going to add but it’s just a random string. The drive I was going to add still shows that it’s an unused disk in the main storage menu. I scrubbed the pool to see if that would help but it didn’t. So I was wondering if there is anything I can do to fix this other than offloading the files off the NAS and starting the pool from scratch again?

Also yes, I do know this is beta software and I knew what I was getting into when I installed it. This isn’t the fault of Truenas that’s why I’m not complaining. I just want to see if there is an answer to this problem before I go through a long process.

Did you test the drive (ie a burn in) before adding it as another member?

Drives sometime die when brand new, hence people recommend running a burn in procedure.

Seems like your new drive may have died.

Electric Eel is not BETA software. It’s in active development and is intended for developers only.

See: Software Status - TrueNAS Roadmap - Open Source NAS Software

BETA will be in August… after that, we’ll be accepting bug tickets.

I just took the drive out and threw it in a hard drive dock and it’s still functional. I guess I’ll just move everything off and start it all over. I took the risk and it didn’t pay off :frowning:

I got ya, I was just seeing if anyone has had this problem and knew of a solution. I saw on reddit a user named iX_Chris suggest using it on the truenas sub. He mentioned the only major problem they know of with the build is app support doesn’t work which is fine since I have a separate NAS for my apps. I’ll probably just reroll this nas for now. Wish it could have workd out.

You are on the bleeding edge until BETA.

I strongly recommend no deployments with production data. Even BETA is a slight risk until its been community tested for a while.

2 Likes

I am expanding my backup sever and it is sitting at 25% to but if you look at your netdata dash board you can see that it is reading and writing. i am asuming that it is distributing the data between all drives and it will take a while if you have a lot of data.

Yes, there is a remapping process. The system is still operational while remapping goes on.

With large HDDs it can take many days… depending on how much actual data is stored.

To me that seems like bad interfacing. You should keep people informed of what’s going on. To be stuck at 25% for hours/days is going to confuse people and cause people to believe that something is stuck so they’re going to reboot. Just show files moving or that there is activity so people know everything is ok.

1 Like

Well, I am not on the developer version, but on “ElectricEel-24.10.1” and the problem with “stuck at 25% - waiting for expansion to start” is still there. Have 5 x 12 TB drives, added drive no. 6 and “nothing” seems to happen. Process “raidz_expand” takes up 50% cpu, but that is it. Would be nice to have that process running in the background, giving some status updates in a log or so.

That’s a reasonable feature request. Its a OpenZFS feature, so it depends on what telemetry in in there.

Let us know if it doesn’t complete.

In fangtooth, we’ve added the RAID-Z expansion acceleration fix to OpenZFS. I’d guess the process will be many times faster.

Great! Thanks for the heads up! I was not so much concerned about the time it takes (even though faster would be nicer!), but there is no real status update in the gui. The “sudo zpool status -v | grep done” in the shell ultimately gave me the info I was looking for.

Glad to hear. I assume its completed successfully??

If you can post the command and the results for people to find, that would be useful.

Not yet - it was 5 x 12 TB in a raidz2 filled at around 80% - slowly getting there :wink:

expand: expansion of raidz2-0 in progress since Thu Jan 16 17:38:46 2025
11.8T / 38.2T copied at 185M/s, 30.91% done, 1 days 17:33:26 to go

System → Shell and in the shell window enter
sudo zpool status -v
The output is rather lengthy, but about 10 lines down from the top, you can see “expand:…” and there you’ll find all the information when waiting for a new drive to be added to an existing pool.

You can also look at the Reporting → Disks section and you can see the reads/writes going on, just no eta on when it will finish

I’d expect it to be >2X faster with Fangtooth