sfdisk yes, sorry.
So it was never added successfully - you had the 1x10T and hit the Expand button, and then it errored on you?
Edit: My lack of a 4Kn disk is complicating things a bit.
Yes, exactly.
To make everything clear, first I tried a different method, without any outside input, just what seemed intuitive.
I went to Storage > First pool / Topology, then Add VDEV. At the data section I selected the 8T disk and went to review. Here I didn’t see a reassuring info that the new capacity will be the sum of the 10T and 8T disks so I was discouraged to go through the VDEV addition. I therefore did NOT update the pool. I clicked on “start over”. It asked me to confirm I want to start over. I didn’t feel confident enough to confirm anything at this point and just clicked somewhere else (like Dashboard) on the side menu to make tough decisions go away. 
This was the point when I started to look for instructions. And this is when I tried the Expand button and things fell apart.
1 Like
Please show the output of zpool status.
Please provide the model number of your drives: except for you HGST your other drives are likely to be SMR, which is bad for ZFS and could be a concause.
Was the pool created on SCALE, on this same system that’s described above?
Can you do zdb -l /dev/sdb1 to see if your ZFS label is visible on the HE10 still?
The output:
pool: Backup
state: DEGRADED
status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using 'zpool replace'.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-4J
config:
NAME STATE READ WRITE CKSUM
Backup DEGRADED 0 0 0
mirror-0 DEGRADED 0 0 0
3fbfda50-62af-4d20-9bc4-af00bfa797ea ONLINE 0 0 0
10101785153192291193 UNAVAIL 0 0 0 was /dev/disk/by-partuuid/d54485de-3df4-4628-a7bf-429f38bfb9ed
errors: No known data errors
pool: boot-pool
state: ONLINE
scan: scrub repaired 0B in 00:00:10 with 0 errors on Wed Jun 19 03:45:12 2024
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
sde3 ONLINE 0 0 0
errors: No known data errors
“Backup” is what I was referring to as the second pool (WD Purple + WD Blue mirrored, Purple pulled). The first pool (HE10, the problematic) is not included anymore in the above report, however still wisible in the UI.
The model numbers:
8T Barracuda: ST8000DM004-2U9188
4T WD Blue: WDC_WD40EZAZ-00SF3B0
The Purple is not plugged in anymore, but it’s WD40PURZ.
Yes, it was created on this exact machine as a straightforward, bare-metal install. There was a little difference though. When I installed TrueNAS Scale (only like 8-10 weeks ago), only two drives were present, the HE10 and the 500GB SSD. The SSD for boot and the HE10 in itself for the only data pool. Later I added the 250GB SSD as L2ARC. Then I installed the two WD drives for the Backup pool. Everything worked perfectly for weeks. Then today happened everything else.
admin@truenas[~]$ sudo zdb -l /dev/sdb1
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
Is this bad news? 
Isn’t it somehow possible to find the error message from the Expand operation in the logs?
Did you implement the lru_gen fix?
I literally laughed out loud reading " so I was discouraged to go through the VDEV addition. I therefore did NOT update the pool. I clicked on “start over”. It asked me to confirm I want to start over. I didn’t feel confident enough to confirm anything at this point and just clicked somewhere else (like Dashboard) on the side menu to make tough decisions go away. " I feel your energy thru the network on that. We ALL have been there. I am currently self guiding my way thru my first TrueNAS setup coming from pure linux server/docker environment… Anyways…Good luck with the issue as it sounds like you will eventually figure this out.
1 Like
The purple is the only CMR among the three of them. Not that this has something to do with your current issue.
1 Like
As it sits, that’s bad, as it indicates that ZFS can’t find a label/header for your partition.
Stand by; I’m doing some experiments here.
2 Likes
Good news, I was able to repro. Better news, I was able to recover after a repro.
I’ll DM you in the morning (my morning) when we can do this interactively but I’m fairly confident I can get your pool back online.
6 Likes
No, what is that?
Thank you so much, I’m looking forward to your DM!
The lru_gen fix is for an over-eager swap algorithm in the Linux kernel.
sudo -s
echo 0 >/sys/kernel/mm/lru_gen/enabled
The second line can also be added as a post-init script in System → Advanced → Init/Shutdown - but the fix is also included in 24.04.1.1 so upgrading to that is also a solution.
And you’ve got a DM as well on your original request.
3 Likes
Learn the lesson - do NOT guess at ZFS administrative tasks as you are more likely than not to screw things up.
*thumb-down*
Why thumbs-down?
Here is a user who set up their NAS with a VERY poor configuration in many respects.
- Poor choice of original SMR disks
- Poor disk layout
Then they removed a drive in an incorrect manner, leaving one pool degraded, and then guessed poorly at how to make the other pool bigger and caused it to fail to load.
In other words, he guessed.
In other words, he realised he didn’t know what he was doing, and he guessed again.
The good news is that we are all trying to help him recover back to a sensible configuration. But that will be wasted if he doesn’t learn not to screw things up again.
Yes, that is all understood. I’m sure more so much by Prusstalc (*I* would have choose to praise HoneyBadger instead of poking at Prusstalc’s wound to convey the message of Read The eFfing Manual [RTFM]).
RTFM was often the response given by most *nix admins and you typically learned your lesson after the first one you got but the point never really added to the “conversation” all that much. For example, software evolves all the time, and a “RTFM” today may be useless 6 months from now (sometimes a better response is “how to think the situation through” instead). Example situation/response: This user deleted his home directory files and (s)he got a RTFM response but the fix for the developer was to fix the manual and program options. refuse systemd-tmpfiles --purge invocation without config file specified on cmdline · Issue #33349 · systemd/systemd · GitHub
Yes, the RTFM still applies yesterday, today, and tomorrow (you should always check the manual) but the user also needs to know which manual to check and needs to learn the underlying problems more so than how to check the manual. …I find teaching someone how to fish is often better than just giving them a fish.
3 Likes
Reminder - let’s not pick on anyone for being new or not knowing how to use TrueNAS.
13 Likes