Pools becomes unavailable after reboot and wipefs didn't help

Hardware: Ugreen DXP4800 Plus
Disk 1, 2 3 are in Raidz1 (bay 4 empty)
two SSDs in mirror

I recently moved from UGOS 1.0.0.1895 to TrueNAS ElectricEel-24.10.0.2. I was hit with this Jira where the disks from the previous UGOS btrfs raid filesystem were used to create the pools. After every reboot, the pools became unavailable. The current work around is to export/import pool each reboot. After doing some more reading I came across possible solutions here where Alexander Motin proposed two solutions

Solution 1: The problem is likely that your disks have some metadata from other file system type, or maybe swap partition that is getting detected by blkid tool before ZFS. It would help to carefully wipe them out without damaging ZFS. I guess you might be able to use blkid to identify the other detected types, and then with wipefs wipe its metadata specifically, hoping to not datamage ZFS in process.

I did see that wipefs showed “linux_raid_member” and went ahead and did wipefs --backup --all /dev/sda1 and sdb1 and sdc1. However, it did not resolve the issue. Anyone else had this issue and were able to resolve it?

Here is my output of bldid…

root@truenas[~]# blkid --probe /dev/sda

/dev/sda: PTUUID="e9f4f67b-1ed4-464f-b25f-07878a4b5b6d" PTTYPE="gpt"

root@truenas[~]# blkid --probe /dev/sda1

/dev/sda1: VERSION="5000" LABEL="volume1" UUID="16443763964528027974" UUID_SUB="6737445603676628705" BLOCK_SIZE="4096" TYPE="zfs_member" USAGE="filesystem" PART_ENTRY_SCHEME="gpt" PART_ENTRY_NAME="data" PART_ENTRY_UUID="365ff50b-a0a9-4f7c-914d-4cba40adecd6" PART_ENTRY_TYPE="6a898cc3-1dd2-11b2-99a6-080020736631" PART_ENTRY_NUMBER="1" PART_ENTRY_OFFSET="2048" PART_ENTRY_SIZE="23437766656" PART_ENTRY_DISK="8:0"

root@truenas[~]#

Solution 2: The problem with that issue is that the pool in pre-release 24.10.0 was not created correctly if it was created on previously used disks, that causes problems for later automatic pool import. Unfortunately the easiest way to fix it is to unload the data, destroy the pool and recreate it on newer TrueNAS version having the fix.

However, this is incorrect, I created the pool with version 24.10.2.

If you read this far, Thanks a bunch!!! Any pointers on how I should proceed?

Thanks
Ramana

What does sudo wipefs /dev/sda1 …/sdb1 … /sdc1 show now?

I think you have to mention the partion type like vfat or linux_raid_member in the actual command, but I guess you just left it out in the post out of convenience?

And just to be safe, wiping the disks before that via the GUI. Might actually be the fastest and easiest solution here…

I messed up and did not do the -t linux_raid when doing the command. The output is shown below

root@truenas[~]# wipefs /dev/sda1 
DEVICE OFFSET        TYPE       UUID                 LABEL
sda1   0x3f000       zfs_member 16443763964528027974 volume1
sda1   0x3e000       zfs_member 16443763964528027974 volume1
sda1   0x3d000       zfs_member 16443763964528027974 volume1
sda1   0x3c000       zfs_member 16443763964528027974 volume1
sda1   0x3b000       zfs_member 16443763964528027974 volume1
sda1   0x3a000       zfs_member 16443763964528027974 volume1
sda1   0x39000       zfs_member 16443763964528027974 volume1
sda1   0x38000       zfs_member 16443763964528027974 volume1
sda1   0x37000       zfs_member 16443763964528027974 volume1
sda1   0x36000       zfs_member 16443763964528027974 volume1
sda1   0x35000       zfs_member 16443763964528027974 volume1
sda1   0x34000       zfs_member 16443763964528027974 volume1
sda1   0x33000       zfs_member 16443763964528027974 volume1
sda1   0x32000       zfs_member 16443763964528027974 volume1
sda1   0x31000       zfs_member 16443763964528027974 volume1
sda1   0x30000       zfs_member 16443763964528027974 volume1
sda1   0x2f000       zfs_member 16443763964528027974 volume1
sda1   0x2e000       zfs_member 16443763964528027974 volume1
sda1   0x2d000       zfs_member 16443763964528027974 volume1
sda1   0x2c000       zfs_member 16443763964528027974 volume1
sda1   0x2b000       zfs_member 16443763964528027974 volume1
sda1   0x2a000       zfs_member 16443763964528027974 volume1
sda1   0x29000       zfs_member 16443763964528027974 volume1
sda1   0x28000       zfs_member 16443763964528027974 volume1
sda1   0x27000       zfs_member 16443763964528027974 volume1
sda1   0x26000       zfs_member 16443763964528027974 volume1
sda1   0x25000       zfs_member 16443763964528027974 volume1
sda1   0x24000       zfs_member 16443763964528027974 volume1
sda1   0x23000       zfs_member 16443763964528027974 volume1
sda1   0x7f000       zfs_member 16443763964528027974 volume1
sda1   0x7e000       zfs_member 16443763964528027974 volume1
sda1   0x7d000       zfs_member 16443763964528027974 volume1
sda1   0x7c000       zfs_member 16443763964528027974 volume1
sda1   0x7b000       zfs_member 16443763964528027974 volume1
sda1   0x7a000       zfs_member 16443763964528027974 volume1
sda1   0x79000       zfs_member 16443763964528027974 volume1
sda1   0x78000       zfs_member 16443763964528027974 volume1
sda1   0x77000       zfs_member 16443763964528027974 volume1
sda1   0x76000       zfs_member 16443763964528027974 volume1
sda1   0x75000       zfs_member 16443763964528027974 volume1
sda1   0x74000       zfs_member 16443763964528027974 volume1
sda1   0x73000       zfs_member 16443763964528027974 volume1
sda1   0x72000       zfs_member 16443763964528027974 volume1
sda1   0x71000       zfs_member 16443763964528027974 volume1
sda1   0x70000       zfs_member 16443763964528027974 volume1
sda1   0x6f000       zfs_member 16443763964528027974 volume1
sda1   0x6e000       zfs_member 16443763964528027974 volume1
sda1   0x6d000       zfs_member 16443763964528027974 volume1
sda1   0x6c000       zfs_member 16443763964528027974 volume1
sda1   0x6b000       zfs_member 16443763964528027974 volume1
sda1   0x6a000       zfs_member 16443763964528027974 volume1
sda1   0x69000       zfs_member 16443763964528027974 volume1
sda1   0x68000       zfs_member 16443763964528027974 volume1
sda1   0x67000       zfs_member 16443763964528027974 volume1
sda1   0x66000       zfs_member 16443763964528027974 volume1
sda1   0x65000       zfs_member 16443763964528027974 volume1
sda1   0x64000       zfs_member 16443763964528027974 volume1
sda1   0x63000       zfs_member 16443763964528027974 volume1
sda1   0x62000       zfs_member 16443763964528027974 volume1
sda1   0x61000       zfs_member 16443763964528027974 volume1
sda1   0x60000       zfs_member 16443763964528027974 volume1
sda1   0xae9ffdbf000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdbe000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdbd000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdbc000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdbb000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdba000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdb9000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdb8000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdb7000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdb6000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdb5000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdb4000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdb3000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdb2000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdb1000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdb0000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdaf000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdae000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdad000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdac000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdab000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdaa000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffda9000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffda8000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffda7000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffda6000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffda5000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffda4000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffda3000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffda2000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffda1000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffda0000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdff000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdfe000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdfd000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdfc000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdfb000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdfa000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdf9000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdf8000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdf7000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdf6000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdf5000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdf4000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdf3000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdf2000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdf1000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdf0000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdef000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdee000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffded000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdec000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdeb000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffdea000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffde9000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffde8000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffde7000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffde6000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffde5000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffde4000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffde3000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffde2000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffde1000 zfs_member 16443763964528027974 volume1
sda1   0xae9ffde0000 zfs_member 16443763964528027974 volume1
root@truenas[~]# 

Well, at least on sda1, the raid_member labels are gone. What about the other partitions, sdb1 and sdc1?

Hope you did not delete more than you should, at least in the thread below the OFFSet after the command also starts at 0x3f000. There are also ways to recover the partion information, but not sure how that works and how safe it is…

[Solved] Pool becomes unavailable after reboot

root@truenas[~]# wipefs /dev/sdb1
DEVICE OFFSET        TYPE       UUID                 LABEL
sdb1   0x3f000       zfs_member 16443763964528027974 volume1
sdb1   0x3e000       zfs_member 16443763964528027974 volume1
sdb1   0x3d000       zfs_member 16443763964528027974 volume1
sdb1   0x3c000       zfs_member 16443763964528027974 volume1
sdb1   0x3b000       zfs_member 16443763964528027974 volume1
sdb1   0x3a000       zfs_member 16443763964528027974 volume1
sdb1   0x39000       zfs_member 16443763964528027974 volume1
sdb1   0x38000       zfs_member 16443763964528027974 volume1
sdb1   0x37000       zfs_member 16443763964528027974 volume1
sdb1   0x36000       zfs_member 16443763964528027974 volume1
sdb1   0x35000       zfs_member 16443763964528027974 volume1
sdb1   0x34000       zfs_member 16443763964528027974 volume1
sdb1   0x33000       zfs_member 16443763964528027974 volume1
sdb1   0x32000       zfs_member 16443763964528027974 volume1
sdb1   0x31000       zfs_member 16443763964528027974 volume1
sdb1   0x30000       zfs_member 16443763964528027974 volume1
sdb1   0x2f000       zfs_member 16443763964528027974 volume1
sdb1   0x2e000       zfs_member 16443763964528027974 volume1
sdb1   0x2d000       zfs_member 16443763964528027974 volume1
sdb1   0x2c000       zfs_member 16443763964528027974 volume1
sdb1   0x2b000       zfs_member 16443763964528027974 volume1
sdb1   0x2a000       zfs_member 16443763964528027974 volume1
sdb1   0x29000       zfs_member 16443763964528027974 volume1
sdb1   0x28000       zfs_member 16443763964528027974 volume1
sdb1   0x27000       zfs_member 16443763964528027974 volume1
sdb1   0x26000       zfs_member 16443763964528027974 volume1
sdb1   0x25000       zfs_member 16443763964528027974 volume1
sdb1   0x24000       zfs_member 16443763964528027974 volume1
sdb1   0x23000       zfs_member 16443763964528027974 volume1
sdb1   0x7f000       zfs_member 16443763964528027974 volume1
sdb1   0x7e000       zfs_member 16443763964528027974 volume1
sdb1   0x7d000       zfs_member 16443763964528027974 volume1
sdb1   0x7c000       zfs_member 16443763964528027974 volume1
sdb1   0x7b000       zfs_member 16443763964528027974 volume1
sdb1   0x7a000       zfs_member 16443763964528027974 volume1
sdb1   0x79000       zfs_member 16443763964528027974 volume1
sdb1   0x78000       zfs_member 16443763964528027974 volume1
sdb1   0x77000       zfs_member 16443763964528027974 volume1
sdb1   0x76000       zfs_member 16443763964528027974 volume1
sdb1   0x75000       zfs_member 16443763964528027974 volume1
sdb1   0x74000       zfs_member 16443763964528027974 volume1
sdb1   0x73000       zfs_member 16443763964528027974 volume1
sdb1   0x72000       zfs_member 16443763964528027974 volume1
sdb1   0x71000       zfs_member 16443763964528027974 volume1
sdb1   0x70000       zfs_member 16443763964528027974 volume1
sdb1   0x6f000       zfs_member 16443763964528027974 volume1
sdb1   0x6e000       zfs_member 16443763964528027974 volume1
sdb1   0x6d000       zfs_member 16443763964528027974 volume1
sdb1   0x6c000       zfs_member 16443763964528027974 volume1
sdb1   0x6b000       zfs_member 16443763964528027974 volume1
sdb1   0x6a000       zfs_member 16443763964528027974 volume1
sdb1   0x69000       zfs_member 16443763964528027974 volume1
sdb1   0x68000       zfs_member 16443763964528027974 volume1
sdb1   0x67000       zfs_member 16443763964528027974 volume1
sdb1   0x66000       zfs_member 16443763964528027974 volume1
sdb1   0x65000       zfs_member 16443763964528027974 volume1
sdb1   0x64000       zfs_member 16443763964528027974 volume1
sdb1   0x63000       zfs_member 16443763964528027974 volume1
sdb1   0x62000       zfs_member 16443763964528027974 volume1
sdb1   0x61000       zfs_member 16443763964528027974 volume1
sdb1   0x60000       zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdbf000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdbe000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdbd000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdbc000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdbb000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdba000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdb9000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdb8000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdb7000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdb6000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdb5000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdb4000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdb3000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdb2000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdb1000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdb0000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdaf000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdae000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdad000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdac000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdab000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdaa000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffda9000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffda8000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffda7000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffda6000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffda5000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffda4000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffda3000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffda2000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffda1000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffda0000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdff000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdfe000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdfd000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdfc000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdfb000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdfa000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdf9000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdf8000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdf7000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdf6000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdf5000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdf4000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdf3000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdf2000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdf1000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdf0000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdef000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdee000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffded000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdec000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdeb000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffdea000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffde9000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffde8000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffde7000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffde6000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffde5000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffde4000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffde3000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffde2000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffde1000 zfs_member 16443763964528027974 volume1
sdb1   0xae9ffde0000 zfs_member 16443763964528027974 volume1
root@truenas[~]# wipefs /dev/sdc1
DEVICE OFFSET        TYPE       UUID                 LABEL
sdc1   0x3f000       zfs_member 16443763964528027974 volume1
sdc1   0x3e000       zfs_member 16443763964528027974 volume1
sdc1   0x3d000       zfs_member 16443763964528027974 volume1
sdc1   0x3c000       zfs_member 16443763964528027974 volume1
sdc1   0x3b000       zfs_member 16443763964528027974 volume1
sdc1   0x3a000       zfs_member 16443763964528027974 volume1
sdc1   0x39000       zfs_member 16443763964528027974 volume1
sdc1   0x38000       zfs_member 16443763964528027974 volume1
sdc1   0x37000       zfs_member 16443763964528027974 volume1
sdc1   0x36000       zfs_member 16443763964528027974 volume1
sdc1   0x35000       zfs_member 16443763964528027974 volume1
sdc1   0x34000       zfs_member 16443763964528027974 volume1
sdc1   0x33000       zfs_member 16443763964528027974 volume1
sdc1   0x32000       zfs_member 16443763964528027974 volume1
sdc1   0x31000       zfs_member 16443763964528027974 volume1
sdc1   0x30000       zfs_member 16443763964528027974 volume1
sdc1   0x2f000       zfs_member 16443763964528027974 volume1
sdc1   0x2e000       zfs_member 16443763964528027974 volume1
sdc1   0x2d000       zfs_member 16443763964528027974 volume1
sdc1   0x2c000       zfs_member 16443763964528027974 volume1
sdc1   0x2b000       zfs_member 16443763964528027974 volume1
sdc1   0x2a000       zfs_member 16443763964528027974 volume1
sdc1   0x29000       zfs_member 16443763964528027974 volume1
sdc1   0x28000       zfs_member 16443763964528027974 volume1
sdc1   0x27000       zfs_member 16443763964528027974 volume1
sdc1   0x26000       zfs_member 16443763964528027974 volume1
sdc1   0x25000       zfs_member 16443763964528027974 volume1
sdc1   0x24000       zfs_member 16443763964528027974 volume1
sdc1   0x23000       zfs_member 16443763964528027974 volume1
sdc1   0x7f000       zfs_member 16443763964528027974 volume1
sdc1   0x7e000       zfs_member 16443763964528027974 volume1
sdc1   0x7d000       zfs_member 16443763964528027974 volume1
sdc1   0x7c000       zfs_member 16443763964528027974 volume1
sdc1   0x7b000       zfs_member 16443763964528027974 volume1
sdc1   0x7a000       zfs_member 16443763964528027974 volume1
sdc1   0x79000       zfs_member 16443763964528027974 volume1
sdc1   0x78000       zfs_member 16443763964528027974 volume1
sdc1   0x77000       zfs_member 16443763964528027974 volume1
sdc1   0x76000       zfs_member 16443763964528027974 volume1
sdc1   0x75000       zfs_member 16443763964528027974 volume1
sdc1   0x74000       zfs_member 16443763964528027974 volume1
sdc1   0x73000       zfs_member 16443763964528027974 volume1
sdc1   0x72000       zfs_member 16443763964528027974 volume1
sdc1   0x71000       zfs_member 16443763964528027974 volume1
sdc1   0x70000       zfs_member 16443763964528027974 volume1
sdc1   0x6f000       zfs_member 16443763964528027974 volume1
sdc1   0x6e000       zfs_member 16443763964528027974 volume1
sdc1   0x6d000       zfs_member 16443763964528027974 volume1
sdc1   0x6c000       zfs_member 16443763964528027974 volume1
sdc1   0x6b000       zfs_member 16443763964528027974 volume1
sdc1   0x6a000       zfs_member 16443763964528027974 volume1
sdc1   0x69000       zfs_member 16443763964528027974 volume1
sdc1   0x68000       zfs_member 16443763964528027974 volume1
sdc1   0x67000       zfs_member 16443763964528027974 volume1
sdc1   0x66000       zfs_member 16443763964528027974 volume1
sdc1   0x65000       zfs_member 16443763964528027974 volume1
sdc1   0x64000       zfs_member 16443763964528027974 volume1
sdc1   0x63000       zfs_member 16443763964528027974 volume1
sdc1   0x62000       zfs_member 16443763964528027974 volume1
sdc1   0x61000       zfs_member 16443763964528027974 volume1
sdc1   0x60000       zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdbf000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdbe000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdbd000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdbc000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdbb000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdba000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdb9000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdb8000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdb7000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdb6000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdb5000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdb4000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdb3000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdb2000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdb1000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdb0000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdaf000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdae000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdad000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdac000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdab000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdaa000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffda9000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffda8000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffda7000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffda6000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffda5000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffda4000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffda3000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffda2000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffda1000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffda0000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdff000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdfe000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdfd000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdfc000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdfb000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdfa000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdf9000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdf8000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdf7000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdf6000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdf5000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdf4000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdf3000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdf2000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdf1000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdf0000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdef000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdee000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffded000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdec000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdeb000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffdea000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffde9000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffde8000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffde7000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffde6000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffde5000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffde4000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffde3000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffde2000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffde1000 zfs_member 16443763964528027974 volume1
sdc1   0xae9ffde0000 zfs_member 16443763964528027974 volume1
root@truenas[~]# 

Also, here is the output of blkid

root@truenas[~]# blkid --probe /dev/sda 
/dev/sda: PTUUID="e9f4f67b-1ed4-464f-b25f-07878a4b5b6d" PTTYPE="gpt"
root@truenas[~]# blkid --probe /dev/sda1
/dev/sda1: VERSION="5000" LABEL="volume1" UUID="16443763964528027974" UUID_SUB="6737445603676628705" BLOCK_SIZE="4096" TYPE="zfs_member" USAGE="filesystem" PART_ENTRY_SCHEME="gpt" PART_ENTRY_NAME="data" PART_ENTRY_UUID="365ff50b-a0a9-4f7c-914d-4cba40adecd6" PART_ENTRY_TYPE="6a898cc3-1dd2-11b2-99a6-080020736631" PART_ENTRY_NUMBER="1" PART_ENTRY_OFFSET="2048" PART_ENTRY_SIZE="23437766656" PART_ENTRY_DISK="8:0"
root@truenas[~]# blkid --probe /dev/sdb 
/dev/sdb: PTUUID="c05abc07-c6dc-495a-a2fb-07b1ad10712b" PTTYPE="gpt"
root@truenas[~]# blkid --probe /dev/sdb1
/dev/sdb1: VERSION="5000" LABEL="volume1" UUID="16443763964528027974" UUID_SUB="7321019067119131946" BLOCK_SIZE="4096" TYPE="zfs_member" USAGE="filesystem" PART_ENTRY_SCHEME="gpt" PART_ENTRY_NAME="data" PART_ENTRY_UUID="ab56dbcb-7b3f-4827-9244-be0fca85a45f" PART_ENTRY_TYPE="6a898cc3-1dd2-11b2-99a6-080020736631" PART_ENTRY_NUMBER="1" PART_ENTRY_OFFSET="2048" PART_ENTRY_SIZE="23437766656" PART_ENTRY_DISK="8:16"
root@truenas[~]# blkid --probe /dev/sdc 
/dev/sdc: PTUUID="03901ed7-3895-4dbb-9b4f-39e428ba5531" PTTYPE="gpt"
root@truenas[~]# blkid --probe /dev/sdc1
/dev/sdc1: VERSION="5000" LABEL="volume1" UUID="16443763964528027974" UUID_SUB="1642051029178993254" BLOCK_SIZE="4096" TYPE="zfs_member" USAGE="filesystem" PART_ENTRY_SCHEME="gpt" PART_ENTRY_NAME="data" PART_ENTRY_UUID="d5359e3e-3707-4a81-bde8-18b503fd0b86" PART_ENTRY_TYPE="6a898cc3-1dd2-11b2-99a6-080020736631" PART_ENTRY_NUMBER="1" PART_ENTRY_OFFSET="2048" PART_ENTRY_SIZE="23437766656" PART_ENTRY_DISK="8:32"
root@truenas[~]# 

Hope someone can help!
Thanks
Ramana

Just to close it out… I ended up recreating the pools.
Thanks
Ramana