Your board doesn’t have any different settings for “RAID” vs “AHCI” mode in the BIOS that would have been defaulted back, correct?
Let’s go spelunking: output of each of these in individually wrapped codeblocks, please:
sudo zdb -l /dev/sda2
sudo zdb -l /dev/sdb2
sudo blkid --probe /dev/sda2
sudo blkid --probe /dev/sdb2
i can check the BIOS again when i get home.
here are the outputs for the commands one thing to note - for some reason sdb2 is not reporting my “boot pool” drive
so i did the commands for sdc2
sudo zdb -l /dev/sda2
root@truenas[~]# sudo zdb -l /dev/sda2
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
root@truenas[~]# sudo zdb -l /dev/sdc2
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
root@truenas[~]# sudo blkid --probe /dev/sda2
/dev/sda2: PART_ENTRY_SCHEME="gpt" PART_ENTRY_UUID="df6b7731-6a5d-11ef-85b2-d43d7e5548d5" PART_ENTRY_TYPE="516e7cba-6ecf-11d6-8ff8-00022d09712b" PART_ENTRY_NUMBER="2" PART_ENTRY_OFFSET="4194432" PART_ENTRY_SIZE="23433576280" PART_ENTRY_DISK="8:0"
root@truenas[~]# sudo blkid --probe /dev/sdc2
/dev/sdc2: PART_ENTRY_SCHEME="gpt" PART_ENTRY_UUID="df5a79cd-6a5d-11ef-85b2-d43d7e5548d5" PART_ENTRY_TYPE="516e7cba-6ecf-11d6-8ff8-00022d09712b" PART_ENTRY_NUMBER="2" PART_ENTRY_OFFSET="4194432" PART_ENTRY_SIZE="23433576280" PART_ENTRY_DISK="8:32"
just for reference when i did lsblk today
root@truenas[~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 10.9T 0 disk
├─sda1 8:1 0 2G 0 part
└─sda2 8:2 0 10.9T 0 part
sdb 8:16 0 119.2G 0 disk
├─sdb1 8:17 0 260M 0 part
├─sdb2 8:18 0 103G 0 part
└─sdb3 8:19 0 16G 0 part
└─sdb3 253:0 0 16G 0 crypt [SWAP]
sdc 8:32 0 10.9T 0 disk
├─sdc1 8:33 0 2G 0 part
└─sdc2 8:34 0 10.9T 0 part
compared to previously - not sure why they changed actually…
root@truenas[~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 10.9T 0 disk
├─sda1 8:1 0 2G 0 part
└─sda2 8:2 0 10.9T 0 part
sdb 8:16 0 10.9T 0 disk
├─sdb1 8:17 0 2G 0 part
└─sdb2 8:18 0 10.9T 0 part
sdc 8:32 0 119.2G 0 disk
├─sdc1 8:33 0 260M 0 part
├─sdc2 8:34 0 103G 0 part
└─sdc3 8:35 0 16G 0 part
└─sdc3 253:0 0 16G 0 crypt [SWAP]
while this shouldn’t impact anything, that seems like it’s not as it should be. All of my drives show as Disabled, NOT FROZEN [SEC1] - and this happening after a reboot makes me wonder if it’s related.
Have you fully powered off the system and unplugged from the wall? Obviously if you’re not physically near the system, you can’t do that now - but I have a little voice in my head telling me this isn’t as it should be and it’s at least leading us to the problem.
Are yours Western Digitals?
All my Seagate Exos drives show this:
ATA Security is: Disabled, frozen [SEC2]
I notice that @ProtoZero also has Seagates.
EDIT: Not trying to sway the troubleshooting in any other direction. Just wanted to add “data” from someone who has a healthy pool, but happens to have the same “ATA Security” output as the OP.
In this system, mine are WD, yes. Every one of the four ZFS labels being “failed to unpack” makes me think there’s some manner of access control failing out here.
@ProtoZero were you using encryption on your pool or drives by any chance?
blkid isn’t even picking up the pool version/label at all.
What version of TrueNAS was this pool originally created on (I’m assuming CORE based on the PART_ENTRY_TYPE being freebsd-zfs) and what version are you currently running now?
I dont believe i was using any encryption on the drives - at least not intentionally thru TrueNAS. the Drives themselves do say SED “self encrypting drive” not sure if thats related
Truenas version right now is: TrueNAS-SCALE-22.12.4.2
When i first created the NAS i tried out CORE - however i quickly made the switch to SCALE.
I dont remember the exact time frame but I’ve primarily been on SCALE since. and i’ve used SCALE with it functioning normally during this time.
I dont recall too much about CORE. Does SCALE Support Pool Imports from CORE? if not then i def made new POOL once i swapped to scale
Is there anything I can test on my end, without putting my precious data at risk, to see if I can get my Seagate drives to reset their ATA Security to “NOT FROZEN”? (Assuming that this might be the reason for the odd behavior of suddenly not seeing the ZFS pool labels.)
I only ever reboot when needed, and I don’t do anything fancy in terms of security with my drives. (Just ZFS encryption. Nothing lower level than that.)
The last time my server powered off was when we lost electricity about 3 weeks ago, but the UPS unit triggered a clean shutdown. The server has not shutdown or reboot since then.
Model: ST12000NM0127
They sure are SED. That is one thing that does stand out.
So this doesn’t get overlooked. 
Even if you “think” you did, assume that you didn’t.
When you power off and physically unplug from the wall, hold down the computer’s power button for about 5 seconds to completely discharge.
Let’s ask them.
sudo sedutil-cli --query /dev/sda
Wut? You mean to tell me that there exist drives out there that would encrypt itself without explicit consent?
root@truenas[~]# sudo sedutil-cli --query /dev/sda
/dev/sda ATA ST12000NM0127 G006 ZJV59MC0
TPer function (0x0001)
ACKNAK = N, ASYNC = N. BufferManagement = N, comIDManagement = N, Streaming = Y, SYNC = Y
Locking function (0x0002)
Locked = N, LockingEnabled = Y, LockingSupported = Y, MBRDone = N, MBREnabled = N, MBRAbsent = N, MediaEncrypt = Y
Geometry function (0x0003)
Align = Y, Alignment Granularity = 8 (4096), Logical Block size = 512, Lowest Aligned LBA = 0
Secure Messaging function (0x0004)
Activated = Y, Number of SPs = 2
Enterprise function (0x0100)
Range crossing = Y, Base comID = 0x07fe, comIDs = 2
TPer Properties:
MaxMethods = 1 MaxSubpackets = 1
MaxPacketSize = 2028 MaxPackets = 1 MaxComPacketSize = 2048
MaxResponseComPacketSize = 2048 MaxSessions = 1 MaxIndTokenSize = 1992
MaxAuthentications = 165 MaxTransactionLimit = 1 DefSessionTimeout = 0
MaxSessionTimeout = 0 MinSessionTimeout = 0 DefTransTimeout = 0
MaxTransTimeout = 0 MinTransTimeout = 0 MaxComIDTime = 0
MaxComIDCMD = 0 RealTimeClock = 0
Host Properties:
MaxComPacketSize = 2048 MaxPacketSize = 2028
MaxIndTokenSize = 1992 MaxPackets = 1 MaxSubpackets = 1
root@truenas[~]#
I will do this when i have physical access tonight
Yes. Enterprise feature. The drive encryts and decrypts automatically with a key that its only known to itself. To safely and instantly dispose of the drive and all its confidential data, an administrator only has to issue the command to reset the key… 
Great, well let’s hope this “no pools found” post wasn’t caused by the self-encrypt because now I have another vendor to avoid next to Western Digital.
The product clearly markets and prints “SED” on the label. It’s not a hidden feature.
Seagate’s own datasheet has different rows and model #'s for non-SED and SED.
They don’t try to hide it, unlike how WD tried to gaslight everyone when suspicions started to develop about their SMR “NAS” drives.
I agree, but it still remains that if this thread turns out to be caused by the self encryption, it does so from a simple power loss; and for that kind of bug that causes total data loss on a drive marketed for enterprise use (supposed to be resilient against things like power loss) is kind of inexcusable.
I guess excluding the whole vendor is unfair, but I’d definitely avoid the whole SED line like the plague.
scary thought
Scary indeed yes, but that is essentially the whole core purpose of encryption. If data can still be recovered… well then it basically failed its whole purpose and quite a lousy encryption product.
All vendors have SED models. You have to select these—and be eligible to buy because these are subject to export restrictions (not to be sold to North Korea, etc.).
Key reset is not triggered by accident, and if the key had been reset there would be no partition table or UUID left to be read: The whole drive would be random blankeness.
(The content of) This drive will self-destruct in 30 microseconds…
We’ll see what other information can be gathered with physical access.
Because of the mention of the CMOS battery dying, and that the motherboard has “RAID” options for the SATA ports, it’s possible that this could be an underlying culprit.