ZFS import crashing system

TrueNAS General
, ,
1

Hi everyone!

I’m having an issue where a perfectly running system died on me. Let me set the scene for you:

Hardware

HPE DL360 gen 9 system with hba to netapp disk shelf. I have 24 3.5 inch drives divided up into 4 raidz1 vdevs of 6 drives each. I have 2 NVME’s as metadata device in mirror, 2 dedup SSD’s and 2 log ssd’s.

Situation

Yesterday morning my system crashed. Truenas wouldn’t boot anymore. Since I didn’t find any specidfic errors in the ILO related to hardware, I assumed the SDcard (yes I know) the OS was installed on died. No problem, I threw in 2 drives and reinstalled truenas to those in a mirrored VDEV for bootpool and took out the SD. Great success! the system boots and i was able to configure Truenas 25.10 normally. Now for the piece the resistance: importing the old pool

Oh no

I tried to do an import using the gui, but the system just resets itself without throwing an error. Fine, maybe someting is wrong with the pool, let’s take a look:

root@truenas[/home/truenas_admin]# zpool import
  pool: Oxidizer
    id: 10025228163071528045
 state: ONLINE
status: The pool was last accessed by another system.
action: The pool can be imported using its name or numeric identifier and
        the '-f' flag.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-EY
config:

        Oxidizer                                  ONLINE
          raidz1-0                                ONLINE
            8e5bd552-1445-4fe9-ade5-ca00279cf522  ONLINE
            bceb9521-187f-48ea-b3e5-be53eb6fa76d  ONLINE
            bfbddaa2-341c-48ac-911c-a54f715fd3b2  ONLINE
            36f67b68-1566-4613-89d2-9d819e7583fe  ONLINE
            3a560bb5-a9b2-4a59-843c-f7da6dba607a  ONLINE
            3b0cd83b-9135-40b2-a9e5-49a7fdf75771  ONLINE
          raidz1-1                                ONLINE
            108f3d35-1ca5-4da9-9947-5b21d1f345de  ONLINE
            5e51406b-ff4a-475e-933d-9eb87303d55a  ONLINE
            289869a5-312c-4b21-acc1-d1234b430695  ONLINE
            863fbe3c-7201-4a99-8ce0-4861d00a26b1  ONLINE
            956a4d1d-0844-42e2-8ff6-dc9e35179e52  ONLINE
            b1f3b40d-e611-4fb1-b96a-448c5540279e  ONLINE
          raidz1-2                                ONLINE
            9162dea6-8284-4c68-a0a2-371bca76e99e  ONLINE
            78b9bf86-67c0-4771-9c2c-a6332227021f  ONLINE
            802ff312-af2a-4ff1-a269-569717392f0d  ONLINE
            a8cefcfe-aa65-495e-9333-c424e70a5e5c  ONLINE
            cbf9d608-c5e0-4dbb-8da9-9951a3c68bc9  ONLINE
            d8e655e4-0cc3-4ca4-a994-9164d23c05a7  ONLINE
          raidz1-3                                ONLINE
            e3feae67-c536-4ac6-9fe3-0308497e6309  ONLINE
            3096d3a6-1679-4ea5-a431-38e0bb2539e3  ONLINE
            8495ec58-cf8e-4727-9691-1a3738ca1548  ONLINE
            eac35c2d-40e9-4458-a838-ba6e60abf3a1  ONLINE
            f629b7de-244f-4530-9def-adfd82dc1051  ONLINE
            b43f90ff-7b5d-4b75-8ea3-b8070ce0bf35  ONLINE
        dedup
          mirror-6                                ONLINE
            a75b850a-4a8c-4176-82c3-97ede4cd76a1  ONLINE
            f040a409-44be-4152-9e2c-91bd51ad9cfe  ONLINE
        special
          mirror-4                                ONLINE
            0b89f764-f06a-4bb9-acfb-b985b8120226  ONLINE
            3eb27884-6106-4ae2-bbec-1eefbb5bb449  ONLINE
        logs
          mirror-5                                ONLINE
            686fafe2-7468-43c2-91a7-b9b7a1609146  ONLINE
            d23078b6-a57e-4b42-ba47-260edc04394d  ONLINE

Looks like all devices are there.. I tried the import -f manually, whilst checking dmesg but i was unable to see any errors showing up. The system just thinks about the import for 3 minutes and resets.

Next steps

I am not sure how to proceed. Importing with special flags like -R, -F don’t help. Is there someone who can help me with formulating a plan? Might it be my HBA? All drives show up, and truenas is stable without errors as long as i dont import the pool.

Bright side

The most important data is backed up offsite,

2

Update

I’m able to mount it read only without issue… the pool looks healthy at first glance:

root@truenas[/home/truenas_admin]# zpool import -o readonly=on -N -f Oxidizer
root@truenas[/home/truenas_admin]# zpool status                                                     
  pool: Oxidizer
 state: ONLINE
  scan: scrub repaired 0B in 1 days 10:44:21 with 0 errors on Mon Oct 20 10:44:25 2025
config:

	NAME                                      STATE     READ WRITE CKSUM
	Oxidizer                                  ONLINE       0     0     0
	  raidz1-0                                ONLINE       0     0     0
	    8e5bd552-1445-4fe9-ade5-ca00279cf522  ONLINE       0     0     0
	    bceb9521-187f-48ea-b3e5-be53eb6fa76d  ONLINE       0     0     0
	    bfbddaa2-341c-48ac-911c-a54f715fd3b2  ONLINE       0     0     0
	    36f67b68-1566-4613-89d2-9d819e7583fe  ONLINE       0     0     0
	    3a560bb5-a9b2-4a59-843c-f7da6dba607a  ONLINE       0     0     0
	    3b0cd83b-9135-40b2-a9e5-49a7fdf75771  ONLINE       0     0     0
	  raidz1-1                                ONLINE       0     0     0
	    108f3d35-1ca5-4da9-9947-5b21d1f345de  ONLINE       0     0     0
	    5e51406b-ff4a-475e-933d-9eb87303d55a  ONLINE       0     0     0
	    289869a5-312c-4b21-acc1-d1234b430695  ONLINE       0     0     0
	    863fbe3c-7201-4a99-8ce0-4861d00a26b1  ONLINE       0     0     0
	    956a4d1d-0844-42e2-8ff6-dc9e35179e52  ONLINE       0     0     0
	    b1f3b40d-e611-4fb1-b96a-448c5540279e  ONLINE       0     0     0
	  raidz1-2                                ONLINE       0     0     0
	    9162dea6-8284-4c68-a0a2-371bca76e99e  ONLINE       0     0     0
	    78b9bf86-67c0-4771-9c2c-a6332227021f  ONLINE       0     0     0
	    802ff312-af2a-4ff1-a269-569717392f0d  ONLINE       0     0     0
	    a8cefcfe-aa65-495e-9333-c424e70a5e5c  ONLINE       0     0     0
	    cbf9d608-c5e0-4dbb-8da9-9951a3c68bc9  ONLINE       0     0     0
	    d8e655e4-0cc3-4ca4-a994-9164d23c05a7  ONLINE       0     0     0
	  raidz1-3                                ONLINE       0     0     0
	    e3feae67-c536-4ac6-9fe3-0308497e6309  ONLINE       0     0     0
	    3096d3a6-1679-4ea5-a431-38e0bb2539e3  ONLINE       0     0     0
	    8495ec58-cf8e-4727-9691-1a3738ca1548  ONLINE       0     0     0
	    eac35c2d-40e9-4458-a838-ba6e60abf3a1  ONLINE       0     0     0
	    f629b7de-244f-4530-9def-adfd82dc1051  ONLINE       0     0     0
	    b43f90ff-7b5d-4b75-8ea3-b8070ce0bf35  ONLINE       0     0     0
	dedup	
	  mirror-6                                ONLINE       0     0     0
	    a75b850a-4a8c-4176-82c3-97ede4cd76a1  ONLINE       0     0     0
	    f040a409-44be-4152-9e2c-91bd51ad9cfe  ONLINE       0     0     0
	special	
	  mirror-4                                ONLINE       0     0     0
	    0b89f764-f06a-4bb9-acfb-b985b8120226  ONLINE       0     0     0
	    3eb27884-6106-4ae2-bbec-1eefbb5bb449  ONLINE       0     0     0
	logs	
	  mirror-5                                ONLINE       0     0     0
	    686fafe2-7468-43c2-91a7-b9b7a1609146  ONLINE       0     0     0
	    d23078b6-a57e-4b42-ba47-260edc04394d  ONLINE       0     0     0

errors: No known data errors

  pool: boot-pool
 state: ONLINE
config:

	NAME        STATE     READ WRITE CKSUM
	boot-pool   ONLINE       0     0     0
	  mirror-0  ONLINE       0     0     0
	    sdg3    ONLINE       0     0     0
	    sdh3    ONLINE       0     0     0
3

You might check the various logs for errors related to this pool.

Plus, while you may not have the space, making a non-ZFS backup now, might copy most of the files okay. (Non-ZFS backups = something other than ZFS Replication / Send & Receive, which won’t generally work on a R/O imported pool.)

4

What log files would that be? I tried importing the pool without log devices attached (just unplugged the SSD’s) but it also crashes

5

I don’t know where the log files are kept. Not yet had a need to look. (My personal Linux computers use a different distro, and different location for log files.)

The frustrating thing with this problem, is that you can import the pool Read Only. However, you can’t run a scrub on a R/O pool, so you can’t see if their are any problems.

While the pool is imported R/O, you might write a script to read every file. Of course this could take a long, long time. Much easier to backup all the files using a non ZFS backup, like RSync.

6

You’re right, that could take forever. But for the external backup, that’s impossible. I do not have an additional 100 tb laying around.

7

Your pool appears to have corruption in it. It is practically impossible for a simple OS crash or power loss to cause ZFS pool corruption:

So for your pool to have this serious level of corruption, either a rare ZFS bug occurred, or something else that I can’t see remotely.

Where you go from here, is first look at the logs. Again, I don’t know where TrueNAS CE / SCALE keeps them. Nor do I have complete skills in reading them.

If you must take action, you can start looking at the latest ZFS TXG, (aka write Transaction Group), on each disk. It should be 100% the same for all disks in a pool. I don’t know about special vDevs, De-Dup vDevs or Log vDevs, but they should also have the same TXG number.

You can look at the ZFS TXGs using:

sudo zdb -l /dev/X9 | grep -i txg | head -1

Replace the “X” with the disk name, like sda. Then replace 9, with the partition number.

If their are a few disks with smaller TXG numbers, you could possibly roll the transactions back a bit to see if the corruption is only in the most recent writes, (aka most recent write transaction groups). But, doing so will drop the most recent writes, (including making those files or file updates disappear permanently).

Having 100TB of data you can’t back up, makes it harder to recommend a solution. If you really need the data, a data recovery service or Klennet might be your last resort.

8

Alright! now we’re getting somewere!
After checking the last TXG numbers and trying to import the pool using the lowest one:

root@truenas[/home/truenas_admin]# zpool import -F -f -T 694409 Oxidizer
cannot import 'Oxidizer': I/O error
        Destroy and re-create the pool from
        a backup source.

This seems to point at a hardware problem? even though the disks all show up as good…

10

So i tried to do a regular import (it did crash) but it also gave me some insight:

kernel log:

Nov 11 13:43:12 truenas kernel: sd 0:0:7:0: [sdax] tag#302 Sense Key : Recovered Error [current]
Nov 11 13:43:12 truenas kernel: sd 0:0:7:0: [sdax] tag#302 Add. Sense: Defect list not found
Nov 11 13:43:12 truenas kernel: sd 0:0:6:0: [sdaw] tag#269 Sense Key : Recovered Error [current]
Nov 11 13:43:12 truenas kernel: sd 0:0:6:0: [sdaw] tag#269 Add. Sense: Defect list not found
Nov 11 13:43:23 truenas kernel: sd 0:0:2:0: [sdd] tag#268 Sense Key : Recovered Error [current]
Nov 11 13:43:23 truenas kernel: sd 0:0:2:0: [sdd] tag#268 Add. Sense: Defect list not found
Nov 11 13:43:23 truenas kernel: sd 0:0:1:0: [sda] tag#274 Sense Key : Recovered Error [current]
Nov 11 13:43:23 truenas kernel: sd 0:0:1:0: [sda] tag#274 Add. Sense: Defect list not found
11

That would be /proc/spl/kstat/zfs/dbgmsg
so something like

tail -n 200 /proc/spl/kstat/zfs/dbgmsg

to investigate (hopefully with insight from @HoneyBadger ).

2 Likes
12

Thanks, on running an import, i do generate a load of errors… Have a loot:

1762871397   ffff948065858000 metaslab.c:1789:spa_set_allocator(): spa allocator: dynamic
1762871397   ffff948065858000 spa.c:6921:spa_tryimport(): spa_tryimport: importing Oxidizer
1762871397   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load($import-ffff948065858000-Oxidizer, config trusted): LOADING
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/0b89f764-f06a-4bb9-acfb-b985b8120226': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/686fafe2-7468-43c2-91a7-b9b7a1609146': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3eb27884-6106-4ae2-bbec-1eefbb5bb449': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d23078b6-a57e-4b42-ba47-260edc04394d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f040a409-44be-4152-9e2c-91bd51ad9cfe': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a75b850a-4a8c-4176-82c3-97ede4cd76a1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a8cefcfe-aa65-495e-9333-c424e70a5e5c': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d8e655e4-0cc3-4ca4-a994-9164d23c05a7': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/36f67b68-1566-4613-89d2-9d819e7583fe': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3a560bb5-a9b2-4a59-843c-f7da6dba607a': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bfbddaa2-341c-48ac-911c-a54f715fd3b2': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/e3feae67-c536-4ac6-9fe3-0308497e6309': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3096d3a6-1679-4ea5-a431-38e0bb2539e3': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8e5bd552-1445-4fe9-ade5-ca00279cf522': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bceb9521-187f-48ea-b3e5-be53eb6fa76d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3b0cd83b-9135-40b2-a9e5-49a7fdf75771': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/eac35c2d-40e9-4458-a838-ba6e60abf3a1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b43f90ff-7b5d-4b75-8ea3-b8070ce0bf35': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f629b7de-244f-4530-9def-adfd82dc1051': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8495ec58-cf8e-4727-9691-1a3738ca1548': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/863fbe3c-7201-4a99-8ce0-4861d00a26b1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b1f3b40d-e611-4fb1-b96a-448c5540279e': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/5e51406b-ff4a-475e-933d-9eb87303d55a': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/cbf9d608-c5e0-4dbb-8da9-9951a3c68bc9': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/78b9bf86-67c0-4771-9c2c-a6332227021f': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/9162dea6-8284-4c68-a0a2-371bca76e99e': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/956a4d1d-0844-42e2-8ff6-dc9e35179e52': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/802ff312-af2a-4ff1-a269-569717392f0d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/108f3d35-1ca5-4da9-9947-5b21d1f345de': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/289869a5-312c-4b21-acc1-d1234b430695': probe done, cant_read=0 cant_write=1
1762871397   ffff948065858000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a8cefcfe-aa65-495e-9333-c424e70a5e5c': best uberblock found for spa $import-ffff948065858000-Oxidizer. txg 1051389
1762871397   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load($import-ffff948065858000-Oxidizer, config untrusted): using uberblock with txg=1051389
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/0b89f764-f06a-4bb9-acfb-b985b8120226': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3eb27884-6106-4ae2-bbec-1eefbb5bb449': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/686fafe2-7468-43c2-91a7-b9b7a1609146': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d23078b6-a57e-4b42-ba47-260edc04394d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a75b850a-4a8c-4176-82c3-97ede4cd76a1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f040a409-44be-4152-9e2c-91bd51ad9cfe': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a8cefcfe-aa65-495e-9333-c424e70a5e5c': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/36f67b68-1566-4613-89d2-9d819e7583fe': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f629b7de-244f-4530-9def-adfd82dc1051': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bfbddaa2-341c-48ac-911c-a54f715fd3b2': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b43f90ff-7b5d-4b75-8ea3-b8070ce0bf35': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3a560bb5-a9b2-4a59-843c-f7da6dba607a': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/eac35c2d-40e9-4458-a838-ba6e60abf3a1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bceb9521-187f-48ea-b3e5-be53eb6fa76d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3b0cd83b-9135-40b2-a9e5-49a7fdf75771': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/5e51406b-ff4a-475e-933d-9eb87303d55a': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8e5bd552-1445-4fe9-ade5-ca00279cf522': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3096d3a6-1679-4ea5-a431-38e0bb2539e3': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/e3feae67-c536-4ac6-9fe3-0308497e6309': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d8e655e4-0cc3-4ca4-a994-9164d23c05a7': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/956a4d1d-0844-42e2-8ff6-dc9e35179e52': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8495ec58-cf8e-4727-9691-1a3738ca1548': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b1f3b40d-e611-4fb1-b96a-448c5540279e': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/863fbe3c-7201-4a99-8ce0-4861d00a26b1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/802ff312-af2a-4ff1-a269-569717392f0d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/cbf9d608-c5e0-4dbb-8da9-9951a3c68bc9': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/78b9bf86-67c0-4771-9c2c-a6332227021f': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/108f3d35-1ca5-4da9-9947-5b21d1f345de': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/289869a5-312c-4b21-acc1-d1234b430695': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/9162dea6-8284-4c68-a0a2-371bca76e99e': probe done, cant_read=0 cant_write=1
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading checkpoint txg
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading indirect vdev metadata
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Checking feature flags
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading special MOS directories
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading properties
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading AUX vdevs
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading vdev metadata
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading dedup tables
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading BRT
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Verifying Log Devices
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Verifying pool data
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Calculating deflated space
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Starting import
1762871398   ffff948065858000 spa.c:9320:spa_async_request(): spa=$import-ffff948065858000-Oxidizer async request task=2048
1762871398   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load($import-ffff948065858000-Oxidizer, config trusted): LOADED
1762871398   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load($import-ffff948065858000-Oxidizer, config trusted): UNLOADING
1762871398   ffff948065858000 metaslab.c:1789:spa_set_allocator(): spa allocator: dynamic
1762871398   ffff948065858000 spa.c:6766:spa_import(): spa_import: importing Oxidizer
1762871398   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load(Oxidizer, config trusted): LOADING
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/686fafe2-7468-43c2-91a7-b9b7a1609146': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/0b89f764-f06a-4bb9-acfb-b985b8120226': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a75b850a-4a8c-4176-82c3-97ede4cd76a1': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f040a409-44be-4152-9e2c-91bd51ad9cfe': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d23078b6-a57e-4b42-ba47-260edc04394d': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3eb27884-6106-4ae2-bbec-1eefbb5bb449': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a8cefcfe-aa65-495e-9333-c424e70a5e5c': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/5e51406b-ff4a-475e-933d-9eb87303d55a': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d8e655e4-0cc3-4ca4-a994-9164d23c05a7': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/956a4d1d-0844-42e2-8ff6-dc9e35179e52': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/e3feae67-c536-4ac6-9fe3-0308497e6309': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3096d3a6-1679-4ea5-a431-38e0bb2539e3': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8495ec58-cf8e-4727-9691-1a3738ca1548': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/36f67b68-1566-4613-89d2-9d819e7583fe': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/eac35c2d-40e9-4458-a838-ba6e60abf3a1': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b43f90ff-7b5d-4b75-8ea3-b8070ce0bf35': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f629b7de-244f-4530-9def-adfd82dc1051': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bfbddaa2-341c-48ac-911c-a54f715fd3b2': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8e5bd552-1445-4fe9-ade5-ca00279cf522': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3a560bb5-a9b2-4a59-843c-f7da6dba607a': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bceb9521-187f-48ea-b3e5-be53eb6fa76d': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3b0cd83b-9135-40b2-a9e5-49a7fdf75771': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/cbf9d608-c5e0-4dbb-8da9-9951a3c68bc9': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/78b9bf86-67c0-4771-9c2c-a6332227021f': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/108f3d35-1ca5-4da9-9947-5b21d1f345de': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/9162dea6-8284-4c68-a0a2-371bca76e99e': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/289869a5-312c-4b21-acc1-d1234b430695': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/863fbe3c-7201-4a99-8ce0-4861d00a26b1': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/802ff312-af2a-4ff1-a269-569717392f0d': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b1f3b40d-e611-4fb1-b96a-448c5540279e': probe done, cant_read=0 cant_write=1
1762871398   ffff948065858000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/5e51406b-ff4a-475e-933d-9eb87303d55a': best uberblock found for spa Oxidizer. txg 1051389
1762871398   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load(Oxidizer, config untrusted): using uberblock with txg=1051389
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/0b89f764-f06a-4bb9-acfb-b985b8120226': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3eb27884-6106-4ae2-bbec-1eefbb5bb449': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/686fafe2-7468-43c2-91a7-b9b7a1609146': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f040a409-44be-4152-9e2c-91bd51ad9cfe': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a75b850a-4a8c-4176-82c3-97ede4cd76a1': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d23078b6-a57e-4b42-ba47-260edc04394d': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f629b7de-244f-4530-9def-adfd82dc1051': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b43f90ff-7b5d-4b75-8ea3-b8070ce0bf35': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/eac35c2d-40e9-4458-a838-ba6e60abf3a1': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3096d3a6-1679-4ea5-a431-38e0bb2539e3': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8495ec58-cf8e-4727-9691-1a3738ca1548': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/e3feae67-c536-4ac6-9fe3-0308497e6309': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/36f67b68-1566-4613-89d2-9d819e7583fe': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3a560bb5-a9b2-4a59-843c-f7da6dba607a': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bfbddaa2-341c-48ac-911c-a54f715fd3b2': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b1f3b40d-e611-4fb1-b96a-448c5540279e': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8e5bd552-1445-4fe9-ade5-ca00279cf522': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bceb9521-187f-48ea-b3e5-be53eb6fa76d': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3b0cd83b-9135-40b2-a9e5-49a7fdf75771': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d8e655e4-0cc3-4ca4-a994-9164d23c05a7': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/863fbe3c-7201-4a99-8ce0-4861d00a26b1': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/802ff312-af2a-4ff1-a269-569717392f0d': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/cbf9d608-c5e0-4dbb-8da9-9951a3c68bc9': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/9162dea6-8284-4c68-a0a2-371bca76e99e': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/289869a5-312c-4b21-acc1-d1234b430695': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/956a4d1d-0844-42e2-8ff6-dc9e35179e52': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/78b9bf86-67c0-4771-9c2c-a6332227021f': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/5e51406b-ff4a-475e-933d-9eb87303d55a': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/108f3d35-1ca5-4da9-9947-5b21d1f345de': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a8cefcfe-aa65-495e-9333-c424e70a5e5c': probe done, cant_read=0 cant_write=0
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading checkpoint txg
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading indirect vdev metadata
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Checking feature flags
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading special MOS directories
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading properties
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading AUX vdevs
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading vdev metadata
1762871399   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load(Oxidizer, config trusted): Read 293 log space maps (448 total blocks - blksz = 131072 bytes) in 777 ms
1762871399   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading dedup tables
13

Sense errors on four different drives isn’t confidence-inspiring.

Question - I’m assuming all your spinners are in the NetApp shelf behind the HBA (good) but how are the dedup and log SSD’s connected? The NVMe cards will be on straight PCIe and not subject to potential interference from an HP SmartArray RAID card, but I’m a bit concerned about the dedup devices specifically. (And the presence of dedup in general, as always, but you’ve at least got the dedicated devices for it.)

Is this the point where it usually buys the farm, or does it continue on after here?

14

There is indeed a lot more:

1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/0b89f764-f06a-4bb9-acfb-b985b8120226': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/686fafe2-7468-43c2-91a7-b9b7a1609146': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3eb27884-6106-4ae2-bbec-1eefbb5bb449': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d23078b6-a57e-4b42-ba47-260edc04394d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f040a409-44be-4152-9e2c-91bd51ad9cfe': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a75b850a-4a8c-4176-82c3-97ede4cd76a1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a8cefcfe-aa65-495e-9333-c424e70a5e5c': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d8e655e4-0cc3-4ca4-a994-9164d23c05a7': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/36f67b68-1566-4613-89d2-9d819e7583fe': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3a560bb5-a9b2-4a59-843c-f7da6dba607a': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bfbddaa2-341c-48ac-911c-a54f715fd3b2': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/e3feae67-c536-4ac6-9fe3-0308497e6309': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3096d3a6-1679-4ea5-a431-38e0bb2539e3': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8e5bd552-1445-4fe9-ade5-ca00279cf522': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bceb9521-187f-48ea-b3e5-be53eb6fa76d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3b0cd83b-9135-40b2-a9e5-49a7fdf75771': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/eac35c2d-40e9-4458-a838-ba6e60abf3a1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b43f90ff-7b5d-4b75-8ea3-b8070ce0bf35': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f629b7de-244f-4530-9def-adfd82dc1051': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8495ec58-cf8e-4727-9691-1a3738ca1548': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/863fbe3c-7201-4a99-8ce0-4861d00a26b1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b1f3b40d-e611-4fb1-b96a-448c5540279e': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/5e51406b-ff4a-475e-933d-9eb87303d55a': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/cbf9d608-c5e0-4dbb-8da9-9951a3c68bc9': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/78b9bf86-67c0-4771-9c2c-a6332227021f': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/9162dea6-8284-4c68-a0a2-371bca76e99e': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/956a4d1d-0844-42e2-8ff6-dc9e35179e52': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/802ff312-af2a-4ff1-a269-569717392f0d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/108f3d35-1ca5-4da9-9947-5b21d1f345de': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/289869a5-312c-4b21-acc1-d1234b430695': probe done, cant_read=0 cant_write=1
1762871397   ffff948065858000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a8cefcfe-aa65-495e-9333-c424e70a5e5c': best uberblock found for spa $import-ffff948065858000-Oxidizer. txg 1051389
1762871397   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load($import-ffff948065858000-Oxidizer, config untrusted): using uberblock with txg=1051389
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/0b89f764-f06a-4bb9-acfb-b985b8120226': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3eb27884-6106-4ae2-bbec-1eefbb5bb449': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/686fafe2-7468-43c2-91a7-b9b7a1609146': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d23078b6-a57e-4b42-ba47-260edc04394d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a75b850a-4a8c-4176-82c3-97ede4cd76a1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f040a409-44be-4152-9e2c-91bd51ad9cfe': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a8cefcfe-aa65-495e-9333-c424e70a5e5c': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/36f67b68-1566-4613-89d2-9d819e7583fe': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f629b7de-244f-4530-9def-adfd82dc1051': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bfbddaa2-341c-48ac-911c-a54f715fd3b2': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b43f90ff-7b5d-4b75-8ea3-b8070ce0bf35': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3a560bb5-a9b2-4a59-843c-f7da6dba607a': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/eac35c2d-40e9-4458-a838-ba6e60abf3a1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bceb9521-187f-48ea-b3e5-be53eb6fa76d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3b0cd83b-9135-40b2-a9e5-49a7fdf75771': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/5e51406b-ff4a-475e-933d-9eb87303d55a': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8e5bd552-1445-4fe9-ade5-ca00279cf522': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3096d3a6-1679-4ea5-a431-38e0bb2539e3': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/e3feae67-c536-4ac6-9fe3-0308497e6309': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d8e655e4-0cc3-4ca4-a994-9164d23c05a7': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/956a4d1d-0844-42e2-8ff6-dc9e35179e52': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8495ec58-cf8e-4727-9691-1a3738ca1548': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b1f3b40d-e611-4fb1-b96a-448c5540279e': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/863fbe3c-7201-4a99-8ce0-4861d00a26b1': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/802ff312-af2a-4ff1-a269-569717392f0d': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/cbf9d608-c5e0-4dbb-8da9-9951a3c68bc9': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/78b9bf86-67c0-4771-9c2c-a6332227021f': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/108f3d35-1ca5-4da9-9947-5b21d1f345de': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/289869a5-312c-4b21-acc1-d1234b430695': probe done, cant_read=0 cant_write=1
1762871397   ffff947dddc16600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/9162dea6-8284-4c68-a0a2-371bca76e99e': probe done, cant_read=0 cant_write=1
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading checkpoint txg
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading indirect vdev metadata
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Checking feature flags
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading special MOS directories
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading properties
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading AUX vdevs
1762871397   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading vdev metadata
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading dedup tables
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading BRT
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Verifying Log Devices
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Verifying pool data
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Calculating deflated space
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Starting import
1762871398   ffff948065858000 spa.c:9320:spa_async_request(): spa=$import-ffff948065858000-Oxidizer async request task=2048
1762871398   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load($import-ffff948065858000-Oxidizer, config trusted): LOADED
1762871398   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load($import-ffff948065858000-Oxidizer, config trusted): UNLOADING
1762871398   ffff948065858000 metaslab.c:1789:spa_set_allocator(): spa allocator: dynamic
1762871398   ffff948065858000 spa.c:6766:spa_import(): spa_import: importing Oxidizer
1762871398   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load(Oxidizer, config trusted): LOADING
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/686fafe2-7468-43c2-91a7-b9b7a1609146': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/0b89f764-f06a-4bb9-acfb-b985b8120226': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a75b850a-4a8c-4176-82c3-97ede4cd76a1': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f040a409-44be-4152-9e2c-91bd51ad9cfe': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d23078b6-a57e-4b42-ba47-260edc04394d': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3eb27884-6106-4ae2-bbec-1eefbb5bb449': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a8cefcfe-aa65-495e-9333-c424e70a5e5c': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/5e51406b-ff4a-475e-933d-9eb87303d55a': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d8e655e4-0cc3-4ca4-a994-9164d23c05a7': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/956a4d1d-0844-42e2-8ff6-dc9e35179e52': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/e3feae67-c536-4ac6-9fe3-0308497e6309': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3096d3a6-1679-4ea5-a431-38e0bb2539e3': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8495ec58-cf8e-4727-9691-1a3738ca1548': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/36f67b68-1566-4613-89d2-9d819e7583fe': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/eac35c2d-40e9-4458-a838-ba6e60abf3a1': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b43f90ff-7b5d-4b75-8ea3-b8070ce0bf35': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f629b7de-244f-4530-9def-adfd82dc1051': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bfbddaa2-341c-48ac-911c-a54f715fd3b2': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8e5bd552-1445-4fe9-ade5-ca00279cf522': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3a560bb5-a9b2-4a59-843c-f7da6dba607a': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bceb9521-187f-48ea-b3e5-be53eb6fa76d': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3b0cd83b-9135-40b2-a9e5-49a7fdf75771': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/cbf9d608-c5e0-4dbb-8da9-9951a3c68bc9': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/78b9bf86-67c0-4771-9c2c-a6332227021f': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/108f3d35-1ca5-4da9-9947-5b21d1f345de': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/9162dea6-8284-4c68-a0a2-371bca76e99e': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/289869a5-312c-4b21-acc1-d1234b430695': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/863fbe3c-7201-4a99-8ce0-4861d00a26b1': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/802ff312-af2a-4ff1-a269-569717392f0d': probe done, cant_read=0 cant_write=1
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b1f3b40d-e611-4fb1-b96a-448c5540279e': probe done, cant_read=0 cant_write=1
1762871398   ffff948065858000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/5e51406b-ff4a-475e-933d-9eb87303d55a': best uberblock found for spa Oxidizer. txg 1051389
1762871398   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load(Oxidizer, config untrusted): using uberblock with txg=1051389
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/0b89f764-f06a-4bb9-acfb-b985b8120226': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3eb27884-6106-4ae2-bbec-1eefbb5bb449': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/686fafe2-7468-43c2-91a7-b9b7a1609146': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f040a409-44be-4152-9e2c-91bd51ad9cfe': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a75b850a-4a8c-4176-82c3-97ede4cd76a1': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d23078b6-a57e-4b42-ba47-260edc04394d': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/f629b7de-244f-4530-9def-adfd82dc1051': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b43f90ff-7b5d-4b75-8ea3-b8070ce0bf35': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/eac35c2d-40e9-4458-a838-ba6e60abf3a1': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3096d3a6-1679-4ea5-a431-38e0bb2539e3': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8495ec58-cf8e-4727-9691-1a3738ca1548': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/e3feae67-c536-4ac6-9fe3-0308497e6309': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/36f67b68-1566-4613-89d2-9d819e7583fe': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3a560bb5-a9b2-4a59-843c-f7da6dba607a': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bfbddaa2-341c-48ac-911c-a54f715fd3b2': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/b1f3b40d-e611-4fb1-b96a-448c5540279e': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/8e5bd552-1445-4fe9-ade5-ca00279cf522': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/bceb9521-187f-48ea-b3e5-be53eb6fa76d': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/3b0cd83b-9135-40b2-a9e5-49a7fdf75771': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/d8e655e4-0cc3-4ca4-a994-9164d23c05a7': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/863fbe3c-7201-4a99-8ce0-4861d00a26b1': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/802ff312-af2a-4ff1-a269-569717392f0d': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/cbf9d608-c5e0-4dbb-8da9-9951a3c68bc9': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/9162dea6-8284-4c68-a0a2-371bca76e99e': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/289869a5-312c-4b21-acc1-d1234b430695': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/956a4d1d-0844-42e2-8ff6-dc9e35179e52': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/78b9bf86-67c0-4771-9c2c-a6332227021f': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/5e51406b-ff4a-475e-933d-9eb87303d55a': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/108f3d35-1ca5-4da9-9947-5b21d1f345de': probe done, cant_read=0 cant_write=0
1762871398   ffff947dddac0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/disk/by-partuuid/a8cefcfe-aa65-495e-9333-c424e70a5e5c': probe done, cant_read=0 cant_write=0
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading checkpoint txg
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading indirect vdev metadata
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Checking feature flags
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading special MOS directories
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading properties
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading AUX vdevs
1762871398   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading vdev metadata
1762871399   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load(Oxidizer, config trusted): Read 293 log space maps (448 total blocks - blksz = 131072 bytes) in 777 ms
1762871399   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading dedup tables
1762871539   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Loading BRT
1762871539   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Verifying Log Devices
1762871539   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Verifying pool data
1762871540   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Calculating deflated space
1762871540   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Starting import
1762871540   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Claiming ZIL blocks
1762871540   ffff948065858000 mmp.c:240:mmp_thread_start(): MMP thread started pool 'Oxidizer' gethrtime 6040742546826
1762871540   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Syncing ZIL claims
1762871542   ffff948148703300 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 6, ms_id 172, smp_length 107240, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 6042760 ms, loading_time 7 ms, ms_max_size 18698240, max size error 18685952, old_weight 600000000000001, new_weight 600000000000001
1762871542   ffff947e903eb300 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 4, ms_id 18, smp_length 205240, unflushed_allocs 1662976, unflushed_frees 2138112, freed 0, defer 0 + 0, unloaded time 6042759 ms, loading_time 23 ms, ms_max_size 7103451136, max size error 7102926848, old_weight 800000000000001, new_weight 800000000000001
1762871542   ffff948148703300 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 6, ms_id 177, smp_length 119384, unflushed_allocs 432930816, unflushed_frees 9023488, freed 0, defer 0 + 0, unloaded time 6042767 ms, loading_time 15 ms, ms_max_size 4915200, max size error 4784128, old_weight 580000000000001, new_weight 580000000000001
1762871542   ffff948148703300 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 6, ms_id 86, smp_length 277512, unflushed_allocs 348737536, unflushed_frees 32768, freed 0, defer 0 + 0, unloaded time 6042783 ms, loading_time 26 ms, ms_max_size 1175552, max size error 1155072, old_weight 500000000000002, new_weight 500000000000002
1762871542   ffff947e903eb300 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 4, ms_id 19, smp_length 341184, unflushed_allocs 44433408, unflushed_frees 93089792, freed 0, defer 0 + 0, unloaded time 6042782 ms, loading_time 60 ms, ms_max_size 7685939200, max size error 7683055616, old_weight 800000000000001, new_weight 800000000000001
1762871542   ffff94814b5cb300 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 6, ms_id 9, smp_length 536000, unflushed_allocs 686379008, unflushed_frees 802816, freed 0, defer 0 + 0, unloaded time 6042810 ms, loading_time 48 ms, ms_max_size 1720320, max size error 1671168, old_weight 500000000000001, new_weight 500000000000001
1762871543   ffff94814b5cb300 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 6, ms_id 46, smp_length 85480, unflushed_allocs 21753856, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 6042859 ms, loading_time 9 ms, ms_max_size 647168, max size error 647168, old_weight 4c0000000000039, new_weight 4c0000000000039
1762871543   ffff947e903eb300 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 4, ms_id 27, smp_length 540984, unflushed_allocs 3747840, unflushed_frees 3596288, freed 0, defer 0 + 0, unloaded time 6042843 ms, loading_time 36 ms, ms_max_size 5607448576, max size error 5605744640, old_weight 800000000000001, new_weight 800000000000001
1762871543   ffff947e2aebb300 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 4, ms_id 28, smp_length 483416, unflushed_allocs 95367168, unflushed_frees 345272320, freed 0, defer 0 + 0, unloaded time 6042922 ms, loading_time 68 ms, ms_max_size 4623736832, max size error 4606246912, old_weight 800000000000001, new_weight 800000000000001
1762871543   ffff947e2aebb300 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 4, ms_id 29, smp_length 383064, unflushed_allocs 381382656, unflushed_frees 52125696, freed 0, defer 0 + 0, unloaded time 6042991 ms, loading_time 53 ms, ms_max_size 5539680256, max size error 5531918336, old_weight 800000000000001, new_weight 800000000000001
1762871543   ffff947e2aebcc80 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 6, ms_id 143, smp_length 277904, unflushed_allocs 0, unflushed_frees 0, freed 0, defer 0 + 0, unloaded time 6043325 ms, loading_time 23 ms, ms_max_size 31965184, max size error 31965184, old_weight 4c0000000000039, new_weight 60000000000000c
1762871543   ffff947e922b6600 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 4, ms_id 30, smp_length 598384, unflushed_allocs 215003136, unflushed_frees 382857216, freed 450560, defer 0 + 0, unloaded time 6043373 ms, loading_time 134 ms, ms_max_size 6116331520, max size error 6110806016, old_weight 800000000000001, new_weight 800000000000001
1762871543   ffff947e922b6600 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 4, ms_id 31, smp_length 303784, unflushed_allocs 238346240, unflushed_frees 120246272, freed 266240, defer 0 + 0, unloaded time 6043508 ms, loading_time 21 ms, ms_max_size 6194794496, max size error 6181347328, old_weight 800000000000001, new_weight 800000000000001
1762871543   ffff947e922b4c80 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 4, ms_id 32, smp_length 470160, unflushed_allocs 101896192, unflushed_frees 190005248, freed 28672, defer 0 + 0, unloaded time 6043532 ms, loading_time 82 ms, ms_max_size 6602928128, max size error 6570754048, old_weight 800000000000001, new_weight 800000000000001
1762871543   ffff947e922b4c80 metaslab.c:2602:metaslab_load_impl(): metaslab_load: txg 1051390, spa Oxidizer, vdev_id 4, ms_id 34, smp_length 181624, unflushed_allocs 4673536, unflushed_frees 290816, freed 102400, defer 0 + 0, unloaded time 6043615 ms, loading_time 14 ms, ms_max_size 8526192640, max size error 8526123008, old_weight 800000000000001, new_weight 800000000000001
1762871543   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Updating configs
1762871543   ffff948065858000 spa.c:9320:spa_async_request(): spa=Oxidizer async request task=1
1762871543   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Starting resilvers
1762871543   ffff948065858000 spa.c:9320:spa_async_request(): spa=Oxidizer async request task=16
1762871543   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Restarting device removals
1762871543   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Cleaning up inconsistent objsets
1762871543   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Cleaning up temporary userrefs
1762871543   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Restarting initialize
1762871543   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Restarting TRIM
1762871543   ffff948065858000 spa_misc.c:2391:spa_import_progress_set_notes_impl(): 'Oxidizer' Finished importing
1762871543   ffff948065858000 spa.c:9320:spa_async_request(): spa=Oxidizer async request task=2048
1762871543   ffff948065858000 spa_misc.c:430:spa_load_note(): spa_load(Oxidizer, config trusted): LOADED
1762871543   ffff947e2aeb8000 spa_history.c:307:spa_history_log_sync(): txg 1051391 open pool version 5000; software version zfs-2.3.3-77-ge3dac4894; uts truenas 6.12.33-production+truenas #1 SMP PREEMPT_DYNAMIC Mon Oct 27 18:40:38 UTC 2025 x86_64
1762871547   ffff947e2aeb8000 dsl_scan.c:3859:dsl_process_async_destroys(): freed 227245 blocks in 1883ms from free_bpobj/bptree on Oxidizer in txg 1051391; err=85
root@truenas[/home/truenas_admin]# client_loop: send disconnect: Connection reset
15

So by default it’s trying to enter your pool at txg 1051391 - attempting to rewind to 694409 is way too aggressive.

Do you have a full zdb -l /dev/sdX you can dump in codeblocks? I’d start with picking a more-recent-but-not-latest.

16

Sure do:

root@truenas[/home/truenas_admin]# for dev in /dev/sd*[0-9] /dev/nvme*n*p[0-9]; do
  [ -b "$dev" ] || continue
  echo ">>> Checking $dev"
  sudo zdb -l "$dev" 2>/dev/null | grep -i txg | head -1
done
>>> Checking /dev/sda1
    txg: 694409
>>> Checking /dev/sdaa1
    txg: 694409
>>> Checking /dev/sdab1
    txg: 694409
>>> Checking /dev/sdac1
    txg: 694409
>>> Checking /dev/sdad1
    txg: 694409
>>> Checking /dev/sdae1
    txg: 694409
>>> Checking /dev/sdaf1
    txg: 694409
>>> Checking /dev/sdag1
    txg: 694409
>>> Checking /dev/sdah1
    txg: 694409
>>> Checking /dev/sdai1
    txg: 694409
>>> Checking /dev/sdaj1
    txg: 694409
>>> Checking /dev/sdak1
    txg: 694409
>>> Checking /dev/sdal1
    txg: 694409
>>> Checking /dev/sdam1
    txg: 694409
>>> Checking /dev/sdan1
    txg: 694409
>>> Checking /dev/sdao1
    txg: 694409
>>> Checking /dev/sdap1
    txg: 694409
>>> Checking /dev/sdaq1
    txg: 694409
>>> Checking /dev/sdar1
    txg: 694409
>>> Checking /dev/sdas1
    txg: 694409
>>> Checking /dev/sdat1
    txg: 694409
>>> Checking /dev/sdau1
    txg: 694409
>>> Checking /dev/sdav1
    txg: 694409
>>> Checking /dev/sdaw1
    txg: 694409
>>> Checking /dev/sdax1
    txg: 694409
>>> Checking /dev/sday1
    txg: 694409
>>> Checking /dev/sdaz1
    txg: 694409
>>> Checking /dev/sdb1
    txg: 718810
>>> Checking /dev/sdc1
    txg: 87962
>>> Checking /dev/sdd1
    txg: 694409
>>> Checking /dev/sde1
    txg: 0
>>> Checking /dev/sdf1
>>> Checking /dev/sdf2
>>> Checking /dev/sdf3
    txg: 71975
>>> Checking /dev/sdg1
>>> Checking /dev/sdg2
>>> Checking /dev/sdg3
    txg: 71975
>>> Checking /dev/sdh1
    txg: 718810
>>> Checking /dev/sdi1
    txg: 694409
>>> Checking /dev/sdj1
    txg: 694409
>>> Checking /dev/sdk1
    txg: 694409
>>> Checking /dev/sdl1
    txg: 694409
>>> Checking /dev/sdm1
    txg: 694409
>>> Checking /dev/sdn1
    txg: 694409
>>> Checking /dev/sdo1
    txg: 694409
>>> Checking /dev/sdp1
    txg: 694409
>>> Checking /dev/sdq1
    txg: 694409
>>> Checking /dev/sdr1
    txg: 694409
>>> Checking /dev/sds1
    txg: 694409
>>> Checking /dev/sdt1
    txg: 694409
>>> Checking /dev/sdu1
    txg: 694409
>>> Checking /dev/sdv1
    txg: 694409
>>> Checking /dev/sdw1
    txg: 694409
>>> Checking /dev/sdx1
    txg: 694409
>>> Checking /dev/sdy1
    txg: 694409
>>> Checking /dev/sdz1
    txg: 694409
>>> Checking /dev/nvme0n1p1
    txg: 694409
>>> Checking /dev/nvme1n1p1
    txg: 694409
17

Can you pick a single pool data vdev member and give a full unfiltered

zdb -ul /dev/sdX

It’s gonna be a wall o’ text.

18

Here is a member disk:

root@truenas[/home/truenas_admin]# zdb -ul /dev/sdy1
------------------------------------
LABEL 0
------------------------------------
    version: 5000
    name: 'Oxidizer'
    state: 0
    txg: 694409
    pool_guid: 10025228163071528045
    errata: 0
    hostid: 1031674282
    hostname: 'srv-nas'
    top_guid: 5944411002399756794
    guid: 3670620725219859415
    vdev_children: 7
    vdev_tree:
        type: 'raidz'
        id: 1
        guid: 5944411002399756794
        nparity: 1
        metaslab_array: 2825
        metaslab_shift: 34
        ashift: 12
        asize: 23991803707392
        is_log: 0
        create_txg: 84372
        children[0]:
            type: 'disk'
            id: 0
            guid: 1478166279156742717
            path: '/dev/disk/by-partuuid/108f3d35-1ca5-4da9-9947-5b21d1f345de'
            whole_disk: 0
            DTL: 7238
            create_txg: 84372
        children[1]:
            type: 'disk'
            id: 1
            guid: 883383624340310875
            path: '/dev/disk/by-partuuid/5e51406b-ff4a-475e-933d-9eb87303d55a'
            whole_disk: 0
            DTL: 7235
            create_txg: 84372
        children[2]:
            type: 'disk'
            id: 2
            guid: 16555192700024296508
            path: '/dev/disk/by-partuuid/289869a5-312c-4b21-acc1-d1234b430695'
            whole_disk: 0
            DTL: 7234
            create_txg: 84372
        children[3]:
            type: 'disk'
            id: 3
            guid: 3764441903778271259
            path: '/dev/disk/by-partuuid/863fbe3c-7201-4a99-8ce0-4861d00a26b1'
            whole_disk: 0
            DTL: 7233
            create_txg: 84372
        children[4]:
            type: 'disk'
            id: 4
            guid: 15252468193385415880
            path: '/dev/disk/by-partuuid/956a4d1d-0844-42e2-8ff6-dc9e35179e52'
            whole_disk: 0
            DTL: 7232
            create_txg: 84372
        children[5]:
            type: 'disk'
            id: 5
            guid: 3670620725219859415
            path: '/dev/disk/by-partuuid/b1f3b40d-e611-4fb1-b96a-448c5540279e'
            whole_disk: 0
            DTL: 7231
            create_txg: 84372
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
        com.klarasystems:vdev_zaps_v2
    labels = 0 1 2 3
    Uberblock[0]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051296
        guid_sum = 6731355438852460797
        timestamp = 1762429388 UTC = Thu Nov  6 12:43:08 2025
        bp = DVA[0]=<4:4e2a736000:1000> DVA[1]=<4:6ea656a000:1000> DVA[2]=<4:824c04b000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051296L/1051296P fill=16119 cksum=00000003a15ec307:00000dfc33fdacf0:001af7b1b94fe43f:22b4d3938addac91
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[1]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051361
        guid_sum = 6731355438852460797
        timestamp = 1762454645 UTC = Thu Nov  6 19:44:05 2025
        bp = DVA[0]=<4:7400068000:1000> DVA[1]=<4:780007b000:1000> DVA[2]=<4:7c00028000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051361L/1051361P fill=16054 cksum=000000032982413a:00000c34e3a64a91:00179612e8bd3bc6:1e6aa9e644aaba8b
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[2]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051362
        guid_sum = 6731355438852460797
        timestamp = 1762455293 UTC = Thu Nov  6 19:54:53 2025
        bp = DVA[0]=<4:48001d9000:1000> DVA[1]=<4:4c0021c000:1000> DVA[2]=<4:7c0000e000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051362L/1051362P fill=16048 cksum=00000003dc202ace:00000edcd4dd2161:001ca58bd9b7ec61:24d9cc71adfdf52f
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[3]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051331
        guid_sum = 6731355438852460797
        timestamp = 1762435295 UTC = Thu Nov  6 14:21:35 2025
        bp = DVA[0]=<4:4800197000:1000> DVA[1]=<4:4c001a3000:1000> DVA[2]=<4:740009a000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051331L/1051331P fill=16077 cksum=000000047d2764d6:000011438bea09a6:00213be19cdf6cf2:2ab2b2e6329fe9e0
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[4]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051364
        guid_sum = 6731355438852460797
        timestamp = 1762456586 UTC = Thu Nov  6 20:16:26 2025
        bp = DVA[0]=<4:6c003e7000:1000> DVA[1]=<4:70000b8000:1000> DVA[2]=<4:74001bd000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051364L/1051364P fill=16046 cksum=00000003d3dd9540:00000ebd4193bc60:001c68fa167f85dd:248c0e3cb9846439
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[5]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051333
        guid_sum = 6731355438852460797
        timestamp = 1762436587 UTC = Thu Nov  6 14:43:07 2025
        bp = DVA[0]=<4:6c0010f000:1000> DVA[1]=<4:7000087000:1000> DVA[2]=<4:7c000c6000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051333L/1051333P fill=16074 cksum=00000005359a1575:0000140c5c90a12e:00269ebc26cece57:31a6c42beb120522
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[6]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051366
        guid_sum = 6731355438852460797
        timestamp = 1762457877 UTC = Thu Nov  6 20:37:57 2025
        bp = DVA[0]=<4:6c006cf000:1000> DVA[1]=<4:70000cc000:1000> DVA[2]=<4:740014d000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051366L/1051366P fill=16042 cksum=0000000512e704b5:000013831096c2be:00258eeaa60cca1f:303fbe7e8604caa2
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[7]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051367
        guid_sum = 6731355438852460797
        timestamp = 1762458526 UTC = Thu Nov  6 20:48:46 2025
        bp = DVA[0]=<4:6c0010f000:1000> DVA[1]=<4:7000121000:1000> DVA[2]=<4:7c00045000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051367L/1051367P fill=16042 cksum=000000042cbe40e3:000010108c781e01:001ef13f6cd5279f:27c69eecfc447bfe
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[8]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051368
        guid_sum = 6731355438852460797
        timestamp = 1762459180 UTC = Thu Nov  6 20:59:40 2025
        bp = DVA[0]=<4:4800184000:1000> DVA[1]=<4:4c0016c000:1000> DVA[2]=<4:7c000d3000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051368L/1051368P fill=16037 cksum=00000004c2acdfdb:0000125536cef9d4:002356dba53ce967:2d76d0e140c64fbe
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[9]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051369
        guid_sum = 6731355438852460797
        timestamp = 1762459819 UTC = Thu Nov  6 21:10:19 2025
        bp = DVA[0]=<4:6c0044d000:1000> DVA[1]=<4:7800128000:1000> DVA[2]=<4:7c00009000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051369L/1051369P fill=16035 cksum=000000047a42506b:0000113e5ec37e3d:00213d7502e1903b:2ac3a4cf37907701
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[10]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051370
        guid_sum = 6731355438852460797
        timestamp = 1762460472 UTC = Thu Nov  6 21:21:12 2025
        bp = DVA[0]=<4:6c00613000:1000> DVA[1]=<4:70000e3000:1000> DVA[2]=<4:7c000d2000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051370L/1051370P fill=16031 cksum=00000004786afacb:0000113a09adc8f4:00213a5e56c23331:2ac65f034c34af8d
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[11]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051371
        guid_sum = 6731355438852460797
        timestamp = 1762461125 UTC = Thu Nov  6 21:32:05 2025
        bp = DVA[0]=<4:6c0020f000:1000> DVA[1]=<4:700011b000:1000> DVA[2]=<4:740014a000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051371L/1051371P fill=16030 cksum=00000004fe306a05:000013319db34558:0024eebf10f75cac:2f6db4c878891e90
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[12]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051340
        guid_sum = 6731355438852460797
        timestamp = 1762441100 UTC = Thu Nov  6 15:58:20 2025
        bp = DVA[0]=<4:4800253000:1000> DVA[1]=<4:4c00275000:1000> DVA[2]=<4:6c002fc000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051340L/1051340P fill=16069 cksum=0000000434173a6c:0000102bc3456762:001f23a49432655f:2804d3baac9e8314
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[13]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051373
        guid_sum = 6731355438852460797
        timestamp = 1762462971 UTC = Thu Nov  6 22:02:51 2025
        bp = DVA[0]=<4:48009dd000:1000> DVA[1]=<4:4c006b8000:1000> DVA[2]=<4:70002fc000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051373L/1051373P fill=16028 cksum=000000035e2d1b4c:00000cf7d02da5fb:0018fe8d938f687f:2026be0187ba0cf8
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[14]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051374
        guid_sum = 6731355438852460797
        timestamp = 1762467630 UTC = Thu Nov  6 23:20:30 2025
        bp = DVA[0]=<4:6c008b9000:1000> DVA[1]=<4:70004c7000:1000> DVA[2]=<4:7400492000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051374L/1051374P fill=16027 cksum=000000028f4fa2c8:000009e33194b582:00131dfc600912f1:18ab5b4ed33021a9
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[15]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051343
        guid_sum = 6731355438852460797
        timestamp = 1762443034 UTC = Thu Nov  6 16:30:34 2025
        bp = DVA[0]=<4:7400109000:1000> DVA[1]=<4:780007b000:1000> DVA[2]=<4:7c0006c000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051343L/1051343P fill=16069 cksum=00000003a1909e17:00000dff53e0fd77:001b0244cb717671:22c840b2559570ec
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[16]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051376
        guid_sum = 6731355438852460797
        timestamp = 1762469733 UTC = Thu Nov  6 23:55:33 2025
        bp = DVA[0]=<4:6c006e1000:1000> DVA[1]=<4:7000467000:1000> DVA[2]=<4:7c003ac000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051376L/1051376P fill=16023 cksum=00000003f68c2d0e:00000f44310e3a34:001d6fd51eacef49:25e1e339831b2c85
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[17]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051377
        guid_sum = 6731355438852460797
        timestamp = 1762471979 UTC = Fri Nov  7 00:32:59 2025
        bp = DVA[0]=<4:6c008fd000:1000> DVA[1]=<4:7000532000:1000> DVA[2]=<4:7c001ba000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051377L/1051377P fill=16023 cksum=000000036f6c4982:00000d3ea7f93cda:00198fb47ff89b17:20ec9db3bd9c55c6
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[18]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051378
        guid_sum = 6731355438852460797
        timestamp = 1762501357 UTC = Fri Nov  7 08:42:37 2025
        bp = DVA[0]=<4:4800aba000:1000> DVA[1]=<4:4c00766000:1000> DVA[2]=<4:780036b000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051378L/1051378P fill=16021 cksum=00000002eb0a14c7:00000b461b01851c:0015cd3d56f2906d:1c236588a3e271de
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[19]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051251
        guid_sum = 6731355438852460797
        timestamp = 1762429287 UTC = Thu Nov  6 12:41:27 2025
        bp = DVA[0]=<4:768bfec000:1000> DVA[1]=<4:824b08e000:1000> DVA[2]=<4:86b4c61000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051251L/1051251P fill=16102 cksum=00000004c48a15aa:00001253bc1d0652:002343795470a073:2d48d03ed9747d76
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[20]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051348
        guid_sum = 6731355438852460797
        timestamp = 1762446262 UTC = Thu Nov  6 17:24:22 2025
        bp = DVA[0]=<4:6c00433000:1000> DVA[1]=<4:700023e000:1000> DVA[2]=<4:740010e000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051348L/1051348P fill=16064 cksum=000000040ea0e111:00000f9c96b2d429:001e11ed0754ea3b:26a7d1413eb1d655
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[21]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051381
        guid_sum = 6731355438852460797
        timestamp = 1762510136 UTC = Fri Nov  7 11:08:56 2025
        bp = DVA[0]=<4:6c00aad000:1000> DVA[1]=<4:70005e5000:1000> DVA[2]=<4:7c000ea000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051381L/1051381P fill=16018 cksum=00000003cdb538a6:00000ea8878e67b1:001c46df3cbdd165:2467d24de0027bde
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[22]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051350
        guid_sum = 6731355438852460797
        timestamp = 1762447546 UTC = Thu Nov  6 17:45:46 2025
        bp = DVA[0]=<4:6c004ad000:1000> DVA[1]=<4:70000be000:1000> DVA[2]=<4:7c000a8000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051350L/1051350P fill=16061 cksum=00000004b3ef74c0:0000121ac2f97105:0022e2fb03c2150f:2cddaff75472780e
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[23]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051351
        guid_sum = 6731355438852460797
        timestamp = 1762448188 UTC = Thu Nov  6 17:56:28 2025
        bp = DVA[0]=<4:6c004d4000:1000> DVA[1]=<4:70001e0000:1000> DVA[2]=<4:7400109000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051351L/1051351P fill=16061 cksum=00000005743fb451:000014f885333b9e:00285c2c3a80633d:33d7463c9f2b2237
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[24]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051320
        guid_sum = 6731355438852460797
        timestamp = 1762429465 UTC = Thu Nov  6 12:44:25 2025
        bp = DVA[0]=<4:82669fe000:1000> DVA[1]=<4:7a83a0c000:1000> DVA[2]=<4:870d7c9000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051320L/1051320P fill=16108 cksum=000000035b7e41d9:00000cef648f1ff6:0018f2221dc4afb5:201bd4865bc33bdc
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[25]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051385
        guid_sum = 6731355438852460797
        timestamp = 1762557338 UTC = Sat Nov  8 00:15:38 2025
        bp = DVA[0]=<4:4800bcd000:1000> DVA[1]=<4:4c008ae000:1000> DVA[2]=<4:8000371000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051385L/1051385P fill=16012 cksum=00000004be0417c7:00001242a9e9aae8:002331f0925101a6:2d45d78cd59d5bf3
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[26]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051386
        guid_sum = 6731355438852460797
        timestamp = 1762558057 UTC = Sat Nov  8 00:27:37 2025
        bp = DVA[0]=<4:4800bfa000:1000> DVA[1]=<4:4c00764000:1000> DVA[2]=<4:7400866000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051386L/1051386P fill=16011 cksum=00000003f892a08c:00000f4a1574abc6:001d778026b080fd:25e6fcfe7cfbda84
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[27]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051387
        guid_sum = 6731355438852460797
        timestamp = 1762560721 UTC = Sat Nov  8 01:12:01 2025
        bp = DVA[0]=<4:6c00246000:1000> DVA[1]=<4:7400083000:1000> DVA[2]=<4:80002cb000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051387L/1051387P fill=16009 cksum=000000025a43b0a1:000009196c31b133:00119deef022acf6:16c39dd49d8c98f8
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[28]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051388
        guid_sum = 6731355438852460797
        timestamp = 1762561732 UTC = Sat Nov  8 01:28:52 2025
        bp = DVA[0]=<4:74000a1000:1000> DVA[1]=<4:78000dc000:1000> DVA[2]=<4:880002a000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051388L/1051388P fill=16007 cksum=000000045249c683:000010a386ab906c:0020114f5bfc572a:293f7dd6899dec13
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[29]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051389
        guid_sum = 6731355438852460797
        timestamp = 1762865092 UTC = Tue Nov 11 13:44:52 2025
        bp = DVA[0]=<4:4800cae000:1000> DVA[1]=<4:4c009e4000:1000> DVA[2]=<4:7000505000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051389L/1051389P fill=16003 cksum=00000003cc1837dc:00000e9f3041701f:001c2eb5f88121a9:2440a88e37518dd5
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[30]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051358
        guid_sum = 6731355438852460797
        timestamp = 1762452697 UTC = Thu Nov  6 19:11:37 2025
        bp = DVA[0]=<4:6c006da000:1000> DVA[1]=<4:7000251000:1000> DVA[2]=<4:740014d000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051358L/1051358P fill=16061 cksum=00000004f868990a:0000131fb45f260b:0024d4a94812a76c:2f5714b4475c4251
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
    Uberblock[31]
        magic = 0000000000bab10c
        version = 5000
        txg = 1051359
        guid_sum = 6731355438852460797
        timestamp = 1762453347 UTC = Thu Nov  6 19:22:27 2025
        bp = DVA[0]=<4:6c00618000:1000> DVA[1]=<4:70000d6000:1000> DVA[2]=<4:7c000e7000:1000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=1051359L/1051359P fill=16059 cksum=0000000550897f89:0000146fb1dc3e4e:002755d2e239ece1:3287a0307a3b047f
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        raidz_reflow state=0 off=0
        labels = 0 1 2 3
19

sdy1 actually has no uberblock for 1051391 :thinking:

Is there one or more of these timestamps that are “before system crash”?

If so please pick the most recent one before it went down.

timestamp = 1762429388 UTC = Thu Nov  6 12:43:08 2025
timestamp = 1762454645 UTC = Thu Nov  6 19:44:05 2025
timestamp = 1762455293 UTC = Thu Nov  6 19:54:53 2025
timestamp = 1762435295 UTC = Thu Nov  6 14:21:35 2025
timestamp = 1762456586 UTC = Thu Nov  6 20:16:26 2025
timestamp = 1762436587 UTC = Thu Nov  6 14:43:07 2025
timestamp = 1762457877 UTC = Thu Nov  6 20:37:57 2025
timestamp = 1762458526 UTC = Thu Nov  6 20:48:46 2025
timestamp = 1762459180 UTC = Thu Nov  6 20:59:40 2025
timestamp = 1762459819 UTC = Thu Nov  6 21:10:19 2025
timestamp = 1762460472 UTC = Thu Nov  6 21:21:12 2025
timestamp = 1762461125 UTC = Thu Nov  6 21:32:05 2025
timestamp = 1762441100 UTC = Thu Nov  6 15:58:20 2025
timestamp = 1762462971 UTC = Thu Nov  6 22:02:51 2025
timestamp = 1762467630 UTC = Thu Nov  6 23:20:30 2025
timestamp = 1762443034 UTC = Thu Nov  6 16:30:34 2025
timestamp = 1762469733 UTC = Thu Nov  6 23:55:33 2025
timestamp = 1762471979 UTC = Fri Nov  7 00:32:59 2025
timestamp = 1762501357 UTC = Fri Nov  7 08:42:37 2025
timestamp = 1762429287 UTC = Thu Nov  6 12:41:27 2025
timestamp = 1762446262 UTC = Thu Nov  6 17:24:22 2025
timestamp = 1762510136 UTC = Fri Nov  7 11:08:56 2025
timestamp = 1762447546 UTC = Thu Nov  6 17:45:46 2025
timestamp = 1762448188 UTC = Thu Nov  6 17:56:28 2025
timestamp = 1762429465 UTC = Thu Nov  6 12:44:25 2025
timestamp = 1762557338 UTC = Sat Nov  8 00:15:38 2025
timestamp = 1762558057 UTC = Sat Nov  8 00:27:37 2025
timestamp = 1762560721 UTC = Sat Nov  8 01:12:01 2025
timestamp = 1762561732 UTC = Sat Nov  8 01:28:52 2025
timestamp = 1762865092 UTC = Tue Nov 11 13:44:52 2025
timestamp = 1762452697 UTC = Thu Nov  6 19:11:37 2025
timestamp = 1762453347 UTC = Thu Nov  6 19:22:27 2025
20

No. 6-11 was the day of the crash.

21

So this is still post-crash?

        txg = 1051251
        guid_sum = 6731355438852460797
        timestamp = 1762429287 UTC = Thu Nov  6 12:41:27 2025