After I exported the pool when requested, the only thing that my GUI shows is an empty page with a create pool option. The import pool button shows me “Murphy | 5933952339070335385” as an option to select but it fails when I press the import button.
I only restarted the system twice after the export, and it still shows me an empty page waiting for me to import or create a pool. So a restart did not change anything displayed before the restart.
The fact that we see the pool as ready to import means that there should be labels present but yet we’re not getting valid ones from zdb is odd.
@NightWatcher so the result of sudo zdb -l /dev/sda1 through sudo zdb -l /dev/sdb1 and on for your disks gives you the failed to unpack?
Hi,
I just double-checked, and the sudo zdb -l /dev/sd<n>1 command is now showing something.
Please note that I am guiding my sister remotely on how to do it, so there is a small chance she misspelled the command the first time and added an unwanted space somewhere.
- sudo /sbin/zdb -l /dev/sd
admin@truenas[~]$ sudo /sbin/zdb -l /dev/sda
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
admin@truenas[~]$ sudo /sbin/zdb -l /dev/sdb
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
admin@truenas[~]$ sudo /sbin/zdb -l /dev/sdc
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
admin@truenas[~]$ sudo /sbin/zdb -l /dev/sdd
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
admin@truenas[~]$ sudo /sbin/zdb -l /dev/sde
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
sudo /sbin/zdb -l /dev/sd1
admin@truenas[~]$ sudo /sbin/zdb -l /dev/sda1
------------------------------------
LABEL 0
------------------------------------
version: 5000
name: 'Murphy'
state: 0
txg: 9294176
pool_guid: 5933952339070335385
errata: 0
hostid: 2061286049
hostname: 'truenas'
top_guid: 17458902741250208291
guid: 16130124137660907058
vdev_children: 2
vdev_tree:
type: 'raidz'
id: 0
guid: 17458902741250208291
nparity: 2
metaslab_array: 76
metaslab_shift: 34
ashift: 12
asize: 90001000366080
is_log: 0
create_txg: 4
children[0]:
type: 'disk'
id: 0
guid: 16130124137660907058
path: '/dev/disk/by-partuuid/3435f367-3b33-4f01-8840-1290d5fc86da'
whole_disk: 0
DTL: 16993
create_txg: 4
children[1]:
type: 'disk'
id: 1
guid: 5060027678964425353
path: '/dev/disk/by-partuuid/fe2af012-d601-45b8-a2ee-69807e123191'
whole_disk: 0
DTL: 16992
create_txg: 4
children[2]:
type: 'disk'
id: 2
guid: 16064810201755477520
path: '/dev/disk/by-partuuid/8821a923-8f37-403e-b398-7b142675292e'
whole_disk: 0
DTL: 16991
create_txg: 4
children[3]:
type: 'disk'
id: 3
guid: 4670711653886402692
path: '/dev/disk/by-partuuid/c6f49e05-23f1-44e7-85d9-2992698893a5'
whole_disk: 0
DTL: 16990
create_txg: 4
children[4]:
type: 'disk'
id: 4
guid: 7773481090224402799
path: '/dev/disk/by-partuuid/355b7bd2-2d89-4789-b13b-81470e7011f4'
whole_disk: 0
DTL: 1937
create_txg: 4
features_for_read:
com.delphix:hole_birth
com.delphix:embedded_data
com.klarasystems:vdev_zaps_v2
labels = 0 1 2 3
admin@truenas[~]$ sudo /sbin/zdb -l /dev/sdb1
------------------------------------
LABEL 0
------------------------------------
version: 5000
name: 'Murphy'
state: 0
txg: 9294176
pool_guid: 5933952339070335385
errata: 0
hostid: 2061286049
hostname: 'truenas'
top_guid: 17458902741250208291
guid: 7773481090224402799
vdev_children: 2
vdev_tree:
type: 'raidz'
id: 0
guid: 17458902741250208291
nparity: 2
metaslab_array: 76
metaslab_shift: 34
ashift: 12
asize: 90001000366080
is_log: 0
create_txg: 4
children[0]:
type: 'disk'
id: 0
guid: 16130124137660907058
path: '/dev/disk/by-partuuid/3435f367-3b33-4f01-8840-1290d5fc86da'
whole_disk: 0
DTL: 16993
create_txg: 4
children[1]:
type: 'disk'
id: 1
guid: 5060027678964425353
path: '/dev/disk/by-partuuid/fe2af012-d601-45b8-a2ee-69807e123191'
whole_disk: 0
DTL: 16992
create_txg: 4
children[2]:
type: 'disk'
id: 2
guid: 16064810201755477520
path: '/dev/disk/by-partuuid/8821a923-8f37-403e-b398-7b142675292e'
whole_disk: 0
DTL: 16991
create_txg: 4
children[3]:
type: 'disk'
id: 3
guid: 4670711653886402692
path: '/dev/disk/by-partuuid/c6f49e05-23f1-44e7-85d9-2992698893a5'
whole_disk: 0
DTL: 16990
create_txg: 4
children[4]:
type: 'disk'
id: 4
guid: 7773481090224402799
path: '/dev/disk/by-partuuid/355b7bd2-2d89-4789-b13b-81470e7011f4'
whole_disk: 0
DTL: 1937
create_txg: 4
features_for_read:
com.delphix:hole_birth
com.delphix:embedded_data
com.klarasystems:vdev_zaps_v2
labels = 0 1 2 3
admin@truenas[~]$ sudo /sbin/zdb -l /dev/sdc1
------------------------------------
LABEL 0
------------------------------------
version: 5000
name: 'Murphy'
state: 0
txg: 9294176
pool_guid: 5933952339070335385
errata: 0
hostid: 2061286049
hostname: 'truenas'
top_guid: 17458902741250208291
guid: 5060027678964425353
vdev_children: 2
vdev_tree:
type: 'raidz'
id: 0
guid: 17458902741250208291
nparity: 2
metaslab_array: 76
metaslab_shift: 34
ashift: 12
asize: 90001000366080
is_log: 0
create_txg: 4
children[0]:
type: 'disk'
id: 0
guid: 16130124137660907058
path: '/dev/disk/by-partuuid/3435f367-3b33-4f01-8840-1290d5fc86da'
whole_disk: 0
DTL: 16993
create_txg: 4
children[1]:
type: 'disk'
id: 1
guid: 5060027678964425353
path: '/dev/disk/by-partuuid/fe2af012-d601-45b8-a2ee-69807e123191'
whole_disk: 0
DTL: 16992
create_txg: 4
children[2]:
type: 'disk'
id: 2
guid: 16064810201755477520
path: '/dev/disk/by-partuuid/8821a923-8f37-403e-b398-7b142675292e'
whole_disk: 0
DTL: 16991
create_txg: 4
children[3]:
type: 'disk'
id: 3
guid: 4670711653886402692
path: '/dev/disk/by-partuuid/c6f49e05-23f1-44e7-85d9-2992698893a5'
whole_disk: 0
DTL: 16990
create_txg: 4
children[4]:
type: 'disk'
id: 4
guid: 7773481090224402799
path: '/dev/disk/by-partuuid/355b7bd2-2d89-4789-b13b-81470e7011f4'
whole_disk: 0
DTL: 1937
create_txg: 4
features_for_read:
com.delphix:hole_birth
com.delphix:embedded_data
com.klarasystems:vdev_zaps_v2
labels = 0 1 2 3
admin@truenas[~]$ sudo /sbin/zdb -l /dev/sdd1
------------------------------------
LABEL 0
------------------------------------
version: 5000
name: 'Murphy'
state: 0
txg: 9294176
pool_guid: 5933952339070335385
errata: 0
hostid: 2061286049
hostname: 'truenas'
top_guid: 17458902741250208291
guid: 16064810201755477520
vdev_children: 2
vdev_tree:
type: 'raidz'
id: 0
guid: 17458902741250208291
nparity: 2
metaslab_array: 76
metaslab_shift: 34
ashift: 12
asize: 90001000366080
is_log: 0
create_txg: 4
children[0]:
type: 'disk'
id: 0
guid: 16130124137660907058
path: '/dev/disk/by-partuuid/3435f367-3b33-4f01-8840-1290d5fc86da'
whole_disk: 0
DTL: 16993
create_txg: 4
children[1]:
type: 'disk'
id: 1
guid: 5060027678964425353
path: '/dev/disk/by-partuuid/fe2af012-d601-45b8-a2ee-69807e123191'
whole_disk: 0
DTL: 16992
create_txg: 4
children[2]:
type: 'disk'
id: 2
guid: 16064810201755477520
path: '/dev/disk/by-partuuid/8821a923-8f37-403e-b398-7b142675292e'
whole_disk: 0
DTL: 16991
create_txg: 4
children[3]:
type: 'disk'
id: 3
guid: 4670711653886402692
path: '/dev/disk/by-partuuid/c6f49e05-23f1-44e7-85d9-2992698893a5'
whole_disk: 0
DTL: 16990
create_txg: 4
children[4]:
type: 'disk'
id: 4
guid: 7773481090224402799
path: '/dev/disk/by-partuuid/355b7bd2-2d89-4789-b13b-81470e7011f4'
whole_disk: 0
DTL: 1937
create_txg: 4
features_for_read:
com.delphix:hole_birth
com.delphix:embedded_data
com.klarasystems:vdev_zaps_v2
labels = 0 1 2 3
admin@truenas[~]$ sudo /sbin/zdb -l /dev/sde1
------------------------------------
LABEL 0
------------------------------------
version: 5000
name: 'Murphy'
state: 0
txg: 9294176
pool_guid: 5933952339070335385
errata: 0
hostid: 2061286049
hostname: 'truenas'
top_guid: 17458902741250208291
guid: 4670711653886402692
vdev_children: 2
vdev_tree:
type: 'raidz'
id: 0
guid: 17458902741250208291
nparity: 2
metaslab_array: 76
metaslab_shift: 34
ashift: 12
asize: 90001000366080
is_log: 0
create_txg: 4
children[0]:
type: 'disk'
id: 0
guid: 16130124137660907058
path: '/dev/disk/by-partuuid/3435f367-3b33-4f01-8840-1290d5fc86da'
whole_disk: 0
DTL: 16993
create_txg: 4
children[1]:
type: 'disk'
id: 1
guid: 5060027678964425353
path: '/dev/disk/by-partuuid/fe2af012-d601-45b8-a2ee-69807e123191'
whole_disk: 0
DTL: 16992
create_txg: 4
children[2]:
type: 'disk'
id: 2
guid: 16064810201755477520
path: '/dev/disk/by-partuuid/8821a923-8f37-403e-b398-7b142675292e'
whole_disk: 0
DTL: 16991
create_txg: 4
children[3]:
type: 'disk'
id: 3
guid: 4670711653886402692
path: '/dev/disk/by-partuuid/c6f49e05-23f1-44e7-85d9-2992698893a5'
whole_disk: 0
DTL: 16990
create_txg: 4
children[4]:
type: 'disk'
id: 4
guid: 7773481090224402799
path: '/dev/disk/by-partuuid/355b7bd2-2d89-4789-b13b-81470e7011f4'
whole_disk: 0
DTL: 1937
create_txg: 4
features_for_read:
com.delphix:hole_birth
com.delphix:embedded_data
com.klarasystems:vdev_zaps_v2
labels = 0 1 2 3
admin@truenas[~]$
All txg values align so this is a good start.
See if zpool import -fF -n Murphy immediately throws an error. If it does, please give me the output of tail -n 200 /proc/spl/kstat/zfs/dbgmsg inside of codeblocks.
If the zpool import does not throw an error and instead tells you “can be rewound to time X” or simply goes to a newline then the next step is to try a read-only import:
zpool import -fF Murphy -R /mnt -o readonly=on
If that works, check your /mnt/Murphy path for your files, and zpool status for health. If it fails, get the tail dbgmsg from above.
If the read-only import works, your files look like they’re present, and zpool status -v says you’re okay - then export the pool with zpool export Murphy before importing for good with zpool import -fF Murphy -R /mnt and that should commit the changes. Once that’s done, issue a scrub, reboot the system and it should auto-mount.
Sorry for late reply, I was asleep
-
sudo zpool import -fF -n Murphy
It shows nothing and return to a new line. -
sudo zpool import -fF Murphy -R /mnt -o readonly=on
It shows nothing and return to a new line. -
ls /mnt/Murphy
Lists my folders:Media Storage ix-applications -
sudo zpool status
admin@truenas[~]$ sudo zpool status
pool: Murphy
state: ONLINE
scan: scrub repaired 0B in 12:17:15 with 0 errors on Sun Jun 8 12:17:16 2025
config:
NAME STATE READ WRITE CKSUM
Murphy ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
3435f367-3b33-4f01-8840-1290d5fc86da ONLINE 0 0 0
fe2af012-d601-45b8-a2ee-69807e123191 ONLINE 0 0 0
8821a923-8f37-403e-b398-7b142675292e ONLINE 0 0 0
c6f49e05-23f1-44e7-85d9-2992698893a5 ONLINE 0 0 0
355b7bd2-2d89-4789-b13b-81470e7011f4 ONLINE 0 0 0
logs
a479fe8b-ff6b-418e-82c4-a05f9c0da709 ONLINE 0 0 0
errors: No known data errors
pool: boot-pool
state: ONLINE
scan: scrub repaired 0B in 00:00:31 with 0 errors on Mon Jun 9 03:45:33 2025
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
nvme1n1p3 ONLINE 0 0 0
errors: No known data errors
-
sudo zpool export Murphy
It shows nothing and return to a new line. -
sudo zpool import -fF Murphy -R /mnt
It shows nothing and return to a new line. -
sudo zpool scrub Murphy
It shows nothing and return to a new line. -
sudo zpool status
dmin@truenas[~]$ sudo zpool status
pool: Murphy
state: ONLINE
scan: scrub in progress since Fri Jun 13 15:33:18 2025
888G / 38.2T scanned at 8.14G/s, 0B / 38.2T issued
0B repaired, 0.00% done, no estimated completion time
config:
NAME STATE READ WRITE CKSUM
Murphy ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
3435f367-3b33-4f01-8840-1290d5fc86da ONLINE 0 0 0
fe2af012-d601-45b8-a2ee-69807e123191 ONLINE 0 0 0
8821a923-8f37-403e-b398-7b142675292e ONLINE 0 0 0
c6f49e05-23f1-44e7-85d9-2992698893a5 ONLINE 0 0 0
355b7bd2-2d89-4789-b13b-81470e7011f4 ONLINE 0 0 0
logs
a479fe8b-ff6b-418e-82c4-a05f9c0da709 ONLINE 0 0 0
errors: No known data errors
pool: boot-pool
state: ONLINE
scan: scrub repaired 0B in 00:00:31 with 0 errors on Mon Jun 9 03:45:33 2025
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
nvme1n1p3 ONLINE 0 0 0
errors: No known data errors
admin@truenas[~]$
Will wait for the scrub to finish, then run zpool status -v, after that is all is well, will reboot the system and it should work? So far pool is not showing as imported in the GUI.
ETA 17h to finish scrub
I would export the zpool via command line before rebooting and after the reboot (if it is not imported) then try to import it in the GUI.
For zpool import, that actually means “success” (and no issue to report)! Hence you could list, show status and initiate a scrub. Let it finish before exporting and reimporting from GUI to be sure.
I agree with both the above comments but tend to agree with @etorix from a slightly safer point of view.
Sorry I was unclear, after the scrub completes export from the command line and then reboot, after the reboot import from the GUI.
It’s worth noting that even after an import from the GUI you will still likely want/need to reboot, because your system dataset was likely on that pool and a number of services will have been rather upset that their backing storage was absent during their past startup. So once it’s back in there with a GUI import, give it a reboot, and ensure that everything is back to normal.
sudo zpool status -v
admin@truenas[~]$ sudo zpool status -v
pool: Murphy
state: ONLINE
scan: scrub repaired 0B in 11:30:35 with 0 errors on Sat Jun 14 03:03:53 2025
config:
NAME STATE READ WRITE CKSUM
Murphy ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
3435f367-3b33-4f01-8840-1290d5fc86da ONLINE 0 0 0
fe2af012-d601-45b8-a2ee-69807e123191 ONLINE 0 0 0
8821a923-8f37-403e-b398-7b142675292e ONLINE 0 0 0
c6f49e05-23f1-44e7-85d9-2992698893a5 ONLINE 0 0 0
355b7bd2-2d89-4789-b13b-81470e7011f4 ONLINE 0 0 0
logs
a479fe8b-ff6b-418e-82c4-a05f9c0da709 ONLINE 0 0 0
errors: No known data errors
pool: boot-pool
state: ONLINE
scan: scrub repaired 0B in 00:00:31 with 0 errors on Mon Jun 9 03:45:33 2025
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
nvme1n1p3 ONLINE 0 0 0
errors: No known data errors
sudo zpool export Murphy
No errors or text shown
I then rebooted, successfully imported via GUI, and then rebooted again.
The pool seems to be working. Will monitor it for the next couple of weeks.
Overall thank you so much everyone! you’re literal life savers.
What could have caused this issue, and how can I prevent it in the future? Could it be because I didn’t select “detach” before powering down the system? Initially, I turned off the system, removed the drive, powered it back on, and let it run for a while with one less drive while waiting for replacement drive to arrive. Later, I shut it down again, installed the new drive, booted up, initiated the replacement process, ran a scrub and waited, and then the issue occurred, possibly before it finished or after.
I’m not 100% sure what triggered this, other than there potentially being a bug somewhere that’s been patched since your version of 23.10.1 is rather old. I’d recommend you jump up a few versions - where to stop will depend on if you have Apps, how they’re configured, and your use (if any) of VMs.
I see, thank you, will update things once I am back home.