Hello
Please HELP
Pool is offline.
H/w config:
Intel(R) Core™ i5-3330 CPU @ 3.00GHz
Gigabyte GA-Z77X-UD5H Motherboard
32GB RAM
6 6TB drives in raidz2 pool with 1 spare 8TB drive
TrueNAS CORE 13.3 on Patriot 32GB USB Stick
List of drives:
ada0: TOSHIBA HDWE160 FS2A ATA8-ACS SATA 3.x device
ada1: ST6000DM001-1XY17Z CC48 ACS-3 ATA SATA 3.x device
ada2: TOSHIBA HDWN180 GX2M ACS-3 ATA SATA 3.x device
ada3: TOSHIBA HDWE160 FS2A ATA8-ACS SATA 3.x device
ada4: ST6000DM001-1XY17Z CC48 ACS-3 ATA SATA 3.x device
ada5: HGST HDN726060ALE610 APGNT517 ACS-2 ATA SATA 3.x device
ada6: HGST HDN726060ALE610 APGNT517 ACS-2 ATA SATA 3.x device
I ran following commands:
# zpool status -v
pool: boot-pool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
da0p2 ONLINE 0 0 0
errors: No known data errors
Then I ran:
# zpool import
pool: storage
id: 13423559279385470407
state: ONLINE
status: One or more devices were being resilvered.
action: The pool can be imported using its name or numeric identifier.
config:
storage ONLINE
raidz2-0 ONLINE
gptid/95632bf3-c2d5-11e5-9d80-94de80a78ddd ONLINE
gptid/610faf79-c2d5-11e5-9d80-94de80a78ddd ONLINE
spare-2 ONLINE
gptid/5bdb0168-c232-11e5-8fb6-94de80a78ddd ONLINE
gptid/c3cc7206-0ac3-11ef-aca8-94de80a78ddd ONLINE
gptid/7c62ff80-c232-11e5-8fb6-94de80a78ddd ONLINE
gptid/75efd457-c2d5-11e5-9d80-94de80a78ddd ONLINE
gptid/43837a79-071d-11e5-9ef5-94de80a78ddd ONLINE
spares
gptid/c3cc7206-0ac3-11ef-aca8-94de80a78ddd
Then…
# zpool import -Fn -R /mnt storage
# zpool import -F -R /mnt storage
cannot import 'storage': I/O error
Destroy and re-create the pool from
a backup source.
Drive list:
# glabel status | sort
Name Status Components
gptid/437b8227-071d-11e5-9ef5-94de80a78ddd N/A ada5p1
gptid/43837a79-071d-11e5-9ef5-94de80a78ddd N/A ada5p2
gptid/5bc9bedc-c232-11e5-8fb6-94de80a78ddd N/A ada4p1
gptid/5bdb0168-c232-11e5-8fb6-94de80a78ddd N/A ada4p2
gptid/60fe2ac7-c2d5-11e5-9d80-94de80a78ddd N/A ada0p1
gptid/610faf79-c2d5-11e5-9d80-94de80a78ddd N/A ada0p2
gptid/75dd4489-c2d5-11e5-9d80-94de80a78ddd N/A ada1p1
gptid/75efd457-c2d5-11e5-9d80-94de80a78ddd N/A ada1p2
gptid/794b8b4b-9d97-11ef-a538-94de80a78ddd N/A da0p1
gptid/7c52403b-c232-11e5-8fb6-94de80a78ddd N/A ada3p1
gptid/7c62ff80-c232-11e5-8fb6-94de80a78ddd N/A ada3p2
gptid/955b1e9a-c2d5-11e5-9d80-94de80a78ddd N/A ada6p1
gptid/95632bf3-c2d5-11e5-9d80-94de80a78ddd N/A ada6p2
gptid/c3b91b46-0ac3-11ef-aca8-94de80a78ddd N/A ada2p1
gptid/c3cc7206-0ac3-11ef-aca8-94de80a78ddd N/A ada2p2
I noticed in the boot log these messages:
Nov 8 00:20:13 jyoti-nas Importing storage
Nov 8 00:20:13 jyoti-nas dsl_scan.c:571:dsl_scan_init(): resilver can't excise DTL_MISSING when finished; restarting in txg 56384479
Nov 8 00:20:13 jyoti-nas spa.c:8392:spa_async_request(): spa=$import async request task=2048
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load($import, config trusted): LOADED
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load($import, config trusted): UNLOADING
Nov 8 00:20:13 jyoti-nas spa.c:6110:spa_import(): spa_import: importing storage
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): LOADING
Nov 8 00:20:13 jyoti-nas vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/610faf79-c2d5-11e5-9d80-94de80a78ddd': best uberblock found for spa storage. txg 56384478
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config untrusted): using uberblock with txg=56384478
Nov 8 00:20:13 jyoti-nas dsl_scan.c:571:dsl_scan_init(): resilver can't excise DTL_MISSING when finished; restarting in txg 56384479
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): read 11 log space maps (11 total blocks - blksz = 131072 bytes) in 53 ms
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_verify found 2 metadata errors and 0 data errors
Nov 8 00:20:13 jyoti-nas spa_misc.c:404:spa_load_failed(): spa_load(storage, config trusted): FAILED: spa_load_verify failed [error=5]
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): UNLOADING
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_retry: rewind, max txg: 56384477
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): LOADING
Nov 8 00:20:13 jyoti-nas vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/610faf79-c2d5-11e5-9d80-94de80a78ddd': best uberblock found for spa storage. txg 56384477
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config untrusted): using uberblock with txg=56384477
Nov 8 00:20:13 jyoti-nas dsl_scan.c:571:dsl_scan_init(): resilver can't excise DTL_MISSING when finished; restarting in txg 56384478
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): read 10 log space maps (10 total blocks - blksz = 131072 bytes) in 32 ms
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_verify found 4 metadata errors and 0 data errors
Nov 8 00:20:13 jyoti-nas spa_misc.c:404:spa_load_failed(): spa_load(storage, config trusted): FAILED: spa_load_verify failed [error=5]
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): UNLOADING
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_retry: rewind, max txg: 56384476
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): LOADING
Nov 8 00:20:13 jyoti-nas vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/610faf79-c2d5-11e5-9d80-94de80a78ddd': best uberblock found for spa storage. txg 56384476
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config untrusted): using uberblock with txg=56384476
Nov 8 00:20:13 jyoti-nas dsl_scan.c:571:dsl_scan_init(): resilver can't excise DTL_MISSING when finished; restarting in txg 56384478
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): read 10 log space maps (10 total blocks - blksz = 131072 bytes) in 23 ms
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_verify found 4 metadata errors and 0 data errors
Nov 8 00:20:13 jyoti-nas spa_misc.c:404:spa_load_failed(): spa_load(storage, config trusted): FAILED: spa_load_verify failed [error=5]
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): UNLOADING
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_retry: rewind, max txg: 56384475
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): LOADING
Nov 8 00:20:13 jyoti-nas vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/610faf79-c2d5-11e5-9d80-94de80a78ddd': best uberblock found for spa storage. txg 56384475
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config untrusted): using uberblock with txg=56384475
Nov 8 00:20:13 jyoti-nas dsl_scan.c:571:dsl_scan_init(): resilver can't excise DTL_MISSING when finished; restarting in txg 56384478
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): read 10 log space maps (11 total blocks - blksz = 131072 bytes) in 38 ms
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_verify found 2 metadata errors and 0 data errors
Nov 8 00:20:13 jyoti-nas spa_misc.c:404:spa_load_failed(): spa_load(storage, config trusted): FAILED: spa_load_verify failed [error=5]
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): UNLOADING
Nov 8 00:20:13 jyoti-nas spa.c:6254:spa_tryimport(): spa_tryimport: importing storage
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load($import, config trusted): LOADING
Nov 8 00:20:13 jyoti-nas spa.c:8392:spa_async_request(): spa=$import async request task=1
Nov 8 00:20:13 jyoti-nas vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/610faf79-c2d5-11e5-9d80-94de80a78ddd': best uberblock found for spa $import. txg 56384478
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load($import, config untrusted): using uberblock with txg=56384478
Nov 8 00:20:13 jyoti-nas dsl_scan.c:571:dsl_scan_init(): resilver can't excise DTL_MISSING when finished; restarting in txg 56384479
Nov 8 00:20:13 jyoti-nas spa.c:8392:spa_async_request(): spa=$import async request task=2048
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load($import, config trusted): LOADED
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load($import, config trusted): UNLOADING
Nov 8 00:20:13 jyoti-nas spa.c:6110:spa_import(): spa_import: importing storage
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): LOADING
Nov 8 00:20:13 jyoti-nas vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/610faf79-c2d5-11e5-9d80-94de80a78ddd': best uberblock found for spa storage. txg 56384478
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config untrusted): using uberblock with txg=56384478
Nov 8 00:20:13 jyoti-nas dsl_scan.c:571:dsl_scan_init(): resilver can't excise DTL_MISSING when finished; restarting in txg 56384479
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): read 11 log space maps (11 total blocks - blksz = 131072 bytes) in 53 ms
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_verify found 2 metadata errors and 0 data errors
Nov 8 00:20:13 jyoti-nas spa_misc.c:404:spa_load_failed(): spa_load(storage, config trusted): FAILED: spa_load_verify failed [error=5]
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): UNLOADING
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_retry: rewind, max txg: 56384477
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): LOADING
Nov 8 00:20:13 jyoti-nas vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/610faf79-c2d5-11e5-9d80-94de80a78ddd': best uberblock found for spa storage. txg 56384477
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config untrusted): using uberblock with txg=56384477
Nov 8 00:20:13 jyoti-nas dsl_scan.c:571:dsl_scan_init(): resilver can't excise DTL_MISSING when finished; restarting in txg 56384478
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): read 10 log space maps (10 total blocks - blksz = 131072 bytes) in 32 ms
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_verify found 4 metadata errors and 0 data errors
Nov 8 00:20:13 jyoti-nas spa_misc.c:404:spa_load_failed(): spa_load(storage, config trusted): FAILED: spa_load_verify failed [error=5]
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): UNLOADING
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_retry: rewind, max txg: 56384476
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): LOADING
Nov 8 00:20:13 jyoti-nas vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/610faf79-c2d5-11e5-9d80-94de80a78ddd': best uberblock found for spa storage. txg 56384476
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config untrusted): using uberblock with txg=56384476
Nov 8 00:20:13 jyoti-nas dsl_scan.c:571:dsl_scan_init(): resilver can't excise DTL_MISSING when finished; restarting in txg 56384478
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): read 10 log space maps (10 total blocks - blksz = 131072 bytes) in 40 ms
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_verify found 4 metadata errors and 0 data errors
Nov 8 00:20:13 jyoti-nas spa_misc.c:404:spa_load_failed(): spa_load(storage, config trusted): FAILED: spa_load_verify failed [error=5]
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): UNLOADING
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_retry: rewind, max txg: 56384475
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): LOADING
Nov 8 00:20:13 jyoti-nas vdev.c:161:vdev_dbgmsg(): disk vdev '/dev/gptid/610faf79-c2d5-11e5-9d80-94de80a78ddd': best uberblock found for spa storage. txg 56384475
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config untrusted): using uberblock with txg=56384475
Nov 8 00:20:13 jyoti-nas dsl_scan.c:571:dsl_scan_init(): resilver can't excise DTL_MISSING when finished; restarting in txg 56384478
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): read 10 log space maps (11 total blocks - blksz = 131072 bytes) in 37 ms
Nov 8 00:20:13 jyoti-nas Importing usb_bkp
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): spa_load_verify found 2 metadata errors and 0 data errors
Nov 8 00:20:13 jyoti-nas spa_misc.c:404:spa_load_failed(): spa_load(storage, config trusted): FAILED: spa_load_verify failed [error=5]
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load(storage, config trusted): UNLOADING
Nov 8 00:20:13 jyoti-nas spa.c:6254:spa_tryimport(): spa_tryimport: importing storage
Nov 8 00:20:13 jyoti-nas spa.c:6259:spa_tryimport(): spa_tryimport: using cachefile '/data/zfs/zpool.cache.saved'
Nov 8 00:20:13 jyoti-nas spa_misc.c:419:spa_load_note(): spa_load($import, config trusted): LOADING
Nov 8 00:20:13 jyoti-nas Pools import completed
Please help.
Thanks so much
Ravi