Cannot reassign drives to existing pool/Existing pool cannot be imported

Troll_CLGT · September 30, 2024, 12:33pm

I have a setup that uses 3 2TB drives in a RAIDZ1 layout. Everything was fine until one of the drives decided that it doesn’t want to work anymore, despite only being 6 months old. The pool got degraded and the drive is listed as faulty. I tried turning off the entire server (which is a proxmox server with a couple of containers and a TrueNAS VM on it with a passthrough SAS RAID card in IT mode) and checking the connections to the drives.

When I rebooted the server, the faulty drive was listed as unavailable, but the pool is gone and can’t be seen in the Datasets section or accessed from my Windows machine. I turned off the server again, this time removing the plugs to the faulty drive.

So here we are, after a couple of reboots, all three drives are recognized by TrueNAS. I’m greeted with this. The drives are unassigned so I thought that I could import the pool like before, since they should still be in the same VDEV. But not this time, it didn’t show up.

Thinking that I should reassign the drives, I tried that to, until I realize I can’t because it might delete all of data and that I can’t press the “Update Pool” button.

A quick trip to the Disk section in the Storage tab and it said that my pool was “exported”, even though I never did.

My setup is as follow:
Intel Xeon E5-2673 v3
2x 16GB ECC REG DDR3
OEM X99 board (more info here)
Dell H310 LSI SAS 2008 RAID card in IT mode

The data in there is really important to me. All of my family photos is in there. I know it isn’t a good redundant setup with backup drives but this was all that I could afford. I never expected one of the drives to be faulty this soon.

etorix · September 30, 2024, 6:15pm

Please put relevant pictures in your post, not as link to an external site.
Hopefufully you have NOT reassigned the drives, so the next step would be the output of zpool import.

Troll_CLGT · October 1, 2024, 4:36am

Hi. sorry for the late reply. I have updated the post with images relevant to the issue and here is the output of zpool import

root@truenas[~]# zpool import
   pool: prox-pool
     id: 1001810977950781727
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

        prox-pool                                 ONLINE
          raidz1-0                                ONLINE
            fd9b1ff9-5fed-412b-baad-ee55f0ff712d  ONLINE
            877645c8-8ea7-4602-bc70-9d40cf905763  ONLINE
            5dc0f711-2501-4824-9689-d427baf98f9e  ONLINE

etorix · October 1, 2024, 6:24am

So zpool import prox-pool should work.
Then a scrub, for safety… and thinking about some backup.

Troll_CLGT · October 1, 2024, 7:04am

root@truenas[~]# zpool import prox-pool
cannot import 'prox-pool': one or more devices is currently unavailable

I think it’s because one of the drives is faulty. I keeps hearing a sort of “tick” sound whenver I tried the command.

Not sure what is the most economical backup option for me. I live in Vietnam so certain part can be hard to find. My option at the moment would be to buy a high capacity HDD and stick it the server itself, doing some manual periodical backup tasks until I can find a way to automate the entire process without leaving the backup drive running all the time.

etorix · October 1, 2024, 9:41am

One failing device should not prevent importing a raidz1 pool, at least in degraded state.
First get some storage space to backup whatever could be accessed, and investigate the health of the drives (SMART reports).
Then we might try zpool import -f prox-pool, unless someone has a better or safer idea.

Farout · October 1, 2024, 10:06am

What the output of zpool import in the proxmox shell ?

Just to make sure its not the proxmox bug.

Troll_CLGT · October 1, 2024, 10:55am

What the output of zpool import in the proxmox shell ?
Just to make sure its not the proxmox bug.

I ssh’ed into TrueNAS when I executed all of those commands.

One failing device should not prevent importing a raidz1 pool, at least in degraded state.
First get some storage space to backup whatever could be accessed, and investigate the health of the drives (SMART reports).
Then we might try zpool import -f prox-pool , unless someone has a better or safer idea.

I’m too well versed about what to do here. Should I just unplug the faulty drive and then try to access the pool with only two drives? And what should I do to backup the data if said data is not accessible to me? I hope that you can show me how to do it since I’m a first-timer here. Thanks in advance.

Farout · October 1, 2024, 11:05am

ssh into proxmox, not truenas, and type zpool import.

etorix · October 1, 2024, 11:10am

Try first zpool import from the Proxmox shell, as requested by @Farout, to check whether this is an issue with the virtualised TrueNAS.
Irrespective of that, SMART reports on all three drives, from any shell (smartctl -a /dev/... with the appropriate drive handles, daN or sdX), cannot hurt.

Troll_CLGT · October 1, 2024, 11:42am

Try first zpool import from the Proxmox shell, as requested by @Farout, to check whether this is an issue with the virtualised TrueNAS.

root@pve:~# zpool import
no pools available to import

In my setup, all of the drives (except for the SSD that boots proxmox and other containers/VMs) are connected through a Dell H310 SAS RAID card running in IT mode. That card is then pass-through to TrueNAS, hence there was no pool on proxmox itself.

Irrespective of that, SMART reports on all three drives, from any shell (smartctl -a /dev/... with the appropriate drive handles, daN or sdX ), cannot hurt.

root@truenas[~]# smartctl -a /dev/sdb
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.1.74-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate BarraCuda 3.5 (SMR)
Device Model:     ST2000DM008-2FR102
Serial Number:    ZFL1B37S
LU WWN Device Id: 5 000c50 0c3941716
Firmware Version: VX14
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
TRIM Command:     Available
Device is:        In smartctl database 7.3/5528
ATA Version is:   ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Oct  1 18:39:34 2024 +07
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (    0) seconds.
Offline data collection
capabilities:                    (0x53) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        No Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 194) minutes.
SCT capabilities:              (0x303d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 32
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   081   064   006    Pre-fail  Always       -       132229515
  3 Spin_Up_Time            0x0023   099   098   001    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   078   078   020    Old_age   Always       -       22732
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002f   088   060   045    Pre-fail  Always       -       573667314
  9 Power_On_Hours          0x0032   095   095   001    Old_age   Always       -       4525h+43m+34.870s
 10 Spin_Retry_Count        0x0033   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       95
180 Unknown_HDD_Attribute   0x002a   100   100   001    Old_age   Always       -       491881173
183 Runtime_Bad_Block       0x0032   100   100   001    Old_age   Always       -       0
184 End-to-End_Error        0x0033   100   100   097    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0 0 0
189 High_Fly_Writes         0x003a   100   100   001    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   064   057   040    Old_age   Always       -       36 (Min/Max 28/37)
191 G-Sense_Error_Rate      0x0032   100   100   001    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   001    Old_age   Always       -       131
193 Load_Cycle_Count        0x0032   087   087   001    Old_age   Always       -       27017
194 Temperature_Celsius     0x0022   036   043   001    Old_age   Always       -       36 (0 27 0 0 0)
195 Hardware_ECC_Recovered  0x003a   081   064   001    Old_age   Always       -       132229515
196 Reallocated_Event_Count 0x0032   100   100   010    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   100   100   001    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   100   001    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   001    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       3445h+55m+39.111s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       5427058184
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       13003256092

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%      1150         -
# 2  Short offline       Completed without error       00%      1082         -
# 3  Short offline       Completed without error       00%       788         -
# 4  Short offline       Completed without error       00%       788         -
# 5  Extended offline    Completed without error       00%        48         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

root@truenas[~]# smartctl -a /dev/sdc
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.1.74-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate BarraCuda 3.5 (SMR)
Device Model:     ST2000DM008-2FR102
Serial Number:    ZFL1B2S9
LU WWN Device Id: 5 000c50 0c3941765
Firmware Version: VX14
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
TRIM Command:     Available
Device is:        In smartctl database 7.3/5528
ATA Version is:   ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Oct  1 18:40:52 2024 +07
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (    0) seconds.
Offline data collection
capabilities:                    (0x53) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        No Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 201) minutes.
SCT capabilities:              (0x303d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 32
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   081   064   006    Pre-fail  Always       -       128189189
  3 Spin_Up_Time            0x0023   099   098   001    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   081   081   020    Old_age   Always       -       19992
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002f   087   060   045    Pre-fail  Always       -       548013215
  9 Power_On_Hours          0x0032   095   095   001    Old_age   Always       -       4525h+23m+41.077s
 10 Spin_Retry_Count        0x0033   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       112
180 Unknown_HDD_Attribute   0x002a   100   100   001    Old_age   Always       -       776770151
183 Runtime_Bad_Block       0x0032   100   100   001    Old_age   Always       -       0
184 End-to-End_Error        0x0033   100   100   097    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   099   000    Old_age   Always       -       0 0 2
189 High_Fly_Writes         0x003a   100   100   001    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   063   057   040    Old_age   Always       -       37 (Min/Max 37/37)
191 G-Sense_Error_Rate      0x0032   100   100   001    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   001    Old_age   Always       -       147
193 Load_Cycle_Count        0x0032   088   088   001    Old_age   Always       -       25551
194 Temperature_Celsius     0x0022   037   043   001    Old_age   Always       -       37 (0 27 0 0 0)
195 Hardware_ECC_Recovered  0x003a   081   064   001    Old_age   Always       -       128189189
196 Reallocated_Event_Count 0x0032   100   100   010    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   100   100   001    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   100   001    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   001    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       3447h+12m+53.830s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       5413693512
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       13042476640

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%      1151         -
# 2  Short offline       Completed without error       00%      1082         -
# 3  Short offline       Completed without error       00%       788         -
# 4  Short offline       Completed without error       00%       788         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

root@truenas[~]# smartctl -a /dev/sdd
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.1.74-production+truenas] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate BarraCuda 3.5 (SMR)
Device Model:     ST2000DM008-2FR102
Serial Number:    ZFL1B2V4
LU WWN Device Id: 5 000c50 0c393fcbf
Firmware Version: VX14
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
TRIM Command:     Available
Device is:        In smartctl database 7.3/5528
ATA Version is:   ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Oct  1 18:41:40 2024 +07
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (    0) seconds.
Offline data collection
capabilities:                    (0x53) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        No Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 198) minutes.
SCT capabilities:              (0x303d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 32
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   084   064   006    Pre-fail  Always       -       238975333
  3 Spin_Up_Time            0x0023   100   098   001    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   086   086   020    Old_age   Always       -       14611
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002f   087   060   045    Pre-fail  Always       -       460520988
  9 Power_On_Hours          0x0032   095   095   001    Old_age   Always       -       4549h+33m+51.642s
 10 Spin_Retry_Count        0x0033   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       165
180 Unknown_HDD_Attribute   0x002a   100   100   001    Old_age   Always       -       1559140005
183 Runtime_Bad_Block       0x0032   100   100   001    Old_age   Always       -       0
184 End-to-End_Error        0x0033   100   100   097    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0 0 0
189 High_Fly_Writes         0x003a   100   100   001    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   062   057   040    Old_age   Always       -       38 (Min/Max 37/38)
191 G-Sense_Error_Rate      0x0032   100   100   001    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   001    Old_age   Always       -       184
193 Load_Cycle_Count        0x0032   089   089   001    Old_age   Always       -       22697
194 Temperature_Celsius     0x0022   038   043   001    Old_age   Always       -       38 (0 27 0 0 0)
195 Hardware_ECC_Recovered  0x003a   084   064   001    Old_age   Always       -       238975333
196 Reallocated_Event_Count 0x0032   100   100   010    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   100   100   001    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   100   001    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   001    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       2831h+02m+42.855s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       5677417956
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       9221306348

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%      4538         -
# 2  Extended offline    Completed without error       00%      2191         -
# 3  Extended offline    Aborted by host               90%      2172         -
# 4  Extended offline    Completed without error       00%      2168         -
# 5  Extended offline    Aborted by host               70%      2162         -
# 6  Extended offline    Completed without error       00%      1175         -
# 7  Short offline       Completed without error       00%      1106         -
# 8  Short offline       Completed without error       00%       812         -
# 9  Short offline       Completed without error       00%       812         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

The above only provides legacy SMART information - try 'smartctl -x' for more

Farout · October 1, 2024, 11:55am

This is good, but unfortunatly proxmox has a bug and it can sometimes import pools even from a passed through controller. So in your case its seems to be working the way it should.

etorix · October 1, 2024, 3:41pm

SMR drives do not work well with ZFS, and the first two drives had no SMART test in the last 3000 hours of operation, but nothing stands out to me here an immediate issue. (SMR drives are an issue, but not “immediate”.)
@Farout, do you have any idea other than trying a ‘force’ import from TrueNAS?

Farout · October 1, 2024, 4:08pm

Unfortunatly no. Other than trying to connect the drives to a bare metal install, eg a live zfs enabled linux distro.

Troll_CLGT · October 2, 2024, 12:26pm

root@truenas[~]# zpool import -f prox-pool
cannot import 'prox-pool': one or more devices is currently unavailable

I don’t know what to say anymore, here’s hoping that replacing the faulty drive might solve the problem.

Do SMR drives usually that bad for ZFS? I read some posts online and while they said it’s bad, as far as I know, SMR only sucks when it’s come to write performance during intensive workload, especially during the resilvering and rebuilding period.

Could you guys recommend some affordable CMR drives, or just links to lists of CMR drives? I guess I start upgrading to CMR drives once I somehow recovered all of the data and gathered sufficient funds.

etorix · October 2, 2024, 1:19pm

You need to import the pool to replace a drive…
SMR drives cause long resilver times, but are NOT supposed to cause that kind of disaster. It may be a Proxmox issue, or bad interaction between ZFS and Proxmox, and you’re not alone in reporting such issues.

WD Red Plus/Pro and Gold, Seagate Ironwolf and Exos, Toshiba N300 and MG drives are all good NAS-grade CMR drives.
Be wary of WD Red, Barracuda, Archive, and all large 2.5" drives.

Troll_CLGT · October 2, 2024, 2:42pm

Maybe I should try reinstalling TrueNAS as a fresh VM.

I’ve done this before and was able to import the pool successfully, albeit the situation wasn’t this bad.

etorix · October 2, 2024, 3:34pm

Or try TrueNAS bare metal.

etorix · October 3, 2024, 7:59pm

The best explanation of the issue is here:

etorix · October 4, 2024, 7:16pm

The more I read, the worse it looks…
Get enough drive space to copy anything you might recover. Then try the desperate options (may lose some data)
zpool import -Fn prox-pool
if the result looks encouraging, run again without the n; if not, try
zpool import -FXn prox-pool
It might need -f and/or -R /mnt as well.