HGST SAS Drive Formatting Troubles

TrueNAS General
1

I’m sort of at my wits ends here.

I recently purchased 10x HGST 4TB SAS hard drives (HUS724040ALS640) off of eBay a couple weeks ago and noticed that they were formatted in 520b sector sizes. I searched around and stumbled onto Daisuke’s post on the old TrueNAS forums describing how to change the sector size using either sg_format or WD’s HUGO utility.

I followed through the post’s instructions on checking for T10 protections on the drives and didn’t see any signs of protections enabled. So I initially started to use sg_format to change the sector size of one drive, but ended up running into a problem where the hard drive stalled the format at 89.99% after three days. I ended up stopping it by closing out the process and physically unplugging the drive for an hour or so and restarting sg_format. It once again ended up getting stalled at 89.99%.

I then decided that I would stop using sg_format and instead try testing HUGO (after verifying that my drive model would be compatible with the utility) on another drive since it’s a WD low level utility. I loaded up HUGO on a Windows 10 live disk and ran the format function on a drive. It seemed to be running just fine and actually quicker than sg_format, so I left it to run overnight. Unfortuantely something seems to have happened either to HUGO or Windows, as I woke up to my server fans running at a high RPM and Windows 10 sitting on a BSOD. Now when I try to initialize that drive with either HUGO or sg_format, I’m met with an error that seems to show signs that it’s bricked.

I’m sorta SOL and really nervous to try anything else on these disks as I don’t want to potentially brick anymore. Has anyone had any luck getting out of this situation?

Tech Info:

  • Dell Poweredge R720 w/ H710 mini on IT mode.
  • 10x HGST 4TB Ultrastar 7K4000 Hard drives (HUS724040ALS640)
    • Hard drives are previously EMC branded drives
    • Running C250 firmware
  • TrueNAS Electric Eel

Please let me know if you need any copies of error messages, script outputs, anything really. I just really want to get this figured out and fixed.

2

This is an easy answer to start with.

Run badblocks on your drives and use tmux to do all simultaneously. This will leave you with drives that have been fully verified to be operational, and blank.

Start there. Your comments about drives not finishing has me concerned. Also what is the cooling like for tha HBA? Is it overheating?

Edit. What is the OS used when using sg_format? What is the exact command line used?

3

I’m away from my server for the weekend, so I’ll run the badblocks test once I’m back.

For the cooling, I believe it’s sufficient enough. It’s the mini HBA card installed default in the server and the fans only ramped up after the BSOD occurred, otherwise I never have issues with cooling.

The sg_format command I used was the one listed on Daisuke’s post

time sg_format -v -F -s 512 /dev/sdd

And the OS used was mini Win 10 through medicat.

4

I generally use sg_format --format --ffmt=1 —size=4096 /dev/sdd

The --ffmt=1 speeds things up considerably. I can format a 24TB drive in approx 1min. It does not erase all user data but resets metadata structures so if you are simply wanting to change a 512 drive to 4096 for example then it’s much faster.

You can run this command from within TrueNAS also.

2 Likes
5

I think I would prefer to format it while running in TrueNAS, or any Linux OS Live OS. That is just me.

He said his sectors were 520 bytes so the fast format probably will not work due to the sector structure. Maybe it will work, I don’t have a drive to test this on.

Here is something Google popped up for me, a formatting guide.

It is an interesting read about changing the block size.

2 Likes
6

Yeah me neither. I can confirm however that going from 512 to 4096 works a treat :wink:

7

I am so sorry, I mixed myself up. I ran the sg_format command through TrueNAS and HUGO through the MinWin10 setup. So the results of sg-format stalling out was through Linux.

I didn’t run it with the -ffmt flag, so I could try that once I’m back.

8

Those were probably in a Dell server as Dell uses 520 bytes. The Level1Tech forum post from Wendel that has a nice info on how to convert them to 512 bytes. Which then could be converted to 4K if desired.

There is mention that the drives can get very hot during the process so make sure both the drives and the HBA has really good cooling or things may go south.

1 Like
9

@joeschmuck I’ll check out the repo you shared, my initial glance looked pretty promising.

@PhilD13 that’s definitely interesting, and thankfully HUGO reported the vendor ID for the drives as EMC. I’ll definitely read into Wendel’s thread. I’ll double check my iDRAC to see if temps jumped at any point during the HUGO format run.

10

Okay, finally at my desk again and able to start running tests. Just for info, this is what sg3_utils reports back with an untouched drive.

# sg_format /dev/sdb
    HITACHI   HUS72404 NEO4000  C250   peripheral_type: disk [0x0]
      << supports protection information>>
      Unit serial number: PEKXXXXX
      LU name: 5000ccaXXXXXXXXX
Mode Sense (block descriptor) data, prior to changes:
  <<< longlba flag set (64 bit lba) >>>
  Number of blocks=7693821210 [0x1ca96651a]
  Block size=520 [0x208]
Read Capacity (16) results:
   Protection: prot_en=0, p_type=0, p_i_exponent=0
   Logical block provisioning: lbpme=0, lbprz=0
   Logical blocks per physical block exponent=0
   Lowest aligned logical block address=0
   Number of logical blocks=7693821210
   Logical block size=520 bytes
No changes made. To format use '--format'. To resize use '--resize

I’m currently running badblocks on a drive that has supposedly formatted to 512 and is reporting to TrueNAS at the correct size, but it’s not looking good.

time badblocks -sv /dev/sdf
Checking blocks 0 to 3907018583
Checking for bad blocks (read-only test): 0 0.00% done, 0:00 elapsed. (0/0/0 err1rs)
...
29970 0% done, 42:47 elapsed. (29970/0/0 errors)

And attempting to run badblocks on other drives that haven’t been touched is concerning too.

badblocks -sv /dev/sdb
badblocks: invalid starting block (0): must be less than 0

badblocks -b 520 -sv /dev/sdb
badblocks: Invalid block size: 520

badblocks -sv /dev/sdb 0
Checking blocks 0 to 0
Checking for bad blocks (read-only test): 0
done
Pass completed, 1 bad blocks found. (1/0/0 errors)
11

I prefer to run badblocks in destructive mode. This allows the drive to erase, write, and read the data. With that said, one would think a read-only test would pass.

However no knowing what the history is of these drives, I would first run a smart long test on them. It is a lot faster than badblocks and if it fails, you know right away you have a real problem.