RAIZ1 pool with 3 CMR and 1 SMR drive, an issue?

Hey,

I recently found out about SMR and CMR. If I understood correctly, SMR and ZFS don’t really play well. I have a RAIDZ1 pool with four 4TB drives:

sdb = WDC_WD40EFRX-68N32N0
sdc = WDC_WD40EFRX-68N32N0
sdd = WDC_WD40EFRX-68N32N0
sde = ST4000DM004-2U9104

Last drive (sde) is the only one that uses SMR.

To be honest, without seeing Wolfgang’s video (Titled: " The Perfect Home Server 2024 – 56TB, ECC, IPMI, Quiet & (kind of) Compact"), I probably wouldn’t ever notice that I have one drive with SMR. Also, I have never noticed anything particularly wrong with my NAS. I had developed some symptoms after being up for 4-6 months, but usually a reboot fixes that.

My write and read speeds hover around 80-113MB/s depending on the file/files. My home network is 1 Gbps.

Truenas Scale version: ElectricEel-24.10.0

My question is, should I be worried about having one SMR drive? Should I switch out to a non-SMR drive? Am I losing some features or performance with an SMR drive?

Whenever you can.

Features, no. Performance, possibly, on sustained writes, and at some point when your pool will fill. The real issue is data safety, as SMR would make for a very long resilver; if you ever have a hardware issue, you’ll be at risk for an extended time.

When I read this, I realized that you already expected the answer would be to replace the drive, sooner than later.

It is a good thing you figured it out before real issues stared happening.

The symptoms and a reboot are probably nothing to do with having an SMR drive.

Writes on an SMR drive are in two stages - first the writes are made to a small area of the disk (typically c. 30GB but the size is not in the disk specs), and when the drive is idle these are then written to the main SMR area of the drive, a much slower process. In other words these drives rely on relatively low usage to do this destaging and to get back the space in the CMR cache.

On sustained bulk writes, the CMR cache fills up completely, and then the drive slows to a crawl, and you would see this in normal operation only if that drive was doing sustained writes of > 30GB as part of the whole pool. So on a 4x RAIDZ1 you can probably write up to 90GB of user data before this happens.

So, under normal low load circumstances, you would never notice the difference. Reads throughput and iops are about the same, and for short bursts of writes throughput and iops are also about the same.

However when expanding a RAIDZ vDev, or resilvering a disk, ZFS does bulk extended bulk writes, and what would take hours or days on a CMR drive end up taking days or weeks on an SMR drive.

The worst time to find out about these performance issues is when you have a drive failure. My advice would be to swap the Baracuda drive out for another matching WD Red Plus drive as soon as you reasonably can.

Thank you @etorix, @joeschmuck and Protopia ! (Can only mention 2 people in a post)

I will get to ordering a new drive ASAP!

The next question is how to swap the drive within the pool. Do you guys know any good tutorials I could follow?

  1. Power down the system
  2. Connect new drive to the NAS
  3. Power on the system
  4. Go to the web UI & login
  5. Go to the ‘Storage’ tab
  6. Top right, there is a ‘Disks’ button
  7. Make note of the SDA/B/C/D/etc. naming of the disks & their serial numbers (the SD# can change after a boot) - identify the new drive & the SMR drive. Double check that you have correctly identified them. Write down the sd# name of each. Double check you wrote it down correctly.
  8. Go back to the ‘Storage’ tab
  9. Find your pool & click ‘Manage Devices’
  10. Expand the vdev (It’ll say ‘RAIDZ1’ in your case)
  11. Find the old drive in the list, referring back to your notes from earlier, click on it
  12. There will be a button labelled ‘Replace’, click it
  13. Find the new replacement disk in the replacement window that pops-up; select the replacement disk
  14. Click ‘Replace Disk’ (shouldn’t be a need to have ‘Force’ checked)
  15. Wait for resilver to complete
  16. Shut down the system
  17. Remove old disk
  18. *Optional: power system back on

…It is a lot of steps when you really write them out like that, but should take like 5 minutes if you’re taking it slow (not accounting for resilver time)

Arguably Step 0 would be to burn-in the replacement drive & make sure it doesn’t suck with a smart long test, a pass of backblocks, and another smart long test. There are guides on the forum better than what I’d quickly list off here.

I see, thanks for the guide!

I will report if anything goes wrong :smiley:

Good luck. Pay attention to the drive serial numbers is the main thing people mess up on. The end up replacing the wrong drive.

2 Likes

@grannymaster

Before telling TrueNAS Scale to do the replace, run the following commands:

  • lsblk -bo NAME,MODEL,ROTA,PTTYPE,TYPE,START,SIZE,PARTTYPENAME,PARTUUID
  • sudo zpool status -v

The first is the most important as it directly ties the disk model to both the device name /dev/sdX and the partition UUID that is displayed in the zpool status and the TrueNAS UI.

I hope this helps ensure that you replace the correct drive.