NVME drive getting thrashed despite not being in any pool

TrueNAS General
,
1

I’ve just put together a new system (25.10 RC) and have a spare NVME drive not assigned to any pool. I noticed that it’s consistently getting writes hammered at it. My TBW has increased by 3% overnight (it’s a consumer grade NVME), it wrote approx 50tb to it.

I can’t find any process in htop or iotop that is causing those writes but the reporting screen agrees with the SMART data showing it’s getting some mega writes.

Restarting hasn’t solved anything so I’ll remove the drive until I can find the reason as I’ll not have much drive left by the end of the week!

thrashed
thrashed1444×476 50.8 KB

Update… after secure erasing the drive in the BIOS and then restarting, it’s now writing at 7GiB/s which is the limit of the drive. My calculation suggests the drive would be cooked in about 30hrs.

image
image1815×458 64.8 KB

Further update… I shut it down for 20 mins and then restarted, since it’s not written a single bite to the drive (verified via SMART) which agree with the GUI report. I also noticed that CPU usage spiked highest during the fastest writes above, although I didn’t catch what htop/iotop was saying during that period.

2 Likes
2

My NVMe, which I use as a SLOG for an NFS share, sees a fraction of a fraction of “writes” that you see on your unassigned drive.

I was going to suggest maybe the GUI is wrong, but you already confirmed it has an effect on the drive’s TBW.

3

@winnielinnie If the drive is truly unassigned to any pool or other use, then what is writing to it?

4

Hence my post… I confirmed the behaviour was still occuring after wiping the drive to be sure. It 100% wasn’t assigned to any pool at the time of the screenshots.

5

That has to be a bug. I doubt you’ll see the same behavior in a live Linux ISO session.

6

It’s not done it again since, which is slightly annoying as I’ve not figured it out. Tempted to write something to monitor the SMART data to shut the nas down if it spots high writes again.

7

Since it went away when you rebooted, I’m curious if there was something running in the background that was doing a diagnostic (a write test?)) until terminated, and it was just … hanging there.

There was nothing about drive activity in the UI graph for that drive?