Just trying to get a general consensus… On these large raidzX pools how often are you scheduling tasks like scrub, long smart tests, etc…
Example, my pool is six 12tb disks wide, one vdev and it takes 7 1/2 hours to run a scrub. Smart Long takes a lot longer. My enclosure can hold up to 12 drives and right now I don’t know if I want to expand further.
I asked the same question not too long ago, but unfortunately received no answer.
IIRC, the default settings for scrubs in TrueNAS Scale is 35 days. I have set mine to monthly at the beginning of the month and every other month a long test in the middle of the month. Also a weekly short test.
The reason I am not running monthly long tests, is that my 20TB drives have an annual rating of 550 TB and I don’t want to get too close to that with my maintenance tasks (see my post referenced above for details).
In my setup, I use new NAS drives (WD Red Pro), so if you use very old drives/shucked drives/desktop drives, I’d run those tasks more often (or - in case of desktop drives - replace them with something more appropriate).
IIRC, long SMART tests take longer, because they also check unused blocks, whereas scrubs only check allocated blocks.
I’m not sure that adding more disks would mean longer durations for scrubs or long SMART tests, because I would expect those tasks to run on all disks in parallel. But my expectation might be wrong…
But even if those tasks were to run longer, what would the problem be with that? You can still use the pool, even though I’d expect performance to be somewhat degraded.
Thank you for the reply! I’m shocked that a missed your post when I read the forum every day, always clicking on Unread and then New… I’m using Seagate Ironwolf drives are a mix of DOM 2020, 2021, 2023 and 2024. Prior to my latest upgrades I ran with 5 Ironwolf 4tb drives (raidz1), guess I never worried about this then, the tasks ran a lot quicker. I am mostly concerned with catching drive errors quickly on the older drives. But having said that I don’t want to be running tasks all the time either, it’s a catch 22. I just wanted to know how others are setup with large pools.
Not sure if I count as having large pools, but here is my setup:
- Scrubs at 03:00, on day 1 and 15th
- SMART Short at 03:00, on day 5, 12, 19, and 26th
- SMART Long at 03:00, on day 8 and 22th
- Snapshots on at 00:00, on day 2 and 16th
I likely stole this setup from someone on the old forums (I’d give credit if I could remember whom). Nothing overlaps, everything seems reasonably timed & spaced out.
You could stagger by a day(s)/hour(s) for different pools if you see any need to do so.
1 Like
Just out of curiosity, how big are your HDDs and what series are you using (e.g. NAS, NAS Pro, Enterprise, etc.) are you using?
Expand ‘See My freeNAS system’ for Fleshmauler post. I just looked and drive models were listed.
2 Likes
Sorry, I missed that. Thanks for the pointer.
Just as an observation: According to the datasheet, the WD Red Plus 8TB (Model Number WD80EFZZ) has an annual workload rating of 180TB.
If I assume that your 8TB drives are half-full, your scrubs will create a workload of 2 * 4 TB per month, and your SMART long will create a workload of 2 * 8TB per months for a total of 24TB per month or 288TB per year. So you are 108 TB above your annual workload rating just buy running your maintenance tasks.
Please note that I’m not raising this issue to chastise you in any way, rather this is a topic that has concerned me, when setting up my pool and I’d really hope that some experts could weigh in on how much of a concern exceeding the annual workload rate actually is.
1 Like
Never even thought of that! In my sample size of me; it hasn’t caused any issues yet & I’m half certain I’m out of warranty on most of these drives by now anyway.
I’m curious if they actually have any telemetry to track the amount of data READ on an HDD, as I imagine this value on their datasheet exists only to deny RMA claims.
Could be way off the mark on it though. I’m not going to lose sleep over it as it ain’t an ssd.
Edit: Seems WD has a white paper on this. If you are worried about it, have 1 less scrub a month.
Look at the TrueNAS Mini info page and expand Drive Compatibility List. Those are the drives tested and approved by iX Systems for ZFS. I would expect them to be fine. I don’t know if they also test them in there Enterprise servers also.
Well, they don’t have the specific 8TB WD Red Plus drives that @Fleshmauler uses in their list, but - as an example - they do have the 10TB WD Red Plus drive (Model WD101EFBX) in their list. That drive also has an annual workload rating of 180TB per WD’s data sheet.
I wouldn’t consider those drives be suitable for Enterprise servers, in fact I’m not even sure that my own WD Red Pro drives would be suitable, rather I’d expect them to recommend WD Gold or Ultrastar drives or Seagate Exos drives for that use-case.
But I’m not sure that saying “I would expect them to be fine.” resolves the issue. I’m sure there are fine for a specific workload and I’m also sure they will not immediately explode, once you exceed the workload, but I am wondering, how that will impact long-term failure rates. And I am also wondering, if the drive tracks lifetime workload in the SMART data for warranty (denial) purposes.
1 Like
My manual Smart Long on my 6 drives finished last night. Almost 16 hours, give or take.
Is that what you guys with large drives see also?
Sounds about right, looks like double the time mine take, which are 8tb.
1 Like