TrueNAS CORE: System Dataset on HDD mirror prevents standby – best layout with SSD boot?

TrueNAS General
, ,
1

Hi everyone,

I’m running TrueNAS CORE 13.0-U5.1 on an HP MicroServer Gen8.

Current storage layout:

  • Boot device: HP iLO Internal SD Card
  • Data pool: p-sg-10tb
  • 2 × Seagate ST10000NE0008 10 TB, ZFS mirror
  • System Dataset (.system) is currently on p-sg-10tb
  • SMART enabled
  • HDD standby currently set to Always On
  • APM disabled

This is a home/archive NAS and it can remain completely unused for many hours or even several days. For this reason, I am considering allowing the two 10 TB HDDs to enter standby after a suitably long idle period.

Before changing anything, I investigated periodic HDD head activity that I could hear while the NAS was otherwise idle.

Using:

gstat -C -p -o -I 1s -f 'ada[01]$'

together with monitoring file modification times under /var/db/system, I found a very clear correlation between the audible disk activity and periodic writes to the System Dataset.

Hundreds of RRD files under .system/rrd are updated in groups at approximately 10-second intervals. This sequence repeats roughly every five minutes, and gstat shows the corresponding write/flush activity on both mirror disks.

So my current conclusion is that the periodic HDD activity is mainly caused by the System Dataset/RRD reporting activity rather than user access to the data pool.

I am planning to replace the current SD boot device with a SATA SSD in the MicroServer ODD bay.

Before changing the configuration, I would appreciate advice from experienced CORE users:

  1. Once the SATA SSD becomes the TrueNAS boot device, would you recommend moving the System Dataset from the HDD mirror to the SSD boot pool specifically to allow the HDD data pool to remain truly idle?

  2. For TrueNAS CORE 13, is keeping the System Dataset on an SSD boot pool considered a sound configuration, or is there a better recommended layout when the goal is to allow a data-only HDD pool to spin down during long periods of inactivity?

  3. If the goal is to let the HDD mirror remain spun down for many hours when unused, what other CORE services should I check for periodic disk access after moving the System Dataset?

  4. For this usage pattern — potentially days without accessing the pool — would you recommend HDD standby at all? If so, what sort of standby interval would you consider reasonable?

My priority is reliability and data integrity first; reduced noise and power consumption come second. I do not want to achieve standby at the cost of excessive spin-up/spin-down cycles or reduced disk-health monitoring.

I have retained the timestamped gstat output and System Dataset audit logs and can provide relevant excerpts if useful.

Thanks in advance for any advice or corrections to my interpretation.

ermanno

2

The best place for the system dataset is on a SSD pool (for jails/VMs/apps); since you don’t have one, and want to spin your drives down, the best place for the system dataset is back on the boot pool. Remember to keep an up to date copy of the configuration file outside of the NAS.

And consider at least upgrading to 13.0-U6.8 :wink: (since you want to spin down, cross-grade to CE is not advisable)

3

…or to FreeCORE…

4

Thank you both — this is very helpful and confirms the direction I was considering.

I will definitely keep an up-to-date configuration backup outside the NAS before making any changes. My plan is first to replace the current SD boot device with a SATA SSD, and only then consider moving the System Dataset off the HDD mirror.

I also take the point about upgrading from 13.0-U5.1 to at least 13.0-U6.8.

One thing I’d be interested to understand better: why would cross-grading to CE be inadvisable specifically when HDD spin-down is one of the goals? Is there a particular difference in disk monitoring or background activity compared with CORE that makes reliable spin-down more difficult?

And thanks for mentioning FreeCORE — I wasn’t aware of it, so I’ll read about it before considering any longer-term migration.

5

Read here:

HDD monitoring in CE has broken standby, and TrueNAS passive-aggressively declines to take up the fix to at least not disrupt the previous behaviour.

6

Thank you, etorix, for the link and the additional explanation.

That clarifies why you advised against moving to CE when HDD standby is one of my main goals.

It’s particularly interesting to see how closely the experience described in that thread matches the periodic disk activity I observed on my CORE system.

I’ll keep this in mind while planning the SSD migration and the subsequent standby tests, without compromising disk-health monitoring.

Thanks again for sharing your experience and pointing me in the right direction!