Seeking Feedback on Power-Efficient Storage Server for Mixed Bioinformatics and Media Workloads

TrueNAS General
,
1

Hi all,

I’m a bioinformatician and currently run a home server that supports both my contract work for various research labs and a range of self-hosted services (home automation, media, and backups). This server is fairly overprovisioned for day-to-day tasks, and I’m considering separating storage and compute to improve power efficiency and system organisation.

My current setup handles everything in one system. For bioinformatics work, the server is typically active four days a week, so I believe I could save on power and improve reliability by moving all storage to a dedicated TrueNAS Scale box. This would allow me to power down the compute node when not in use, while keeping essential services and data online.

I’m also considering reorganising my ZFS layout. Currently, I have a 12-disk pool configured as 4 vdevs of 3-wide RAID-Z1. This evolved from an initial 6-disk setup and was expanded over time. In the new setup, I’m thinking of splitting it into two pools, one smaller 3-disk pool for media (sequential access), and a 9-disk pool for bioinformatics data (a mix of sequential and random access workloads).

Current Work Server (Proxmox):

  • CPU: AMD EPYC 7702P
  • Motherboard: Tyan S8030GM4NE-2T
  • RAM: 8 x 64 GB DDR4 ECC*GPU:
  • 2 x Nvidia RTX 4090, 1 x Intel Arc A750 (for media encoding)
    Storage:
  • 12 x 14 TB Seagate Exos (ZFS pool: bioinformatics + media)
  • 4 x 3.84 TB Intel DC D5 P5530 (VMs)
  • 2 x 280 GB Intel Optane 900P (boot / host)

Proposed Storage Server (TrueNAS Scale):

  • CPU: Intel Xeon E3-1240L v5 (25W TDP)
  • Motherboard: Supermicro X11SSH-F
  • RAM: 4 x 16 GB ECC UDIMM
  • NIC: Mellanox ConnectX-4 (10GbE)
  • Boot Drive:M.2 SSD (TBD)
  • Additions: LSI HBA for HDDs, potential SSD-based L2ARC or ZIL (SLOG) as needed
  • Use case: Always-on storage for both bioinformatics and media workloads, accessed via 10GbE from Proxmox compute nodes.

Home Services Node:

  • System: Minisforum MS-01 (Intel i9-12900H), 64 GB RAM, 2x1TB m.2 SSD (vms)
  • Networking: 10GbE for media access

Questions:

  1. Storage Layout: Would splitting the pool into a 3-disk RAID-Z1 for media and a 9-disk RAID-Z1/RAID-Z2 for bioinformatics workloads be reasonable? Are there performance or reliability trade-offs I should be aware of?
  2. RAM & Caching: Is 64 GB of ECC UDIMM sufficient for the storage workload described, particularly for mixed-access bioinformatics tasks? Would adding SSDs for L2ARC or a dedicated SLOG provide measurable benefits?
  3. Power Efficiency: Any feedback on the E3-1240L v5 as a low-power option for this storage server? Would another platform offer better performance-per-watt in this context?

All feedback is welcome. I’m especially interested in advice around ZFS tuning, memory planning, and cache device placement given my workload profile.

Thanks in advance!

2

Posting links on basics, pool layout and special vdevs.

I don’t know what your Bioinformatics workload looks like or it’s size. See pool layout for your tradeoffs. The more random access or database like interactions, the better Mirrors or NVMe pools look.

HBA needs to be in IT Mode or plain HBA, no RAID.
L2ARC may help for reads. You have to look at the ARC stats using arc_summary once you are up and running with your regular data.
SLOG can be useful but only for synchronous writes and has suggested requirements as PLP, power loss prevention. That’s covered in the primer and online documents, hardware guide section. L2ARC and SLOG can be removed from a pool without destroying it. Special Vdevs, require pool destruction and recreation, usually. Should be covered in the links below.
You might consider a different system or motherboard as you won’t really have much for expansion with your current proposed.

BASICS

iX Systems pool layout whitepaper

Special VDEV (sVDEV) Planning, Sizing, and Considerations

3

IMHO, i was in the same situation, had to decide on that CPU or a 1260L v5 (L series are nor available on v6 :weary: ). At the end i take the 1260L, because the power increment was quite remarkable and they still idle at the same power consumption (90% of the time). Take the 1240l only if you plan to put on It a passive cooler ( despite the low 45w TDP fit the spec of the arctic alpine 12 passive i had, It was not optimal to properly cool that CPU on long full load).

L2arc can be added later without issue, Is not pool critical, so you can decide without rush using the system and on what arc summary will say.

4

Quick note: I would research use of TrueNAS under Proxmox VERY carefully if the workload is mission critical. Running TrueNAS on anything but bare metal is not supported by iXSystems and I recall multiple iterations where even allegedly-properly-configured TrueNAS systems using “pass-through” configurations led to SILENTLY-CORRUPTED data.

There be dragons down that rabbit hole. Silently-corrupted data is my biggest bugaboo because then even your backups eventually get corrupted too.

Folk here have managed to do proxmox in a server and run TrueNAS within it, but if its mission-critical, man, I would rethink that a few times over or at least get in touch with someone who understands deeply why those other folk got into trouble. I’m no such expert.

If it was my set of systems, I’d configure the TrueNAS as a pure file server on something bare metal and then have the proxmox fun on a different machine.

5

To be really nitty-gritty/precise: The HBA needs to be flashed to IT mode firmware. Switching a RAID-supporting firmware to IT mode is not sufficient.

2 Likes