Recommendations for vDev setup

TrueNAS General
,
1

Howdy,

I currently have a TrueNAS Core 13.3 setup and am looking to finally make the jump to Scale. I’ve held off for a few reasons, but am planning to increase my overall storage capacity, so felt doing so would be a good time to roll with a fresh start/install.

I am trying to decide on what the best vDev layout would be for my use case.

Current Setup:

  • Pool 1:

    • 6 x 12TB in a single vDev

    • 2 Drive Redundancy

    • 43TB useable

    • This pool also has a 1TB NVME Cache Drive

  • Pool 2:

    • 2x 8TB in a single vDev

    • Mirrored

    • 7.28TB useable

Future Plan:

  • Pool 1:
    • 9 x 12TB

      • Not sure what the best vDev setup for this is

      • Single RaidZ3 or Single RaidZ2?

      • Leaning towards 3 x vDevs, each with 1 drive redundancy

        • From what I’ve read, this would cut down on Resilvering time if (when) a drive fails, but also comes at the cost of perfromance.
    • 3 x 8TB

      • RaidZ1 with single drive drive redundancy

      • Again, from what I’ve read, it seems I the vDev’s do not all have to be the same size to be in a storage pool. Meaning I can add the 8TB vDev in with the 12TB ones for increased total capacity.

What will I use the server for?

  • Mostly data storage

    • Music, Document Backups, Media Backups, etc.

    • I also keep backups of Installers for apps I install for both my PC’s and Mac’s (data hoarding, I know)

  • Video and Photo editing project backups

    • Not planning to directly edit from the server, but once I am done with a project I tend to dump capture footage and project files, etc. to the TrueNAS server

  • Plex

  • Likely install Tailscale

  • Possibly also install a RustDesk server

The Hardware:

  • Intel 4790k (current)

  • 32GB DDR3 Memory (current)

  • MSI Gaming Z97 Motherboard (current)

  • LSI 9300-16i (new)

    • Thinking about ditching this in favor of a PCI-E SATA Expansion card with a Marvell 88SE9215 chipset
      • The LSI card generates a lot of heat and uses a fair bit of power
      • I have been using a PCIE SATA x4 Card for 5 years now with 0 problems

  • TP Link TX201 2.5GBE Network Card (new)

  • 9 x 12TB Ironwolf Pro NAS Drives (6 old, 3 new - will likely be 5 old & 4 new, as I think one drive is degraded)

  • 3 x 8TB Ironwolf Pro NAS Drives (2 old, one new)

I do not plan to migrate the existing pools or data, as I have everything backed up on External HDD’s and plan to manually rewrite the data (I plan to make some dataset structure changes as well). Critical data I have backed up to multiple External Drives.

I would be thankful for any feedback, thoughts, or suggestions on the best setup for my use case.

Thank you

2

Actually the opposite. More vdevs = better performance.

Just keep in mind: if you lose 2 drives in the same RAIDZ1 vdev, ALL you data is gone.

Same problem when combining a RAIDZ2 vdev with a RAIDZ1. If the RAIDz1 vdev fails, ALL data is gone.

For 9 drives, maybe later larger ones, I would go for a 1x RAIDZ3 9-wide.

3

Would you recommend against mixing the 8TB vDev into the same Data Pool as the 12TB vDev?

4

I would avoid mixing RAID types in the same pool.

Mixing same RAID type with different sized drives is ok.

5

ZFS will balance the load between the vDEVs according to the relative size of the vDEV (not a single drive in case of raidz). So, the bigger vDEV would have a higher overall load, which can be fine for sequential workloads but can be inappropriate for random IO[1]. But you plan to store media, so…

If I were you, I would create 2 pools:

  1. Main pool. 3xRaidz1[2] of 3x12TB drives with a total usable space of 72TB. Or would buy one more 12TB for 2xRaidz1 of 5x12TB drives with 96TB total. IOPS would be ~1.5x less in the 2nd case.
  2. Backup pool. 1xRaidz1 of 3x8TB with 16TB usable.

Then, I would set up a replication task from main to backup for the most precious datasets (personal media library and docs, I suppose). Thus, starting the glorious journey of building personal 3-2-1 storage.

  1. This doc is very useful for understanding pool layout trade-offs. ↩︎

  2. Raidz1 is not very popular amongst this forum, especially with bigger drives. However, imho, having a proper backup is more valuable than having extra redundancy. Strictly speaking, raid(z) is not a tool for preventing data loss; it’s a tool for providing uptime during a drive loss. ↩︎