ZFS Layout advice please!

TrueNAS General
, ,
1

Hello all!

Looking for assistance with configuring my ZFS Pool before I migrate my data to it.
I am planning to virtualise the TrueNAS Core deployment, with Proxmox as the bare metal OS.

Workload:

  • Plex Data store (Movie/TV)
  • Archive file store
    –Above currently 5.5Tb
  • Potentially, VM VHD store for Proxmox
  • Nothing life-ending if I loose it.

Hardware:

  • Supermicro H12SSL-i
  • AMD EPYC 7532 34c 65t, 2.4Ghz t3.3Ghz
  • 256Gb DDR4 2133 ECC (32Gb*8)
  • 1x 2Tb Crucial P5 (Proxmox drive)
  • 6x 6Tb Seagate Skyhawk SATA
  • 2x 2Tb Intel DC P4510 U.2
  • 4x 1Tb Samsung Evo SATA
  • 2x 1Tb misc M.2 SSD
  • Various PCI-E adapters (M.2, U.2, SAS MiniHD, LSI HBAs in IT mode)
  • SlimSAS-8x SATA cables, other cables to suit adapters and disks.

image
image697×336 55.7 KB

I’ve currently got 100Gb RAM allocated to the Truenas, but have headroom to increase if required.

HDD Layout:
I was thinking of one of these two layouts for the 6x6Tb Disks:
I understand the first pool has less fault tolerance, and would not be ideal.
Am I correct in my understanding that I can expand the second pool with an additional 3 disks?
image
Or:
image

SSD Layout
I was thinking about assigning my U.2 SSDs as a Metadata vDEV with small file size set to potentially 128k? This would fill out the 2tb mirror Niceley if my guesses prove to be accurate.

Output of sudo zdb –LbbbA –U /data/zfs/zpool.cache TEMPVDEV

image
image546×664 67.2 KB

TIA!

2

Although on the face of it, the first pool has less fault tolerance, if a disk fails the resilvering time on a single VDEV (and thus extra load on the remaining disks) will be much less than with the second pool where all the disks are involved.

As ever though, there is a large degree of YMMV - personal preference, hardware specs, load types etc.

1 Like
3

Can I expand the pool with only 3 disks if I go for the 3 wide?

4

Traditionally, the way to expand the pool would be to:

Add another raidz1 vdev (i.e. another 3 disks) and/or
Remove and replace each disk with a larger one, one at a time. Once the last disk is replaced the space available can be increased. (I actually had to do this when I discovered the Toshiba drives I bought originally were crap - replaced them with Seagate Ironwolfs).

However, I’m not fully up-to-date on the latest features of TrueNAS / ZFS - I believe that widening vdevs may be possible in the future, so worth a bit of homework on that one or someone here can advise.

1 Like
5

Raidz vdev expansion is expected to land in Electric Eel (24.10). Caveats apply, and it is NOT possible to increase raidz level (Z1 → Z2) so any enlarged pool would remain with a rather low redundancy.

Raidz2 and 3-way mirror for special vdev, if used, are advised for safety.

2 Likes