Initial pool structure with an eye toward expansion

TrueNAS General
1

I’m planning out my first NAS and intend to start with 2 16TB drives in a 4-bay box. Most of the storage will be for a media server, but some backup tasks will also be stored here. Anything I can’t easily replace will be part of a 3-2-1 backup scheme.
I want to make sure I’m setting myself up for making good use of the two other bays. There are three situations I can foresee myself getting to:

  1. IOPS is a bottleneck so I’d want to move to a 2x2 mirror structure
  2. Capacity needs expanding, but one drive at a time, so I’d want to move to a 3-wide RAIDZ1
  3. Capacity needs expanding, adding two new drives, resulting in a 4-wide RAIDZ1

I don’t see myself wanting RAIDZ2 or moving to 2x2 for redundancy reasons. The data’s not that critical, and I’d probably prefer the extra capacity. I also don’t know how long it’ll be before I need to expand.
Given those constraints, am I better off starting with the two drives mirrored or in a RAIDZ1? Up until the last few months the former was pretty obvious, but now that VDEV expansion is on the horizon I could see starting in RAIDZ1 to make options 2 and 3 much easier. The downsides, as I see them, are slightly lower performance and making option 1 more difficult (although I see that as the least likely option).
The main reason I’d expect to expand is if the existing 16TB is filling up, so I don’t want to assume I’ll have a place to replicate it while restructuring. I’m ok taking on some risk of data loss while the migration is in progress, though.

The above has all the details of the question, the rest is just how I imagine the migration would go in each scenario. Feel free to skip.

  • Mirrored option 1
    • Easy, add the two new drives as a new vdev. Set up striping. Done
  • Mirrored option 2
    • Degrade the mirror to free one drive for setting up a new RAIDZ1 vdev with the new drive
    • Copy data from the original vdev to the new one
    • Drop the old vdev and expand the new vdev to include the old drive
  • Mirrored option 3
    • Possibly the same as option 2, but starting the new vdev with 3 drives.
    • Otherwise create a new RAIDZ1 vdev with the new drives.
    • Copy the data from the old vdev to the new one
    • Drop the old vdev and expand the new vdev to include the old drives (one drive at a time)
  • RAIDZ1 option 1
    • Create a new vdev with two new drives, mirrored
    • Copy data from old vdev to new vdev
    • Detach old vdev and restructure it as mirrored
    • Set up striping
  • RAIDZ1 options 2 or 3
    • Easy, expand the vdev (one drive at a time)
2

We are not planning at this time to expose 2-wide RAIDZ1 as an option through the UI or API.

3

Hello and welcome to the forums.

I’d start with a mirror of your two 16TB drives. Then when needed get another two and add an additional mirror. When you hit the next point where you need more space go get some 24TB drives (or whatever is big then with a fair price point) and one at a time replace your 16TB drives.

With only four bays your options are limited but this seems to me the most obvious config.

4

IMHO Raidz1 have sense only if you want to start immediately with 3 disk maximizing the storage, but i wouldn’t plan to use the 4th slot; in case of need i would just replace all 3 disks without adding anyone.
So i think that the mirror layout will be much better: more flexible, more resilient, more iops, and you can frag the cost of the disk on 2 purchases (2 now 2 later, only on Need)

5

I find it very likely that I’ll want a RAIDZ1 for my final expanded configuration. It feels really wasteful to have an extra drive worth of redundancy given my risk profile. And given that the main task this will be dealing with is streaming media, it seems unlikely IOPS will be a bottleneck (containers will mostly be running on another device). Is there no good way to get from 2 drives to 3 or 4 in-place even with vdev expansion? Are there significant drawbacks to expansion that I’m not aware of?

6

Yes, there are. vdev expansion doesn’t change stripe width for existing data. If the data becomes changed later then new data is written with the new stripe width. As your use case is media streaming then you likely won’t be changing your media files very much. None of this may matter to you, however, so its all up to you.

Also, and depending on what you meant by “extra drive worth of redundancy”, this may be well known to you already, two or more mirrors doesn’t allow you to lose ANY two drives. So you should consider multiple mirrors to offer single drive protection only.

Ideally, you’d get one more drive and start with a three drive raidz1, imo.

Good luck!