Educate me on ZFS and capacity

TrueNAS General
1

I keep seeing conflicting things about usable space in my case, and I could not find an online calculator that allows for different-size disks. My NAS will have
2 x 2TB SSD
2 x 6TB HDD

What do you think is the best configuration to use this space, with as little wasted space and with some protection? I could really use some help here.

My understanding is that if I pick Z1, then it will be like I own 4 x 2TB and useable space will be just 6TB (correct?)

If I do dual mirrors striped, then I would have 8TB space in a quite secure and fast running system. (Relatedly, is striping two mirrors a good idea?)

Thanks!

2

Correct.

More secure, and more performant, than RAIDZ1, as well as more capacity.

No real reason not to, at least not that I can think of.

1 Like
3

I feel like striping SSDs and HDDs is just asking for issues. Would the performance not work out to be somewhere between the SSD and HDD? I feel like the SSDs would spend a lot of time waiting for the HDDs to catch up with striped reads and writes.

Granted I have no idea how ZFS would handle this, but it seems to me like two separate pools each with a single mirror would work out better, you could store media/documents/backups etc on the drives where speed is less of a concern, and you could run apps/VMs (or any other performance necessary bits) on the SSDs.

1 Like
4

I don’t know that it would cause problems as such, but it probably isn’t a very good configuration–I’d missed that the 2 TB drives were SSDs. Given that, I’d agree with you that two separate pools would likely be the better plan.

1 Like
5

Wow, that’s a lot of useful information. I’m a noob on NAS and ZFS, I appreciate all the assistance.

Looks like 2 vdevs with paired storage is the way to go. Follow-up questions on caching.

I have a NVME 500gb drive, and 16gb ram. I installed Truenas on a partition of 128gb on the NVME, and planning on using the rest for L2ARC. Also planning on using 12gb of the physical ram on ARC. I did not open a swap partition.

  1. How do I configure the system to use 12gb physical ram for ARC? Asking because I’ve read that Truenas uses 50% physical ram by default.
  2. Is L2ARC useful at all for read and write? How do I use the remaining partition of NVME for L2ARC? Also read somewhere that there is a better way to increase performance by using some sort of different caching, can’t recall what it was… excuse my ignorance.

Basically the question is, what’s the best way for me to use a 500gb boot drive NVME and 16gb physical ram?

6

TrueNAS CORE uses (almost) all available RAM as ARC. So does the current release of TrueNAS SCALE. You don’t have nearly enough RAM to make use of L2ARC.

7

Current version of both Scale and Core use almost all free memory for ARC.

This can be adjusted, but why?

8

via GIPHY

9

I see, I didn’t know that… Thanks for clarification.

L2ARC is not useful in my use case? Did I just buy 500g boot drive for nothing? :slight_smile:

10

Eh, pretty much. But was it a significant cost delta compared to something smaller?

TrueNAS is designed to have a dedicated boot device. Yes, it’s possible to work around that design. But I really don’t recommend it.

11

Why wouldn’t L2ARC improve any performance for my system?

12

Ah, now I remember… special vdev vs L2ARC… is that of any use for my build?

13

ARC is very good.

L2ARC needs space in the ARC for record keeping, thus reducing ARC effectiveness.

You should run your system for a while, then see what your ARC hitrate looks like. It might be 99.5% which means only 0.5% would be eligible for speeding up by L2ARC.

If the memory used by the L2ARC would’ve resulted in increasing ARCs hitrate then the L2ARC possibly degraded performance.

14

Yes. You need at least two devices to make a special vdev (assuming you want to have a pool which is not one failure away from total loss)

15

That’s not how “human” works.

Someone could be gifted (for free) a 1-TiB, ultra fast NVMe, but they have to sign a contract that says “Only for use as a TrueNAS boot disk”. And they’ll still shelve it, never to be touched ever again, in favor of paying out of pocket for a cheap 64-GiB NVMe, because “its potential isn’t being wasted.”

Source: me (a human)

1 Like
16

I can totally relate to that…that’s why I keep dying in video games where you have some special power that charges every 30 minutes… I never use that one cause it might go to waste :smiley:

1 Like
17

Yeah, I know. But in this case it’s a tendency that should be resisted. See:

1 Like
18

I can at least use the remaining partition of NVMe for storing movies etc to stream right? Stuff I would not care if I lose basically. Assuming of course if I lose the boot nvme, my valuable data in the remaining 4 drives would not be lost. (Confirm?)

19

Boot devices are dedicated (by design they are easily replaceable, so no user data is stored there exc. system configuration). Any repartitioning is completely unsupported by iX and you won’t get a lot of people willing to support if something goes wrong.

Assuming you lose the boot nvme, no, you don’t lose any data on your pool(s). Just make sure you have a configuration backup saved somewhere. While I don’t have configuration backups automated, I make a point to download a new backup after any major change, and keep two old ones on hand so I can revert bad config at any time (I just love breaking things).

You could use a USB-SATA converter and use a cheap 64GiB SSD as the boot drive, that would be fine. You’d then be totally free to create a pool with a single device VDEV to store absolutely non-critical data, with the option to then expand that to a mirror in the future (or move the data onto your SSD pool, destroy your single device vdev pool, and add it and a second device as another mirrored vdev to your SSD pool)

2 Likes
20

Also, you could periodically replicate your fast pool to your hd pool