Educate me on ZFS and capacity

Wow, that’s a lot of useful information. I’m a noob on NAS and ZFS, I appreciate all the assistance.

Looks like 2 vdevs with paired storage is the way to go. Follow-up questions on caching.

I have a NVME 500gb drive, and 16gb ram. I installed Truenas on a partition of 128gb on the NVME, and planning on using the rest for L2ARC. Also planning on using 12gb of the physical ram on ARC. I did not open a swap partition.

  1. How do I configure the system to use 12gb physical ram for ARC? Asking because I’ve read that Truenas uses 50% physical ram by default.
  2. Is L2ARC useful at all for read and write? How do I use the remaining partition of NVME for L2ARC? Also read somewhere that there is a better way to increase performance by using some sort of different caching, can’t recall what it was… excuse my ignorance.

Basically the question is, what’s the best way for me to use a 500gb boot drive NVME and 16gb physical ram?

TrueNAS CORE uses (almost) all available RAM as ARC. So does the current release of TrueNAS SCALE. You don’t have nearly enough RAM to make use of L2ARC.

Current version of both Scale and Core use almost all free memory for ARC.

This can be adjusted, but why?


I see, I didn’t know that… Thanks for clarification.

L2ARC is not useful in my use case? Did I just buy 500g boot drive for nothing? :slight_smile:

Eh, pretty much. But was it a significant cost delta compared to something smaller?

TrueNAS is designed to have a dedicated boot device. Yes, it’s possible to work around that design. But I really don’t recommend it.

Why wouldn’t L2ARC improve any performance for my system?

Ah, now I remember… special vdev vs L2ARC… is that of any use for my build?

ARC is very good.

L2ARC needs space in the ARC for record keeping, thus reducing ARC effectiveness.

You should run your system for a while, then see what your ARC hitrate looks like. It might be 99.5% which means only 0.5% would be eligible for speeding up by L2ARC.

If the memory used by the L2ARC would’ve resulted in increasing ARCs hitrate then the L2ARC possibly degraded performance.

Yes. You need at least two devices to make a special vdev (assuming you want to have a pool which is not one failure away from total loss)

That’s not how “human” works.

Someone could be gifted (for free) a 1-TiB, ultra fast NVMe, but they have to sign a contract that says “Only for use as a TrueNAS boot disk”. And they’ll still shelve it, never to be touched ever again, in favor of paying out of pocket for a cheap 64-GiB NVMe, because “its potential isn’t being wasted.”

Source: me (a human)

1 Like

I can totally relate to that…that’s why I keep dying in video games where you have some special power that charges every 30 minutes… I never use that one cause it might go to waste :smiley:

1 Like

Yeah, I know. But in this case it’s a tendency that should be resisted. See:

1 Like

I can at least use the remaining partition of NVMe for storing movies etc to stream right? Stuff I would not care if I lose basically. Assuming of course if I lose the boot nvme, my valuable data in the remaining 4 drives would not be lost. (Confirm?)

Boot devices are dedicated (by design they are easily replaceable, so no user data is stored there exc. system configuration). Any repartitioning is completely unsupported by iX and you won’t get a lot of people willing to support if something goes wrong.

Assuming you lose the boot nvme, no, you don’t lose any data on your pool(s). Just make sure you have a configuration backup saved somewhere. While I don’t have configuration backups automated, I make a point to download a new backup after any major change, and keep two old ones on hand so I can revert bad config at any time (I just love breaking things).

You could use a USB-SATA converter and use a cheap 64GiB SSD as the boot drive, that would be fine. You’d then be totally free to create a pool with a single device VDEV to store absolutely non-critical data, with the option to then expand that to a mirror in the future (or move the data onto your SSD pool, destroy your single device vdev pool, and add it and a second device as another mirrored vdev to your SSD pool)


Also, you could periodically replicate your fast pool to your hd pool

All things considered, I think I’m tempted to replace my 500gb NVMe with a 2TB NVMe, so that with the other 2TB SSDs, I would do a RAIDZ1.

So the final config would be
3 x 2TB SSD → RAIDZ1 vdev
2 x 6TB HDD → Mirror vdev

Buy a cheap 64gb emmc to use for boot drive + apps instead.

Again, the boot drive is only the boot drive. Put the apps on the SSD pool.


@oacikgoz, you can choose to listen to @dan. :-1:

Or you can choose to listen to your heart. :blush:

1 Like

Ok now, the last important point, can anyone donate money for a new 2tb NVMe? :smiley:

1 Like