Our family is buying a new NAS and it has some slots free.
I’m a run-of-the-mill interested home user. The current NAS has two pools:
pool1: 4 VDEV’s in RAIDZ1. (4 x 6 TB HDD), for ISO’s, photo’s etc.
pool2: 2 VDEV’s in mirror. (2 x 2 TB NVMe), for docker apps etc.
The new NAS has 96 GB RAM, a decent processor and 2 x 10G NIC’s.
It has an additional HDD and two M.2 slots unoccupied. Do you have any tips on what would be a good use of the free slots? I was thinking of moving to RAIDZ2, but I have no idea what I should do with the M.2 slots. I’ve read about L2ARC, Special VDEV and SLOG, but I cannot figure out what would improve the speed of the NAS the most for the usage of an average family and a geek father.
One thing: we have a large collection of digitized CD’s and SACD’s and we are using Roon. The current NAS had hiccups when playing Plex movies and SACD’'s RIPs (DSD-format) at the same time. That’s the reason why we moved up and we do want to avoid that issue.
Posting a detailed hardware listing of the new system would help.
A VDEV is a collection of one or more drives. You can have more than one VDEV in a pool. You just listed it a bit off.
pool1 is a single VDEV of 4 drives in Raid-Z1. Raid-Z1 is okay for that size drive but you probably want Raid-Z2 or Mirrors if you go to larger drives,
pool2 is a single VDEV also.
Take a look at the pool layout paper and figure out what your priority is for the pool. I am guessing random read since you are playing movices and DSD format. The ARC or L2ARC would help with repeated reads of data but not necessarily single reads of things like movie playback.
That’s 4 HDDs in one pool with one raidz1 vdev, and 2 NVMe in another pool with one mirror vdev.
Comfortable RAM. L2ARC is unlikely to be of use; you can run arc_summary to evaluate (post the output as formatted text, </> button). You do not appear to have workloads with sync writes, so no use for a SLOG. You do not have enough ports for a special vdev (needs at least two drives).
Most likely nothing. It looks about as good as it gets.
Good idea for better data safety, but there’s no way to change raidz level: The procedure is:
Backup. Delete the old pool. Create a new pool. Restore.
At this stage, further spendings should probably go first to a backup solution (second NAS?). “ZFS is not a backup.”
I would offload the geeky stuff (Roon and Plex) to yourself (the geek) and let the rest of the family do their stuff via streaming with whatever services they like
That way as “the geeky Dad” you don’t get any eye roll or blow back when things break and don’t work (which they invariably do) with firmware upgrades and the like
As a fellow geek father - been there done that
I gave them all access to things like a Family spotify and Netflix account
Both my daughters have moved out of home and they still use them…
(And I still pay for them!)
Thank you very much for taking the time to help me out and setting me straight.
I have two M.2 slots free. (The system has five m.2 slots in total.) Can I use these two available slots for a special VDEV and does it make sense in my case?
You gave me a good idea: using the old NAS for local backup. Everything unrecoverable is already backed up on Glacier but I don’t exactly look forward to digitising our CD collection again. We’re talking multiple terabytes.
Thanks for the tip, The Roon is very much a family thing. I bought a lifetime subscription on day 1 because I knew of the guys behind the project. We also have a Tidal subscription and use it through Roon as a multiroom audio solution. Roon has an excellent Tidal integration: streaming music and local music is treated the same in the interface. Unless you notice the icon, you will not know wether you select a local source or a streaming source.
You would need to check if M2 slots are sharing with any of your SATA
As Etorix said, he didn’t think you would get much, if any benefit from SLOG, or adding L2ARC since you have a good amount of RAM. I would just run the system as you have it. I would stay away from special VDEV. You need the same or better redundantcy as your pool and you can’t take it away without having to destroy and recreate the pool without it. If you want to play with L2ARC on your pool you can add it, decide it is not a good use of storage and then remove it without harming your pool. Just be careful as L2ARC does take up some space in ARC and thus RAM. I think it is explained in the Memory Sizing section in the Hardware Guide in the Documents for TrueNAS
You could, although three slots would be better.
You could add a persistent (this is now the default) L2ARC to get the same benfits with respect to reads (not writes) without the drawback of irreversibly adding a critical vdev to the pool.
But with 96 GB RAM, if your NAS is always on, it is likely that all metadata information is already in ARC so it probably does not make sense.
If the NAS has been in use for some time, run arc_summary and look at the hit ratios.
Maybe, retiring the whole, current NAS (Togehter with the HDDs) would be a better idea.
And buy completely anew. 6TB drives are rather small nowdays.
Also, OP has to identify, why did they have those “hickups” (is it CPU, RAM, HDD or LAN bottleneck) and address that particular issue.
(I have a transcoding suspition)
I would also separate the NAS and the other functions too. Maybe reduce the spec for the NAS and build another machine for the other stuff.
BUt, I am also a fellow “geeky Dad”.
ANd I am just rethinking my home lab.
(I moved from TN on bare metal to under Proxmox)
You are right, but 24 TB is plenty for us. The old CPU is a Celeron J4125. The new CPU is a Ryzen 7. Cinebench results: from 2,900 to 16,232. If that doesn’t solve our transcoding issues, I switch from geeking to gardening. It is probably more age-appropriated too.
Thank you so much. I have no desire to spend money for a marginal or even imaginary gain. I’ll see the reports first, as you said, and then decide. May I bother you with one additional question? What is a good OS boot solution in 2025, please?
I found three solutions:
— Using a Samsung Evo 970 Plus 250 GB, as it keeps the OS probably in the SLI’ve been looking at those IC cache so the wear and tear on the drive is minimal?
— using an Intel Optane 16GB that are going for 20 USD on eBay, but 16GB doesn’t feel future-proof.
— the new NAS has an internal USB 3-slot. In my professional life, we have superb experiences with the Swissbit industrial USB’s. They cost approx. 80 euro for 32 GB, but we install them in the field in harsh conditions and they just keep working.
The Swissbit is my preferred solution for the reliability (double the MTBF compared to the Samsung), but is it fast enough? I don’t mind a slow boot, of course.
I’m regularly away from home for weeks, so reliability is essential. While I can resolve most issues remote, the system has to be able to boot. I have JetKVM’s to do OOB.
My system, that provides our family Plex server is a Xeon E5-2650L v2 CPU with 64 GB of RAM on an Ubuntu VM.
This CPU is able to provide about 7 parallel streams even through the internet , but as soon as I include any transcoding, it struggles even one stream.
FOr an OS drive, it depends on what OS you want to use and for what purpose.
WIndows in general, if you want it future proof too, I recommend 1TB NVMe-
For a linux anything above 64 GB is usually OK (I dont have now any dedicated Linux machines, only VMs on a 20 GB virtual drive.
For TrueNAS, if you only want to use it as a NAS appliance (no Vms, no, containers) 16 GB is theoretically sufficient. (currently that is the minimal requirement)
Usually TN is not to picky about the boot drive.
You can even use a simple USB drive for boot up.
You dont even have to worry about failures, because, you can save the configuration of your system and restore it from that file in minutes. (assuming, you save the configuration file to another location)
Regarding Optane in general, it is a discontinued product from Intel, and as far as I know, the really cheap ones are 16GB in size (bigger ones are relatively expensive) and they limited to PCIe Gen3.
(I bought 3 of them second hand from China like a year ago cheper then 5 EUR each.)
If you want to run VMs and Container (Used to call them Apps in TN), you definitely need more storage, but as I remember, it is not mandatory to keep them on the boot drive.
(I dont use TN neither as a hypervisor not a host for containers)
Anything small and cheap enough that you don’t care about “wasted space”…
I do use 16 GB Optane M10 as boot drives for CORE. For SCALE CE, a somewhat larger drive (ca. 100 GB) may be useful so that a swap partition is created on it.
I bought a lifetime subscription on day 1 because I knew of the guys behind the project. We also have a Tidal subscription and use it through Roon as a multiroom audio solution. Roon has an excellent Tidal integration: streaming music and local music is treated the same in the interface. Unless you notice the icon, you will not know wether you select a local source or a streaming source.