I’m building out a new TrueNAS Scale implementation, this will be my first experience with the product and ZFS in general, so looking for some wisdom please!
The NAS will be mixed use, from on the fly 4k video editing, container storage, media server, lightweight containers etc.
Spec wise i’m pretty happy with and I’m looking at the below;
96GB RAM, 2x10GB, Ryzen 7
Disks;
6x 24TB HDD
4x 2TB NVME (size tbc)
1x 512GB NVME Boot drive.
All data will be backed up to the cloud, so I don’t mind losing a little resiliency as it can be restored with relative ease.
The bit i’m mainly looking for advice on is how to get the best performance out of the NVME whilst retaining the bulk of the storage on HDD.
I see there’s no tiering available on the data, so i’d be looking at ARC or similar?
It’s possibly I can migrate data regularly between the pools?
Any advice on this please and what would you choose for performant storage please?
“96GB RAM,”
Go for ECC, if possible.
Its OK, if you dont plan to use high demand VMs, only containers.
“2x10GB,”
This is the network speed, I guess?
" Ryzen 7"
I assume, this will be OK
“Disks;”
“6x 24TB HDD”
Use RAID Z2 (your capacity will be 4x24TB and you can lose any two drives)
RAIDZ1 is not recommended at this HDD size.
Use only CMR drives, SMR and ZFS are not friends.
“4x 2TB NVME (size tbc)”
For speed, you should go with 4x 2TB striped (total of 8TB with ZERO safety)
“1x 512GB NVME Boot drive.”
Way to much.
16GB is the minimum, but For future proofing, I would say, 64 GB is more then enough.
“The bit i’m mainly looking for advice on is how to get the best performance out of the NVME whilst retaining the bulk of the storage on HDD.”
For fast storage use a striped array of the 4x NVMe M.2 drives.
BUT:
Separate a 12TB dataset of the main Array for snapshoting and backing up the NVMe array.
Safer option to go for a RAIDZ1 that will provide 6TB and 1 disk scurity.
You can create two datasets on the NVMe array, one for the 4k editing, another one for the VMs.
You must pick your motherboard wisely for this purpose.
You will need the following slots/cards:
dual 10G NIC, it will likely take up a 8x PCIe Gen 4 slot
If you dont buy a gamer board, you will need a proper HBA in IT mode that is minimum PCIe 8x
You will need a PCIe to 4x M.2 card, to host the 4 high speed NVMe-s (and I assume M.2 here, not Oculink, or similar) That will need a full, 16x PCIe Slot, that is capable ot 4x4x4x4x bifurcation. (wise MoBo selection)
You might need to also use a GPU for media server transcoding, that will also need at least PCIe 8x slot (but 16x is better)
SO, in total you will need 1 or 2 16x slot and two other 8x slot
That is at least 48 PCIe lanes.
Plus 4 lanes for the boot SSD.
This is at least 4 slots that forces you to the ATX or e-ATX form factor
You need to buy a suitable case for that too.
“I see there’s no tiering available on the data, so i’d be looking at ARC or similar?”
It depends on the expected load. For close family usage (like less then 3-4 people) I would not bother with any additional Drives for ARC or anythig else.
The only thing I would do is use separate datasets for all your different kinds of data:
For multimedia storage, deduplication is absolutely unnecessary.and compression is also not too beneficial
For backups, you can use both, but you have to keep the dataset size the minimum. ( dedup is a real RAM eater.)
For the NVMe array backup (the 12TB one) I would not use deduplication and neither compression.
For any other data, I would only enable compression.
“It’s possibly I can migrate data regularly between the pools?”
Why would you do that?
