Critique my setup

TrueNAS General
,
1

I’m new to all this, but I finially have a TrueNAS scale server running. I’m still in the experimating & early learning stage. Befreo I go too much further along, I’m hoping I could get some advice.

My primary goal is to have reliable storage, but secondarily I may use the server to run some apps and-or VMs: Emby, NextCloud, maybe quickbooks server (database) and say minecraft server. Who knows what else, we’ll see where it goes in time.

For storage, my case has 16 HDD bays so I’ve set-up:
– 4 pairs of 16TB mirrored for primary pool (64TB)
– 5x HDD 16TB Raid 5 for onsite / online backup (64TB) (I plan to have an offline backup also)
– 2x HDD bays remaining (for Hot, Cold spares, other?)

I’ve got a x16 HBA.

The motherboard is a Supermicro X12SCA-5F with
– 128 GB ECC Ram
– CPU: Xeon w-1370 8 cores
– 2x PCIe 3 x4 M.2 NVMe
– 1x PCIe 4 x4 M.2 NVMe
– 6 Sata 3.0 Ports
– 1x SATA DOM power connector

I have other hardware gathering dust, like some Optane p1600X M.2s, 118 & 58gb that I’d like to make use of. I’ve been thinking of using those in the M.2 NVMe sockets, so: a 2x 118gb optane mirror for either metadata or dedup; 1x 58gb for SLOG. Then, put the TrueNAS OS on a SATA DOM or install some sata SSDs.

The optane is PCIe 3.0. If I use the PCIe 4 socket for SLOG, would I be better off to use the optane or get a small PCIe 4 SDD?

My current data has a lot of duplicate files, so I thought about experimenting with ZFS dedup. However, I hope to someday maybe manually dedup that so I wonder if ZFS dedup is worth the resources. Also, I won’t have ZFS dedup setup for the 5 disk raid backup so it seems that whatever space savings I have in the primary resulting from ZFS dedup would not apply to the backup. I suppose I could instead use the ZFS Dedup on the backup pool instead.

Or, I could use the 2x 118gb optane for special vdev metadata. I don’t think I really need special metadata, but then again I’ve got the hardware.

I’ve also read about putting VMs on SSDs. Would makinf use of the SATA 3 ports for additional SSDs be worthwhile?

2

are you using ups for your build? since you are aiming for reliability. So in the event of a sudden power loss, it will then gracefully shut down.

i would prefer running as few hard drives as possible. if you need more capacity just get higher capacity drives so you can run fewer. just thought that before you add a ton of drives, you don’t forget there will be a cost for running so many drives especially if it’s a 24/7 setup with no spin down. but if you are ok with that fine. just thought i’d remind you. more is not necessarily always better ^^

i see you got your backup covered, good :partying_face:

deduplication zfs may help if you have a lot of duplicate data. but dedup requires a lot of ram. and a ups to stay safe in case of sudden power loss.

may be some caveats doing so. so weigh the pros and cons. maybe someone who uses it can explain better :thinking:

that’s my feedback :slightly_smiling_face:

3

With large 16 TB drives, neither 2-way mirrors nor raidz1 are resilient or reliable.
2-way mirror for special vdev and/or dedup vdev is not very resilient either. Dedup is probably NOT worth tickering for a home NAS.
Do you have any use case for a SLOG? (My bet: No!)

Also, a second pool in the same system is, at best, half of a genuine “backup”. If, say, the PSU goes rogue, both pools could be lost in the same event.

My advice would be to get another system for backup and set both primary and backup as raidz2. Up to 6-wide, use motherboard ports and skip the HBA.

4

Thanks for the pointers!

I already most have the HDDs needed to get started. There;s enough HDD slots I could setup Pool 2 (backup) as Z2 & also have a hot spare for Pool 1 (4 pairs of 2 mirrored drives.)

To be clear, I understand not to rely on Pool 2 as my only backup; I plan to add an offline backup before its all done.

Regarding SLOG, I will end up running a database eventually… probably mongo and an a SQL database.

The thing is I have 2x PCIe Gen 3 x4 M.2 slots, 1x PCIe 4 x4 M.2 Slot and Optane P1600 M.2 so I might as will use them for something. I also have a 256gb Gen 4 M.2 SSD (not optane).

Not sure I I would be better off using the Gen 4 M.2 SSD as SLOG or an Gen 3 Optane.

I am toying the idea of using 2x Optane in the Gen 3 M.2 slots as dedup for Pool 2 backup, as a test. If its proves not worth the trouble (which I think will be the case), then I guess I’ll repurpose the 2x Optane as metadata for Pool 1.

Or, would I be better off using the M.2 slots for a VM pool (Pool 3, 1 mirrored pair)? (I have the SATA ports also.) The main reason I might use a VM is to contain services / apps that need root access.