I want to build a new TrueNAS Scale system and would like to take your experiences into account.
Below are the rough specifications of the system:
Data Disks
6 HDDs with 24TB each – (I orientend at the Qnap HCL list)
Raid-z2 (double parity – comparable to Raid6)
Connected via SATA
What about those possible disks?
Seagate Exos X - X24 24TB, 24/7, 512e/4Kn / 3.5" / SATA 6Gb/s ST24000NM002H h ttps://geizhals.de/seagate-exos-x-x24-24tb-st24000nm002h-a3050687.html
or
Western Digital Ultrastar DC HC580 24TB WUH722424ALE6L4 h ttps://geizhals.de/western-digital-ultrastar-dc-hc580-24tb-wuh722424ale6l4-0f62796-a3106003.html
Operating System (OS)
mirrored (Raid1)
Connected via M.2 or SATA (depends on connectors of the mainboard)
What about those possible disks? (depending on price)
Samsung SSD 870 EVO 1TB, 2.5" / SATA 6Gb/s h ttps://geizhals.de/samsung-ssd-870-evo-1tb-mz-77e1t0b-a2458826.html
or
Samsung SSD 990 PRO 1TB, M.2 2280 h ttps://geizhals.de/samsung-ssd-990-pro-1tb-mz-v9p1t0bw-a2795961.html
Cache
Is a Cache (L2ARC?) possible or useful in this constellation to improve the performance of the system?
Connected via M.2
Hint: I have 2 x Samsung 990 PRO NVMe M.2 SSD, 4 TB already in stock
I also heard, that there is a possibility that “consumer” NVMe are worn out when used as L2ARC. Does someone have experience with Samsung 990 PRO as L2ARC?
Mainboard
If possible: Low energy mainboard
Which NIC-Chipset should be taken? The below named have the choice between Realtek RTL8125BG and Intel Killer E3100G
With the above-stated requirements, I am currently looking for a mainboard with 4 x M.2 and 8 SATA ports
Or is it more beneficial to look for a smaller mainboard and attach a PCIe-adapter-card for M.2 and / or SATA ports?
What about those possible mainboards?
ASRock Z790 PG Riptide h ttps://geizhals.de/asrock-z790-pg-riptide-a2820266.html
or
ASRock Z790 Pro RS h ttps://geizhals.de/asrock-z790-pro-rs-a2820217.html
RAM
Is 128 GB sufficient?
192 GB would be possible with the above named mainboards, but then RAM is twice as expensive
With the above named mainboards, only non-ECC RAM is possible. Any experiences or recommendations here?
CPU
I thought about Intel Core i7-14700K, 8C+12c/28T, 3.40-5.60GHz
also to have enough resources for virtual machines running on the TrueNAS system (see below)
Deduplication
Is Deduplication possible with this setup?
Power Supply
What about this PSU?
be quiet! Pure Power 12 M 850W ATX 3.1 h ttps://geizhals.de/be-quiet-pure-power-12-m-850w-atx-3-0-bn344-a2884020.html
Remote Management / KVM
Intel vPro was too expensive (or only available with older chipset z690)
(Mainboards like Supermicro have KVM installed and also ECC-RAM - but are really way more expensive)
Maybe I am trying JetKVM
Virtualization
I want to run Proxmox Backup Server (PBS) as virtual machine on top of TrueNAS scale
.
.
What do you think about the listed components?
I would appreciate your advice, tips and feedback.
Deduplication, just don’t use. It has to be set up just right.
L2ARC, doubt it would help much with that much RAM and light usage. A small or medium office of users, maybe. If necessary, you can add later.
Suggest staying away from RealTek and that Intel Killer, if possible. Regular Intel, Chelsio or Solarflare are three NIC recommendations.
You don’t have to have mirrored boot drives as it is easy to keep a saved configuration backup, replace a failed boot device, reinstall TrueNAS and reload your configuration file.
Sorry if this was misunderstandable.
The data pool consists of 6 HDDs with 24TB each.
The two Samsung 990 PRO would only be for L2ARC Cache, if that has any improvements to the setup.
Regarding the 1TB boot SSDs - Ok, thanks for that hint. I didn’t do any tests yet in regards of virtual machines. If the 1TB boot SSD are too big, is it possible to place the virtual machine storage also on the boot devices, or only on the data pool (consisting of the 6 HDDs).
Ok, so the RAM also boosts up the performance of the system without an installed L2ARC Cache?
Do you have any experience regarding “wearing out effect of consumer NVMes” when used as L2ARC?
This is a hard one, because the next matching mainboard which does not use Realtek RTL8125BG or Intel Killer E3100G is the ASUS ProArt Z790-Creator WIFI which has the NIC Intel I226-V on board and is twice as expensive as the before mentioned mainboards.
If the NIC really is a problem, I probably should take a look at a PCIe NIC.
h ttps://geizhals.de/?cat=mbp4_1700&xf=1244_8~18947_4&sort=p#productlist
Ok, thanks. I didn’t do any tests yet in regards of backup and restore of the TrueNAS scale config. I gotta have a look at it.
Nothing wrong with the drives. Mirrored boot is probably unecessary. L2ARC can be added afterwards, and it is not obvious that you would benefit from it; if you do, however, 4 TB looks like way too much of it with “only” 128 GB RAM.
This does not look like it would require the latest and greatest in CPU.
For a system with over 100 TB of storage, and over 100 GB of RAM as well, it would be pity not to use ECC RAM. Consider going for server-grade hardware of an older generation, possibly second-hand or refurbished to keep costs down, or look into AMD Ryzen rather than Intel Core.