After many (10+) years of using my QNAP TS-563 (5 x 3TB WD RED in RAID6) I finally decided to move on and build something more technically advanced and increase performance and storage without using a lot of power.
I’ve watched a lot of YouTube videos, read a lot of articles and asked on Discord. My original plan was to buy a Gigabyte MC12-LE0 and a used Ryzen Pro, but unfortunately prices have risen beyond what I was willing to pay.
For now, a fried offered me an “old” SuperMicro X11SSL-F with an Intel Xeon E3-1230V6 (3.50 GHz, 4 cores, 8 threads, 8 MB) and 16GB ECC DDR4-2400 RAM for ~€150.
The board comes without NVME and offers 6 SATA ports:
Now my questions:
Should I use a SATA SSD for the boot drive and use the remaining 5 SATA ports for hard disk storage or better use a PCiE dual NVME extender card and use all 6 SATA ports for HDD storage?
If using a 2 slot NVME PCI card, better a mirrored boot pool or boot plus zfs cache/app storage? And any brand advice? As far as I can tell from the SuperMicro docs the PCI Bifurcation will work with the X11SSL-F.
Hard to find any absolute statements on which brand of storage to buy. As far as I can sum up, go CMR and everything is fine?
As I don’t want to completely blow my budget (1000 - 1500 €) on storage, I think 4TB is the sweet spot?
Any recommendations on the PSU? I assume 400W would be enough.
The motherboard has two Gbit Ethernet ports, would you upgrade to 10G straight away? I’m not sure if this set of hardware will provide more than 100Mb/s. At the moment I only have a 2.5G PC and would need to buy some new switches and cables.
Thanks for reading and the opinions of the mighty truenas wizards are greatly appreciated!
The X11SSL-F IMHO Is still a viable choice, my systems are both on similar hw and i’m pretty satisfied. What you can find usefull Is buy more RAM, but ECC Udimm modules are quite expensive…
The bifurcation should can be possible with x8x4x4, so you can buy a passive cheap card and put directly 3 NVME, but don’t waste 2 disks for boot… keep 2 in mirror for apps and a small one for boot (space on boot pool will be not used for anything else, optane 16/32gb NVME are pretty cheap). The mainboard also provide an internal USB port that you can use for boot from a SATA SSD with an USB adapter. 6 SATA ports free for a resiliente 6x raidz2
About cache, normally Is not needed: for slog depends on your use case (heavy sinch write), and you don’t have enough RAM for l2arc.
The wattage of the PSU not mean everything… But the most of ~450/500w Gold PSU can handle without problem this kind of load.
You should still have 1 Pci-ex slot for think about a 10gb upgrade in future, but as you mentioned you will not benefit now (better use budget for buy more RAM IMHO)
As soon as I find a good oppertunity on 32 or better 64Gb RAM I’ll buy it!
Will be looking for some 3 or 4 way nvme pcie cards. Right, I don’t plan to use the NAS for apps/VMs, but better be prepared when I want to
Found a good looking card on german amazon (Quad NVMe PCIe Adapter, RIITOP 4-Port NVMe auf PCI-e Express 4.0/3.0) für about 50€.
Looks pretty solid with the passive heatspreader and not too expensive.
I just upgraded my SuperMicro NAS with new disks, my SuperMicro MoBo has 6 Sata & 1 NVME slot.
I was advised to go with 4 large drives vs 6 medium/small.
So ended up buying 4 x 16TB Segate IronWolf’s (spinnng rust), and put them in in a RAIDZ2 configuration.
It was an “outch $$” experience, but in the end it made sense.
I now have room for two 16TB disks more if i run out of storage.
@oxyde mentioned you have an internal USB Port …
I would use that one as boot disk, so you have room for 6 SATA disks.
Don’t use an usb stick for boot, but get a USB → NVME adapter, and a cheap 128GB NVME disk … I use Patriot P300 as boot disk.
Remember you have to mount/fit the USB–>NVME adapter inside the chassis.
Remember if you (in the future) want to run 2.5Gb or 10Gb you’d need a PCIe slot for that card.
But probably will work the same!
(Btw this one, 15€ on AliExpress, Little more the 4 NVME version… they are the same from Amazon at less than half of the price).
Or if really you don’t want use apps (but trust me, when you have the possibility you will ) just skip this way and go for the SATA/USB option / single NVME adapter and forget It (for now obv).
For the RAM, i have bought mine here, now the price Is 10€ less
You were offered a fair alternative. But note that at the current 100-150€ the MC12-LE0 is still excellent value for a new, boxed, server motherboard—AM4 server boards typically cost 300-500€.
Mirrorred boot is of very limited use for a home NAS. You do not have enough RAM to even think about L2ARC.
NVMe boot from a small and cheap drive in the x4 slot leaves all 6 SATA ports for storage.
For an application pool, check how bifurcation might work with two x8 slots from a x16 root port which can do x8x4x4 or x4x4x8 but not x4x4x4x4.
Thank you for your consideration. I’ve ordered 6 x 8TB Toshiba N300 drives.
I’m still unsure about the PCI bifurcation. All I could find on the SuperMicro homepage was a FAQ entry for a setup with 2 NVME and the appropriate BIOS settings.
Sorry for my idiotic questions, but I can’t find out which card to use with which maximum size of NVME drives.
@oxyde mentioned that the CPU can’t bifurcate for 4 drives. So how do I install more than 2 drives?
Look into the actual BIOS options of your board rather than at generic documentation.
I would expect that it can pass x4x4 to one slot (so two NVMe drives in a passive riser) and x8 to the other (one drive).
For more than three drives you’d need a PCIe switch—which is the easiest but most expensive option.
) just skip this way and go for the SATA/USB option / single NVME adapter and forget It (for now obv).