Need hardware guidance for my first NAS build

[Nasdara]

Hello,

Before I start I will list a few bullet points to give some more requirements and context to what I need:

• • ECC is a must.
• • Intel CPU only
• • Full disk and data encryption (not sure if there are any specific hardware requirements for that)
• • Not necessary right away as I have no use for it, but an option for 10Gb NIC card if not on the board.
• • Intel Quick Sync (or so I thought, more on that below)
• • The lower the power consumption the better
• • 99% set on using Proxmox with TrueNAS as a VM (unless there are serious reasons not to)

I know it’s not ideal - but I don’t really know what I will be using the server for other than data storage. I work with Linux Servers (Debian VPS), Docker and VMs on a daily basis but none of them are for home things like PiHole, HomeAssistant and a bunch of other cool stuff you see people using their NAS for (feel like a Raspberry PI would do for most though). Web services are run on VPS, and when I need a VM for whatever reason I spin it up on my gaming PC. So might be a bit delusional for looking at higher end hardware but good opportunity to get into like the homelab hobby territory (server racks and all)

Everything below is based on the research I’ve done for the past week and kind of pigeonholed due to the fact I was set on needing Quick Sync for transcribing (+ECC so limited options). Now that I am writing this I realise this could be easily solved with like an Intel Arc card…

After some quick reading on this and the old forum I decided on the ASUS Pro WS W680-ACE IPMI motherboard which I can get for £380 brand new + a 12th gen Intel i5 12600k or i7 12700k for about £225, total about £600 lets say.

Then I started reading every post about the W680 and TrueNAS on Reddit and Forums - and some say it’s not the greatest option and that a Supermicro Server board would be better suited. For example, this whole thread: just a heads up, finally ASUS w680 available. | TrueNAS Community

So then with zero knowledge about any of the Supermicro boards or their naming schemes, I noticed that the X10SDV-TLN4F comes up pretty often as a recommendation. Being in UK my options for used Supermicro boards are pretty limited in general and there is no used X10SDV-TLN4F for sale and a new one cost over £1000! Plus the D-1541 seems a lot less powerful than a W680 with an i5 12600k (so X10SDV is weaker and costs nearly twice as much over here?) So my first question is: am I correct in that if those two were the only option - then the W680 route is the way to go? (I could also get Supermicro W680 X13SAE-F for a bit of a premium (£440))

I am not really familiar with the different non-consumer grade Intel products so I don’t even know what other board from Supermicro to target as an alternative. What would be the closest Supermicro board and Xeon combo that is on the level of the W680 with 12600k or 12700k?

I am also open to a reality check, as I might be going a little bit overboard with the compute power and wasting money, but again I am not really able to estimate how much I will need as this is my first build. I am worried that if I go with a weaker system it will hurt me in the long term, when my needs grow and I’ll be stuck with outdated hardware (however if there would be a significant cost reduction I’d be more than happy to compromise so open to recommendations)

Plus my gaming PC is kind struggling on an 8th gen Intel and GTX 970, so one benefit I see from the W680 setup is I could buy a graphics card and potentially use it for gaming, anti-cheats won’t allow a Proxmox VM (without risking a ban) so would probably have to boot from a separate drive to actual Windows, but that’s just a thought - I don’t want it preventing me from getting a better suited NAS build.

[etorix]

@Nasdara

The lower the power consumption the better

THAT is a stronghold of the Xeon D-1500. 45 W when churning at 100% on all cores—and with 10 GbE in the SoC.

@Nasdara

99% set on using Proxmox with TrueNAS as a VM (unless there are serious reasons not to)

There is strong suspicion that Proxmox sneakily corrupts pools beyond repair even when properly implemented…

Can't import datapool

Hi all, I have a server that is showing the datapool is FAULTED. Luckily all the disks are showing back online. I have this setup in ProxMox as a VM. I’ve tried several commands “zpool import data -FXf, zpool import -f, etc”. I can’t get it to import. Anyone know how to fix this issue? See the screenshots. 16 disks total. Thanks! [image] [TN 2]

@Nasdara

I don’t really know what I will be using the server for other than data storage

Very, very strong argument to go for bare metal and leave any app/container/VM on your other server. Plus you can skip a HBA and save its power draw.

@Nasdara

I decided on the ASUS Pro WS W680-ACE IPMI motherboard which I can get for £380 brand new + a 12th gen Intel i5 12600k or i7 12700k for about £225, total about £600 lets say.

Add the cost of DRR5 ECC UDIMM to have the complete picture. This should bring you closer to £1000.

@Nasdara

So then with zero knowledge about any of the Supermicro boards or their naming schemes

Learning a bit about the naming scheme is useful, as you can then work out the name of the board you’d want and lookup if it exists in the lineup!
But do not forget to looks elsewhere. AsRock Rack makes some interesting server boards and their website is easier to search than Supermicro’s (not a high bar, admittedly).

@Nasdara

Being in UK my options for used Supermicro boards are pretty limited in general and there is no used X10SDV-TLN4F for sale

Keep looking on eBay. Years ago, I actually got a D-1541 from a UK refurbisher, as a complete system featuring a (Datto-rebranded) AsRock Rack D1541D4U-2T2R. If it weren’t for the Java-based IPMI it would still be nice board.

@Nasdara

am I correct in that if those two were the only option - then the W680 route is the way to go?

If you want Quick Sync transcoding on the CPU (iGPU) and/or run high compute apps/VMs, then yes.
If you want pure storage at low power, a Pentium D-1508, Xeon D-1518 or higher is your best call. Bonus: These use DDR4 RDIMM that is really, really, cheap second-hand. The D-1500 could still do Quick Sync by adding a cheap Arc dGPU, and provide enough cores to play with quite a number of apps/VMs on top of NAS duties (all the “fine on a Raspberry Pi” stuff will positively thrive on a 2 GHz Broadwell-D Xeon!).

@Nasdara

I am not really familiar with the different non-consumer grade Intel products so I don’t even know what other board from Supermicro to target as an alternative.

The alternatives are the intermediate generations and the Xeon E-2000 corresponding to the Core consumer CPUs you may be familiar with (we discussed some of these here). To keep with your “ECC and Quick Sync” requirements:

• Comet Lake. C256 chipset (Supermicro X12STH, AsRockRack E3C256D4U) and Xeon E-2300G.
• Cascade Lake. C246 chipset (X11SCH, E3C246D4U) and Xeon E-2100G/2200G or Core i3-8100/8300/9100/9300 (these do ECC!).
  Not the same Quick Sync generation, not as much compute power as Alder Lake (but you’ve not made a great case for needing this kind of computing power on your NAS), DDR4 ECC UDIMM should be cheaper than DDR5 ECC UDIMM, and there are plenty of E3C246D4U2-2T boards on eBay (£500 total with CPU and RAM?).

[Constantin]

Following up on @etorix’s awesome post, a great D-1508-based SOHO file server board is the x10sdv-2c-7tp4f, a motherboard with 20 SATA ports built in (2 of which are SATADOM), a low-core but high speed CPU (good for SMB, since SMB is a single-thread application per client), two SFP+ ports, two PCIe 3.0x8 slots, mSATA as well as miniPCIe x4 slot, SuperMicro quality + support, low wattage. But, it’s flex ATX (i.e. a fairly large board), the two cores will not host a billion VMs, and the CPU is embedded so you cannot change your mind later. At $400 used (with shipping to UK), or $800 new from a reputable seller.

[Constantin]

For something tiny, take a look at the X570D4I from Asrock Rack. Not my favorite company due to its spotty end-user support, but it is Mini-ITX, it has two Oculink ports that you can use either for 8 SATA or NVME, 1 PCIe 4.0x16 slot, 10GbE Copper, and the freedom to select among billions of Ryzen CPUs. For low-wattage and validated ECC support, scour eBay for a -G or -GE CPU on the Asrock Rack CPU QVL list. For $400 new (and likely $500 w/shipping to UK), this board offers a lot of flexibility in a very small space.

But, it’s not all unicorns and roses, as this build report attests. While Serve the Home was able to get full bifurcation on the x16 slot, @directhex in his build was not, apparently due to limitations re: the Cezanne-series CPU. Also note the unusual heat sink / cooling solutions needed for this board. Lastly, this board may not be suitable for a performant all-flash setup due to bus bandwidth limitations between the CPU and the X570 chip (only four PCI 4.0 lanes, which is lame)

[etorix]

AMD does not fit the “Intel only” requirement.
As for bifurcation with Ryzen, for a change, it is quite simple in AMD parlance:

• Desktop CPUs can do x4x4x4x4, and most support ECC (but check for the odd exception in recent models).
• Laptop/mini-PC-oriented APUs can only do x8x4x4 (like Intel Core) and only the “PRO” versions support ECC. Due to their monolithic design, they have lower idle power than the CPUs.

The “non-ECC” Ryzen CPUs are actually recycled APUs with a disabled iGPU (= Intel ‘F’).

[Constantin]

Whooops, over looked that. Thank you for correction.

[Nasdara]

First, thank you so much both of you for taking time and giving in depth replies. I was worried my post will be too long for anyone to bother to read (originally was twice as long but managed to trim it a little bit lol)

@etorix

There is strong suspicion that Proxmox sneakily corrupts pools beyond repair even when properly implemented…

Yeah, I’ve read all the recent posts after I posted. Will probably stick to bare metal for the time being and maybe switch to Proxmox once it is investigated more or I have a proper separate backup strategy in place (this will be my only storage for the time being so can’t be risking loosing it all )

@etorix

Add the cost of DRR5 ECC UDIMM to have the complete picture

Well, trying to match DDR4 ECC memory that is recommended by Supermicro seems to work out more less the same due to no availability of used sticks, and the fact that all of these parts new are much more expensive vs the US.

Unless I am doing something wrong? I will post some example build tables and their prices below so you can see what I mean.

@etorix

Learning a bit about the naming scheme is useful

I have been referencing their naming scheme guide thing I found on their website, but obviously still gets a bit overwhelming and confusing cross checking every letter until you’re fluent in it haha.

@etorix

Comet Lake. C256 chipset (Supermicro X12STH, AsRockRack E3C256D4U) and Xeon E-2300G.

Here is two example builds one W680 and one C256:

W680	Name	Price	Notes
Motherboard	ASUS PRO WS W680-ACE IPMI	£449.44	Price went up
CPU	Intel Core i5-12600K	£170
RAM	DDR5 Kingston 32GB ECC x2	£300
	Total:	£919.44

C256	Name	Price	Notes
Motherboard	Supermicro X12STH	£353.00	New (no used for sale)
CPU	Intel Xeon E-2378G	£420	£545 new
RAM	MEM-DR432MD-EU32 (32GBx2)	£202	None for sale in UK…
	Total:	£975.00

There is literally only like four used E-2300G for sale at the moment in the whole of Europe on eBay

Used alternatives found are:

• E-2386G for £522
• E-2336G for £318 (so total £873+++ as there is no cheap RAM)
• E-2324G for £222 (so total £777+++ as there is no cheap RAM)

So again, it seems like the W680 not only is more powerful but also cheaper in this scenario, unless I am seriously missing something here? (also wonder what the power consumption difference is?)

I will look for C246 options and Xeon D-1500 later (not confident in the performance of that one to be fair), and post my results.

@Constantin

a great D-1508-based SOHO file server board is the x10sdv-2c-7tp4f

Modern X10SDV-2C-7TP4F replacement? | ServeTheHome Forums This guy seems to struggle with encryption on this one.

More importantly: Importing from the US or anywhere outside of the UK for that matter is kind of a lost cause unless it is extremely discounted - as anything that costs over £135 will get taxed a minimum of 20% extra VAT by the lovely customs mafia + any multiple of different custom fees and duties which aren’t too bad but the VAT is a killer. This is probably the reason why most of these niche server things brand new are resold for so much in UK (especially if you’re not a business and have to pay the VAT)

On top of it all we got the energy price working against us currently at $0.29 USD per kWh (and artificially capped by the government so no reason for it to not go up even higher soon)

Hoping to be able to offset those by cancelling the 2TB Apple iCloud (£8.99 a month) for backing up my phone and a 2TB Google Drive (£7.99 a month) for backing up photos or replacing it with Backblaze or something

[etorix]

@Nasdara

So again, it seems like the W680 not only is more powerful but also cheaper in this scenario, unless I am seriously missing something here?

No, that’s the Xeon tax at work. It should work better with the C246 scenario, provided you also shop for used ECC UDIMM (rare but not inexistent) and do not try to follow a much too short and much too old QVL. There are reports of weird cases, but I’ve personally never had any issue with any Micron/Samsung/SK Hynix-branded module of ECC RAM, UDIMM or RDIMM, in any server/workstation motherboard. It just works.

@Nasdara

Xeon D-1500 later (not confident in the performance of that one to be fair)

For storage, more than enough! There’s a reason why these ten year old boards are STILL sold new for eye watering prices: They do the job, and their succesors are actually less suited to storage tasks (much higher power for X11SDV, somewhat lacking in basic I/O for X12SDV).