I am sick to the back teeth of my very very expensive QNAP NAS. I bought it mostly because I didn’t want to have the hassle of a self-managed piece of hardware and assumed that buying a properly supported product would be fewer headaches in the long run. I was wrong.
Another hardware problem a couple of days ago has had me writing a shopping list for a DIY NAS based on TrueNAS. I would appreciate any suggestions for improvements or optimisations to my (potential) build.
Case: Supermicro CSE-743TQ-903B-SQ
Mobo: ASRock Rack AMD B650D4U
Processor: AMD Ryzen 5 9600
Memory: 64G of non-ECC memory(1)
HBA: Some sort of 9300-8i(2)
Disks: WD Red pro 6T x 6
I would take two cache SSDs and two cache NVMe drives from my QNAP NAS and also use them in this build. The SSDs as mirrored boot, the NVMes probably for VM.
Hello forum, and tell me where I’m wrong!
(1) I simply cannot justify ECC memory at current prices, even if I dropped this to 32G it still stupid amounts of money. I could be talked into buying a tiny amount of ECC memory now, but the QVL for the motherboard is limited.
(2) My least comfortable component as I’d be buying refurb, but this model seems very well supported and understood. The current series of this controller is remarkably expensive, as are the breakout cables.
Do you have a “strong” reason to stay with DDR5?
Because if you plan on a DDR4 platform, you can achieve ECC, and probably also avoid the HBA (Is very common for those motherboard to have at least 6 SATA ports). Everything for less money
It’s a bit hard to give advice because we don’t know what the system will be used for. Is it mostly just a SMB share with a small, single VM possible or do you need all that memory to run larger VMs? You need to plan for cooling the HBA unless it is installed in a server with server level cooling. HBA cards call for 150-200 linear feet per minute on air flow. A lot of users install a fan blowing right over the heat sinks.
If this is your primary or secondary source of data storage, I would suggest going with ECC. We have posts in the forums where users have pool issues, test the memory and find it is bad. The more copies of the data you have, the easier it is to deal with and restore from a backup. You have to decide how much your data is worth and if you are willing to risk it.
You can do a search for ‘memtest’ and read the posts where there were pool recovery problems. Memory problems tend to cause errors across all the drives. If they are not caught early and taken care of, the pool is usually lost along with all, or most, of the data.
You don’t need to have a mirror boot pool. As long as you keep a current backup (download) of the system configuration, you can install the OS back to a new boot device, reload the System Configuration and be back to a normal system quite quickly.
Yeah, but mirrored boot pool means I get to do it on my terms, when I’m free, not immediately, and that stuff always happens at the worst possible time.
But yes, it’s not exactly an amazing benefit when sata port limited.
The QNAP is used for bulk media and home directories with some smaller VMs. I would probably run more VMs on it if I could trust it, which I can’t and don’t. Extra annoying because it has 128G of ECC memory, but SO-DIMMs. I use BackBlaze for cloud backup of everything; I know this doesn’t protect against data corruption.
As the boot SSDs would be reused, and good quality, hardware, there’s no extra cost involved so I see that as a free “nice to have”.
In that case you might look into motherboards with suitable SO-DIMM slots… (DDR4?)
If the VMs are light and could do with ca. 2 GHz clocks, you might go for some high-core Xeon D-1500 and DDR4-2133/2400 RDIMM. Otherwise, would you be ready for Xeon E5 or some old Scalable?
This had occurred to me, but I don’t really want to build what would be fairly important machine on hardware that isn’t new or trivially replaceable.
If I could find a new AM5 mobo capable of using the RAM then I’d very seriously consider that over a more server-orientated board. The one board I’ve found that is close will take only 64G (which is still probably okay) but it’s very unclear as to whether ECC is properly supported or not.
.