This is where I parted company. My always on primary server draws 20 watts with a two drive mirror; my backup servers draw closer to 70 (both X10SDV) but I turn them on only when I need them. You have to have a backup but reversing them would add $135 in electricity a year.
2 Likes
Actually, my UPS showed a percentage value, and I forgot to perform the VA → W transition 
. So, my estimation should have been 220VA * 0.8(PF?) - 70W = 106W.
I’ve performed basic tests – it idles around 94-101W. Used to be 93-97W at the start. The only explanation I can come up with is that managing (relatively) big ARC sips some power. I will post more detailed numbers later.
maybe an alternative route (for max power efficiency) is to get one of those mini pcs that comes with lots of nvme slots. I have seen ones with 4 - 8 slots.
if you are not going much beyond hosting files, a n100/n150 class cpu is more than adequate.
if you want to be cheap, ECC DDR3 RDIMM is quite cheap now. but the chips usually consume lots of power.
if you use lots of mechanical drives, pick a system that allows you to put them to sleep.
Careful with those. See the issues with the Beelink Mini.
Not recommendet with ZFS. See latest T3 Truenas Tech Talk.
I finally pulled the trigger on a system. It doesn’t match my original requirements perfectly, but it’s close enough:
Odroid H4 Ultra
-
Supports 64 GB RAM, despite what the documentation says (confirmed by many users)
-
Not “real” ECC, but DDR5 in-band ECC — a compromise I was willing to make
-
No 10 GbE on board, but includes a Gen3 x4 M.2 slot with bifurcation support (adapter cards available). With a adapter, I can use two NVME drives (mirrored) as boot drives and then use the other two lanes for a 10GbE NIC… not sure if possible but I’m happy with 2x 2.5Gbe as on the board for now…
-
4× SATA ports
-
2x 1TB nvme drives (boot drives)
-
Intel Core i3 N305 — similar to the N97, but slightly more powerful and very energy-efficient
What do you guys think? Also, do you think the performance would be still ok if I run TrueNas as a VM on Proxmox on it? I won’t be running a lot of other VMs on it but I like the idea of having it as part of a cluster. I have two other MiniPcs with Proxmox on it that run my webserver and nextcloud… in case one of those goes down, I could have them automatically spool up on the Odroid system (High availability).
It will let you bifurcate to x2x1x1? And I don’t think I’ve seen a 10 GbE NIC that didn’t need at least 4 lanes.
You do you, but even 128 GB (as small as I could find) is gross overkill for a boot device. Keep in mind that you can’t use it for anything else.
1 Like
Totally unnecessary in a home NAS, IMHO.
I think I’ve seen PCIe x2 Aquantia NICs. Whether these are a good choice with TrueNAS is another matter…
Oh… is that the case if you install TrueNas bare metal or do you also need a dedicated boot drive when installing it in proxmox? I was going to use the 1TB drives as the boot drive and also store VM’s etc. The board would also have an emmc slot… not sure if that is a good idea to use as a boot drive…
From what I just gathered, there are lot’s of options. It is a bad idea to use USB flash drives as boot drives, but connecting external SSDs to USB appears to be working just fine. Else I would have to give up on the idea of 10Gbe, and use the internal m.2 slot to connect the 2x 1TB NVMEs plus the truenas boot drive (2x 128GB mirrored drives).
So basically you asked here for advice. You were given sound advice but decided for whatever reason to buy some random - excuse my french - junk ?
And now you are talking about proxmox ???
2 Likes
Yup. Ask for advice, change his mind (fair enough), and goes for consumer-grade hardware which goes against most of the initial requirements (RDIMM, 10G, “SATA SSD for the OS (no need for NVMe)”… nope, only the 4*SATA made it), for use as virtualised TrueNAS which was never mentioned.
Plonk!
1 Like
So the board you bought is about the same price as the one I suggested, but it doesn’t meet a single one of your stated requirements (other than, I guess, low power consumption). You do you, of course, but what about this one appeared better to you than the X10SDV?
1 Like
Here is the way I see it:
Power efficiency was my number 1 requirement, and the board/processor combo I went for perform better than anything else suggested. Beats a Xeon D-1521/X10SDV with quite wide margins in performance and power consumption, and isn’t 10 years plus old and hard to come by (dealing with used boards and all the potential issues).
Meets the requirement of 64gb memory, and partially the one of ECC memory. (I wanted RDIMM mostly because I can get it super cheap on the used market, didn’t care about it being “registered” other than the price) The Crucial 64gb module I got is cheaper than regular ECC memory as well. In band ECC is good enough for me. So all in all this is the best option for me.
I can run SSDs (sata) for boot with an adapter or NVMEs. Just said that SSD sata drives would be good enough, not that I absolutely don’t want NVMEs… In this regard, the board I chose is more than adequate.
Also can connect my 4x 18TB drives over the built in SATA conectors.
Literally the only requirement it doesn’t satisfy is the 10Gbe port… I might get it working but it will be a hack. Like using an adapter to 2x m.2 2 lanes each, one with a 2x sata adapter (boot drives) and the other with a 10gbe adapter. I decided that I will be fine with 5Gbe or 2x 2.5Gbe for now.
Still think it was the right decision.
I got a PCIe4.0x1 & works ‘okay’ - some issues on boot that require a few commands that I haven’t figured out how to easily cron… other than boot it is solid.
These are the requirements you stated in your OP:
- => 64 GB DDR4 ECC RDIMM capacity
- You say your choice does 64 GB; its product page says a max of 48 GB. It doesn’t do DDR4, it doesn’t do ECC, it doesn’t do RDIMMs.
- 10 GbE
- Your choice doesn’t have it. I guess it has 2.5 GbE? Nothing on the product page actually says this though.
- SATA boot SSD + 4 spinners
- Nope, just 4 SATA ports. You could add more via the NVMe slot, but that’s your only expansion option.
- Power draw < 70W at idle
- I guess it meets this? Again, the product page doesn’t say anything about idle power draw, but it does recommend a 133W PSU if you’re using spinners.
If this board meets your needs, that’s great, but it also means you did a remarkably poor job of stating your requirements in your OP, because this board meets only the “low power draw” one.
Don’t care what the product page says, plenty of users say it works without any issues. That is what counts. So, yes, it meets the 64GB requirement. Also partially meets the ECC requirement as it supports in band ECC. It’s not RDIMM but that was not a hard requirement as I only cared because of the cheap price.
Doesn’t have 10 GbE on board, only 2x 2.5GbE. There are ways to get 10GbE with an adapter. Good enough and therefore meets my needs.
Also can be done with an adapter (M.2). Two of the Gen3 PCIe lanes for the boot SSD, 2 lanes for the Ethernet. Therefore meets the requirement.
From what I have read, it should definitely meet the power draw requirement.
So the only thing I’m really not getting is RDIMM… More than happy to compromise as I’m getting more power, better energy efficiency, and a new board that I can order from the local shop and have it in my hands the same day.
BTW, here are some power draw numbers. This beats pretty much everything else by a wide margin: https://forums.unraid.net/topic/167669-odroid-h4-intel-n97-2x25gbit-4x-sata-1x-m2-ddr5-max-48gb-with-ecc/
There’s plenty of users reporting similar numbers. At 3.5USD/W per year, this will save probably hundreds of dollars over its lifespan. Also, I won’t have much noise because there is not a lot of heat that needs to be expelled from the case…. I’ll report once I have it all built.
I haven’t read everything yet, but isn’t 10 GbE a bit overkill if you’re only connecting four spinning drives? (Unless you’re hoping the use case or workload will really make good use of the ARC cache and therefore saturate your bandwidth.)
BTW … AFAIK: an additional 10 GbE NIC could make your modest power-consumption setup go up in smoke….I suspect it could at least double your idle draw, these things can consume a fair amount, even when idle. So it’s worth to have an eye on this topic too…
You are absolutely right, this is one of the reasons I haven’t bought a 10 GbE NIC yet. I do some film editing and therefore deal with a lot of large files. Also downloading camera footage would benefit from 10 GbE, however, it is not like I’m using it daily. I was afraid that my power usage would go way up, so I’m holding off on 10 GbE for now and just see how it all goes.
1 Like
Even having a 10 GbE NIC installed, you’re not going to saturate that link with just four spinning drives….neither on reads nor on writes. Unless your workload happens to hit the ARC just right, say the 64 GB (minus whatever) you’re actively working on, you’ll never see that kind of throughput.
(Unless you’re planning to stripe those four HDDs together,…in which case you could, in theory, get somewhere around 600–900 MB/s…
But I’d suggest forgetting about that idea before some of the forum members go into anaphylactic shock just from reading it. 
2 Likes