Months have passed since I decided against W680 platform, yet I am still no closer to deciding on hardware.
Well maybe that’s not completely true, following a lot of the good advice here has gotten me looking at “real server” platforms, currently I am enamored with X11SPI-TF (or similar) as my “all in one” box due to real ECC (RDIMM) support (for the ZFS NAS part), and plenty of expansion options (for adding an Arc 310 for Frigate, etc.) plus who knows what else in the future.
My math shows that the above option can currently be put together for a little over half the price of the “2 separate boxes” option, where I think I would go with the oft-recommended D-1500 based system (for the ZFS / TrueNAS) and then something like an Optiplex 7080 MT perhaps (10th Gen Intel), for the Frigate box.
The only real question I keep wondering about is power usage. These numbers can be hard to come by, I have scoured the Interwebs but only found a couple mentions, one saying that the X11SPI-TF can idle “in the 10s of watts” but I have also read 50-100W, so I’m not sure what to believe.
I suppose Optiplex “idles” at maybe 10-15W (perhaps less?), and I don’t know about D-1500 setups either, so I would welcome any numbers on those as well.
Kindly mention how many HDDs you have, and other things which may affect power usage (10G copper Ethernet, HBA cards, etc.).
I doubt that the mobo itself drains 50W, especially idle.
I can’t answer your questions. But IMO, the whole power consumption scalping topic is overhyped (and makes sense only for devices running on battery). As a general rule of thumb, newer hardware has lower consumption for the same performance because of a newer lithography and thus lesser heat generation.
Home hardware is rarely used under full load (tbh, rarely used under non-idle load even). And I have high doubts that you can save up more than 10-20W at idle with, let’s say, 5 years newer HW. 20W → 24 * 365 * 20 → 175kWh a year. That would save some tens of $/€ per year. And mb would save a few hundred over the whole HW lifetime period in the best-case scenario.
So, IMO, price/performance ratio is generally a more important factor than power consumption.
I don’t think it is simply a matter of the age/generation of the hardware. These TinyMinyMicro / office machines (Optiplex, et al) are optimized for low power usage, where it seems to me that “real server” hardware, often, is not (D-1500, Atom, are the exceptions to this, and why they are so often recommended for an always-on NAS).
Based on real numbers (for me anyway), if there is 70W difference between the two scenarios I laid out in OP, we are talking $110.70 per year (or $9.22 per month) difference in power cost. Is that the end of the world? No.
Now, a correction from OP. Prices of D-1500 seem to have gone down since I last looked. Now the price difference between the two scenarios is more like $200.
So, it would take me roughly 2 years to recoup the higher initial cost, on a straight up cost analysis. Since I expect this system to last longer than that, it would seem the “2 boxes” might be the way to go after all?
But I am still guessing as to D-1500 power numbers.
I am going to try and quantify power usage on your additions, to try and get down to a rough estimate of base system usage.
RAM: ??? but I don’t think significant, 1-2W per stick (no idea)? total 8-16 W
boot SSD: minuscule (?)
3 x HDD @ 5-10w each (?) = 15-30 W
SSD: 1-2 W (?) again, no idea, call it 1 W
NVMEs: <1W at idle (?)
10G (copper) NIC: 5-10W (?)
Quadro P1000: I only spent about 30 seconds, but it seems like this is a 47 W TDP dGPU? I know that doesn’t equate to power usage, but it gives a clue. Sounds to me something comparable to an Arc perhaps (at least in terms of power usage, as Arc is also a 50W TDP part). Anyway, call it 20-30W for arguments sake (?). I know, it really depends on usage.
Intel Optane: 5-10W (?) again, based on 10 seconds research
Corrections on any of the above are welcomed.
So, we might have somewhere between 54 W (low) and 97 W (high, probably too high?) worth of accessories, which might make the base system something like 18-61 W. So, maybe “some 10s of watts” is not unreasonable after all.
Of course, different mfrs can implement things differently, I suppose. But it’s some kind of data point, at least. So, thanks for sharing it.
I would like to add, this configuration included a number of SSDs, to wit:
6x Intel DC S3700 400GB
1x Samsung XP941 512GB
1x Intel DC P3600 400GB (add-in HHHL card)
How much power usage that adds? Probably not a lot, but maybe some few watts?
D-1500 still seems like very power efficient system.
So let’s say, for arguments sake, 20-25 W at idle (without SSDs). Plus 10-15 W idle for the Optiplex system. We are at 30-40 W for the “two boxes” scenario. I’m afraid that’s going to be really hard to beat.
However, based on my calculations above, Farout’s base system might be anywhere between 18-61 W. Which makes the two scenarios not really so far off from one another after all, I’m afraid.
Although I thank you very much for taking the time! For science!
I’m curious, have you ever made any efforts to lower power usage (checking BIOS settings, turning off unused peripherals, checking C-states, etc.)?
I also can’t help but to wonder how different Supermicro’s implemention might be, using the same CPU slot…
Another data point, I wanted to point out that, in order to have an apples-to-apples comparison, we have to include the Arc 310 power usage in the “one box” scenario (as, with “two boxes” that work is being done by the iGPU on the 10th Gen Core CPU on the Optiplex).
According to theseposts at least, it seems we might expect anywhere from 10-40 W usage by the Arc 310 alone (depending on load).
I want to note that Farout’s EPC621D8A appears to be based on C621, where the X11SPI-TF I am considering is based on C622, a slightly newer chipset. Not sure how much difference that might make (in power usage)?
Other than that, looks like a nice board, with lots of expansion options, maybe I should consider it as well (although, that power usage! ).
EDIT:
Studying the specs of the AsRock board further, I see 4x Gigabit Ethernet on-board, no doubt contributing to the power consumption.
In fairness, Supermicro board sports 2x 10 Gigabit, I’m sure those are burning some watts as well.
In either case, I would hope to be able to turn those off and instead install an inexpensive network card and use a DAC, in order to lower the power consumption.
So, to summarize where we are now:
The “one box” solution might burn anywhere from 60-130 W (that’s 10-40 for Arc + 50-90 for the mobo).
And the “two box” solution seems to be holding steady at (an estimated) 30-40 W.
So, perhaps not an insignificant difference (30-90 W)!
I still think more data points are needed (hint, hint).
From STH reviews, a Xeon D-2100 (X11SDV), which is the SoC version of C621, idles at 50-60W, and I would expect a discrete C621 chipset to drain a bit more.
Idles at 62w with all four bays filled. I’ve seen it get up to about 110 when it (tries to) transcode 4k video in software. Only the main 1G NIC and the IPMI port are plugged in. I don’t have any faster switching gear yet so I haven’t tried the 10G NICs.
Chipsets do make a difference, but as usual “it depends”.
For example, the NIC on my motherboard is built in and its efficiency is likely in the x520 range given the age of the motherboard. A more modern x710 NIC consumes 5w, or less than half of a 520.
Similarly, the HBA here (a LSI 2116) is also fixed to the motherboard uses about 13W at idle. A modern version like the 9500-16i runs on 6W at idle.
Lastly, the D1537 is ancient, and has been ancient for a while. A 5650G(E), 8300G, or like chip could sip power but the comparison is difficult because so many C6 states, ASPM, etc actually have to be activated to make it happen.
I have zero doubt that a motherboard built with modern components would idle normally at 15w less than my current rig. But it doesn’t make sense to switch just yet. The biggest hurdle is the current cost of DDR5 memory, which makes 128GB about as expensive as the motherboard itself.
That to me is the main reason to look into older boards that have very little going on peripherally (ie no 10GbE or HBA built in) but adequate PCIe slots to host same and the ability to insert a more modern CPU. Not sure what that board might be, however.
For science! 
).
