I am currently redoing everything in my homelab and aiming for low power consumption.
Currently I am on TrueNAS Scale
Hardware: Lenovo m920q mini PC with LSI HBA which runs to 3 12tb HDDs
TrueNAS virtualized on proxmox with PCI passing through the HBA
Definitly a frankenstein build lol
My questions:
What are your thoughts on the ASM1166 SATA host controller? (m.2 to sata) I’ve seen mixed things on these forums, and I want some up to date thoughts. Are they reliable? Is it fast enough if I got more drives down the road? I understand that these allow C8 pkg so very low power consuption compared to my LSI cards I’ve tried. And they are cheap, as I am not wanting to spend a ton.
If using a ASM1166 is out of the question, has anyone been able to get low power consumption with anything else? As I mentioned I’ve tried a few different HBA’s and it doesn’t seem like any of them support PCI power management, not to mention them by themselves draw ~10-15w
Any tips on keeping the drives spun down in TrueNAS? I see a lot of people who use Unraid can use a cache to prevent plex from spinning up the drives when scanning.
What is best for power consumption/power management, Scale or Core? I like the UI of scale, but I do not use any of its features. Only thing it does is store and share files over the network.
Based off what I see online I think I could get my entire homelab into this one little PC and that would draw about 10-20w at idle. That’s down from 200w on my previous setup! But I want it to be reliable. I am willing to sacrifice some speed if necessary.
Off topic you may remember me from: HELP ZFS Pool data recovery (on the old forum)
I still owe you guys one for helping me out like I’ve never been helped before. I’ve learned a lot since that and I have had no issues since. Thank you!
I’d say that a machine that is tuned off uses the least.
…but in all seriousness, are you a prepper ?. What is the idea behind lowest power consumption ?. Dollar by dollar, how much is the new setup and how much are you spending in electricity ?
Because 7200 RPM drive uses the same 24 TB or 1 TB. That actually, that old 1 TB may use more.
If you’re gonna put it in a car then you’d go all solid state.
Power where I live is crazy expensive in the summer. So that’s the main reason. And the other reason is because it’s fun! I love going down the rabbit hole of low power servers, and I like the idea of it. The changes I’ve made using this Lenovo mini PC are already noticeable on my electric bill and I’ve been really enjoying working on it. It’s crazy how much you can pack into the m920q. Check out the mods people do on google some time.
If you don’t care about reliability ditch the HBA. The 1166 is known to drop drives on reboot, it has a mux, only supports 4 drives natively and only uses 2 pcie lanes. It is very dependent on your motherboard as to the C states it can achieve, best if the slot is connected to the CPU. The 10W used by an LSI is money well spent. I’ve been using HBA’s in unraid, proxmox and SCALE going back to supermicro PCI-X 8 port SATA 3gb and have never had any errors I could blame on a card.
oooh, OP must be feeling brave posting about non-HBA non-LSI+ solutions…
reality for me, I’ve been using an ASM1166 as part of a heavily used server (Dell T340) for the past 2 years, it’s a PCIE x1 variant as that’s the slot I had left so it runs at the expected ~900MB/s shared… fine for extra spinners. I’ve never had an uncorrectable caused by it, hotswap works fine, 4kn works fine. and as a bonus my card has a set of blinky lights. and no issues at least for me with the above “drops drives on reboots”
this same server is also using an ASR-78165 Adaptec HBA… it’s configured in “HBA Mode” with write cache disabled, ie obvious things. feeding both internal and external drives at the same time… hotswap works fine, 4kn works fine. SAS/SATA both work fine, expanders work fine. and of course never had an uncorrectable actually caused by it… and most importantly it fills a very useful niche of “an HBA that has internal and external ports” (this server is socket 1151, so it’s quite limited on pcie lanes)
TLDR, I’m sure you are fine… there are hardware combinations that don’t work well with Linux or BSD systems… don’t use via or jmicron for example… but regarding a lot of hardware that this forum would find objectionable it seems a little moot IMO.
Currently the thing preventing me from going to a higher C state is the HBA card. I take it out and I can get C8. with it in C2 if I’m lucky
I hear the silverstone frimware for these is very good with power management, haven’t heard about which one works well with TrueNAS tho
Its good to see there are some people that its been working for. Are you running Scale or Core? Have you had to do anything to it to make it work or maintain it?
I do have the flexability to use it in the PCI express slot instead of an M.2 if you think that would help things. It would also be directly connected to the CPU that way.
Another thing I wanted to say is the data on these drives is not life or death, and I have backups in the cloud(which I don’t want to have to use) but that’s why I’m looking into using the ASM1166 as an option.
this server is running Scale, for me the ASM1166 was completely plug and play. the form factor of it shouldn’t really matter. just make sure it has a heatsink.
I will say that having that wires hanging from an M.2 might not be the best idea as they have a tendency of flexing side-to-side when you do that, but that’s a matter of preference.
about the value of the data, you should still want to buy something that works and isn’t defective in some way. but honestly if I can host half a petabyte on “the wrong” controller cards your system will be fine
This, in itself, now appears to be a cause for concern.
The 1166 would be another one.
What about turning things the otherway around? TrueNAS bare metal, no HBA, no ASM1166, drives on motherboard SATA ports, and other VMs/apps running as containers/sandboxes/VMs under TrueNAS.
The ASM1166 isn’t as bad as claimed under SCALE. There was an issue with the driver quirk in the newer kernel where it would always report having 32 ports instead of the correct hardware count, but it’s not on the same level of “don’t ever use these.”
The potential issues regarding quality control of the physical card the chip is installed on remain though - as pointed out, the M.2 form factor ones are subject to a lot more torque from their connectors (especially the ones that use SFF-8087 SAS-style plugs) so a weak PCB will likely crack and result in bad/no contact internally - I believe a couple YouTube creators had this problem. The relative lack of airflow/cooling is a second factor, as a standalone PCIe card has a lot more surface area, so a larger heatsink can be used, there’s generally better airflow available in the PCIe section, and the cables are less likely to restrict what airflow is available.
We had a rather long discussion here about them, I’ll link to my post which was marked as the “Solve” and I feel summarizes things pretty nicely.
That’s good to hear, I did end up ordering one. It’s got a big heatsink as well as it’s pci express instead of m.2. I may need to toy with the firmware to find one that has good pci power management. But it’s high quality based on reviews and I think it’ll hold up okay. It will also have some active cooling on it.
If you guys are interested I can keep updating this thread on my journey.
Any tips on keeping the drives spun down in TrueNAS? I see a lot of people who use Unraid can use a cache to prevent plex from spinning up the drives when scanning.
I know some about TrueNAS and you can create a cache drive, would this acutally help with the drives not spinning up for random plex scans or maybe a linux iso download?
I do spin down my hard drives where large media files are stored and I have l2arc ssd for it.
I use Emby and when it does the scan, once a week, the disks do spin up.
But since it is only once a week, I’m fine with that.
As a server admin, I do understand the cons and pros of spinning down disks, but electricity is also quite expensive for me, my hard drives are only accessed a few times per week and I have my main pool on SSDs with frequently used stuff.
I’m on CORE 13 and my biggest problem with it is a recurring task called sync_db_keys in /usr/local/lib/python3.9/site-packages/middlewared/plugins/pool.py which does “zfs load-key” with “-n” every 24 hours from boot up for all native encryption pools, even when the keys are already loaded.
Since I don’t have issues with encrypted pools needing the keys reloaded frequently, I created this small patch file which needs to be applied after installation or system updates:
Quick update (not final update) so I recently got setup with the ASM1166 pci card. Got one from ali express and it’s great. High quailty, big heatsink, and its been working good for the past two weeks. Proxmox passthrough to the truenas scale VM. Plug and play it brought my power consumption down too. Now (without much tuning) My server which does much more than just TrueNAS (docker, home assistant, windows VM) draws 14 watts idle. (excluding drives) I have yet to measure the total power draw with the drives as they run on a seperate pico PSU. I can’t image it would be much when they’re spun down though. I was also able to upgrade it to 2.5gb nic utilizing the wifi card slot.
What’s left is to pickup a low power 2.5gb switch instead of a big cisco POE one that draws a bunch at idle. I will also get a small fanless POE switch for just cameras.
I will get NVR software setup on my other server that runs OPNsense. Instead of using a NVR from the vendor. Then I can have more features, and combining it with my OPNsense server means I don’t need to run another appliance which would cost more power.
After that it will just be some tuning. But so far very happy with the results. I should have a powerful Lab with two nodes, 12 cores each, 64gb of ram each, 2.5gb LAN, and a total idle power draw of <50 watts (including switching, router, NVR, etc).
Can you pinpoint how much power the ASM1166 card consumes?
My Background is I have a proxmox pve running now and one of my VMs is a freebsd stock server which is the way I’ve been building my NAS since around 200x. I normally have one or two SAS9200-16e running in there but now that I am consolidating a couple of machines running around the house I’m trying to get the power down. I was quite happy with the system idling at around 26W but adding just one LSI card ramps the power up to 45W and that feels entirely unacceptable.
Since I normally hook up 16 - 24HDDs depending on the backups that I’m doing I was looking at a nvme PCIe card with 4x M.2 slots and filling that card up but I can’t find anything on the TDP of the ASM1166 chip and the datasheets seem to be offline.
Since you said you like the rabbit hole, you could change the drives out for an all NVMe system. Low power, low heat. Not cheaper hardware wise but if power is that expensive, this might be a consideration.
Hi! I’m about to build similar setup including the asm1166, truenas, proxmox passthrough, but on topton-18 with N300. Power consumption is important for me as well and I totally understand you. I plan to run my nas/multiple purpose server from solar + battery buffer for the night. Batteries are pricey so power consumption matters very much.
Had there been any problems with the controller so far? Does it support direct smart reading? Hotplug works all ok?
