Yeah. That’s again a different thing.
Drives/Controllers must correctly implement sync write.
Controllers could have a memory buffer, but as long as they only respond when the write is completed.
Again, this is part of my point. People are going to use these things (and the PCIe equivalent). Which ones are broken, chipset wise, vs those that are merely crap.
Two things.
First, part of the problem with RAM cache on disk controllers is out of order writes by the disk controller, followed by a crash where the complete ZFS transaction is not finished.
Yes, rare. But, when it happens, the pool has to be rolled back to a prior transaction. Maybe not complete pool corruption. though recent data loss. Their is little telling if a cheap SATA controller does optimized out of order writes.
Next, LSI, (aka Avago, IBM, Broadcom, etc…), HBAs have a ton of time on them. Like billions of hours from millions of individual controller instances, (at a guess), on ZFS. Where the new cheapo SATA controller chip is generally used for, well cheap MS-Windows or Linux computers. (Real, Data Center servers would not use an M.2 SATA controller, especially not one with a SATA port multiplier.)
And, to answer the “why do you recommend using motherboard SATA ports but I can’t use this oddball SATA adapter I found?” question:
Like the LSI controllers, the Intel or AMD SATA controllers baked into chipsets have also been used for many millions of “storage-years”.
Chipset controllers don’t use port multipliers without telling you in the manual.
Another example of things going bad with the ASM1166 chipset. Could be lack of heatsink
Reddit discussion on above 6x Asm1166 adapter
One major note: this poor little asmedia chip runs HOT, and that’s only 3 drives running at 250 MB/s each. Other comments/reviews also mention the chip gets hot. I’m certain this is the cause of premature deaths, so get a heatsink+airflow for it. The chip is only about 1/2 an inch square, so any small pch heatsink will be more than enough.
Thanks for documenting that this one is bad even without a downstream port multiplier…
That’s the thing, with SCALE and ZoL, iXsystems is moving from a relatively small world of well “supported” hardware platforms and add-on hardware that FreeBSD has drivers for into the Linux ecosystem, where a much larger scale and scope of allegedly-compatible hardware exists.
Hence, I expect more edge case issues and like problems to affect SCALE users as bugs in Linux implementations are shook out. Sort of like the much smaller Apple hardware ecosystem vs. Windows / DOS. Think the ongoing Realtek 2.5GbE driver issues, etc.
These issues are less likely to affect enterprise customers because those folk generally buy known-good equipment from qualified suppliers, but I expect a lot of pain on the enthusiast end due to the much wider set of platforms / hardware that Linux allegedly supports.
So this will likely affect iXSystems less in general but the scale of support issues brought up in the forum may readily increase over time.
It will influence how the appliance is perceived. Casual users want it to run on whatever they want.
But that’s always been the case, partially fueled by YouTube videos telling people they can turn their castoff Pentium 4 into a NAS with FreeNAS. FreeBSD’s hardware compatibility has never been a significant limit on this–Linux might expand it a little bit, but this is nothing new.
Exactly, but with FreeBSD we had at least the clear borders of “if it’s supported it works, otherwise it doesn’t”; Linux has plenty of gray area in this regard, plus attracts way more casual users because… “linux is cool” I guess?
Hardly. Realtek NICs are “supported” under FreeBSD. So are 3Ware RAID controllers, as well as a bunch of other RAID controllers.
It’s true that Linux supports more stuff, but FreeBSD supports plenty of dodgy hardware. Including, I’d be willing to bet, the SATA controllers that are the subject of this topic. Edit: I don’t have to bet; it does: 5x / 6x / 8x SATA M.2 PCIe Cards - #25 by Stux
Anyway, not my problem… at least for the time being. Going to stick with CORE up until I am forced to change.
bit late to thread - “Interesting” - no… more like tempting and seducing. Simply just “plug and pray” it works without any thought.
Of course, they’re all nominally AHCI devices, they all use a universal driver.
Fundamentally, the situation is that dodgy SATA controllers have been around for decades. There is no market for a new design of a non-dodgy SATA controller - between NVMe and SAS HBAs, there is no niche for anything new, there’s only space for the same old dodgy crap. Cramming it into an M.2 card is technically novel, but it’s still crap.
You can get m.2 to PCIE adapters that have power plugs to inject power into it. That isn’t really anything new or novel. Everyone on here complaining that people are trying different inexspensive solutions is hilarious. Many playing with cheap not well tested hardware aren’t doing mission critical things. There is definitely a place for trying out these things and seeing how they work. Sure, port multipliers are bad for Truenas generally, but you know what, they don’t have to be inherintly. Perhaps one day better port multiplie support will exist in Truenas. Anyway, I say everyone should get off their high horses and stop bitching about what everyone should or shouldn’t be doing.
Sorry, I dont think advice on buying a used HBA for 35$ is that outlandish.
Why use ZFS and Truenas then at all ? There is a dozen other free NAS software out there where u can use your port multiplier until it melts. 
Yes, so they always say. Then they come crying when they lose their data, and find it was more “mission-critical” than they thought it was.
If you don’t care about your data, why use TrueNAS? There are other NAS OSs out there that are easier to use by a wide margin. But if you do care, do it right and use proper hardware. It isn’t hard to find, it doesn’t have to be expensive, but it does take a little bit of research.
@Farout There are reasons for not wanting a proper HBA in some builds. For example, building a low power, tiny footprint nas with a used mini pc where your only option is an M.2 slot. This means either an m.2 to PCIE adapter with an HBA or a m.2 to sata card. The latter takes up less space, is less complex and has fewer points of failure. Non-port multiplier m.2 to sata cards with heatsinks exist and have been used reliably and extensively on unraid and have become more common with truenas users. And many of us prefer Truenas over unraid for a myriad of reasons if if may in the community are ignorant to reality.
