There are ones that work reliably, it just requires some research and thorough testing. Plemty of people have home lab builds that they use for non-critical data. Home lab or not, ciritical data or not, it is still a NAS. I have done several non-supported, non-reccomended builds for personal use that are still working great many years later and proved to be far more resilient and reliable than your standard off the shelf NAS RAID solutions. What it all really boils down to is people like you saying Truenas is not for DIY home build use unless done the same way one would deploy an enterprise solution. The reality is that neither you, nor anyone else gets to decide that. You are of course welcome to your own opinion and to caution all day long as why people shouldnāt.
Also, I just want to say a big thankyou to @HoneyBadger for responding with something informative, thoughtful and helpful! While I have come acrosss most of this in my own research I never really got to that point of expaining any of this through the barage of fairly condecenting messages saying Truenas is not for that. Not to mention being suspended unjustly for having disagreeing opinions with a moderator. We need more people like you offering helpful advice and information instead of just condecendingly telling people not to do unsupported builds in their home lab environments. Sure, caution against it poltiely if you will, but the rudeness needs to stop. Thanks again for a breath of fresh air on here.
There is nothing wrong with someone building a TrueNAS system with 4 18TB disks on the oldest, slowest, 64-bit Intel processor you can find, only 8GB of non-ECC memory, an ancient RAID HBA with SATA 1 channels, and without any redundancy whatsoever i.e. a terrible spec. which is asking for trouble PROVIDING that they understand just how badly it will perform, just how at-risk their data is and just how little support they will get when it all goes wrong.
Of course if money was no object, and physical space was no object, and electricity usage was no object, we would all have Rolls-Royce rack-mounted bespoke-built enterprise-server-based solutions with RAIDZ3 and hot spares and dual power supplies and dual network infrastructure, and building-wide UPS. But home users, small businesses, hobbyists, generally have constraints on the money, space, time, effort etc. and are willing to make some compromises on security to get what they need within the cash and physical space etc. they have.
These compromises are mostly going to mean less redundancy than you might like, perhaps single points of temporary failure, and requiring double failures to lose data. Since many of us store e.g. media files which are non-critical and still have much better redundancy than a non-redundant single disk that the data was previously on, these can easily be levels of risk that we can accept because they are already significantly lower risk that we previously accepted.
There is nothing wrong with making compromises to get a useable system with acceptable levels of redundancy vs. risk within the budget and space you can afford - just so long as you understand and accept the risks created by the compromises you choose.
P.S. That said, I think it is probably quite easy for inexperienced users to cobble together a system that has the right number of ports but which is going to perform poorly or put their data at risk of corruption. In other words, they wonāt understand the risks they are creating.
Indeed, despite having been in IT since before DOS PCs existed, and despite having done masses of projects including huge RAID subsystems and complex networks - even I almost succumbed to the "throw together whatever hardware has the right number of ports) and it was only the fortunate fact that the motherboard I first planned to use was dead-dead-dead that pushed me to buy a RAID appliance instead and have a couple of limited compromises rather than something that was architecturally flawed.
It is far easier to give a simple rule-of thumb to use LSA HBAs flashed to IT mode for SATA ports and avoid a whole bunch of unknown potential gotchas resulting in data loss at a later date.
Thanks for the kind words @William_Steele - I agree with both you and @Protopia and I even had to challenge a few of my own ingrained hardware biases when SCALE released with significantly better support for hardware that was a no-go under CORE, including the ASMedia cards and Realtek NICs.
Just throwing my 2C in here as well. We should remember to all be professional and respectful at all times. Regardless of what particular hardware choices we all choose to make. At the end of the day we each have different risk tolerances and opinions about everything. I suspect most of you would be horrified at what I run for my own personal home-brew rigs as well
P.S: Iām happily using a Realtek 2.5Gb onboard NIC here and itās been working flawless on SCALE for quite some time now.
Well, if @kris thinks itās OK to use compromise solutions (and if anyone knows the risks it will be him) then that is pretty much an expert endorsement that itās OK (so long as you understand the compromises and risks you are taking).
Butā¦
I still think that these can be distilled down to a set of rules of thumb, but these rules of thumb need to be checked every so often.
I still read in various places a rule of thumb that you should either use MB SATA ports or LSA HBAs flashed to IT mode. But is this still the case?
I also still read in various places a rule of thumb that you should only use Intel Ethernet ports. But is this still the case?
Some of us are intentially not looking to do things the easy way. I think most using Truenas for DIY homebuilds understand the risks. Of course there will be those that donāt and I think as long as they are politely reminded or informed of the risks and not scolded or talked down to, which seems to happen alot on this forum, I think this will be a more welcoming open source community. What can I say? Some of us are just masochists
I want to make something 100% clear here: nobody was punished for disagreeing with me. I took action, as a moderator, in response to clear and unambiguous violations of the forum rules and basic etiquette in general.
Speaking for myself, I do not particularly care what random user #248 does. Theyāre presumably an adult and itās their problem. I do care when bad things are misrepresented as being good, in the face of ample evidence that they are bad, and with zero substantiation of the claims being made. Because some clueless person is going to read that and get the wrong idea, and they will suffer because of it.
Overly dramatic? Tell that to the folks who lost their children in a fire, only to come very close to losing all their pictures as well because they just didnāt know what was safe for their data. Iām sure plenty of others have had meaningful losses to go along with their lost data, and, at the end of the day, Iām not a psycho who gets a kick out of telling people āitās fineā when I know (or should know) that it isnāt.
I disagree. Despite my decades of relevant experience my first design was very naively probably going to use the wrong sort of SATA adapter to expend the number of SATA ports - but fortunately my first intention to āthrow stuff together using the tower system I was giftedā got abandoned when the MB turned out to be dead.
But the compromises I did accept in the end - using a USB SSD (not a flash drive) for the boot drive and using the boot drive for a second apps pool - were carefully researched and risks understood.
I think that the complexities of PCI channels and multiplexed SATA ports and all the other exceedingly low-level stuff that goes into knowing whether a SATA port solution is a good one is beyond most people - most people donāt want to have to understand all this complex detail, they just want a simple low-cost rule of thumb they can follow to buy solid hardware first time and get it up and running.
I disagree. Despite my decades of relevant experience my first design was very naively probably going to use the wrong sort of SATA adapter to expend the number of SATA ports - but fortunately my first intention to āthrow stuff together using the tower system I was giftedā got abandoned when the MB turned out to be dead.
Based on your response I am not sure what you disagree with specifically. That some of us are very experienced (myself included) and not looking to do things the easy or reccomended way when it comes to personal use? Iād imagine the majority of people willing to do a DIY open source NAS solution are fairly competent, with a fair amount of technical skill and problem solving abiltiy.
But the compromises I did accept in the end - using a USB SSD (not a flash drive) for the boot drive and using the boot drive for a second apps pool - were carefully researched and risks understood.
Regarding compromises, compromises exist in any technical solution no matter how much expertise, time and money you have. Sometimes, itās not compromising at all. Itās just experiementing, testing, benchmarking, amusement, etc. These people exist period and I am one of them, so I guess there canāt really be a disagreement here.
I think that the complexities of PCI channels and multiplexed SATA ports and all the other exceedingly low-level stuff that goes into knowing whether a SATA port solution is a good one is beyond most people - most people donāt want to have to understand all this complex detail, they just want a simple low-cost rule of thumb they can follow to buy solid hardware first time and get it up and running.
Again, I think there are plenty of DIYāers that understand what they are doing and capable of determining if a SATA solution is good. The information is out there. Port multiplication happens in the enterprise all the time with SAS expanders. Different technology, but same concept. Again, I am not advocating for or against them and I donāt know how everybody keeps focusing on this tiny piece of a broader conversation. If I had a nickel for every time someone spent more time telling me why I shouldnāt do something rather than just answer a question, I would be rich. Just because you donāt understand my reasons, doesnāt mean they are not valid. We can just agree to disagree on the number of DIYāers just looking to not have to understand all the complex detail.
Yowsers, has this thread gotten off topic! Thatās partially my fault I guessā¦
Yes - you are experienced and knowledgeable and happy to go down a rabbit hole.
I am also experienced and perfectly capable of understanding the details, but despite that I was about to do something stupid without having understood the technicalities and I was saved from myself by fate. But now I know what I would need to understand, I donāt personally want to take the risks of deciding for myself whether an e.g. M.2 to SATA adapter would be safe - I want a lower level of risk and I would rather use a known safe adapter.
But many many home / small business users donāt have the experience or technical skill to understand whether an e.g. 8x SATA M.2 PCIe Card will work reliably (which IS the topic) - they want a simple piece of advice like āUse an LSI HBA card and flash it to IT modeā or if there are other safe choices then a clear bulleted list.
SAS expanders are much better served from a bandwidth perspective (typically a bare minimum of 4x 6Gbps SAS lanes to the controller) and the SAS protocol is inherently switchable and queued (along the lines of the FIS-based SATA port multiplication) but without the nastiness of the āsingle offline device conks out the whole expander.ā
A SAS expander is a visible device on the SAS bus itself, and theyāre much more robust and well-understood on the SCSI/T10 standards level. Think of them akin to a network switch with multiple trunked uplinks to the next level.
They dont though, because end users who buy these sorts of things are that, end users. TrueNAS is an enterprise level NAS solution, so most of their users actually use more trusted reliable and solid brands for their builds, even for many home users.
There is an influx of DYI home users sure who think running TrueNAS on a desktop gaming board with 16GB of non ECC ram and cheap SSDās and a realtek nic is fineā¦ but by no means are they the focus.
I by no means am saying they are equivalent in terms of capability, standard implementation or reliability, just that they are kind of being generally villified as a technology around here and were designed to serve a similar purpose. There are some well though out implementations that would be acceptable for some non enterprise use cases if it was implemented in a more standardized way and perhaps it would have evolved more if it had found a larger audience. Iām just pushing back on the general concept that anything not enterpise sucks and canāt be relaible. I used SATA port multipliers a few decades ago in such a way that it wasnāt at all bottlenecking and single drive for a customized large scale storage solution to house data for my media asset managment startup and it worked flawlessly for our purposes until better more scalable cost effective solutions came along.
It is fine and this is an open source platform, so at the end of the day people are going to use it for non-enterprise use cases and as that base grows so will the community of people who use it for DIY home projects who can support eachother. This argument that enterprise software has no place on commodity hardware or in the home just doesnāt really work for me.
I did. I ended up going with a 10GTek. I plan to seriously load test it and measure temps. If it gets to hot I can always replace with a heatsink and fan. And for whatever reason it proves to be too problematic I do have a m.2 to PCIE adapter and a SAS HBA laying around somewhere. It will get even jankier with my wiring up a custom power supply I pulled from an old egpu enclosure and a home made SATA backplane.
SATA port multipliers and SAS expanders are the āsame conceptā like a Yugo and a Formula 1 racer are both cars.
SAS expanders do not block all other devices when one device is slow to respond. SAS expanders use a āsum of bandwidthā system, where devices can get more bandwidth than a single channel, and that multiple-channel connection extends back to the the HBA. This also allows redundant paths to the same device. And, it also means that although a device has to communicate over the wire that connects it to the expander, once the expander gets the data, it can pass it to another device over any channel that eventually leads to that device. This allows both time-based and channel-based multiplexing of data, neither of which is supported by SATA port multipliers.
SAS expanders support a superset of the SCSI command set, which allows the HBA to send commands to one device to transfer data to another device, and the data never flows through the HBAā¦it goes from device to expander to device.
Simply put, if you want to eventually lose access to data on your disks, connect them with an SATA port multiplier. If you donāt like restoring data from backup, then donāt use port multipliers.
I am not sure why we are still discussing port multiplers and your analogy of Yugo vs Formula racer is a little over the top. Depending on what mode of SATA port multipler is being implemented a single non responsive device isnāt going to hang everything up (Iām not referring to Truenas). Iāve said multiple times that I am not a proponent of Port Multipliers in this day in age, nor have I claimed that it is better or on par with SAS expansion. Itās clear that there is just a general disdain for SATA from you and others and seem to be here to just harp on anything not deemed enterprise. In reality, SATA port multipliers can be used with relative reliability if well implmented on the hardware and when it isnāt stretched beyond itās limits. I reliably used it 20 years ago for large pools of drives for my startup until better options existed. But sure, letās continue to split hairs all day on something that really is rather irrelivent today. This seems to be a real productive use of time.
While I donāt expect Port Multiplication to be something that ever gets implemented on Truenas, you can expect the group of users who reliably use Truenas on commodity non enterprise hardware to continue to grow. I have a hyper-v virtualized truenas system thatās been reliably running and migrated between several generations of desktop hardware for nearly 6 years without a hiccup and Iām about to do frankenstein build using a N6005 thin client with an M.2 to SATA adapter. Whether or not you or others think there is a place for that here isnāt going to change a thing.
