What’s the difference between the two TrueNAS versions?
I’m sorry, but I don’t understand what these acronyms mean. HBA in IT mode?
I already have an idea of which CPU and motherboard to get except for the RAM. To be honest, I was really leaning toward getting the UGREEN NASync DXP4800+ and then installing Unraid/TrueNAS on it. It seems very affordable and has good specs like the G8505 CPU and 10GbE. The lack of ECC is why I’m considering this other more expensive route. I know the odds of bitrot is low and I guess I could still go back to that route with a properly configured OS and mitigation factors.
Can you elaborate on what you mean by a 4-wide raidz2 and a lone drive? How many drives is that? 5? So from a hardware standpoint, 5 drives configured in the manner that you have described will best mitigate any concerns about bitrot? I know the probability won’t be zero.
Thank you! A lone drive like any external USB HDD that has a copy of the data stored in another physical location? If so, is it a good idea to use 7-Zip or WinRAR to store the data with parity files? If so, what’s the best method for reliability in this case?
Three drives mirroring each other. Capacity: 1; redundancy:2.
Mirrors aren’t limited to two…
Host Bus Adapter (basically a SAS controller) is only applicable if you have SAS drives or need more drives than your motherboard can support.
Yes. My point was to take five drives as an example and compare:
all five drives in a very secure raidz3 (but if sh!t happens, you may still lose all); and
four drives in raidz2, and using the fifth as external backup (if sh!t happens to the main system, you still have the backup; of course a single drive may not be enough to hold everything but let’s keep it simple…)
So, having established that not all the money is flowing to the NAS and that there will be a backup, the minimal number of drives to put in the NAS is 2 using a (2-way) mirror, 3 for raidz1, 4 for safer raidz2. The optimal number of drives also depends on the capacity you need (6-8 would be optimal for raidz2). And the layout and degree of redundancy depends how much you want to avoid restoring from backup other than in terminal disaster… If you’re happy with getting back to backup, 2-way mirror or raidz1 may be acceptable; if not, two degrees of redundancy, meaning 3-way mirrors (expensive) or raidz2 is the minimum to confidently survive ONE incident (you resilver with still one degree of redundancy left!).
Look up the Qualified Vendor List (QVL)…
That would be Xeon E-2400, or Xeon 6300P as it has been renamed. But Current Core i5 and higher also support ECC when used with the corresponding W680 or W880 chipset.
This will likely come out more expensive than Ryzen/EPYC 4000. For “cheapest”, look into older lines. You do not need to buy new, or to buy the latest, to make a safe NAS.
When I set up my NAS a few years ago, I went with TrueNAS Core.
TrueNAS Core is based on FreeBSD and has long been valued for its stability.
However, active development has shifted to TrueNAS SCALE, the Linux-based edition.
Core still receives maintenance updates, but new features arrive primarily in SCALE.
Since I don’t do anything fancy with this NAS, I currently have no reason to change,
so I’m sticking with this build for now.
By HBA, I mean a Host Bus Adapter that is installed in a PCIe slot.
I’m using an older but very reliable model: the LSI 9211-8i.
IT mode (“Initiator Target mode” means the card’s firmware has been flashed so that it no longer provides its own RAID functionality, but instead passes the drives directly and unchanged to the operating system – exactly what TrueNAS needs.
With an LSI 9211-8i, you can directly run 8 SAS lanes, i.e. 8 ports for drives.
By using daisy-chaining and so-called SAS expanders, you can theoretically connect up to 256 drives.
This technology comes from the server world and is useful if the mainboard doesn’t have enough SATA ports or – as in my case – if you are using SAS hard drives.
EDIT: sorry, just saw, that etorix answered most of your questions
Please bear with me as I’m still learning. I noticed that text is formatted like this and I don’t understand why and what it really means other than it means three drives.
Blockquote
Three drives mirroring each other. Capacity: 1; redundancy:2.
Mirrors aren’t limited to two…
When you say current i5, do you mean the 14th-gen CPUs? Or are all 12th, 13th, and 14th Gen i5 including the cheapest i5 support ECC RAM with the W680 chipset?
I’ll have to do a bit of research on this. I’ll go with whichever is cheaper not just in costs, but power usage as well. I prefer to buy new parts rather than used. I hope we can agree to disagree on this point. It doesn’t have to be the current generation, but it’ll be easier to find. I only intend to use this as a NAS so absolute performance isn’t necessary and I will have to get the ECC RAM, which is expensive!
No. Best is to use that drive with ZFS. Because ZFS has all of that built in. A ZFS to ZFS replication is guaranteed to be correct - in case it’s not, you will get a notification about that.
Drives A, B, C are in a 3-way mirror vdev. B is a full copy of A. C is a full copy of B, A is a full copy of C…
The three-drive vdev has the capacity of one drive, but two degrees of redundancy: It can lose two drives and still retain data.
I think you have some homework to do reading the ZFS primer…
Core i5 and above from 12-14th gen with W680.
i5 and above from Core Ultra(2) with W880.
No problem: It’s your money!
Low power being your avowed priority, you can also buy an Atom C3000 board (Supermicro A2SDi): These are very low power, not last generation but still sold new with the expected ten year support that is typical for embedded boards.
The question is how you’d want to use a backup. How you keep your backup current. And how you use it to restore in case of a disaster.
The easiest way is ZFS replication, to a second TrueNAS system, and/or to an external (e.g. USB) drive (which is then a single drive pool… if your data fits on one big drive). Easy incremental backup. To restore, you replicate in the other direction. But the backup is not directly readable from a Windows or macOS client (unless you install OpenZFS on Windows/OpenZFS on OS X), you have to set up a new NAS. No zip, no RAR: ZFS has built-in compression.
If you want to access your files directly from the backup, then you would format the external drive as NTFS or HFS, plug it into a client PC (not the NAS) and use rsync or some other backup program to periodically copy the content of the NAS to it—and zip/rar might be useful. Easy access to restore; more complicated backup.
I think you are focusing on this the wrong way. If you are worried about family vids and photos, which are cold storage items, you should be focused on the backups first. TrueNAS is phenomenal at replicating your data to other servers. Your first server you will keep mostly on, so build a low power 2 drive mirror with twice the space you need. The more drives you have, the more power you use. Then focus on an off-site second server, maybe out of spare parts, that is your backup of your data. If your first server goes kaput (including fire, etc), you can rebuild it and then restore the data from the backup server. If a drive on the first server fails, Amazon will have a new drive to you in a few hours. Then also a third backup, which could be a USB drive, online storage, or a third server. Power on the second and third servers regularly (I do weekly), and replicate from the first server. For me, that takes around five minutes and negligible power. Only after that, in my highly inexpert view, should a homelab user be scoping out a lot of drives.
“ZFS loves RAM”, so more is better (until it doesn’t make a difference) but you do not need 200 GB RAM to manage a 200 TB pool. How much also depends on network speed and whether there are further services running on the NAS (which is not the case here).
I’d guess (but really just guess) that 64 GB is comfortably safe. Possibly even less, but if you spend for 200 TB of storage you should not be trying to map the minimal amount of RAM which works well enough and save on RAM what amounts to pocket change.
In-band ECC is not an Intel feature: It is required by DDR5 standard.
And it is NOT “ECC” as discussed in relation with ZFS, as it only attempts to guard against errors during transfer but cannot correct bit flips—for that you need the extra set of memory chips which comes with genuine ECC RAM modules (and extra cost).
IMHO, the introduction of on-die ECC (or in-DRAM ECC) in the JEDEC DDR5 standard was more of a necessity to handle the increased error rates caused by higher speeds and bandwidth.
DDR5 is inherently a bit more robust than DDR4 without ECC, but it’s still not the same as “real” system-level ECC.
That’s already causing — and will likely continue to cause — a lot of confusion among consumers.
I hope that the whole DDR5 “on-die ECC” narrative doesn’t push motherboard and CPU vendors to abandon real ECC support in the consumer segment — otherwise, affordable and reliable DIY NAS builds could become a thing of the past.
Consumers will really have to pay close attention to what they’re buying, because marketing departments can be very creative with their wording…
Anyone who truly wants real ECC should, when buying new hardware,
always look for wording such as “ECC & Non-ECC, Unbuffered (UDIMM) — Supported (requires CPU and motherboard support)” in the specifications.
I think it’s the other way.
On-die ECC silently corrects errors inside of memory chips.
On-die ECC happens at a lower level than true ECC memory and does not report any details about whether errors are detected, unlike externally-controlled ECC.
In-band ECC is like a RAID-5 for DDR.
A memory controller calculates parity and stores it in ram at the cost of 1/32th of capacity and some performance. It can detect, fix and report memory corruption.
There was a nice article about this on anandtech, now lost
“True” ECC RAM uses extra memory chips for this purpose, so there is no impact on capacity and performance.
