I had, but I have not made any comparison to assess how useful that was.
Most of the issues were told you by others already.
- Avoid SMR drives at any cost! that is the most important
- As you established in the end, you should add a smaller SSD raid for video editing (maybe a M.2 NVMe would even be better. <especially, if you want to move to 10GB ethernet in the beginning, or later)
- I think, if you dont want to use this box for anything else, then pure storage, the 2x32GB ECC RAM will be more than enough. If you want to enable deduplication on your pool, you should consider 1GB of RAM for each TB of disk data as a rule of thumb. (this means, that for a 16TB pool, 16GB of RAM will be used for this purpose and only the rest will be available for all other functions)
- One thing, that nobody mentioned and will influence your noise level is the fact, that TrueNAS WILL set yout fan speeds to 100% by design (if they did not abandon this concept recently) that means, that your system and CPU fans , if they are connected to the motherboard, will always run at 100% speed.
You cannot override this function, even by the built in BIOS control. If you are lucky, you MIGHT be able to do it by the IPMI interface of the motherboard, (if there is any). Otherwise, you have to buy a separate fan control circuit of override this behavior. Therefore, I recommend to buy a PSU that has the ZERO fan option, (the fan is off, when the temps are OK AND to buy 120mm or even 140mm system fans, that will run on much lower RPM-s. Also to buy a CPU cooler that has a 120mm fan on it (which will be difficult for this small case) - One other option is to not to install TrueNAS on bare metal, but over Proxmox instead, then you will at least get the BIOS control of fans available.
5 GB RAM per TB, validated by my personal experience.
Dedup is a memory hog, and I’m not confident that “fast dedup” (aka. “less slow dedup”) will do enough to change that. Just don’t use dedup.
Well, I only tried it once.
Did not really like myself either.
And on top of that, you cannot just simply turn it off, if you dont like it on the dataset later.
So ever since I separate my data to the category, where dedup is pointless (media files, and such) and to the data that might benefit from that (windows backups) This worked for me really well so far. (16TB for media (6x4TB in RAIDZ2, and 6TB for backups (2x6TB mirror) )
But returning to your point, 5GB is even worse. However, ECC RAM prices are luckily getting really dirt cheap. (for example on Aliexpress, you can buy 128GB DDR3 ECC for like 90 EUR. Now the bottleneck for DDR3 is getting literally that you can hardly buy any Mobo that supports more than that.
Don’t worry about custom building something. The single most important thing is how much data do you need to store on your NAS. Your required disk space will show you how much storage you actually need. That will give you some indication of how many drives you need for your NAS. Also regarding disk space, I strongly suggest over estimating by at least 20%. It’s much better to have too much space than run out!
After using a NAS with your various computers and devices, THEN look at a permanent solution. For the first one, just put something together out of any parts you have sitting around. Most people screwing around with a NAS have an older box sitting around… An issue most people forget about is the speed and bandwidth of your network. The faster the network, the less time you’ll spend waiting for that computer backup to finish. My main computer is a massive system that takes 3-5 days to fully back up, so it’s configured back up using several smaller backup activities…
I highly recommend the “Sagittarius” 8-bay case found on AliExpress. It uses mATX motherboards and can take a whole ATX PSU if your CPU cooler isn’t too large.
You could expand your motherboard options if you simply hook a SAS controller up to it, giving you loads of SATA connections: SAS HBA is LSI 9300" is a recommend option, 50€ on amazon.
note: they need to be flashed with a firmware that doesn’t put your drives in any kind of raid array, but just a bunch of disks instead
Thanks a lot for your help and suggestions. I really appreciate that.
Definitely: I will start with 4 HDDs in raidz2 and evaluate afterwards (in the process of using it) if an SSD pool is necessary for my purpose.
Thanks a lot for that insight. That is indeed quite a peculiar behavior. But I plan on buying a motherboard with IPMI, so that should cover it (hopefull).
That seems quite excessive. But I don’t plan on using deduplicatin anyway - more so in sight of your comments.
I would love to do that: tinkering around a bit and explore what I actually need. But unfortunately I have no old gear lying around at the moment. So it has to be a fresh build with new equipment for me and if I already need to take the time/energy to set it up, I thought I’ll do it properly
I’m quite intreged to see how this will pan out for me. But if I opt for the A2SDI-H-TF I have at least the possibility to upgrade my entire network to 10GbE in the future. My OPNsense and my other server have the appropriate ports already, but I would need to upgrade switches. Yet, they are quite expensive and power hungry.
This I will keep definitely in mind. Although With 32TB of storage (4x16TB in raidz2) I feel that have more than enough space for the next few years.
I came accross the Sagittarius during my research, but read in a reddit post, that it is quite burdensome to build. This was actually the post, that brought my attention to the Jonsbo N3 (see comment of OP). Can you share that view or did you have a different experience with the case?
That thought crossed my mind several times. But the options are already quite limitless and overwhelming so I was quite happy to narrow it down to a view boards. Yet, do you have a certain configuration in mind? Always happy to take another viable option into consideration.
You’ve missed the point. You are talking about RAM, not DISK space. RAM is very important, but I was specifically referring to DISK space. RAM is necessary, but not as critical. If you don’t have enough RAM, your backup will be done with whatever RAM TrueNAS can use. This RAM is used as a cache, to temporarily store what is being written to the actual NAS, which is DISK space. That disk space can be hard drives and/or SSDs, but in either case, it is not the same as the 32GB of RAM you were talking about.
To calculate how much disk space you need, you need to determine what you want to store. IS your NAS going to be used for storing backups only? Is it going to be used to house a database of movies? Is your NAS going to be used for loading/installing/reconfiguring test systems, to learn more about computers and networks? All these use DISK space. You need to add how much DISK space you will be needing, then add at least 20% to that. 5 years ago I created a 120TB NAS with an array of 15TB harddrives. That was more than double what the backups of all my computers needed. Since then, I’ve had one disk that wore out and had to be replaced, but the NAS has never been completely broken. I’m now up to about 75% usage, but still have room to grow, if necessary. A NAS is very useful, but there really ARE other options. IF you have less than 12TB of need, then maybe use portable hard drives and backup to them. If you need a NAS for other reasons, again you’ll need to size it appropriately. By size it, I mean DISK space, not RAM size. RAM is one of those things that there really isn’t an upper limit. The more the better. It makes the caching bigger and faster overall…
One final thing, make damned sure you have your NAS connected to a UPS. An Uninterruptable Power Supply is a CRUCIAL part of a successful NAS. IF you have a power outage at a critical time, you can lose a lot of data that’s in the RAM cache… A UPS will give your NAS a chance to finish its writing before it need to shut down, due to some kind of power interruption… Seriously, consider what I’ve written here…

I came accross the Sagittarius during my research, but read in a reddit post, that it is quite burdensome to build. This was actually the post, that brought my attention to the Jonsbo N3 (see comment of OP). Can you share that view or did you have a different experience with the case?
I’ve seen this post before. I didn’t think it was that burdensome at all, though admittedly it’s a little bare-bones. The N3 is a no-go for me because of the motherboard limitation: it’s impossible to find a decent motherboard in that size at an affordable price, whereas mATX motherboards are commonplace and inexpensive, including on the secondhand market. The N3, like all Jonsbo cases, has a really crappy system for mounting drives, since it uses those stupid rubber pull straps that can break, but also put a lot of force on the drive as you pull it because of the stretchiness of the rubber. The Sagittarius avoids that with simple metal pull-plates. It’s true you need to route your SATA cables and power cable between the two sides of the case before you add most other parts. The other big issue is that front-facing SATA ports on an mATX motherboard will interfere with any fan there. I avoided this with a motherboard that only has top-facing SATA ports. I didn’t have any trouble with SSD drives because I didn’t use any: I only have HDD and NVMe drives. The advice about avoiding PSUs with very long cables is probably good; I used an SFX with modular cables so I didn’t have any trouble there, but it seemed to me that an ATX PSU with modular cables shouldn’t be that bad either, as long as the CPU cooler isn’t too tall.

You’ve missed the point. You are talking about RAM, not DISK space.
Sorry for the confusion, that was simply a typo on my side. Of course I meant TB and not GB (I edited the original post). I am well aware that you are talking about disk space rather than RAM
So I’m planning to start my NAS with 32TB of space and I will be mostly using the NAS for backups as well as a database for movies, series and music.

One final thing, make damned sure you have your NAS connected to a UPS.
I definitely will! Good thing is, I already have an UPS, that powers some of my other critical equipment.

The other big issue is that front-facing SATA ports on an mATX motherboard will interfere with any fan there.
Just to make sure, with front-facing SATA ports, you mean SATA ports on the side of the motherboard (facing the same direction as normal USB ports), right?

I used an SFX with modular cables so I didn’t have any trouble there
Thanks for the advice. I am also planning to use a SFX PSU.

Just to make sure, with front-facing SATA ports, you mean SATA ports on the side of the motherboard (facing the same direction as normal USB ports), right?
That’s correct: on the opposite side of the MB from the rear-panel ports (display, network, etc ports). The Reddit link you had shows a photo of the problem. Lots of MBs have forward-facing SATA ports because they presumably want to avoid interference with long video cards.

Thanks for the advice. I am also planning to use a SFX PSU.
Be sure you get an SFX-to-ATX adapter plate, because there isn’t one included with the Sagittarius.
4 HDD is not the best layout for a RaidZ2.
You should use 4n+2 Drive for optimal performance (This means that 6 drives is the most efficient layout for a RAIDz2 (5 drives for RaidZ1)
With 4 drives, it is better to start with two striped mirrors or maybe RaidZ1.

is better to start with two striped mirrors or maybe RaidZ1
Except that that would preclude expanding the raidz2 by adding additional disks.

You should use 4n+2 Drive for optimal performance
There no longer is any kind of “optimal width” for performance. Power-of-two, multiple-of-four, or whatever.
The “Optimal price per TB” is still a valid point of view.
A 4 member RaidZ2 is using half of its capacity for redundancy. a 4+2 does only 1/3 (etc.).
Also, there is always a sweet spot of HDD prices and there is the availability of the sATA ports on the MoBo and the available drawers in your case.
You’re moving the goal posts!
It was performance, now it’s price-for-capacity.
I suggest Supermicro H12SSL-i with either 7402P or 7C13 Epyc and 256GB of ram.
Thanks a lot for your replies, support and insights, which greatly helped me to tailor the NAS to my presonal needs.
I now finally pulled the trigger on the individual parts:
Component | Final choice |
---|---|
Motherboard | Supermicro A2SDi-H-TF |
CPU | embedded: Intel Atom Processor C3758 with a Noctua NF-A6x25 PWM on top of the heat sink as suggested by etorix and explained here, here and here. |
HDD/Storage | 4x Toshiba Enterprise MG09 16TB: MG09ACA16TE in Raidz2 |
Boot Drive | cheap SSD: KingSpec PCIe 3.0 NXM 128GB |
RAM | 2*32GB ECC RDIMM |
PSU | Fractal Design Ion+ 2 - 560 Watt |
Case | Fractal Node 304 |
I will go with this (minimal) setup and add a 2-way mirror SSD pool for ongoing edits if needed/performance of th HDDs is not sufficient.
Running solnet-test-array on my own A2SDI-H-TF revealed that the individual SATA ports have higher throughput than the SFF-8443 (MiniSAS HD) ports. Use a breakout cable for the HDDs and put the SSDs on SATA ports.