Thoughts about upgrading my TrueNAS Server

TrueNAS General
,
1

I am currently running TrueNAS Core on an HP ProLiant Microserver Gen 8 as my personal home file server with

  • one SSD to boot from
  • 4 HDDs with 12 TB each as storage (53% used)
  • 16 GB RAM
  • CPU: Intel Celeron G1610T @ 2.30GHz (2 cores)

I’d like to upgrade the server hardware while keeping the storage disks.
This poses several questions, for which I am looking for guidance.
Links to other resources are as welcome as direct answers :slight_smile:

The server functions as our family file server at home. There are no VMs on this server (I am running Proxmox on another server for this), only two jails for Syncthing an Restic Restserver.

Hardware

Which hardware should I buy?
The hardware must be available in Germany, which means I cannot use the TrueNAS products.

I found some components and I am willing to invest money where it makes sense, but I also like to pay less if it’s possible without having to compromise too hard.

Case

I am thinking about a case for 8x 3.5" HDDs, so I can keep my current disks and am more flexible when it comes to extend the storage or replace existing disks.

What I found: SilverStone SST-CS381 v1.1 Mini-ITX Storage with a 500 W Power Supply (about 520 €).

Any other recommendations?

Main board

If I am using the case mentioned above, I need a Micro-ATX, Mini-DTX or Mini-ITX board.

I think I’d prefer Micro-ATX, because there I can get 3 PCIe slots to extend the system with additional SATA ports or faster LAN (currently all my switches are 1 GBit/s, but this might change in the future).

One M.2 slot should be enough for the system disk (since this system is not critical, I don’t need redundancy here).
Of course the board needs to support ECC RAM.
More SATA ports are better, but PCIe SATA cards aren’t very expensive, so this doesn’t seem to be very important.
IPMI would be nice to have, but an HDMI or DP port is more important to me (not VGA only).

I am unsure if I should invest in a server board (ASRock Rack?) or if some PC board (ASRock or Asus?) would be sufficient.

As far as I can see, I could get a PC board like ASRock B550M Pro4 for under 100 €, while server boards like ASRock Rack X470D4U cost well over 300 €, or even over 500 € to get some of the better features.

Are there good reasons to use a server board for home use?

CPU

The choice of CPU is tightly connected to the choice of main board.
At least 4 CPU cores.
I’m indifferent about AMD vs. Intel, but I think I might get more power for the same money from AMD.

Are there any special CPU features I should be sure to get for a TrueNAS system?

RAM

Type of RAM has to be decided after choosing the main board, but it will be at least 32 GB ECC RAM.
Would I feel any difference if I put in 64 GB?

Anything else?

Did I miss some vital hardware component in my list?

Software

I hope it won’t be too complicated to install TrueNAS Core on the new system and import a settings backup from my old one to get the same configuration up and running with the existing data disks.

Is there some guide for this you can recommend?

Are there any good reasons to switch to TrueNAS Scale?
The BSD based system is running fine for a decade now, so I don’t have any urge to switch to Linux. But if I should ever do this change, the new setup would be the perfect time to do it.

Sorry for the long text and thank you very much for any help you can provide! :slight_smile:

2

While it is nice to upgrade, you should ask yourself if the current server is fine right now.

You should make a more definitive list of requirements. For example: Must be u-ATX or smaller, must be able to contain eight 3.5" HDDs, etc.

What is the main goal you desire to achieve over the current system?

The one thing people want (we see it a lot) is low power, high processing capability, low cost. You generally get one or two of these, never all three.

AMD is the most bang for the buck and it supports ECC (in it’s own way). AMD also typically has a built in TPM in the CPU which makes it a good plus if you need it. I don’t know if Intel CPUs have that, never looked into it.

CORE will be around for a while now so there is no rush to switch over to SCALE, and if you were changing out your hardware, I would not recommend switching to SCALE until after you have a bit of time on the new hardware because if a problem occurs, is it the new hardware or SCALE? Avoid doing this if you can. Stay on CORE for now is my best advice. And you can read up on SCALE and how it is doing. 24.10.1 is having some issues for some people. 24.10.0.2 seems to be doing very good. And SCALE 25.04 should be out in Beta in February. It is expected to have a few nice improvements. If you use any jails right now, I would absolutely tell you to wait as “containers” is expected to be incorporated, with is like jails.

I built a system last year that may fit your needs, you can reference it as an example of what you may desire for a CPU, Motherboard, RAM. The case would be different based on what you desire.

A word of advice: When it comes to HDD’s and placement in a system, they need proper cooling. Pay close attention to that because some of those smaller cases that are appealing, have terrible airflow and drives can really heat up.

The motherboard I have for this build does have 3 PCIe slots, however one is a x1 slot so it is not as useful as I would like. It sits empty unfortunately. If it were another x4 slot, I would have been highly recommending this motherboard. So plan out what your slots “could” be used for in the future and actually locate cards that would work. You don’t need to buy them but at least you would know what would work when you did desire to purchase.

I’m a proponent of buying RAM, all that you feel you would ever need, when you purchase your system. With that in mind, 32GB is the minimum I would recommend. It may be a little overkill however RAM is a critical resource and when you run short, the system suffers. If you plan to run VM’s on TrueNAS, then buy 64GB.

Do not be stuck on M.2 as a boot drive. 2.5" SSDs are much cheaper and they do not need to consume a drive bay, you can literally use hook/loop tape (Velcro) to stick it anywhere. I prefer to drill mounting holes before I put anything in the case. I take my time and plan it out, otherwise (I’ve been there) I have to pull everything apart and then drill holes. Never leave your electronics installed, metal dust can really make a bad day. Wipe out the case with a moist cloth to pick up all the debris. Again, if you go down this path.

Does your system need to be quiet? We all like them to be quiet but does it need to be? My NVMe system currently sits next to my bed and does not make a bit of noise. Most of the time the fans running at slow speed provides all the cooling required. the fans only seem to go into high speed during power on selftest, so for a brief few seconds upon power on.

I know I didn’t answer many of your questions but I hope I at least gave you stuff to think about. And if you can support an ATX motherboard, the doors open up a lot for you. It isn’t that much larger, not really. Consider it.

One piece of advice is, if you ask questions like “Will this work hardware work with TrueNAS?”, you may get a yes, a no, or maybe. You really need to understand that you take the risk here. If you desire something that has been tested to work, look at peoples signatures, see what hardware they are running. If you have a question about that hardware, send them a message and try to ask specific questions. I like to ask “what would you do different” as most people do find minor issues after they build the system. They may not be show stoppers but like my x1 PCIe slot, to me is seems useless in my system. I can’t even put a 10Gb port in it. A x4 PCIe port is what I want. that is a shortcoming in my opinion of the motherboard I purchased.

I highly recommend that you run the burn-in tests on your new system before installing your HDDs. Let it run for days, do not be in a hurry if the data on the drives is really important.

Well crap, I’ve said enough.

Last thing, Welcome to the forums.

3

Thank you very much for your thoughts on this!

My main goal is getting more CPU power, since usage of the system has increased over the years and I can see that computing power seems to be the bottle neck, e.g. for syncthing processing large new files.

Upgrading the RAM size just seems a logical step when upgrading everything else.

I think my most important requirements can be narrowed down:

  • Size does not matter too much. Might use ATX as well. But the space is not suited for a 19" rack, so I’d prefer a from factor that stands independently.
  • Noise does not matter (it’s standing in the cellar).
  • I’d like to have 8 slots for 3.5" HDDs. If I think about it, hot-swap-access from the outside would be nice, but not really important.

I am having a hard time to find an adequate ATX case for this - probably because there are so many cases for desktop and gaming PCs. Any recommendations for places to look for such things would be appreciated! :slight_smile:

I have similar problems finding a main board. I did find the Asrock Rack ROMED8-2T/BCM, which seems to have everything I could wish for - but it’s over 700 €.

Are there any recommendable ATX boards that work well with TrueNAS Core?
Preferably for an almost-recent AMD CPU?

Does it make sense to use a gaming board that supports ECC RAM or is this the kind of decision people tend to regret?

4

“seems” is a word that puts me on high alert. My somewhat provocative response in such case is “thinking means not-knowing”. In this case, a higher CPU load can be caused by the CPU waiting for I/O from the disk or network. Meaning you don’t have a CPU bottleneck at all.

Without additional analysis you may end up with no performance gain at all.

5

With 8 disks I would recommend something with hot-swap. I don’t have it and it is the single biggest source of frustration. 8 disks without hot-swap usually means discrete cabling, which is a nightmare and increases the risk of failure.

For a NAS I always recommend to look closely at used enterprise gear. Any server board that is not older than 10 years will be more enough. Hell, I am running all my servers on Supermicro X9SRi-F, which are about 12 years old. And that is in a business context. Today I would go for X10 because they have DDR4 RAM.

Gaming boards are relatively expensive and the factors that increase their price are not needed for NAS. So not recommended from where I stand.

ECC is cheap if you buy used RDIMMs for, again, used enterprise gear. I paid about 20-25 Euros per 32 GB DDR3 RDIMM. Today I would generally recommend to go for DDR4, but that is not so much more expensive either.

6

You are absolutely right to be alerted by this word :wink:

Some more aspects of this:

I have 1 GBit/s LAN and usually I get over 100 MB/s when copying from the NAS to my PC.

While running syncthing on a folder with large video files, the CPU went to 100% and the copied data rate slowed down over an order of magnitude (I don’t remember the exact numbers).

I think it’s reasonable to assume that this was caused by syncthing calculating hashes before it could copy the data and that this would be improved by more CPU power.

7

I’m not sure about the enterprise gear, even though buying good stuff used sounds like a very good idea.

I assume most of this would be 19" stuff, which might be hard to fit into the space I am currently using for my servers.

And in most cases this gear will probably use a lot of energy, which I’d like to avoid as far as feasible.

8

First of all, you are going to get different opinions and that is perfectly fine. These are viewpoints from different perspectives.

I dislike hot-swap, unless you buy a high quality cage. My perspective, it is so much easier and safer to power down. You can have easily swappable drive carriers, and easily pull a drive out and pop a new one back in, but shutting down is just so much safer in my book.

There is a joke in there somewhere, but that is good. There is no need to a rack system, you can get a decent ATX case with a built in 8 drive swappable bays.

Glad you have a cellar, it’s probably nice an cool there as well. I liked having mine in a basement when I had one, now I don’t.

Gaming boards do not generally have IPMI so odds are you would regret that. But gaming boards can be fine, depending on the board. A RealTek NIC is a problem, you should ensure it has an Intel NIC or better.

@ChrisRJ made a lot of good points as well.

You should take a look at the GUI reports to see what the disk I/O was, RAM, SWAP, etc. Maybe you ran out of RAM.

This is how you can keep the price down, sometimes very considerably.

If you would prefer something newer, look around.

That is a serious difference in hardware and expensive. I highly doubt you need that unless you are going to use something like ESXi and fill that thing with 256GB RAM and very top end CPU. Keep looking.

9

Right now, you can find Gigabyte MJ11-EC1 (mini-ITX, embedded EPYC, 8 SATA, 1 GbE) and MC12-LE0 (micro-ATX, AM4 Ryzen, 6 SATA, x16+x4 slots) boards for 100€ or less.

For 8 SATA and 10 GbE, AsRock Rack E3C246D4U2-2T are readiliy available from eBay (micro-ATX, LGA1151-2 the last Intel socket with ECC-enabled Core i3).

I thoroughly hate Silverstone designs, their many tiny screws and countless design flaws.
There’s some discussion about NAS cases here:

Unfortunately, the Inter-Tech NAS-8 turned out to have some design flaws as well:

I’m not making any recommendation. You need to narrow it down somewhat on explicit requirements (CPU power, number of drives) and implicit requirements (size, noise, pool layout and how it’s going to eveolve…).

10

Just as a suggestion:

From a price standpoint this is an absolutely fantastic offer. This machine has got all the goodies you can have: 10GB Ethernet, IPMI, great build quality, lots of RAM,…
You need to change the raid controller for an HBA, like this:

1:1 swap, easy to to. The SAS backplane contains SAS expander, so 8 channes can drive 12 disks.
In order not to loose hdd slots to boot media you should add this
https://www.delock.de/produkt/89473/merkmale.html?g=1140
and one or two small M.2 SATA SSDs for booting. The server is too old to boot from NVME, so it must be SATA.

I’ve bought multiple of these machines as TrueNAS Scale servers at work, and they just run fine 24/7.

1 Like
11

Wow! Thanks a lot for all those replies!

You gave me a lot to think about and to research further…

This led me to a very small first step to increase the performance: I ordered a better CPU (Xeon E3-1265L for 31 €) which I will install instead of the current Celeron.

I hope this will be good enough for two or three more years before I have to revisit this topic again :slight_smile:

It’s a pity that HP decided to put only two DIMM slots into this server, and limit RAM to 16 GB!

1 Like