First Build - Maybe Ryzen

TrueNAS General
1

I’ve done a lot reading and searching lately and found a few posts about ryzen. I was hoping someone would point me in the right direction.

I currently have a ASUS ROG Strix B550-F motherboard that I took out of my son PC and was thinking of using it for a new build. I was wondering do I get a AMD 5650G just so I don’t have to put in a discrete GPU. I will be using the build for mostly storage but also want to host a plex/Jellyfin server. I usually am the only one using the server but I have times where I need it to transcode and I like the idea of having a transcoded on board. If I didn’t have the motherboard already, I would probably go Xeon. Should I just get like 3700 cpu with like an intel Arc GPU or stick with the AMD 5650G?

Also I currently have a synology with 4 - 4 TB drivers that are probably 6 years old. I was thinking of getting 4 or 5 - 8 TB drives and just transferring all the data over. I was thinking of going raidZ2 and then either brining the 4- 4TB drivers over and making a new vdev but feel like it would be overkill on space for me. Should I keep the synology as a offsit back up, or sell the synology and just stick with the 5 - 8 TB drives?

Last question, I currently have 16 gigs of non-ECC ram that came out of my son’s PC. I assume I would be ok for the time being to use that RAM and get some ECC ram in the future. The only reason I would use the RAM is because I already have it, and with buying a new CPU and drives I would be taped out for now.

2

I run two systems, both on AM4 with Ryzen processors and I’ve had a great experience thus far. One is from ~early 2023 and the other I just spun up about a month ago.

There is obviously a lot of variation in these decisions in regards to how many services will you be running (just plex/jellyfin forever, or maybe more?), how much will you be storing, how critical is your data, etc. I guess it’s probably best to start small than overcommit a ton of money into something you might not even stick with.

For the parts that are easier to comment on:

  1. ECC isn’t a must, although it is recommended, especially if data is critical. Personally I’d say it’s nearly a must to put the machine on a UPS (which is ideal anyway) if you aren’t going to use ECC memory. I tend to prefer CyberPower units but there are other manufacturers that are good also. If you’re not putting anything else on it you can get a smaller unit for less money.

You can interface the UPS directly with TrueNAS and have the machine automatically clean shutdown when there’s a power loss.

ECC memory and power-loss prevention don’t tackle the same issue or anything, but it’s better to have some of your bases covered rather than none.

  1. 16GB isn’t much for ZFS but if you only store those few TB and only run plex/jellyfin it would be enough to get you rolling at least. Swapping out RAM isn’t exactly hard so it’s not a big deal if you start out that way.

  2. Some Ryzen boards/CPUs actually do support ECC memory, and your board does, so you could potentially upgrade to that in the future. Both of my machines use DDR4 3200MHz ECC memory. ECC requirements are as follows:

  • Board calls out support for unbuffered ECC (I’ve never seen any other ECC type supported on consumer boards). The B550-F Gaming says it can in it’s manual and on its product page
  • All DIMMs support ECC (don’t mix)
  • Your Ryzen CPU is either a non-APU model or a PRO APU model

so either CPU you mentioned should actually support ECC in theory, meaning you could transition to it as part of a RAM upgrade in the future. It can be more expensive, but sometimes you get lucky. Personally I’d recommend Crucial or Kingston over no-name stuff.

  1. Used hardware is the name of the game for the home lab, even if it is consumer grade (as long as it’s in good shape), so save money where you can. RAM/CPU are particular points of focus for used hardware. Even HDDs can be a decent used option if you stick to recertified ones. On this same note, if you do start with on of the mentioned Ryzen CPUs and you start to outgrow it, you can go all the way up to a 5950x on that platform down the road which should be more than enough power for anything you’d likely ever do. My main machine houses a 3950x and most of the time its really overkill

  2. Related to the last point, I recommend enabling ECO mode on whichever CPU you pick. Save the power, the performance loss is minimal.

  3. I know for a bit AMD APUs didn’t work in at least Plex (no idea with Jellyfin), but I’ve seen people using it anecdotally. No idea on performance, but if you go discrete, an A310 is a really good budget option. Personally I’d probably lean 3700 for the extra cores and an A310/A380 over the 5650G if you can manage it.

  4. RAIDZ2 is a pretty good baseline. Some people swear by a pool of mirrors, but of course you lose more space that way. I mainly run RAIDZ2 (14x8) and I’m able to saturate a 1Gb link so I’m not bottlenecked by storage performance. I do make use of the Special Metadata VDEV though so the whole thing is really performant even for random I/O. My second machine doesn’t have the Special VDEV and so far at least it’s not much noticeably worse.

  5. I’d either say keep the Synology as a backup destination or sell it. Unless you find yourself becoming a data hoarder (whistles) then you’re probably right that keeping all of it would be overkill.

I’ll add more later if I can think of anything.

3 Likes
3

The latest generations of Xeon E and Xeon 6300P no longer have an iGPU…
If you’re the sole user you may well get away with transcoding on the CPU; otherwise, a small Arc dGPU together with a Ryzen CPU/APU increasingly looks like the most attractive option.

3 Likes
4

So I like the idea of the 3700 and the arc gpu.

I’m looking for ECC ram.

I know the crucial is slower but at this point does it really matter for the price of $100?

5

The Crucial RAM you show is Registered (RDIMM), so it won’t work with your CPU. You need UDIMM.

2 Likes
6

So funny enough (and this does get really annoying while searching for ECC memory), the larger contributor to why the latter is cheaper is because it’s Registered ECC (RDIMM) vs Unregistered ECC (UDIMM) (and it won’t work on your board).

I’m not 100% certain, but IIRC the situation is that RDIMM is much more common as its generally the ECC implementation of choice for servers, and since there are a lot of server platforms out there, it benefits more from economies of scale. Consumer platforms supporting ECC is more uncommon, and AFAIK they only ever support UDIMMs, I think because they’re less complex to support; however, since UDIMM use is less common, they tend to be more expensive.

So annoyingly, unlike the norm, you sometimes need to pay more for ECC on consumer hardware.

You are on the right track though going after two sticks vs one, so that you can get the benefits of Dual Channel.

OWC is one of those “no name” brands I was talking about that I considered. I ended up going with Crucial because they had a really good price on 128GB of 3200 back when I got it (where it cost basically the same as the others); however, as you can see people do use those lesser named brands with success so you can give it a shot if you want XD. I was considering it until I found the basically expensive (at the time) Crucial sticks. Their direct store was pretty good, but they recently got rid of it…

I went with Kingston on my second build because of that. I don’t know if you have a Microcenter near you, but I was able to get one of the sticks at half price because it was open box. Works flawlessly. It would require some luck but if you do happen to have one it could be worth a peak.

One other thing to note is that (again at least with UDIMM) unlike normal Desktop memory its rare to find them sold in kits. Generally you just get however many single sticks you need and they’ll work since they’re only at JDEC speeds anyway. Though the two examples you gave happen to be exceptions.

I was able to get my 128GB (32 x 4) sticks for about $450 and that seemed below average for that much so that was good at ~$3.50/GB. At the 32/64GB target you can probably do closer to $2.50/GB, but it won’t be as cheap as normal desktop memory sadly.

@etorix Beat me with the short version :slight_smile:

7

The cheapest name brand UDIMM ECC stick I know of that you can buy without any fuss is this:

The next speed down is basically the same price.

So if you want to save money and don’t want to futz with looking for used sticks/bargins, you could pay nearly half by trying OWC. I think they seemed like the most promissing lesser known brand.

You could buy them, use something like a Linux Live USB to run some memory stress tests overnight and confirm that ECC is functional (sometimes sticks will work but the ECC function won’t, due to some kind of compatibility problem) and if they pass all that they’re probably good. If not, just return them, it’s Amazon after all :man_shrugging:

Probing for ECC function can be a little tricky as the tools to do so have changed over time, and some of the older ones show false negatives (i.e. it looks like it’s not working when it is) for UDIMMs on consumer platforms, but TrueNAS itself can tell you:

image
image546×398 15.5 KB

There are also memory stress tests out there designed for verifying ECC correctness as well.

2 Likes
8

Note that the listing has plenty of options to choose from, including UDIMM, but that the product image is actually RDIMM: chips in “multiples of 9 plus one” in the middle (the “register”.
ECC UDIMM is plainly “multiples of 9”, plus some trace spaghetti in the middle.
Non-ECC UDIMM is “multiples of 8”.

9

In the context of ECC RAM for AMD Ryzen boards it is critical to understand that there are two levels of support for ECC:

  • Typical: The machine will boot and work with ECC RAM, but the additional protection of ECC RAM is inactive
  • Rare: The additional protection of ECC is being used (“full support”)

I don’t have an AMD system (other than a 15 year old Phenom), but IIRC you need a Ryzen Pro for full support.

On most systems the extra cost for ECC is wasted because it does not offer anything, relative to non-ECC RAM. So please double-check the details.

2 Likes
10

@ChrisRJ speaks no lies, and AMD while many of the CPUs “support” ECC, the motherboard chipset didn’t. That was many years ago but I still pay close attention to the motherboard and if it really can support ECC RAM. This is why a server motherboard is likely the better choice.

Always check and double check your hardware compatibility and capability before buying. Most of us have bought what we “thought” we were getting but ended up with something much less.

1 Like
11

When i was initially putting together the first system, I was aware of this debacle, though with Ryzen I’ve pretty much seen anecdotes anywhere from “it never functions at all” to “it absolutely functions as long as the CPU and motherboard you’re using state support for it”. I believe some of this is FUD due to complications with early Ryzen, or some just assuming because the CPU supports it that it will work with any motherboard, when of course that’s not the case; however, it is fair to question whether its actuality functional given its importantance and this uncertainty.

It appears that confirming ECC function before any errors is basically impossible, even on professional systems, like a Xeon platform. It largely comes down to checking for correctible errors bejng logged either by your system (varies depending upon OS) or BIOS, if your system is more advance and has something like IMPI (usually server hardware only). Otherwise you’re still essentially using blind faith that ECC is working, even on a server platform, albeit with a much stronger track record and reason to believe in that case.

Funny enough Windows has a pretty clear cut error that gets put into the event log when a correctable error is detected, but of course I’m not using Windows on these systems, and doing so briefly wouldn’t be adequate enough as a test.

In rare cases some software can inject ECC errors on purposes for testing purposes, but it seems that many BIOSes lock out this functionality.

For Ryzen specifically, the motherboards that claim support it generally have a seciton under AMD settings for enabling ECC, though it may appear regardless of board for alll I know (will check). Passmark (not that they’re gospel) includes some details about Ryzen under its page for ECC testing that imply they’ve tested at least some hardware from the platform that showed ECC was working properly.

I am curious about this matter myself, so as I can get around to it I’ll poke and prod both my systems to see if I can find more evidence of ECC working other than any tests where the RAM is just being checked for supoort.

12

I’m not certain about that. I don’t recall which one off the top of my head but I think MemTest86 supports ECC testing, but it is the paid version that injects faults to test it out. And I have no idea how reliable that is as well.

EDIT: Also from the same company is an ECC Tester program. Passmark Software.

13

I just dipped my toe into the water not more than 2 weeks ago, but then dove in harder than I ever thought possible. I picked up a steal of a deal on a used enterprise workstation for well under $100 and figured I’d give TrueNAS a try since I was curious about Immich. I ended up loving Immich and TrueNAS (and jellyfin, and wireguard, and home assistant, etc. etc. etc.) much more than I expected as well as it all being easier to setup than I expected. So I packed that machine full of SSDs and ram and nics to use a services machine and then went to town spec’ing out another machine to function as a NAS.

That first machine is a 6-core Intel 8th gen (i5 8500), now with 32gb ram (non-ecc). It idles around 10-11 watts while running about 10 services all from SSDs and has HD630 graphics on the processor, which Jellyfin and Immich seem to be able to utilize well for hardware transcoding. I’m very happy with it.

Coming this week, I decided to go with a used Ryzen Pro 4650GE which has 6 cores and 35w TDP. Seems like what it lacks in raw horsepower, it will make up for with flexibility and low power. Plus, it has ECC capability. I paired it with an Asrock X570 mobo due to ECC support, some extra PCIe lanes, and enough SATA ports on the mobo for me to not have to jack around with a HBA right away. Again, avoiding an HBA should help me keep power consumption down. All of this hardware was cheap-ish, available, and low power, but still has some power and flexibility. I should have everything within a few days and will see how that all pans out. The one minor concern of mine is that some of those used Ryzen Pro cpus are vendor locked. Mine shouldn’t be, but… it’s a possibility.

14

The paid version of Passmark MemTest can indeed inject ECC errors on certain platforms, and subject to a possibly well hidden BIOS setting. Carefully read the product note for supported sustems: Even some Xeon platforms are NOT.
When everything works, I suppose it is the closest one can get to certainty that ECC works as intended. (Short of building a home cyclotron to stress test the hardware…)

Short concluding remarks:
EPYC 4004/4005 is now a fully certified ECC platform build on desktop Ryzen.
Somehow, ECC always involves an element of faith from the user.

15

I haven’t read through all of the above responses, so appologize if this is redundant. Be careful of the CPU you choose if you want ECC RAM support. I am not sure about earlier generations of AM4 Ryzen CPUs but at least in the last couple of generations, CPUs with built in graphics do not support ECC RAM, unless they are the Pro version. recently built a server based on the Gigabyte B550i Aorus Pro AX board, Ryzen 5 Pro 5650GE and Nemix DDR4 3200MHZ 2Rx8 288-PIN ECC Unbuffered UDIMMs. The thing runs like a beast, and has full ECC support

image
image487×243 5.18 KB

16

Sorry if this wasn’t clear but you said pretty much what I meant.

You need to perform testing to try to force an error in order to confirm ECC function, and Memtest86 paid does in theory allow that. I was saying it’s impossible to confirm full ECC function ahead of time, i.e. before any errors have been detected or forced :slight_smile:

@amacal1 Sounds like a nice system! The lower power use, noise, and occasionally even cheaper price (depending on goals) is what makes using pro-sumer systems over proper server class ones, when it’s safe to get away with it, attractive in the first place after all!

@Louie1961 Yes, I had touched on this before. In theory to use ECC on Ryzen (AM4 at least) you need:

  • A non APU model or a PRO APU model
  • Motherboard the explicitly states it supports ECC DIMMs
  • UDIMMs instead of RDIMMs

But the remaining question is while on paper that should mean full ECC support, can we find any evidence of it working in practice? I’d say I’m a believer… but I also believe in “trust but verify”.

Like I said, I’ll try digging around system logs, using various utilities, and maybe even giving Memtest86 with injection support a go to see if I can’t provide any proof.

17

One of my last systems about 10 years ago (desktop workstation) had a lot of “gremlins”, but worked normally most of the time. It started with an occasional blue screen from time to time. Occasionally I’d get some weird Windows errors from corrupted system files but nothing a Windows SFC and/or installation repair couldn’t fix. But, it started happening more frequently so I resolved to figure out what was going on. First thing I checked was the ram, but a quick check didn’t find anything. I had a very old SSD in it, so I replaced the SSD and re-installed Windows but within a short period I had more corrupt system files. Finally, I did a more thorogh RAM check and somewhere many, many hours into the check it detected tons of errors on one of the sticks. Replaced the stick and it was seemingly fine after that.

That was my experience with a “bad stick” of RAM where the system was able to function seemingly normal for most of the time, but files were being corrupted and “data” (system files, for all I really know, but surely more than that) was being lost. Again, had it not been for corrupt system files that Windows was yelling at me about and the occasional blue screen, who knows how many “important” files would have been harmed. For my workstation, this is maybe acceptable. A damaged file would be a file I’m working with actively and I can fix in the moment or repair/replace in the moment. For a NAS? The thought of dealing with this and discovering years later that some percentage of my files or filesystem has been borked is nightmare fuel.

1 Like