Advice on Hardware for media vault and streaming

TrueNAS General
41

Well PCIe 3.0 x8 is 8GB/s so i think it will be fine :slightly_smiling_face:

Edit: what I dont know is if that pci x8 slot can supply enough power. The tesla can draw up to 75 watts.

42

some research says PCI-e can support up to 75w? So this may not be an option. Could go with a P40 which has a dedicated power plug from the power supply I believe. Would just be wasting the P40’s potential on PCI-E x8 but according to you peeps that isn’t needed anyways and the price for the P40’s isn’t much different than the P4’s. Thoughts?

43

So you want a relatively new motherboard (do not go with the newest and latest unless you want to pay more, and possibly have the privilege of dealing with teething issues for the price) with 8 SATA ports (skip the HBA!), a PCIe slot for your dGPU (open x8 would be enough) and enough PCIe lanes for NVMe drives for your VMs. Plenty to choose from.
Put some figures on the number of cores and the amount of RAM you’d want for said VMs and we could discuss options.

1 Like
44

Don’t confuse base clock and turbo clock.

If you are being bottlenecked by a single thread then a modern intel CPU will turbo up that thread to the turbo speed or close to it.

The base clock speed is the speed you are guaranteed even under maximum load (provided cooling is sufficient)

For that to be the speed you achieve you’d have to be running all the cores flat out anyway.

2 Likes
45

Screenshot 2024-04-23 at 10.38.44 PM
Screenshot 2024-04-23 at 10.38.44 PM1406×838 132 KB

i was talking about base clocks in this case

46

Yes, and the turbo clocks are 2.3 vs 2.6hhz.

On an otherwise idle system you’ll probably be hitting be turbo clock on a single hot thread.

I think quadruple the cores is worth it :wink:

But j like running apps, services and VMs :slight_smile:

1 Like
47

etorix, thank you. So if I’m using 7 10TB drives in a RAIDZ2 config with 1 drive as a spare how much ram would I need? 64GB? Would 64GB be enough for Truenas and 2 VM’s? Or should I just get 128GB and be done with it? I would prefer a newer board honestly. But I completely agree with you, I don’t want teething issues, I want tried and true but not old. As for cores, I’m not sure what I need for SMB, PLEX, and a VM. But I do see lots of Xeon chips for reasonable prices with up to 16 cores.

I realize the ram useage on VM’s is dependent on what i’m doing with them so I want future ability to do what I want.

48

The (very) rough rule of thumb is 1 GB per TB of storage. But it relaxes as pool and RAM grow.

So your 50 TB (or 70 TB, it’s that rough…) pool would likely do fine on 32 GB, and comfortable on 64 GB.
How much more do you want for VMs? What class of NIC?

If it all fits within 128 GB, then UDIMM platforms are in the game.
AsRock Rack X570D4U + Ryzen
C256 (Supermicro X12STH) + Xeon E-2300
C246 + Xeon E-2100/2200 (Supermicro X11SCH are getting hard to find or overpriced, but there are many AsRock Rack E3C246D4U2-2L2T on eBay, these are likely “new old stock” of a board which was released too late in the C246 cycle and did not sold as expected… it depends on what you accept as “new”)

For more power (in all accpetions: computing power and wall power), RDIMM platforms. But these will get expensive new. Maybe a new X11SPM-F/TF/TPF board with a refurbished 1st/2nd gen. Xeon Scalable and RDIMMs?

2 Likes
49

This is great info, much appreciated. I’ll look in more detail but the Rack E3C246D4U2-2L2T with an Intel E-2278GE looks like a killer combo. Wouldn’t have to sacrifice GPU speed in future with PCI x16 (don’t use slot 4 lol) and has 7 SATA ports. Lots of functionality. The E-2278 seems fast and has 8 cores so this could be good for a while and allow to play with VM’s and not limit me. Not cheap but between the board, CPU, and 128BG ECC ram it would be around $1200. The fact the E-2278 has onboard graphics would allow for some flexibility I would imagine?

1 Like
50

Do not bother with the GE version: It will idle at the same level as a non-E.
The iGPU is not useful for TrueNAS (there’s IPMI to go into BIOS settings, or the VGA port from the BMC), but you may use it for transcoding or pass it to a VM.

51

I couldn’t find a G version (vs GE) at a reasonable price unless it was a G ES version with a lower clock speed (2.6 GHz) but WAY cheaper. These are confusing LOL. Understood on the iGPU, my thoughts were on the VM front.

1 Like
52

I did not expected GE to be more readily available than G… In any case, stay clear of Engineering Sample versions; beside possible quirks and lower specs, they may need some specific boards on a specific BIOS version.

1 Like
53

Like you said, that ASRock board, at least the 2L2T version and CPU are few and far between. Good to know on the ES version CPU’s, didn’t know that.

There many of the 2T versions but I’d have to use my HBA. the 2T has 2 missing SATA on boards

54

The other option you mentioned would be E-2378G with a SUPERMICRO MBD-X12STH. So this option has more parts available and is a Cheaper package!!! This seems like a good option.

1 Like
55

Good to know that 12th generation could become a good option. I mostly buy refurbished/second-hand for my NAS, so I may not be up to date on prices for new parts.

56

do you find refurbished/second-hand reliable and in general a good experience?

57

So far, yes.
Typical sources of second-hand server hardware are professional refurbishers (on eBay, or directly… once you’ve found the local ones) and like-minded amateurs (“For Sale” section of the ServeTheHome forum). That helps.

58

So Far the configuration on paper looks like this:

  1. Case: Silverstone Technology CS380B (temps questionable)
  2. HD: 8 X 10TB Seagate Enterprise Capacity 3.5 HDD (RAIDZ2 + Spare)
  3. GPU: Tesla P40 with 3D printed cooling solution
  4. Motherboard: Supermicro MBD-X12STH-F-B
  5. CPU: Xeon E2378
  6. ECC RAM: A-Tech 128GB 2666MHz ECC kit
  7. Power Supply: EVGA SuperNova 650 P5, 80 plus Platinum 650W
  8. Boot Drive: Kingston 128GB SSD PCIe 3.0 M.2 NVME
  9. CPU cooler: Noctua NH-D9L
    10: ?

Any Glaring issues or comments? I don’t know if A-Tech RAM is good or not.

For future VM’s, I could use one of the PCI-e slots for an expansion card for SSD’s?

59

Sorry to go off-topic a little (this is a TrueNAS forum, not Plex/GPU, but it feels relevant to the thread)

I’ve been researching adding a basic GPU to my own Plex server recently, and the accepted wisdom seems to be that, for transcoding, your GPU doesn’t need to be in a high speed PCIe slot. Even if the GPU is spec’d for x16 lanes it will typically be quite happy in a x4 slot, or even x1. I guess the large lane count is more useful for loading large textures into vRAM quickly for gaming rather than streaming video data through for [en|trans]coding

I can’t speak from experience as I’ve not tried it yet, but thought this may be helpful if you’re short of wide slots.

Of course, OP’s reqs are for multiple simultaneous 4k transcodes (are you crazy? don’t transcode 4k😂), so bandwidth may be more important in their case.

60
  1. Seeing what appears to be the same drive cage as the DS-380 I do not question temps: This is going to be a cooker and/or a screamer… Quite possibly both.
  2. Do not use a hot spare with a single raidz2 vdev: Go straight for raidz3 if you’re paranoid.
  3. / 5. What about a Xeon E-2378G and iGPU transcoding? Keep it simple.
  4. No idea either. I only buy Micron/Samsung/SK Hynix-branded modules (possibly Kingston or rebranded Dell/HP)—second-hand.
  5. D14 looks oversized where a U12 would largely do.

Xeon E can only bifurcate x8x4x4, so you could have up to three drives in the x16 slot.