Platform Recommendations

TrueNAS General
,
1

I’ve been running with the following platform for the last 7-years and it has been rock solid, however I’m now at the stage where the platform is starting to show it’s limitations, namely memory - it supports a max of 32gb.

Asrock Rack E3C236D2I
Xeon CPU E3-1245 v6
32gb DDR4 2400 ECC

I use the machine as a proxmox host and have a TrueNAS VM on it, the advantage of this board is that 6 of the 8 SATA ports are on a separate controller, so I just use PCIE passthrough to pass the controller through directly to TrueNAS.

Memory is the biggest problem for me right now as I have to be very very strict about memory allocation many of my VMs have <= 1gb of memory.

I’d like to begin doing some LLM stuff and it would be nice to have a Windows VM as well for gaming via parsec, so I’m thinking discrete GPU and then partition that with vGPUs that are passed through to VMs.

As far as TrueNAS goes I’m looking at a RAIDZ2 of 6x 12tb disks and then a seperate pool of 2x Mirrored VDEV of 2x 4tb SSDs.

Here are my requirements:

  • A separate controller to do passthrough and accommodate the above number of SATA ports.
  • 2-3 m.2 nvme slots
  • Support for 128gb+ ECC memory
  • 65-120w TDP CPU
  • m-ATX
  • 2-3 2.5gbe-10gbe Intel NICs (must be intel - not Broadcom or Realtek)

I’m looking for a cost effective option here - so I don’t necessarily need server or workstation grade hardware (as long as it supports ECC), as things like onboard IPMI are overrated in my opinion; I’ve had onboard IPMI with my current platform for 7-years and the console is horrendously outdated and requires Java web client which no browser supports anymore. Instead I plan to use a JetKVM with ATX power control.

Just to preempt any comments such as “don’t virtualise TrueNAS” or “a storage server should be a storage server” - I know and the primary motivation for doing it this way is space; I live in an apartment and therefore have always done “do everything” style servers; if I ever decide to buy a house, then the first thing I will do is a buildout a rack.

2

Have you tried using bigger memory modules? It often happens that they work despite the the specs limit (due to reasons).

Made a quick search:

  • EC266D4U might be an option.
  • D1541D4U-2T8R as well.
  • X570D4U-2L2T/BCM, IF you manage to find it a PRO CPU.

These however are pricy. Your requirements are quite hard to combine, you will have to compromise on something (ie price)

  • X11SCH-F is cheaper, but will require a network card for 10Gbit.
3

Thanks for the reccomendation

Yeah I did try bigger memory modules a couple of years ago - I tried 32gb dimms and whilst it did post, it became wildly unstable.

I was also looking at X570D4U-2L2T/BCM, however it’s really difficult to track down.

The Asus PRO WS W680M-ACE SE looks like it may be an option, slightly less difficult to find, but very expensive.

4

As far as I know, 4 slots of DDR5 is challenging to run and might not even post, but I have no experience with this motherboard.

5

Newer platforms and their HTML5 IPMI will have no such issue.
For RAM capacity look into X11SPM-TF (-TPF)—you may have to buy the board new, but DDR4 RDIMM are cheap and 1st/2nd gen. Xeon Scalable are commodities by now (Silver and Gold 5000 for lower TDP).

1 Like