TrueNAS Scale Remote Backup Server Build

TrueNAS General
, ,
1

Recently I build a new Remote Backup Server based on the AMD Ryzen 7000 Platform. The specs of the build are:

  1. CPU - AMD Ryzen 9 7900
  2. Motherboard - Supermicro H13SAE-MF
  3. Memory - Kingston ECC UDIMM 96GB 5600MT/s
  4. BootPool - 2x PNY CS900 120GB (Mirror)
  5. StoragePool-1 - 2x Seagate Exos x20 10TB (Mirror)
  6. StoragePool-2 - 6x Seagate Exos x20 20TB (Yet to decide the best layout, still testing various arrays)

For the PCIe Add-On Cards I have 3 Options, 2x Gen-5 x8 Cards and 1x Gen-4 x4 Card. For the HBA I went with LSI 9300-8i for connecting all the 8x Hard Disks.

For the x8 and x4 Slots I have 2 Cards lying around QNAP QXG-10G2T-X710 (QXG-10G2T-X710 | Two 10GbE Ports with SR-IOV and iSCSI, Block-based, Supports Multiple Virtual Disk Modes | QNAP) and QNAP QM2-4P-384 (QM2-4P-384 | Flexible and versatile, boosts performance and functionality | QNAP) both work fine with Windows 11 and Ubuntu Desktop 24.04 LTS (Tested). Wondering if it would work with TrueNAS Scale 24.04?

My Goal is to add 2x 10Gbe Ports and at least 2x Additional M.2 NVMe.
H13SAE-MF with the above parts mentioned leaves me

  1. 1x Gen-5 x8 Slot
  2. 1x Gen-4 x4 Slot
  3. 2x Gen-5 x4 M.2 NVMe Slots
2

That’s a lot of computing power for a “backup” server. I find it stange that you put any stress on Gen5 and Gen4 slots when all of your devices, especially the QM2-4P-384 card, are Gen3…
As for the pool layout, 6-wide raidz2 or raidz3 are the only safe and realistic options; three 2-way mirrors is not safe and two 3-way mirrors is a waste of space on a backup. The purpose of the separate mirror pool is unclear.

What is your question?

1 Like
3

I want to add at least 2x more NVMe M.2 Drives and 2x 10Gbe Ports for quick restore in an emergency, with the cards available QNAP QXG-10G2T-X710 & QNAP QM2-4P-384 can I achieve this or I need anything else?

4

Assuming you won’t be re-designating the backup as a primary in an emergency, and thus maybe running bock storage, the only thing that really makes sense is to use raidz. Pick 1,2 or 3 based on your level of paranoia.

6-way RaidZ2 seems like a good compromise

The pair of 10tBs almost seem a waste, but I guess you could even backup extra critical data twice.

5

10TB Mirror is primarily for docker backups, next cloud backup and a few databases.
While 20TB array will be for Media Backup and VM Backup Images.

6

Gday, have you actually built the server to this spec? Asking, as I bought a similar setup, Supermicro H13SAE-MF, Ryzen 9700, and same set of 2x Kingston 48 Gb ECC 5600 MT/s. The problem I ran into, this motherboard does not want to run two of these modules (probably because of 5600 MT, which is not supported by the motherboard). One module runs fine, but two do not play together. I am working with Supermicro and Kingston support in that, but so far, the only result was removal of this motherboard from compatibility list for 48 GB ECC module on Kingston’s website.

7

Did you try setting the ram speed in the motherboard bios manually rather than auto detect when both sticks are in ?

8

Supermicro BIOS does not have that setting.

I kept troubleshooting it, and eventually figured it’s an issue with their BIOS that happens only with Ryzen 9000. Then, I found that 7000-series had default RAM speed set at 5200MT/s (which is also listed as max allowed speed for H13SAE-MF). And Ryzen 9000 bumps this default to 5600MT/s.

Which leads to really weird behaviour, even NEMIX RAM rated at 5200MT/s, if I run two DIMMs, runs at 5600MT/s (and, strangely, works, while Kingston rated 5600MT/s doesn’t).

But when BIOS detects 4 DIMMs (2DPC), it overrides the default with 3600MT/s. Hence when I install 192 Gb, it works fine at 3600 MT/s.

It also works if I have Ryzen 7600X with 2x48 Gb. The only combination that fails is Ryzen 9000 and 2x48Gb DIMMs.

I submitted that data to Supermicro, and hope they eventually fix the BIOS issue. Meanwhile, I guess I’ll keep 192 Gb, even though I don’t really need it. Well, larger ZFS cache doesn’t really hurt, except for extra ~4W energy consumption.

9

Well that’s annoying, but hey at least in theory they can fix it with a bios update, I see they have the bios 2.2 on their website is that what’s on your board already ?

I’m actually interested in this board to replace my aging X10SL7-F.

Are you a fellow Aussie by chance ? If so who did you purchase your board through ?

10

Yes. I tried both versions, made no difference. On the good note, running cheaper NEMIX modules (rated 5200MT) works just fine (even with 2 modules).

I replaced my aging Xeon-E home server. In the process of doing so I realised it is a dead end, and I should’ve done differently, - instead of running one big server, I would’ve been better with 2x smaller boxes. Costs the same, uses less power, and enables HA.

Indeed I am :australia:! Although at the moment I’m in the US. Bought it from Newegg Business. Turned out they don’t require you to be a business to buy from them. They deliver to Australia, although exchange rate nowadays is… embarrassing.