NAS Hardware for a new build, Advice?

TrueNAS General
,
1

I’d appreciate advice! Especially if there is any very big mistake?

Build Specifications
ATX Mid-Tower

ATX Power Supply (500+ Watts/Modular-as budget and power requirements dictate, thanks to Fleshmauler :))

Thermaltake Toughpower GF A3 Snow Edition 1050W 80+ Gold Full Modular SLI/Crossfire Ready ATX 3.0 Power Supply; PCIe Gen.5 600W 12VHPWR Connector Included; PS-TPD-1050FNFAGU-N; 10 Year Warranty
$156.99 - Yikes!

Main Board Asrock Rack E3C246D4U2-2T
CPU Intel Xeon E-2288G Processor

Memory Crucial 64GB (4X 16GB) DDR4 2666MHz PC4-21300 ECC UDIMM Memory Ram CT16G4WFD8266

SAS Host Bus Adapter SAS9305-16I LSI SAS 9305-16I 16 Port PCIe 3.0 x8 12 Gb/s
Hard Drives

8 (Eight) Seagate ST8000NM0075 Exos 7e8 8tb 7200rpm SAS 12Gb/s Dual Port 256mb Buffer 512e 3.5inch HDDs/Refurbished

I like the board because it supports the Xeon server processors that are supported by Windows 11, and it supports ECC memory (DDR4), and the board has two, 10Gbs Nics. I really wanted to use an ASUS MB because I’ve built a half dozen machines with ASUS and those things work well forever. But they’re 5 times the cost for similar specs as compared to the Asrock.

I’m thinking of using the LSI SAS host on one or even two of the PCIe x 8 slots, because those cards and the Exos SAS drives support 12Gbs data rates. Is that useful? I installed Cat5e in my SOHO (Small Office / Home Office) around 20 years ago. Happily, my use shows that Cat5e can transmit at 5+Gb/s readily, I think it can do more actually. The 1Gb/s “limit” for Cat5e is a rating or certification, not a limit. Cat5e is capable of 10Gb/s for runs up to 45 meters. I haven’t tested 10Gb/s in my SOHO, but it won’t be long.

There’s one, M.2 slot on the mainboard. Does it make sense to install the TrueNAS / OS on an M.2 drive, or would it be better to install a pair of SSDs in hardware/bios raid (mirrored) for the TrueNAS OS.

Thanks in advance!

I’m going to build two of these, I think. I have two power supplies and two mid tower cases on hand, so I don’t have to buy those. I’m delighted that the rest of the hardware, without the SAS HDDs will cost less than $550, on ebay. I’m pretty happy about that actually.

Eight of those Seagate Exos HDDs will cost around $700 (Refurbished/recertified). So, all in less than 1,300 USD for a 64TB NAS. Is that a good value, or am I doin’ it wrong?

Skol! Rick

2

I think you need unbuffered memory (udimms), not registered (rdimm). Your budget is going to take a hit on that swap.

3 Likes
3

Thanks for your comment

I think the ECC RDIMM memory that I have generally costs more than ECC UDIMM memory due to the additional circuitry required for higher density work loads, no?

I’m using ECC RDIMM because I think it makes the machine more stable.

I’m usually wrong :frowning: Happily, I’m due to be correct about something early next year, because no one can be wrong all the time :wink:

Okay, so ECC UDIMMs they are, I’ve corrected that choice by the original topic post, thanks to Habitual1 and Flesmauler!

4

Unfortunately I wasn’t referring to ecc/non-ecc, see Unbuffered versus Registered ECC Memory - Difference between ECC UDIMMs and RDIMMs

Then check the manual for the E3C246D4U2-2T and you’ll see it specifies udimm/unregistered for both ecc and non-ecc…

2 Likes
5

@habitual1 is correct, takes ecc udimms, not ecc rdims. I’d also consider a beefier psu if you can spring the extra few $

1 Like
6

Okay, UDIMM is good for the board.

So, I’ll try this?

Hey, you’re both right so I absolutely need UDIMM memory.

So!

Crucial 64GB (4X 16GB) DDR4 2666MHz PC4-21300 ECC UDIMM Memory Ram CT16G4WFD8266

That is $50 more than what I had, but the damn drives cost $850 with tax and shipping. So the UDIMM won’t scuttle the project.

Now to check the CPU / Chip Set specs to be sure the memory works with the apartment.

Thanks again!

1 Like
7

If you have SAS drives, you need a HBA. But no spinning drive will come close to saturating even 6Gbs… 12Gbs is for SAS SSDs and for putting lots of drives behind expanders.
If you have only 8 drives, go for a 9300-8i. And for more drives, you can always add an expander rather than a second HBA: No PCIe lane required.

Better still: Get SATA drives and just use the motherboard ports.

2 Likes
8

Hey, thanks for paying attention to this build, etorix! I really appreciate the help, although I’m processing your advice with some difficulty. I don’t think I understand your recommendation in a way.

I found these Host Bus Adapters for SAS, SAS Host Bus Adapter SAS9305-16I LSI SAS 9305-16I 16 Port PCIe 3.0 x8 12 Gb/s. There sixty bucks. The 8TB SAS drives that I found, are a value, I think? They’re 12Gb/s SAS drives, with dual port connectors. I don’t really understand what dual ports are, but I think I need to see the drives then choose the correct cable type to connect the HBA and the SAS drives?

My board has only 8 SATA ports…I have room in the case, if I install newer, better, quieter fans, for twelve drives. I really like the idea of Z2 or Z3, even with hot spares. I’m going to start with 8, and maybe a couple of hot spares, and another pair of hot spares on the shelf?

What do you think of the drives I found/these: Hard Drives 8 (Eight) Seagate ST8000NM0075 Exos 7e8 8tb 7200rpm SAS 12Gb/s Dual Port 256mb Buffer 512e 3.5inch HDDs/Refurbished

Thanks again!

Rick

9

There are refurbished drives, which are used and checked for function by a vendor. They may or may not last very long and may or may not have any warranty.

Then there are recertified drives, which have been repaired and serviced by the manufacturer and usually come with warranty.

Since hard disks are mechanical devices I don’t trust refurbished drives enough for my invaluable data. Recertified drives might be an alternative if they are cheap enough.

You don’t get any performance increase by buying SAS drives.

2 Likes
10

That’s cheap for a 9305. I would have expected more, hence the suggestion for a cheaper 9300.
In either case, make sure the HBA gets airflow: It needs cooling.

An enterprise feature you will NOT use with the “Community Edition”.

Without a backplane, you’ll need breakout cables from what’s on the HBA (should be SFF-8643 “MiniSAS HD” for the 9300 family) to 4xSATA or 4xSAS, as applicable.

It would depend on the price, I guess, but like @fchk I prefer to have new drives.
And 8 TB is way too small for my taste with spinners. I’m currently on 12 TB, and got a 20 TB (second hand with <100 hours) to replace one drive which had a SMART error.
Given the choice between SAS and SATA, I would also go for SATA, for ease of deployment (no HBA) and better SMART reporting.

1 Like
11

Hey, thanks again etorix, I hope I didn’t make a silly mistake with the drives, they have dual ports. They look like every other SAS port that I’ve seen, so I hope I can connect them to the HBA…Yikes, I hope so!

For the Seagate ST8000NM0075 Exos 7e8 8tb 7200rpm SAS 12Gb/s Dual Port 256mb Buffer 512e 3.5inch Hard Disk Drives,

I paid: $90.00 x8=$720.00 a bit less after a 5% discount, then more including taxes and shipping. So, around 12USD/TB? I’m an engineer (more truly an architect) so I’m not great at mach :wink:

Yikes, dual ports. I just pulled the trigger, I hope I can return if I can’t get them to spin up!

Thanks,

Rick

12

Hey, fchk, thanks for your response.
So, I’ve likley made a mistake buying “Refurbished.” From the 8 drives I bought, I was thinking to use 6 drives in a Z3 pool, with a hot spare or two? I expect to use less than 6TB of data on the main NAS pool. Hopefully that will get me started. If the drives start failing then maybe I can remortgage my house (or this NAS?), to replace/resilver the drives with either new or Seagate recertified drives? I pulled the trigger, so I have to deal with the drives. I’m not sure that I can return them, but if they don’t work, Amex will refund the price, Amex also doubles the warranty.

Also, I don’t understand the comment about performance of SATA vs SAS. SATA III has a 6Gb/s rate, while the SAS HBA, and the SAS HDDs both have 12Gb/s rates. The HBAs are PCIe x8, and the PCIe slots on the board are PCIe x8 vIII, so I don’t think that’s a bottle neck?

The board has 2@10Gb/s NICs, so that’s the slowest transfer rate. My SOHO has Cat5e cable, which I know from testing can tranfer more than 5Gb/s in the very short runs that I use. So, on the LAN, 6Gb/s might be a practical limit, but wouldn’t the performance within the NAS be better with the 12Gb/s SAS/HBAs on PCIex8 slots, as compared to 6Gb/s SATA? For replicating a pool, or resilvering or just working as a NAS? I don’t know why I try, I’m so out of my depth!!!

Where is the bottle neck? Thanks again! Rick

13

The limitation are the drives themselves. One drive makes 120 revolutions per second, and you have n 4kB sectors per track. A drive can only read from a single head at a time. The whole mechanics results in physical read/write speeds of maybe 150MByte per second. Even the fastest drives available can’t saturate a 3Gb SATA-II link. Ok, transfer from and the the internal cache is faster, but the transfer to and from the disk itself is not.

This is the ultimate limitation. You won’t get any improvements from 12G SAS compared to 3G SATA-II on mechanical drives. SSDs are a whole different story. There even PCIe Gen 4 x4 may be too slow to saturate the device.

2 Likes
14

That’s the interface speed. The drives behind, being spinners, could not even saturate 3 Gbps SATA 2/SAS 2 with actual data transfer. So the distinction is irrelevant.

15

Okay, lesson learned. Next time SSDs!

16

Well, SSD isn’t always the answer. It really depends on your use case; particularly if your use case is just to have a NAS for bulk media storage. That use case rarely ever needs fastest speed possible and price/TB tends to be a more important factor. Very few people in the home scene needs ultra high read speeds out of their NAS.

Also, if your network is limited to 1G (or worse WiFi), then your bottleneck isn’t even the drives, it’s the network speed.

2 Likes
17

Hey, Whatteva, thanks for your reply!

My use is for three SOHO activities. I’m an architect, so I use CAD (2D mostly, some rendering/graphic design) at desktops/workstations in the LAN. These are hypervisors actually, fedora / kvm. I run communications for a local political campaign, that’s sometimes graphics intensive, but mostly just something between home and a small secondary school classroom (not the media lab, though).

We’re going to do much more with video in the next year. Both for the Architect’s office/my office and for the campains. So If the server is quick, that will be a real benefit.

We’ll see how it goes. Hopefully the mainboard, cpu and memory can survive changes to the HBA and HDD specifications. And hopefully things will work for a while? I need to fly to Brazil to sell a kidney to buy more disks :wink: They’re not worth so much anymore, the kidneys…

18

Hey, Fleshie, look what you’ve made me do???

Thermaltake Toughpower GF A3 Snow Edition 1050W 80+ Gold Full Modular SLI/Crossfire Ready ATX 3.0 Power Supply; PCIe Gen.5 600W 12VHPWR Connector Included; PS-TPD-1050FNFAGU-N; 10 Year Warranty
$156.99 - Yikes!

Goodness! I did a calculation following jgreco’s excellent post, that sticky post about PSUs. The calculation indicates 1050 Watt power supply! I’m shocked (pun intended). To think I was eye-balling 500 or maybe 750 Watts??? Caramba!

Now a UPS? Who knew? Wow. 'Any recommendations there? For UPS?

Thanks again!

Rick

2 Likes
19

I have had very good success with APC, and I like getting the network monitoring one for alerts when things happen.

20

I think if you want to save money, you can buy a used brand name one (eg. APC, Eaton) and replace the battery.