I’m interested in building a storage server. To save on cost, I’m doing it myself instead of buying a pre-build one. The server will have the following configuration with a case that support up to 12 SAS/SATA disks:
Motherboard: Asrock SIENAD8UD-2L2Q
2 x M.2 Samsung ssd’s for TrueNAS scale.
1 x Samsung enterprise ssd for caching (optimized for lots of reads and writes)
12 x Seagate SAS drives
1 x Broadcom 9400-16i Tri-Mode (to connect the SAS disks)
Case: Inter-Tech IPC 2U-2412 2HE STORAGE
This board has a few MCIO ports, and as I’m fairly new to this, I’m not sure how exactly these work. If I’m correct, I can connect the SAS drives (SFF-8087) to the HBA controller. That will solve the connectivity for the 12 SAS drives. One of my question is if the PCIe slot that the HBA card is plugged into, has enough lanes to support full speed of 12 SAS disk (Seagate Exos X24 SAS)?
My other question is, can I use the MCIO connector to connect the ssd for caching?
I’m not sure you’re going to save money compared with buying an older refurbished server.
That’s what the HBA is for. But there’s no SFF-8087 in there (this one was the 4-port connector of the 9200 generation).
Should be the least of your concerns…
Do the math: You have 12 SAS drives, which as spinning drives may, at best, push about 250 MB/s into their 12 Gb/s interface; that’s 3 GB/s coming to the HBA. The HBA has a a PCIe 3.1 x8 connection at 3 GT/s per lane, i.e. 24 GT/s.
The short answer is yes, but as described you have no RAM, so nothing to cache…
[quote=“etorix, post:2, topic:21108”]
That’s what the HBA is for. But there’s no SFF-8087 in there (this one was the 4-port connector of the 9200 generation).
[/quote] The SFF-8087 is included from the backplane of the case. The HBA has SFF-8643, but should work with an adapter I assume? Would you recommend another HBA card instead?
Perfect thanks. I’m thinking to use a combination of SSD’s and SAS HDD depending on the use (simple data share, or VM)… I might end up using 4 SATA SSDs for VMs and the spinning drives for shares, and general storage.
I didn’t specify the ram, as I thought it would be obvious, but the plan is to have 192gb of RAM paired with an AMD EPYC 8124P cpu. The reason why I wanted a SSD for caching was to use only spinning SAS drives paired with an SSD, so I would have massive amount of cheap storage, and have the benefit of fast storage of the caching for the VMs (iSCSI).
You only need SFF-8087 to SFF-8643 cables. A 9305-16i (or 9300-8i if the backplane has an expander) may serve you as well for less than the Tri-Mode 9400.
“Cache” may not mean or do what you think with ZFS.
Read here for your iSCSI setup:
I was able to place an order, the Tri-Mode 9400 was somehow cheaper than the one you recommended.
Also looked up the article you referenced, I realized I made a mistake in my previous post, as I meant NFS instead of iSCSI. Did some more reading and set up a small lab environment to play around with read/write caching in trueNAS. in the end I decided to buy 2 non-enterprise level SSDs for the caching, which in my use case did had a significant impact on the performance.
