That’s a weird mix of excessively new and powerful motherboard and CPU with a trusty but old NIC and old oversized SAS HBA. For just four drives SATA HDDs directly on motherboard ports would have served you better, and if you had a deal on SAS drives a 9300-8i would have been enough. Make sure to plug power into the 9300-16i and cool it well: This thing cannot feed on the PCIe slot alone and runs HOT.
Water coolers are generally not advised for NAS builds—obvious point of failure due to eventual leak.
I know it’s going to sound like piling on, but really, what is the point of this post/thread? “What hardware would you recommend for this use case,” “what do you think of this hardware,” or even “I already have this hardware, how will it work,” would all make sense. “I just bought this mishmash of hardware and am going to build a TrueNAS box with it!” Uh, OK?
I know it isn’t much of a welcome, but this thread really looks like a prime example of doing things backward.
Here in Brazil, it’s a bit complicated to buy more specific hardware, unfortunately.
The HBA controller and the network card are cheap here for us, or rather, affordable to buy, and both are PCIe x8.
The reason for choosing the motherboard was precisely because it has the PCIe Bifurcation feature, allowing each card to actually work in its PCIe x8 specification.
I didn’t find any other new motherboards with a warranty that were more affordable than this Z890 Aero G, so I ended up choosing it. Incredibly, it was cheap here.
The choice of the HBA was basically for two reasons: support for up to 16 SAS drives and relatively low cost here in Brazil. Initially, the project will be with 4 HDDs, but more drives will be added later (at least 6).
This NAS will initially be used by 8 to 12 machines for video file storage, all gigabit, which is why I also opted for the Intel X520-DA2 connected to the switch via SFP+. The idea is to avoid network bottlenecks later with more drives to handle the load.
Thanks for the tip about the hot card; I’ll get a fan to put directly on it to avoid overheating problems.
Good point, I ended up choosing this water cooler because I already have it in 3 setups here and it’s never given me any problems, and it’s cheap here in Brazil.
But I’ll consider an air cooler; initially this NAS will be used for 2-3 months, I believe I won’t have problems with the water cooler, but I’ll get an air cooler
I needed a motherboard with PCIe Bifurcation support to deliver x8/x8 to the LSI HBA and the Intel X520-DA2.
Here in Brazil, it’s difficult to find affordable parts, and incredibly, this motherboard and processor were very cheap compared to simpler motherboards and processors from older generations.
Initially, it will have 16GB, unfortunately, but I believe it will be sufficient to serve the files, as it won’t have deduplication or other resources, just an SMB for file delivery.
Another point is that later, 6 to 8 more disks will be added, and then this setup, in principle, will be sufficient, simply requiring the addition of the disks and perhaps more memory.
Even DDR4 memory here in Brazil is absurdly expensive, and it’s impossible to find motherboards with support for two PCIe x8 slots.
That’s great, buddy! The topic was for sharing information, especially since I noticed that, being a new piece of hardware, there isn’t much information available online, and I wanted to share my experience as I progress.
Here in Brazil, it was the cheapest hardware I could find to meet the demand. I explained some of the reasons in previous answers.
Indeed, this x8/x8 setup is now a rarity and this was a good criteria—but the recommendation here would still be an older motherboard, such as the AsRock Rack E3C246D4U2-2T (new old stock on eBay from China if you can import it without too much hassle… and it already has a more modern 10GBase-T on-board).
Honestly, this Aero G will be best used as a work PC (or Hackintosh ) than as a NAS.
For adding drives, note that you may also use the motherboard ports for SATA drives. ZFS does not care splitting a pool across several controllers.
ZFS uses RAM for performance. Faster networking uses more RAM.
16 GB is perfectly adequate with 16 TB of raw storage at Gigabit speed—and with only four drives you are NOT going to run 10 Gb/s at full line rate. As you add more storage, and more clients, you’d want to have more RAM (32 GB RAM is traditionally the recommended minimum for 10G).
Another potential issue: Video files are large. Raidz# is more space efficient, but ideally you’d want to all drives at build time and create suitably wide vdevs rather than rely on raidz expansion. But if you have 8-12 clients accessing the NAS simultaneously, you want more IOPS, which means more vdevs, and would then favour mirrors (50% space efficiency only).
A ZFS vdev may have the sequential throughput of several drives, but it always has the random access performance (IOPS) of a single drive.
In short, there are two potential bottlenecks: RAM and pool design.
Unfortunately, here in Brazil there are many taxes to import products from China. The motherboard you mentioned would be more expensive than the Z890 G that I bought new, with warranty and everything.
In addition, there would be the shipping time, which I also couldn’t wait for at the moment. But it’s good to know about this card, and I’ll keep an eye out for it in case one becomes available at a good price here.
I knew I’d seen your name somewhere before, it was on some Hackintosh forum, hehe, that’s cool!
Good to know, I’ll keep an eye on that! Thanks for sharing
I believe that for the initial pool, which will consist of 4 disks, I’ll use 2 mirrors, each with 2 disks, which might improve the concurrency, right?
2 vdevs = IOPS of 2 drives
Then you can grow the pool by adding further mirror vdevs, so 2 drives at a time. Easy, even more IOPS, but 50% space efficiency.
The alternative is to go for 4-wide raidz (preferably raidz2 if this is to grow wider and wider), use raidz expansion (still IOPS of one drive) until 6-8 drives and add new vdevs, requiring several drives in one go.
Ther’s nothing magic about this particular board… except that it’s now on liquidation sale by multiple Chine sellers. I don’t expect that it will other markets.