I’ve setup a new system and have a question about HW Raid vs ZFS+RAIDZ[1-3] performance.
System Details
MB: Supermicro X11SDV-C8+TLN2F
Ram: 4x32 DDR4-2133MHz PC4-1700- ECC RDIMM 2Rx4
SSD: 4x1Tb
I’m using one of the SSD for boot, and the other three in their own RAIDZ1 pool.
HDD: 2x16Tb
The 2x16TB are currently setup as RAID0 in the hardware controller - I have two additional disks arriving, and plan to ultimately use RAID5, but for now for testing, that’s how they are setup.
RAID: Supermicro AOC-S3108L-H8IR-16DD
Enclosure: Silverstone DS380B with 8 3.5" bays, and 4 internal 2.5" bays
The question I have is that in many forum posts with similar topics, there are statements to the effect of TrueNAS/ZFS needs direct access to the drives to function optimally. This makes sense generally given what I understand about the benefits of ZFS and assuming no HW RAID. However, in most of those posts, people were working in Virtual Machines, so I’m not sure that advice applies in my situation where I built a dedicated TrueNAS server.
It seems to me that a dedicated HW RAID controller would be faster at managing RAID R/W operations than a ZFS+RAIDZ[1-3].
This may be a moot question as I don’t see a setting in the RAID controller to allow direct access to the disks to the BIOS and from there to TrueNAS. If I delete the RAID0 setup and don’t configure something else, the disks are not shown in TrueNAS (as expected). Hypothetically, I could RAID0 each of the disks independently, but then it seems that I would double the read/write overhead from ZFS and the HW-RAID.
What are your thoughts?
Thanks in advance.
Josh
ps. I think I do need the HW RAID card to use the 8-drive backplane on my enclosure - the MB only has 4 SATA slots, which are filled by the SSD and the MB also seems finicky - the first controller I tried was not RAID (LSI-SAS 9300-8i model H5-25573-00) and did not function at all - the MB wouldn’t even post with it installed.
Read enough entries on this forum and the old one and you will see:
Loads of warnings about flashing to IT mode and NOT using RAID hardware in RAID mode with ZFS, even as JBOD.
Loads of examples of people ignoring these warnings and losing their data.
So, based on this, my advice would definitely be DON’T TRY IT.
But it’s your system and your data and your decision, and as long as you are prepared to own and live with the consequences, do feel free to ignore all the above advice and see whether your own luck is better than everyone else’s.
Sorry, but there’s exactly nothing I like in your setting.
X11SDV: high idle power, and not enough ports.
Hardware RAID: enough said!
Silverstone DS-380: Could not sufficiently cool a full set of HDDs in there, even with screaming pressure-optimised fans on the side. And a pain to work with.
You’re in for serious trouble not so far down the road…
Ok - well, you and @dan were succinct but convincing. I’m going to return the RAID card and try to figure out which HBA will work with my MB. It may be that I just got a bad card the first time… but I’ll re-do the research and try something new. Perhaps you have an opinion on whether LSI-SAS 9300-8i model H5-25573-00 should work with my MB?
Looking at the Supermicro site of compatible cards for my MB, AOC-S3008L-L8e is compatible and is reasonably priced on Amazon. Do you have any specific concerns about that card?
