I have some old Areca Raid-cards, they work well, and have been working well for close to two decades now. But does anyone have any experience with using such cards for TrueNAS? Any things too look for? They have a “pass-through”-mode, but not sure how much the actually pass through.
All reported experiences have either been terrible or mercifully cut short by the cards’ complete inability to even pretend to function.
2 Likes
Thanks for the feedback, do you have any more details? What hasn’t functioned?
It’s been 10 years of these and others like them, so the details have blurred together. The bottom line is that they are unknown quantities and offer no advantage over an LSI SAS3 or SAS3 HBA.
Well, the very real advantage for my use case is that I actually have the Areaca-cards, so i don’t have to use a lot of money on new cards.
One issue is that some RAID cards don’t pass SMART through, even in “pass-through” mode. This can affect trouble shooting and pre-fail detection. This is easy to check.
Next, some RAID cards, even in “pass-through” mode, still use either elevator seeking or write cache without consent.
The first breaks one of ZFS’ data integrity methods. ZFS writes data in a specific order:
1 - Data, to free space
2 - Directory entries for that new or updated data, to free space
3 - Metadata that makes those 2 writes effective, to circular list
If written out of order and a power loss or software crash occurs, this can damage the pool. Which won’t happen with proper, write barrier implementation in the HBA or drive firmware.
The write cache issue is similar, as ZFS makes assumptions that when told a write completes, it actually completed. Then ZFS can continue other writes, maintaining the ZFS pool’s integrity at all times. If hardware lies about writes being completed, (using write cache and faking the response that the cache was cleared because it’s a RAID card with battery backed RAM), all bets are off for ZFS pool integrity.
Last, some low end RAID cards are not intended for the extended continuous traffic that ZFS can cause. During a ZFS data scrub every block in use is read, which may cause a low end RAID card to over heat. Even if that does not cause over heating, disk replacement could do so.
All in all, using storage connection devices that are known to work properly, (Intel SATA, LSI HBA, etc…), means that during trouble shooting, you don’t have yet another potential fault to investigate.
3 Likes
Refurbished LSI 9200/9300 HBAs do not cost a lot of money.
1 Like
I get that, but the plentiful bad experiences and lack of compelling need have led the community at large to not spend time on these things, given the availability of LSI HBAs.
Not passing smart i can live with i think, the card itself lets me check that.
Not sure what elevator seeking is, can you explain that a bit more? Is there some test to check if this happens? I also think the write cache thing would be fine(?) as that can be turned off and on for the card.
No, you cannot.
Maybe, but if smartmontools can’t speak with the disks, TrueNAS will have no way of actually monitoring their SMART data, which massively increases your exposure to risk when faced with a failing disk.
2 Likes
I see they are about 60 dollars on ebay from china, that is reasonable i agree, i would need two as the card is 24 slots. I will consider it.
Why not? The card warns of any smart trouble, it even mails you when it happens.
In TrueNAS? That’s not likely, have you actually tested that? It’s also ugly as hell in the best case, HBAs have no business at all dealing in emails or any networking, really.
Depending on your setting you may consider a single 9305-24i or a -8i HBA with an expander ($30) over two HBAs.
1 Like
Not in trueNAS (or not as far as i know), but the point is getting the message quickly, right? The cards have their own network ports and can be interacted with through it.
1 Like
@Leverpostei - Elevator seeking is what it sounds like. During normal elevator usage, an elevator stops at the floors requested as it passes them.
So, person A gets on floor 1, and wants floor 10. Person B gets on floor 3 and wants floor 7. The elevator goes from 1, to 3, to 7 and last to 10. This is not the order requested. Person A has to wait for person B’s requests before getting to floor 10.
But, ZFS REQUIRES it’s writes to be in order, IF you want normal ZFS data protections. How does this relate to elevator seeking? Same as the example above.
1 - ZFS to write data to free space at block 10
2 - ZFS wants metadata written to free space at block 7
3 - ZFS has the critical metadata on blocks 2-5
4 - The disk is currently at block 1
RAID controllers can “optimize” writes by writing blocks in optimal order;
- Blocks 2-5
- Block 7
- Block 10
This breaks the COW, Copy On Write because the critical metadata activates before the directory entry and even before the data was written. If a power loss occurs, pool damage occurs. Potentially fatally.
Now do I know for sure your Areca HW RAID card does elevator seeks?
No.
But, let’s pose the question another way.
Do you know for sure your Areca HW RAID card won’t corrupt ZFS pools?
Regardless if it is corner case firmware bugs, or power losses?
To clarify this for newer ZFS users, ZFS was thoroughly designed & tested for graceless, (aka un-planned / unexpected), power losses. And I mean REALLY designed for it. Any data in flight can be lost, just like any other file system. But, ZFS pools are supposed to remain consistent, at all times. Partly due to COW, Copy On Write. Though other methods are used too, like a circular buffer for the Uber blocks, (aka critical metadata).
Of course hardware faults, (or hardware RAID controller firmware), can damage a pool that would normally survive a graceless power loss.
2 Likes
The most insidious issue with faulty sata controllers, or raid cards, is that everything looks like it’s working perfectly until you have an issue (a power event for instance) and then they corrupt your pool.
It’s your data, you can do what you want, but it’s very hard to certify these things.
But LSI HBAs do work.
They can be had for even cheaper and probably be genuine if you scope out ebay long enough.
YMMY.
Can you elaborate? Not using a backplane with an expander?
So is elevator seeking the same as NCQ? I have never heard the expression before, is there some way to test this?
Nothing is a 100%, you can ask if i am sure using any controller, and i would have to say no. The question is how good/bad it is compared to alternatives.
Not using a backplane at the moment no, they are pretty expensive.