Unexpected slow performance, where did I go wrong?

TrueNAS General
1

Hi, I expecting that I have done something really daft here but looking for some pointers on where I have gone wrong.

Running Truenas Core with 15 disks HPE raid controller in passthrough mode.

Reliability of data is the most important thing to me but I also need throughput. So my pool looks like this:
5 mirrors each one with 3 disks all striped.
What I can’t really understand is why it is a complete dog, compared to similar on native HPE raid - i.e. Raid 10ADM I am seeing tiny throughput about 1/4 of the speed of a single drive. Drives are only modest Samsung SSDs but I see great performance with HPE raid.

Machine should be fast, 2 x 12 core 128GB of RAM - old DL380s are cheap on account of the deafening fans.

Did I get the wrong end of the stick here? If so how should I have set it up, I’m struggling to find the right documentation.

I have enough space to pull every third drive and use to move stuff about but need to understand where I have gone wrong. Ideally want to take advantage of thin provisioning for workloads, I’m running some VMs on this from separate box with XCP-ng.

LAN is 10Gbit

2

Maybe it is the HPE raid controller, even if it is in passthrough mode.

Generally, LSI HBAs in IT Mode are recommended. Truenas/ZFS needs full control of the disks.

Which OS?

If you have not done it, maybe googling the exact name of the controller and truenas to see if it shows up a solution/confirms the problem.

1 Like
3

If not already done, I suggest reading What's all the noise about HBAs, and why can't I use a RAID controller? | TrueNAS Community.

Writes? Reads? I suggest reading Introduction to ZFS | TrueNAS Community and ZFS Storage Pool Layout - Resources - TrueNAS Community Forums.

Which NIC? Which version of TN are you using, CORE or SCALE? Exact version please.

Please provide us more informations, thank you.

4

Thanks Googling, the controller was a bit of a horror show.

It is an HP P440AR seeing some others talk of poor performance too but the non standard backplanes may make using an alternative very challenging.

I was hoping at least 1GB /s read the speed of 3 disks which should be easily achievable on a 12GBit controller.

5

Hi Davo,

NIC is HP 669281-001 2-port 10Gb 560FLR-SFP

It is Core TrueNAS-13.0-U5, seems that this is now being deprecated, do you guys recommend moving to scale?

6

Right now? No, things are still WIP there. I would suggest starting from updating to 13.0-U6, knowing that in a few weeks 13.3 will be out.

Oh, it’s SFP. Well, things change a bit then.

For starters share the output of hpssacli cmd -q "controller slot=0 show config detail”… this should let us know what your RAID controller is doing.

Then, if it’s not in IT mode, you should be able to force it using hpssacli cmd -q “controller slot=0 modify hbamode=on forced" if it’s in the slot 0.

Start from there, then we can ask for more testing… like iperf and fio.

7

Even this sort of server typically has standard connectors on at least one end, at least for some configurations. You might lose access to proprietary mezzanine slots, but arbitrary SAS controllers should not be hugely difficult.