i just ordered a connectx-3 and a connectx-4 and will compare direct power draw, as well as overall impact on the system. I’ve read that with the connectx-3s sometimes ASPM doesn’t work and systems can’t reach high C-states.
we’ll see, for now the BCM57810 seems to be doing great.
Just curious why both if you have concerns on the ConnectX3?
I remember having a weird problem with a ConnectX2 but don’t remember running into any issues with the 3s. Although I’m probably one of the least power conscious users here so that doesnt mean much.
because i don’t know for sure yet and will find out 
i’d prefer the more future proof 4 but if that acts up or uses way more power than the 3, i’ll go with the 3, if it works (better).
also something to consider for future builds of mine/retrofitting my systems is, that the 4 costs around 80-90€ here. the 3 I can get for around 45€.
Shows that this seems to have come from internal GC tasks or whatnot in the drive. I did a secure erase of all of them and now reads so far look like this:
What I’ve done on each drive:
hdparm --user-master u --security-set-pass p /dev/sda
hdparm --user-master u --security-erase p /dev/sda &
friggin’ awesome:
Array's average speed is 540.327 MB/sec per disk
Disk Disk Size MB/sec %ofAvg
------- ---------- ------ ------
sda 1920383MB 540 100
sdb 1920383MB 540 100
sdc 1920383MB 540 100
sdd 1920383MB 540 100
sde 1920383MB 541 100
sdf 1920383MB 541 100
sdg 1920383MB 540 100
sdh 1920383MB 542 100
sdi 1920383MB 541 100
sdj 1920383MB 540 100
sdk 1920383MB 540 100
sdl 1920383MB 540 100
ASM1166 is a 6 port chip (no sata bridge) and connects over pcie 3.0 x2.
I use it with 3 WD red plus disks for about 6 months in truenas scale without issues. On the contrary, some enterprise hgst disks had communication issues and got lots of smart data retrieval warnings.
what are you referring to? while your statements are true, they seem a bit unrelated…
There’s a plethora of sellers on eBay…
I can give a little update on this - my iSCSI storage is still going strong, at full performance. Doing the secure erase apparently genuinely solved this issue.
Hmmm. This is a classic behaviour seen on SSDs as they are written and re-written
In theory using TRIM should avoid. Ensure that you’re either periodically trimming the drives, or you have the pool set to auto-trim.
afaik the PM863a I’m using here have some quirk in truenas which says “trim broken” so they don’t trim regularly.