think i found my issue and my fix
i have also stumbled upon another possibility.
this article
is talking about needed to turn off Native Command Queuing for all his WD gold 16tb drives by setting the queue depth to 1, and it has links to other articles and discussions about flawed NCQ in WD Golds in ZFS github from 2020…
I am using WD Gold 18TB drives…
i am going to use this script to set the queue length to a value of 1 for ONLY my WD gold 18TB drives. this will leave my micron 1.92TB SSDs and my WD Purple drive (for Frigate Surveillance) alone at their default queue lengths of 32.
#!/bin/sh
for i in /dev/sd? ; do
#echo "$i"
model=$(smartctl -i $i | grep "Device Model")
if [[ "$model" =~ "Micron" ]] || [[ "$model" =~ "PURZ" ]]; then
echo "skipping disk: $i --> $model"
else
echo "Disabling NCQ for disk $i"
echo 1 > "/sys/block/${i/\/dev\/}/device/queue_depth"
fi
done
so my plans are:
1.) set queue depth to 1, test system
2.) if still errors, then try setting libata.force=3.0G. while this will not affect one of my pools since it will always be running off an HBA, it would allow me to test my other pool connected to the motherboard SATA controller. this pool also gets errors on heavy loads so it will be worth testing. Assuming this fixes the issue, then i will worry about configuring the HBA controllers.
3.) if still errors, then i will try testing with BOTH libata.force=3.0G and the queue depth set to 1
so far after more than 30 hours, no errors on either pool when setting the queue depth to 1. i am going to keep testing, and trying to increase the queue depth to 2 and or 3 etc and see if anything errors out.