I am seeing my pool as degraded now after being on 25.04.0 for a relatively short period of time and I’ve noticed others are having similar issues.
At fist I thought I had a disk going bad but all of the smart tests pass without an issue. Also, if I reboot the system, the degraded disk seems to change every time. Sometimes it tells me the disk is degraded, other times it tells me the disk is faulted.
Here is what I’m seeing right now…
Any idea where to start?
Thanks,
John
Here is my system…
MB - Gigabyte 990FXA-UD3.
AMD FX™-8350 Eight-Core Processor, 16 GB ram
3 WD 4TB - WDC_WD40EFRX-68WT0N0,
Boot Disk - TOSHIBA-TL100 SSD - 120GB
One million errors is a pretty severe amount. Can you share the disk detail pane on the right to show if it’s Read/Write/Checksum?
Something like this is usually related to a controller or cable failure. Check for loose or damaged cables, as well as dirty contacts.
Checksum errors usually are symptomatic of things like cabling and connectivity - either data or power.
Check for obvious faults at the physical cable level, and then we can have a look at the tail end of dmesg to see if you’re seeing device resets.
OK, I am going to change out all the cables.
And possibly add a secondary card that I used prior to getting this machine that adds 6 more drive ports. Just to see if I have an issue with one of the on board ports.
Thanks
John
Careful with those, they can be more problematic than helpful:
If it’s something using the ASM1x66 chipset and crucially not a port multiplier then it will likely be okay. If it’s one of the “two-port controllers with a 1:5 multiplier” then I recommend you don’t use it. 
I didn’t put the card in… I can’t find it right now anyway. 
I did go ahead and put in brand new cables. I am still seeing the pool as Degraded, but this time it’s “sdd” and not “sdc”. Only showing 99 errors this time.
The other thing that I haven’t mentioned is that I do have them in an AMS Venus ATA 5 bay backplane that allows for hot swapping the drives if I want. I have the hot swap function turned off in the BIOS right now though.
I have 2 of these (using 1) and have not had an issue thus far.
My thought is that I can try and go direct to the drives to see if that makes a difference. Although I have not tried cycling all the power connectors yet. I should probably try that first.
Thanks!
This could be the issue as well - not speaking to any quality, but just that a backplane is a point of failure.
I’d suggest removing as many intermediate devices as possible - run the SATA cables straight to the drives for a test.
So, I’ve bypassed the backplane and have everything wired direct. The pool is still showing degraded, now on “sdb”. Here what it shows. No errors…
I’m running a SCRUB right now, but I’m not sure if that will do anything. I’m not sure why it is showing “Degraded”…
Thank you!
So the scrub is about 90% done. The drives are now showing…
sdb = 762
sdc = 146
sdd = 762
So, I’m guessing there is something else going on…
it happened to me too. it more of a bug, i still shut down the server and ran the disks through windows crystaldiskinfo. Everything was fine.
At first it was just one. then, another one. for no reason
So, check sata power cable and sata data cable. make sure everything is well.
then. what i did was put another disk where it said it was degraded and use the other one “degraded” in another vdev.
meaning i bought another disk, used that for 1 degraded dved. it resilvered it, no problem.
then i added the other one (that was previously degraded , according to truenas), added it to another degraded vdev, it resilvered and now everything is fine.
So, really , just ran them through a program to make sure everything is fine. if it is , swap them and it will resilver and everything will be ok.
It was for me at least, because i had no errors of consequence
Is anyone else having this type of issue? I am really thinking this is a glitch. Like @rc722, I pulled all the drives and ran WD software on them to check each disk. They all passed with no issue and no issues on any smart tests within TrueNAS.
I also added a secondary SATA card just to check if there might be an issue with the onboard controller, but the card had the same issues…
I’ve also tried multiple different SATA cables… Brand new and used…
It is also odd that I am seeing identical checksum errors on 2 drives with the 3rd showing something different. It’s not always like this but more times than not…
Also, the “degraded” drive is not always the drive with checksum errors. Sometimes, I see errors on the other 2 drives and ZERO on the “degraded” drive.
I have tried running scrubs and they always complete successfully.
Just looking for confirmation that this is a potential bug…
Thanks!
I’ve been running 25.04 since - well, 2025-04 - and no CKSUM errors to report here, with regular scrubs ongoing.
Have you checked your RAM using memtest or a similar program?
Does your dmesg log show anything in relation to disk timeouts or things like WRITE_FPDMA_QUEUED errors?
No, I haven’t checked memory. I will do that now…
I’m not sure how to view the dmesg log. Can you tell me how?
Thank you!
sudo dmesg will dump the whole thing, sudo dmesg | less will let you scroll through it, sudo dmesg | grep -i SomeText will look for SomeText in the output.
Here is what I see when I search for each of the 3 drives…
2483.377125] sd 5:0:0:0: [sdf] tag#23 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 2483.377130] sd 5:0:0:0: [sdf] tag#23 Sense Key : Illegal Request [current]
[ 2483.377135] sd 5:0:0:0: [sdf] tag#23 Add. Sense: Unaligned write command
[ 2483.377139] sd 5:0:0:0: [sdf] tag#23 CDB: Read(16) 88 00 00 00 00 00 00 40 02 a0 00 00 00 e0 00 00
[ 2483.377142] I/O error, dev sdf, sector 4194976 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 2483.866078] sd 5:0:0:0: [sdf] tag#14 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 2483.866087] sd 5:0:0:0: [sdf] tag#14 Sense Key : Illegal Request [current]
[ 2483.866093] sd 5:0:0:0: [sdf] tag#14 Add. Sense: Unaligned write command
[ 2483.866100] sd 5:0:0:0: [sdf] tag#14 CDB: Read(16) 88 00 00 00 00 00 00 3f fa a0 00 00 00 e0 00 00
[ 2483.866104] I/O error, dev sdf, sector 4192928 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 2486.274425] sd 5:0:0:0: [sdf] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 2486.275310] sd 5:0:0:0: [sdf] tag#0 Sense Key : Illegal Request [current]
[ 2486.275319] sd 5:0:0:0: [sdf] tag#0 Add. Sense: Unaligned write command
[ 2486.275329] sd 5:0:0:0: [sdf] tag#0 CDB: Read(16) 88 00 00 00 00 01 d1 c0 ba 20 00 00 00 e0 00 00
[ 2486.275339] I/O error, dev sdf, sector 7814036000 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 2486.658192] sd 5:0:0:0: [sdf] tag#8 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 2486.658200] sd 5:0:0:0: [sdf] tag#8 Sense Key : Illegal Request [current]
[ 2486.658207] sd 5:0:0:0: [sdf] tag#8 Add. Sense: Unaligned write command
[ 2486.658213] sd 5:0:0:0: [sdf] tag#8 CDB: Read(16) 88 00 00 00 00 01 d1 c0 bc 20 00 00 00 e0 00 00
[ 2486.658218] I/O error, dev sdf, sector 7814036512 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 4507.444574] sd 5:0:0:0: [sdf] tag#7 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4507.444579] sd 5:0:0:0: [sdf] tag#7 Sense Key : Illegal Request [current]
[ 4507.444582] sd 5:0:0:0: [sdf] tag#7 Add. Sense: Unaligned write command
[ 4507.444585] sd 5:0:0:0: [sdf] tag#7 CDB: Read(16) 88 00 00 00 00 00 00 40 00 a0 00 00 00 e0 00 00
[ 4507.444588] I/O error, dev sdf, sector 4194464 op 0x0:(READ) flags 0x800 phys_seg 27 prio class 0
[ 4507.512804] sd 5:0:0:0: [sdf] tag#11 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4507.512808] sd 5:0:0:0: [sdf] tag#11 Sense Key : Illegal Request [current]
[ 4507.512811] sd 5:0:0:0: [sdf] tag#11 Add. Sense: Unaligned write command
[ 4507.512815] sd 5:0:0:0: [sdf] tag#11 CDB: Read(16) 88 00 00 00 00 00 00 00 00 a0 00 00 00 e0 00 00
[ 4507.512817] I/O error, dev sdf, sector 160 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 4508.044634] sd 5:0:0:0: [sdf] tag#30 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4508.044639] sd 5:0:0:0: [sdf] tag#30 Sense Key : Illegal Request [current]
[ 4508.044642] sd 5:0:0:0: [sdf] tag#30 Add. Sense: Unaligned write command
[ 4508.044646] sd 5:0:0:0: [sdf] tag#30 CDB: Read(16) 88 00 00 00 00 00 00 00 00 a0 00 00 00 e0 00 00
[ 4508.044650] I/O error, dev sdf, sector 160 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 4508.097787] sd 5:0:0:0: [sdf] tag#3 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4508.097796] sd 5:0:0:0: [sdf] tag#3 Sense Key : Illegal Request [current]
[ 4508.097803] sd 5:0:0:0: [sdf] tag#3 Add. Sense: Unaligned write command
[ 4508.097809] sd 5:0:0:0: [sdf] tag#3 CDB: Read(16) 88 00 00 00 00 00 00 40 00 a0 00 00 00 e0 00 00
[ 4508.097813] I/O error, dev sdf, sector 4194464 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 4508.656632] sd 5:0:0:0: [sdf] tag#8 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4508.656637] sd 5:0:0:0: [sdf] tag#8 Sense Key : Illegal Request [current]
[ 4508.656640] sd 5:0:0:0: [sdf] tag#8 Add. Sense: Unaligned write command
[ 4508.656645] sd 5:0:0:0: [sdf] tag#8 CDB: Read(16) 88 00 00 00 00 00 00 40 02 a0 00 00 00 e0 00 00
[ 4508.656647] I/O error, dev sdf, sector 4194976 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 4508.764951] sd 5:0:0:0: [sdf] tag#12 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4508.764956] sd 5:0:0:0: [sdf] tag#12 Sense Key : Illegal Request [current]
[ 4508.764959] sd 5:0:0:0: [sdf] tag#12 Add. Sense: Unaligned write command
[ 4508.764963] sd 5:0:0:0: [sdf] tag#12 CDB: Read(16) 88 00 00 00 00 00 00 00 00 20 00 00 00 e0 00 00
[ 4508.764965] I/O error, dev sdf, sector 32 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 4509.328730] sd 5:0:0:0: [sdf] tag#8 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4509.328734] sd 5:0:0:0: [sdf] tag#8 Sense Key : Illegal Request [current]
[ 4509.328738] sd 5:0:0:0: [sdf] tag#8 Add. Sense: Unaligned write command
[ 4509.328742] sd 5:0:0:0: [sdf] tag#8 CDB: Read(16) 88 00 00 00 00 00 00 00 02 a0 00 00 00 e0 00 00
[ 4509.328744] I/O error, dev sdf, sector 672 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 4509.536774] sd 5:0:0:0: [sdf] tag#16 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4509.536778] sd 5:0:0:0: [sdf] tag#16 Sense Key : Illegal Request [current]
[ 4509.536782] sd 5:0:0:0: [sdf] tag#16 Add. Sense: Unaligned write command
[ 4509.536786] sd 5:0:0:0: [sdf] tag#16 CDB: Read(16) 88 00 00 00 00 00 00 00 02 a0 00 00 00 e0 00 00
[ 4509.536788] I/O error, dev sdf, sector 672 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 4509.640947] sd 5:0:0:0: [sdf] tag#19 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4509.640951] sd 5:0:0:0: [sdf] tag#19 Sense Key : Illegal Request [current]
[ 4509.640956] sd 5:0:0:0: [sdf] tag#19 Add. Sense: Unaligned write command
[ 4509.640960] sd 5:0:0:0: [sdf] tag#19 CDB: Read(16) 88 00 00 00 00 00 00 40 02 a0 00 00 00 e0 00 00
[ 4509.640968] I/O error, dev sdf, sector 4194976 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 4510.421789] sd 5:0:0:0: [sdf] tag#1 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4510.421798] sd 5:0:0:0: [sdf] tag#1 Sense Key : Illegal Request [current]
[ 4510.421804] sd 5:0:0:0: [sdf] tag#1 Add. Sense: Unaligned write command
[ 4510.421811] sd 5:0:0:0: [sdf] tag#1 CDB: Read(16) 88 00 00 00 00 00 00 00 02 20 00 00 00 e0 00 00
[ 4510.421815] I/O error, dev sdf, sector 544 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 4512.458003] sd 5:0:0:0: [sdf] tag#22 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4512.458843] sd 5:0:0:0: [sdf] tag#22 Sense Key : Illegal Request [current]
[ 4512.458850] sd 5:0:0:0: [sdf] tag#22 Add. Sense: Unaligned write command
[ 4512.458857] sd 5:0:0:0: [sdf] tag#22 CDB: Read(16) 88 00 00 00 00 00 00 3f fc a0 00 00 00 e0 00 00
[ 4512.458864] I/O error, dev sdf, sector 4193440 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
[ 4512.757927] sd 5:0:0:0: [sdf] tag#1 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 4512.757935] sd 5:0:0:0: [sdf] tag#1 Sense Key : Illegal Request [current]
[ 4512.757942] sd 5:0:0:0: [sdf] tag#1 Add. Sense: Unaligned write command
[ 4512.757948] sd 5:0:0:0: [sdf] tag#1 CDB: Read(16) 88 00 00 00 00 01 d1 c0 bc 20 00 00 00 e0 00 00
[ 4512.757953] I/O error, dev sdf, sector 7814036512 op 0x0:(READ) flags 0x800 phys_seg 28 prio class 0
This drive is now showing “Faulted”…
The plot thickens…
BTW… Nothing shows up when I search for “timeout” or “WRITE_FPDMA_QUEUED”
Running MemTest86 right now.
@HoneyBadger the men test is almost through the 4th pass and 0 errors, so I’m guessing the memory is fine.
I wonder if we’re looking at a power state management bug.
Do this line in a shell to force your SATA links to never go to sleep.
echo maximum_performance | sudo tee /sys/class/scsi_host/host*/link_power_management_policy
Let’s see if this corrects it.
