Removing L2 Cache locks up kernel

I have a strange issue happening. I have 2x NVME drives setup as a mirror or stripe set (can’t remember) for Cache of my zpool (L2 Arc). I’m trying to remove them, but I can only remove one at a time. From the web interface it locks up and freezes my machine, everything halts including VMs and docker container, have to hard reset. I also tried to remove it from the command line “zpool status RAID” and then “zpool remove 1a072803-a1e3-40bb-bad2-93decf330567” and it times out and freezes the machine. How can I get rid of both of these drives at once?

When running this command from the shell:

zpool remove RAID 1a072803-a1e3-40bb-bad2-93decf330567 c6е38511-ad86-46a2-93e8-cead17bc4197

It also freezes the system and I have to hard reset:

How can I remove the 2x cache stripe set? Is there a low level command to nuke it from the pool?

I actually don’t think it is your L2ARC. Normally, removing a L2ARC is pretty harmless and straight forward.

However, your data drives are in a Striped configuration. You call it “RAID”, but their is zero redundancy in the data side.

Not sure why that would affect removing L2ARC devices, unless their is a problem with the data pool devices. Except the zpool status does not show any problem with them.

If you intended to make a Striped pool, this may be a consequence of doing so. However, if you did not, then backup everything and redo your pool.

Yes, I intended to make a stripe set. And I have added and removed log and l2 vdevs several times as I was experimenting. It was pretty seemless, especially deleting the cache vdev. Then I settled on making a 2x NVMe cache (also a stripe set) and left it like that for the last 3+ weeks. Just now I tried to delete it again to do some more experiments and it’s a no go. Kernel craps out. Now I think I’ve had 1 or 2 TrueNAS Scale updates since then, I’m running the latest version, and I did notice a bug with the display of the core/thread temps (known bug), so maybe something they changed in this last update broke something else?

Hmm…

If you are not running TrueNAS as a VM, then I would report this as a bug. Either from the TrueNAS GUI or from the “Report a Bug” link at the top of any forum page.

Perhaps someone else will have something else to try.

Already opened a bug and provided logs, no updates yet: Jira

1 Like