I figured it out. For some reason, my system was swapping, even though I have ~240gb of ram. I ran swapoff -a
and then disk usage slowed way down. Bonus- it seems faster too. Not sure why it was using swap. You can tell in the graph below exactly when I turned off swap.
edit:
adding another graph to show when swap turned off:
I’m feeling quite confident that swap was hitting the boot drive for some reason. I dunno why, but turning it off made my boot drive shut up.
I added this as an init script in the UI so it applies next boot. Good idea?