Another follow-up. Turns out the instructions should work for Jellyfin, but are not sufficient for Plex. Plex will show your hardware, but not use it. Switch on debugging in Plex and you’ll find you’re missing the driver.
The solution is to install the nvidia-container-runtime for Docker to use when launching containers. See this guide.
I finally see the magical “(hw)” when transcoding a stream in Plex and my CPU is no longer used!