NVIDIA GPU Previously Detected in nvidia-smi, No longer detected

I am having this really bizarre issue. I am running Proxmox + TrueNAS with GPU Passthrough.

When I originally installed it last night passthrough was working and TrueNAS could output the GPU via nvidia-smi. By all accounts everything was working as it should.

Some stuff happened with the plex app on TrueNAS apps and the driver stopped working. The gist of it was, the plex app container couldn’t access the GPU and I couldn’t see the GPU allocation. What I usually do is go to the GPU isolation section, add it and then it errors and I remove it, then if the plex container is stopped it will show up and I can allocate. Since then I cannot see the GPU in nvidia-smi, even on a new install (which is bizarre since its a new VM).

Since apt isn’t supported on scale, I am not sure how to go about fixing this. Doing the exact same VM but with Ubuntu Server + installing the nvidia drivers nvidia-smi shows the GPU. However on my current TrueNAS install + reinstall + new TrueNAS SCALE VM, I get the error:

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

The card does show in lspci -v as shown below, but just wont work anymore.

I have tried removing the card restarting, adding it back in, physically removing the card, starting, then shutting down and putting the card back. I cannot get past the communication issue.

The GPU is an NVIDIA A2000 which is supported by the current TrueNAS SCALE Nvidia driver version.

Proxmox allocation below:

Utterly bizarre. It just fixed itself 5 minutes after posting this. I basically took the ubuntu VM had that the driver installed. Installed TrueNAS SCALE on it, ran nvidia-smi. It gave the same error as before.

I removed it from that VM and put it back to the original TrueNAS Scale VM, started it up and it started working again.