Join an idiot for some live troubleshooting on a lovely cold Saturday

We got quite a few problems to resolve, the first one being:
After major hardware upgrades (AMD EPYC platform, SuperMicro H12SSL-I board) the 24.10.0 “Electric Eel” Boot ENV doesn’t boot with the “cannot import boot-pool” error.

Reddit user mentioned disabling CSM as possible resolution. Let’s see!
Possible solution 1: disable CSM
Result: error persists.

Possible solution 2: Thanks to the snapshot feature


we can boot to ENV 24.04.2.3 just fine, set it to “keep” and attempt an upgrade to the newest 24.10.0.2, however exactly that has been reported problematic in the past: https://ixsystems.atlassian.net/browse/NAS-131890?focusedCommentId=282032
It reads:

We have fixed this moving forward starting in EE but upgrades have the potential of running into this issue. There really isn’t an easy solution once you’re into this scenario. The path forward is to save a backup of your configuration to your local PC. And then fresh install EE-RC.2 to the boot drives again. During the installation process we wipe all partitions and ensure any zfs filesystem labels are cleared out as well.

→ Oh boy, let’s hope a fresh reinstall isn’t necessary in my case.
There also is an ongoing discussion about the issue here: https://forums.truenas.com/t/clean-install-of-24-10-0-results-in-cannot-import-boot-pool-no-such-pool-available/21993/19
Result: SUCCESSFUL Upgrade TrueNAS-24.04.2.3 → TrueNAS-24.10.0.2. Booted up like :butter:. That’s a :clap: to the devs right there:

Next up: WTH are my apps? It seems that automatic migration of Kubernetes Apps to the new Docker backend is not possible if the apps dataset is encrypted (should’vce read the memo). Should be a relatively easy workaround to temporarily move the apps data to an unencrypted dataset then.

Also next up: Figuring out why this happens and GPU passthrough to VMs isn’t possible despite BIOS being configured to do so:

Nevermind… In 24.10.2 the NVIDIA driver is gone and seems like you have to install it separately?

That’s correct, you really should read the release notes… hint hint, nudge nudge

Indeed I read those:

  • Starting in 24.10, TrueNAS does not include a default NVIDIA GPU driver and instead provides a simple NVIDIA driver download option in the web interface. This allows for driver updates between TrueNAS release versions.Users can enable driver installation from the Installed applications screen. Click Configure > Settings and select Install NVIDIA Drivers. This option is only available for users with a compatible NVIDIA GPU and no drivers installed or for users who have previously enabled the setting.

If I look under Apps → Configure → Settings there is NO “Install NVIDIA drivers” option anymore. The RTX 3090s show up in “lspci | grep -i nvidia” output but not recognized in the TrueNAS UI anywhere! Apps service is running.


What is this mystery? I meet the mentioned requirements:

  • compatible NVIDIA GPU :white_check_mark:
  • no drivers installed :white_check_mark:

Not sure why that would be.
Try force reloading the page.

1 Like

The GUI just wouldn’t show me the option.
Manually installing the driver as mentioned here via

midclt call -job docker.update '{"nvidia": true}'

installed the NVIDIA driver just fine, ‘nvidia-smi’ shows output as expected.

May be worth reporting that as a bug.

1 Like

Absolutely, once I figure out the root cause of all the weird issues. After installing the driver manually and force refreshing Firefox it still doesn’t show me any GPUs to isolate, not even the onboard ASPEED chip. Will report if progress is made. Let’s give her a full reboot for a start.
btw the UI option in the Apps tab resurfaced as checked after the install via CLI:

Edit: split up the missing “install NVIDIA drivers” and missing “isolated GPUs” into different bug reports. Waiting on input from the devs for now.

1 Like

The missing GPUs under “Isolated GPU Device(s)” has already been reported here: No GPU´s in isolation list
Will move over there regarding this to avoid duplicates.