HOWTO: Keylase Nvidia Patch

Bringing over this thread from the old forums.

Dragonfish and Electric Eel mount /opt and /usr among other volumes as ro on boot. It is done to secure the system and make it more immutable. It will require the following changes to get the patch working moving forward.

!!WARNING!!

Doing this is of your own volition and not supported by IXsystems in anyway. You will likely be denied support if you submit a ticket with it enabled. However, it is simple to revert and only a single library is being patched.

You can mess your system up! Make sure to have a fresh configuration and data backup in case you need to restore.

If you are having GPU issues with this installed, make sure to revert and then test the issue you’re having to make sure it’s nothing to do with the patch.

Why?

Nvidia limits the number NVENC encoding sessions to 8 on consumer grade GPU’s. This patch unlocks this number as it’s an artificial limit. This is especially useful in applications like Plex and Jellyfin that can offload encoding from the CPU to the GPU.

See the following reference from Keylase’s repo:

NVENC patch removes restriction on maximum number of simultaneous NVENC video encoding sessions imposed by Nvidia to consumer-grade GPUs.


Scripts

Post Init

You can run these scripts ad-hoc as well as creating a Post Init script on boot using the advanced settings. Running this script will require root or sudo rights on the system.

Note: Create a zfs dataset to run the scripts from.

Script for Dragonfish:

#!/bin/bash
if [ ! -d "/tmp/nvidia-patch" ]
then
    git clone https://github.com/keylase/nvidia-patch.git /tmp/nvidia-patch \
    && mount -o remount,rw /opt \
    && mount -o remount,rw /usr \
    && bash /tmp/nvidia-patch/patch.sh \
    && mount -o remount,ro /opt \
    && mount -o remount,ro /usr
else
    git -C "/tmp/nvidia-patch" pull \
    && mount -o remount,rw /opt \
    && mount -o remount,rw /usr \
    && bash /tmp/nvidia-patch/patch.sh \
    && mount -o remount,ro /opt \
    && mount -o remount,ro /usr
fi

Script for Electric Eel:

Thanks to @sean6541 for the initial script. This one has been modified with a few fixes and converted to bash.

#!/bin/bash

set -x

PATCH_DIR="$PWD"
KEYLASE_GH="https://raw.githubusercontent.com/keylase/nvidia-patch/refs/heads/master/patch.sh"
KEYLASE_SCRIPT="$PATCH_DIR/patch.sh"

# Download latest keylase script
wget -O $KEYLASE_SCRIPT $KEYLASE_GH && \

# Update backup_path
sed -i -r "s,^backup_path=.*,backup_path=\"$PATCH_DIR/libnvidia-encode-backup\",g" "$KEYLASE_SCRIPT" && \

# Add output path if it doesn't exist
[[ ! -f $PATCH_DIR/patched-lib ]] && mkdir -p "$PATCH_DIR/patched-lib"

# Patch libs
PATCH_OUTPUT_DIR="$PATCH_DIR/patched-lib" bash "$KEYLASE_SCRIPT" && \

# Get current mounted nvidia libs
NV_MOUNTED_LIBS=`awk '/libnvidia/ { print $2 }' < /proc/mounts`

cd "$PATCH_DIR/patched-lib" && \

# Mount patched libs
for file in * ; do
   suffix="${file##*.so}"
   name="$(basename "$file" "$suffix")"

   # Check if old libraries are mounted
   # and unmount if so, else continue
   # mounting.
   if [[ ! -z $NV_MOUNTED_LIBS ]]; then
      i=0
      for NV_LIB in ${NV_MOUNTED_LIBS[$i]} ; do
         ((i++))
         umount $NV_LIB
         NV_LIB_BASE=`basename $NV_LIB`
         mount --bind "$NV_LIB_BASE" "$NV_LIB"
      done
   else
      mount --bind "$file" "/usr/lib/x86_64-linux-gnu/$file"
   fi
done && \

# Link libs
ldconfig

Reverting

Dragonfish

Run the following command to revert the patch and remove all files from the server.

sudo mount -o remount,rw /opt \
&& sudo mount -o remount,rw /usr \
&& sudo bash /tmp/nvidia-patch/patch.sh -r \
&& sudo rm -r /opt/nvidia \
&& sudo mount -o remount,ro /opt \
&& sudo mount -o remount,ro /usr \
&& sudo rm -rf /tmp/nvidia-patch

Electric Eel:

NV_LIB=`awk '/libnvidia/ { print $2 }' < /proc/mounts` && sudo umount $NV_LIB

If you created a Post Init script, delete that startup script from advanced settings otherwise you’ll be patched again on boot.


Changelog

01.01.2025

  • Added Electric Eel support
  • Removed manual steps
4 Likes

The root filesystem on DragonFish is read only. If you want to make modifications to it you wiill need to first run /usr/local/libexec/disable-rootfs-protection.

NOTE: once the base install has been modified, the TrueNAS server will be considered an unsupported configuration from the standpoint of triaging bug reports.

1 Like

Yes, I’m aware now. Thank you.

I was going to try and mount -o remount,rw then back mount -o remount,ro just to install the library.

Ok, OP updated with how I got it working. Good luck out there!

Do you know if it’s possible to install this on the latest version of TrueNAS Scale ElectricEel-24.10.1?

Yes, it’s working. You need to install the drivers first in EE, then once you reboot it should work fine.

In the latest version? It gives me this error

Detected nvidia driver version: 550.127.05
libnvidia-encode.so
76d18dfeedc058a21c2ce5981f4fb742cc21b97d  /mnt/apps/misc/nvidia-patch/libnvidia-encode-backup//libnvidia-encode.so.550.127.05
./patch.sh: line 406: /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.550.127.05: Read-only file system

In 24.10.X you need to remove the protection as can be seen here:

Be sure to read the warnings and implications of doing this before you go ahead.

Doesn’t work when doing this. Does it work for you?

root@server /m/a/m/nvidia-patch (master)# sudo systemd-sysext unmerge
Unmerged '/usr'.
root@server /m/a/m/nvidia-patch (master)# sudo ./patch.sh
nvidia-smi utility not found. Probably driver is not installed.

Did you do this:

The systemd-sysext you’re unmerging contains the nvidia driver, so that’s not terribly surprising.

2 Likes

Do we need to install it using Nvidia’s package or through the Apps configuration? Because if it’s via the Web UI it’s already installed.

I’m sorry but I don’t know.

Untick the option for drivers then save it, then tick the box and save to reinstall the driver.

Did not work on Electric Eel 24.10.1.

This is what I’m doing now in a post-init command in Advanced Settings in EE and it’s working. Make sure you have the Nvidia drivers installed beforehand.

sleep 30 && /mnt/pool/data/scripts/patch-nvidia-driver.sh

So when it’s run as a post-init command, the filesystem isn’t read-only?

The script mounts -o remount,rw and runs the commands on the directories where it needs access then -o remount,ro when completed.

Sorry I was confusing patch-nvidia-driver.sh with your patch.sh for some reason.
Are you able to run that script “ad-hoc” or was that only with previous scale versions? I still get mount: /usr: cannot remount sysext read-write, is write-protected. when I run it by itself.
I added your post-init command too and it still doesn’t seem to work. I don’t see any logs for it in for the post-init job either.

Do you have issues if you run the script manually after boot?