HOWTO: Keylase Nvidia Patch

Bringing over this thread from the old forums.

Dragonfish mounts /opt and /usr among other volumes as ro on boot. It is done to secure the system and make it more immutable. It will require the following changes to get the patch working moving forward.

!!WARNING!!

Doing this is of your own volition and not supported by IXsystems in anyway. You will likely be denied support if you submit a ticket with it enabled. However, it is simple to revert and only a single library is being patched.

You can mess your system up! Make sure to have a fresh configuration and data backup in case you need to restore.

If you are having GPU issues with this installed, make sure to revert and then test the issue you’re having to make sure it’s nothing to do with the patch.

Why?

Nvidia limits the number NVENC encoding sessions to 8 on consumer grade GPU’s. This patch unlocks this number as it’s an artificial limit. This is especially useful in applications like Plex and Jellyfin that can offload encoding from the CPU to the GPU.

See the following reference from Keylase’s repo:

NVENC patch removes restriction on maximum number of simultaneous NVENC video encoding sessions imposed by Nvidia to consumer-grade GPUs.


Getting Started

Testing out the patch:

sudo bash /tmp/nvidia-patch/patch.sh
Detected nvidia driver version: 545.23.08
libnvidia-encode.so
Attention! Backup not found. Copying current libnvidia-encode.so to backup.
mkdir: cannot create directory ‘/opt/nvidia’: Read-only file system

Volume Mounts

We need rw access to /opt and /usr. /opt is where the library will get backed up to and /usr is where the library will get patched. Both are ro on boot.

mount|grep "boot-pool/ROOT/*"|grep -E '/opt|/usr' 
boot-pool/ROOT/24.04.0/opt on /opt type zfs (ro,relatime,xattr,noacl,casesensitive)
boot-pool/ROOT/24.04.0/usr on /usr type zfs (ro,relatime,xattr,noacl,casesensitive)

Manual Process

Mounting /usr and /opt as writable is enough for now.

sudo mount -o remount,rw /opt \
&& sudo mount -o remount,rw /usr

Run the Keylase patch script, then when you’re done, just mount the volumes back to ro:

sudo mount -o remount,ro /opt \
&& sudo mount -o remount,ro /usr

Results:

sudo bash /tmp/nvidia-patch/patch.sh
Detected nvidia driver version: 545.23.08
libnvidia-encode.so
2fe3cbed1eda7f3dbd9090f23430ff1d6a490e1e  /opt/nvidia/libnvidia-encode-backup/libnvidia-encode.so.545.23.08
74a78bcd74f3da01210f275b6f77638f607682eb  /usr/lib/x86_64-linux-gnu/nvidia/current//libnvidia-encode.so.545.23.08
Patched!

This is temporary and will be revert on reboot. More permanent options are below.

Scripted Approach

“Oneliner”

This method will require you to run it after each boot, but this way is great for testing out the change without making it permanent.

sudo mount -o remount,rw /opt \
&& sudo mount -o remount,rw /usr \
&& git clone https://github.com/keylase/nvidia-patch.git /tmp/nvidia-patch \
&& sudo bash /tmp/nvidia-patch/patch.sh \
&& sudo mount -o remount,ro /opt \
&& sudo mount -o remount,ro /usr

Script to run Post Init

You can run this script ad-hoc as well as creating a Post Init script on boot using the advanced settings. Running this script will require root or sudo rights on the system.

#!/bin/bash
if [ ! -d "/tmp/nvidia-patch" ]
then
    git clone https://github.com/keylase/nvidia-patch.git /tmp/nvidia-patch \
    && mount -o remount,rw /opt \
    && mount -o remount,rw /usr \
    && bash /tmp/nvidia-patch/patch.sh \
    && mount -o remount,ro /opt \
    && mount -o remount,ro /usr
else
    git -C "/tmp/nvidia-patch" pull \
    && mount -o remount,rw /opt \
    && mount -o remount,rw /usr \
    && bash /tmp/nvidia-patch/patch.sh \
    && mount -o remount,ro /opt \
    && mount -o remount,ro /usr
fi

Reverting

Run the following command to revert the patch and remove all files from the server.

sudo mount -o remount,rw /opt \
&& sudo mount -o remount,rw /usr \
&& sudo bash /tmp/nvidia-patch/patch.sh -r \
&& sudo rm -r /opt/nvidia \
&& sudo mount -o remount,ro /opt \
&& sudo mount -o remount,ro /usr \
&& sudo rm -rf /tmp/nvidia-patch

If you created a Post Init script, delete that startup script from advanced settings otherwise you’ll be patched again on boot.

3 Likes

The root filesystem on DragonFish is read only. If you want to make modifications to it you wiill need to first run /usr/local/libexec/disable-rootfs-protection.

NOTE: once the base install has been modified, the TrueNAS server will be considered an unsupported configuration from the standpoint of triaging bug reports.

1 Like

Yes, I’m aware now. Thank you.

I was going to try and mount -o remount,rw then back mount -o remount,ro just to install the library.

Ok, OP updated with how I got it working. Good luck out there!