Install NIX on TrueNAS SCALE

TrueNAS General
1

Since TrueNAS is quite locked down in what you can do, I figured NIX would be an ideal way to have flexibility of getting any package I want vs. not breaking TrueNAS. People who don’t fully understand this, please familiarize yourself with NIX - its awesome.

To install NIX you need to do few very straight-forward steps :

  1. I use single-user mode as it’s very simple.

  2. Figure out where you want to place NIX store (I placed it on NVMe array), create /nix (you will need mount -o rw,remount / for this)

  3. Create systemd mount unit that bind-mounts your chosen NIX store location into /nix (google how to create mount unit files, its easy). Enable and start it.

  4. Since we will be installing NIX as root, you need this :
    echo "build-users-group =" > /etc/nix/nix.conf

  5. Install NIX
    TMPDIR=/root/nix-install sh <(curl -L https://nixos.org/nix/install) --no-daemon

  6. Don’t remember if it auto-creates .bash_profile but you need below in it. I guess it can also go into .bashrc :

[[ -f /root/.nix-profile/etc/profile.d/nix.sh ]] && . /root/.nix-profile/etc/profile.d/nix.sh

  1. I use S3 cache for NIX stuff via MinIO that runs on TrueNAS. To configure you need :
    a. install MinIO and create s3 access keys (google how)
    b. Create :
.aws/config
[default]
region = us-east-1
s3 =
    signature_version = s3v4
endpoint_url = https://<MINIO HOST>:9000

.aws/credentials
[default]
aws_access_key_id = xxxx
aws_secret_access_key = xxxx

c. Create .config/nix/nix.conf

cat .config/nix/nix.conf
substituters = s3://nix-cache?scheme=https&endpoint=<MINIO HOST>:9000 https://cache.nixos.org
experimental-features = nix-command flakes
post-build-hook = /root/bin/upload-to-cache.sh

where upload-to-cache.sh looks like

# cat /root/bin/upload-to-cache.sh
#!/bin/sh

set -eu
set -f
export IFS=' '

server="<MINIO HOST>:9000"

echo "Uploading paths" $OUT_PATHS
exec nix copy --to "s3://nix-cache?scheme=https&endpoint=${server}&compression=zstd" $OUT_PATHS

  1. Source nix-profile . /root/.nix-profile/etc/profile.d/nix.sh

  2. And start installing stuff like there is no tomorrow

nix-env -iA nixpkgs.git
nix-env -iA nixpkgs.awscli
nix-env -iA nixpkgs.hddtemp

Fully functional NIX without breaking TrueNAS at all.

2

…or so you pretend.
Do not install stuff (other than scripts) on your root/admin account, make a VM or sandbox instead.

1 Like
3

Its just a nas what can possibly go wrong.

I guess it can be installed as non-root user too, I was just too lazy to do it.

to be honest, I did it because I needed hddtemp

4

How is is going? Any regrets?
I am about to do the same.
I hope the be able to limit the impacts to a nix dataset and the /nix bind mount.
Maybe something in /etc for the builders.

Actually, first off, I might attempt to install nix in an lxc, or vm, and mount /nix into there for all the building/maintenance of nixpkgs. There are just one or two apps I want from nixpkgs, although I find that a debian incus vm allows me to install packages which can at times do what I want within the VM and also run at system level – there were some thunderbolt tools (some had to be built from github) I wanted as my system has thunderbolt attached storage.

5

Works perfectly fine.

You just need to redeploy systemd unit that bind mounts between SCALE upgrades and that’s about it

6

Cool. I just installed nix into a debian lxc container, and did a

nix profile install nixpkgs#btop

This lxc container has my $HOME mounted and a user created (it was a pain in the ass to get the uid mapping working correctly).
I installed nix as me, not root.

btop runs in the lxc, but has, natch, a limited view (but all the CPUs).
I exit the lxc shell, and do the following:

# mount -o rw,remount /; mkdir /nix; mount -o ro,remount /
truenas% sudo mount --bind /var/lib/incus/storage-pools/rpool/containers/tbtools/rootfs/nix /nix
truenas% . ~/.profile
truenas% btop
truenas% which btop
/home/dap/.nix-profile/bin/btop

That was really easy. I like this method better as I don’t have to dork with permissions again as I would have had to do had I mounted a /mnt/dozer/nix volume in the lxc.
Whenever I need to add anything more, I just do sudo incus exec tbtools bash followed by su dap and I can then do nix things which are reflected in my user profile in dom0 (to borrow a xen term).

I have a script in /root/bin/mounts.sh which is run after the system comes up, so this should survive upgrades.

truenas% sudo incus exec tbtools -- su - --command "nix profile --extra-experimental-features 'nix-command flakes' list" dap
Name:               btop
Flake attribute:    legacyPackages.x86_64-linux.btop
Original flake URL: flake:nixpkgs
Locked flake URL:   github:NixOS/nixpkgs/12a55407652e04dcf2309436eb06fef0d3713ef3?narHash=sha256-N4cp0asTsJCnRMFZ/k19V9akkxb7J/opG%2BK%2BjU57JGc%3D
Store paths:        /nix/store/2h5716mnxvdq9pq9ahi5264q4vkgiqz4-btop-1.4.3

Name:               nix
Store paths:        /nix/store/xlg2s2hdngc2gjb6kfwac1g7b4q2pcg5-nix-2.28.3
truenas%