Great post! Looking forward to your automation and documentation.
2 Likes
Automated config file incoming… I will post it soon and edit this post above with the new steps…
1 Like
Post updated with automated config. This will get you running with Docker running using the TrueNAS docker host binaries and Nvidia GPU support. 
There are still issues to figure out, like updates in a GPU jail.
1 Like
Done. Updated the post with the automated method.
2 Likes
If you’re doing that, you’re running an older version of jlmkr.py.
1 Like
FYI:
Thank you! I’m sure your config/post will help (many) others and hopefully set a precedent but I’m not thanking you for that (I’ll let other people do that :)). But, I am for something else; you’ve inspired a thought/idea for me to make a change (in something else unrelated–not jlmkr related). I always try to reach out–even if it’s just with a simple “thank you” whenever I can…but you’re not going to get attribution in a code header or anything (that just wouldn’t make any sense). 
I don’t know if you’re a dev (I get a feel that you are just based on your comment style), but I just wanted to drop a quick note (even if it doesn’t make a whole lot of sense): *thumbs-up* Thanks (“I’m stealing that”).
2 Likes
Just for edification/verification purposes, what is the latest version we should be using and where would one get it?
First comment in this thread links to @Jip-Hop repo.
I’m a devops engineer and have been a systems architect for many years.
1 Like
*blink-blink* Oh good gawd!? Do you happen to need a friend?
1 Like
ty das for the guide. i’m sure many would appreciate it. it certainly filled a hole that was not filled in before. now we know.
that does sound concerning.
so how do you update the stuff within the jail like the debian base, docker? also i’m guessing you can’t just install tools like nano editor in the jail either after using the nvidia graphics config?
it’s not like you have to update debian and docker everyday, but it does require doing so every few months. any solution to that? cauz not being able to update that seems like a big con to me.
might just be better to follow stux guide and go straight to the meat and potatoes (just use truenas docker integration directly via the truenas > apps, and just ghost jailmaker altogether for those needing to use nvidia graphics card)
to patch or update the jail, you’ll have to disable the GPU and comment out all the extra mounts in the config file to do so at this point.
or did you mean, whenever you want to update, just stop the jail, comment out, start jail, go to jlmkr shell docker (or whatever u called ur jail), then run the update
sudo apt update && sudo apt upgrade
then stop the jail, undo the hashed out lines for the graphics. save. start the jail (this loads back the nvidia config), and it works.
Is that what you meant? if that’s the case that is workable if slightly annoying.
Yeah, just comment out the extra mounts for docker, etc, and disable GPU passthrough, then you can update. That’s the workaround for now.
I’m thinking about a way to modify jlmkr to be able to just pull in a new rootfs into it’s cache and just drop that in over the jail of choice. Restarting the jail would pull in the new binaries. That might require some fundamental changes to how the jails work though… I have to think about this for awhile.
1 Like
What if: a simple—supplemental—script that reads the current config, parses (aka: ignores the bad stuff), writes a new config, and then rewrites the original back?
The supplemental could take the arguments of any extra tasks to preform besides actual mundane update steps.
In my BSD jail making script I wrote two configs like this so, my thing did: 1. Simple config 2. Did stuff. 3. Final config. But also I thought it was easier to destroy my jail for major upgrades and remake with a new userland so I didn’t implement an update mechanism like this though.
1 Like
I see what you’re suggesting here and it’s a good idea! I would prefer a non-destructive update process for the jails not requiring a complete reconfiguration of existing jails. Something down the lines of an immutable jail that pulls in the updates on reboot.
I’m thinking about seeing if systemd-nspawn will allow for running the update process on boot as well before all the bind mounts are bound.
I can also make the current automated config universal for both GPU and non-GPU jails by just expanding the logic in the initial_install var.
I may rewrite jlmkr into bash as well, I prefer it over other languages.
My idea for the rewrite and functionality:
- Use all native binaries and controls for all functionality to make it easy to understand and use. Would also allow for a universal script that should, in theory, work on any distro with
systemd-nspawninstalled. Would need to detect FS in use to disable/enable ZFS configuration. - Create/Edit/Destroy
systemd-nspawnmachines.- Allow GPU integration
- Pull in LXC images
- Control (Start|Stop|Restart)
nspawnmachines. - Leverage ZFS like
jlmkrdoes by creating adatasetorzvol.zvolhas some advantages as it could be converted to a VM easily in SCALE. - Immutable (
ostree?)overlayfsto allow for package installation inside jails that have host bind mount points.
1 Like
You almost need something “before boot”. …I wish I was better at linux so I’m just tossing out gibberish to be honest.
If you do, go /bin/sh. I’m dipping my toes in shell script and I could use the experience to help you and help get in my 10,000 (a mentor told me to “not talk until I wrote at least 10,000 line of XYZ”).
1 Like
Nice ideas, just not sure if it’s worth it.
Next major release will include Incus (LXC) so jailmaker will not be needed anymore.
But yeah, next release is still 6 months away.
2 Likes
Do you have any information on this?
Well, if you just lurk on GitHub or Jira you can see pretty well whats going on 
Let me just post a few links.
https://ixsystems.atlassian.net/browse/NAS-130251
3 Likes
This looks great! I haven’t messed with Incus too much.
Incus is being integrated in Fangtooth (25.04).
I believe it’s already in the nightlies.
I would suggest checking it out, rather than rewriting jailmaker.
What is needed is a script to migrate a jailmaker jail to an incus one 
Incus already has an LXC to Incus migration, and Jailmaker jails use LXC images.
3 Likes