I’m running plain FreeBSD and BSD Pot is turning out to be a very nice jail manager. The tool is mainly built to be a docker type of tool (via Nomad and/or Ansible) but it obviously does jails too. With the Nomad job file, you’re supposed to “grab the latest images” but, at the moment, I’m building my own templates for plain old jails which is working well. I’ve gone through a few different configurations for my jail templates, but I think I’ve settled on a nice setup–which I plan to automate a bit.
Essentially you write a shell script to do what you want (install packages, create users, create file, etc) as though you are configuring from within the jail/container (I say container, because it essentially works the same for Nomad as it does the Jail). With Nomad you pass the VARs upon start with a JSON file (the job description file) but with the jail, I’m just configuring them outright. I can still pass variables to the Jail upon run time with the rc.d/script framework but I really don’t need the layer of complexity at the moment.
The automation I’m planning will bring my jail templates up to more like a “jail config file setup” so I can utilize the same template for multiple configurations, but this will still build static template files to create jails from. The static template files will give me the most flexibility for what my current plans are (I want a repository of templates which I can pull from or share).
The one most important thing for me when creating and installing jails is the ability to mount data upon initial install, and upon reinstall without wrecking the currently present data. Docker does this nicely with it’s containers, and the scripts I use have the ability to check if data is present, and if it is, it will skip some initialization steps.
Are you planning on integrating the above with your templates?
Mounts? Oh sure, mounting is done at creation or on the fly. But that is sort of per user kind of stuff. I will share a simple jail template when I can. The Nomad side is far more dynamic than the jail stuff though.
I just got done running a test to “modularize” my template. I can keep a directory in my ‘jail-setup’ directory for basic modules like “user setup” “ssh setups” (which are rc.d scripts). and I can copy-in those into different jail templates. I wanted to run this test to see if I can keep one set of these basic things and edit once and apply (copy-in) to all my templates. My test/idea works but I still have to test out my “automation idea” which will make this latest test obsolete.
The idea that most of this stuff is standard shell script make this highly portable!
I got to thinking about my automation plan last night and I think I’m going to scrap the idea and keep a “master file” I just copy every time I need a new jail template. This is very low-tech but I don’t plan to have that many jails so if I ever decide to update my master, I can update my other templates easy enough.
These can only be used with their corresponding manager.
Scripts that can run in jails make it so much easier, but there are some things that need to be run outside using a jail manager. Things like mounting, jail properties, etc…
I was planning on automating my template creation. BSDPot has “flavors”. These templates are just there for me in case I wanted to create another or redo. I have a directory for each template and I have all the mountings, configs, etc. in there. So, if I decide I like DOAS instead of SUDO I can change my template(s), destroy the jail, and create another and be back up and running.
Right, Bastille, iocage, etc, preform a "jexec " upon the jail from the host. You can also do “pkg -j” or “ifconfig -j” upon a jail from the host.
I’m assuming jexec does mounting, properties, commands, etc…
Jail managers are just wrappers on top of jexec as I understand it.
Makes me wonder if it wouldn’t be easier to just writes scripts that can be run across the board regardless of which jail manager you used, by using jexex commands.
I haven’t looked into the native jexec/jail system does mountings, but I think it’s nullfs not fstab. So, I imagine you’d still need that wrapper. But yes, most of the mangers I’ve looked at are running the setup UPON the jail not WITHIN the jail. That’s what attracted me to Pot. When I saw that the setup scripts were scripts written in the first-person, I was kind of excited. I could take snips from almost anywhere on the internet (about setting up PF or sysctl.conf and apply it to my jail template).
My current generic template does:
admin user setup: - .cshrc and .nexrc
ssh setup: - sshd_config and user’s pub keys
network tunings: misc stuff
jail slimming: removing unnecessary binaries like CC and whatnot.
It would be cool to pass you a script if you needed a jail like mine. You could create the jail (with any manager you want), log in, “fetch” my script and run it. Obviously, the mountings and the network parts would make that an almost impossible idea, but you get the point.
yes. I couldn’t get it to work as smoothly as I would like but seemed very, very robust! I believe it is still installed on my machine too. CBSD has huge potential and there are TONS of cool things on their radar. K8S alone is a big one. I do want to give it another round of trials after I get a few more jails replaced.
Good question because I was just thinking about that. Currently I have: Plex and Git replaced. And now, I’m thinking about keeping Nextcloud on my NAS (Nextcloud hasn’t been updated in a VERY long time and is technically good where it’s at).
I still want to create some more templates like Emby, Gitlab, Nexcloud, etc to share with others though.
Most things are fairly straightforward. As for emby, that a simple pkg install.
The one thing I’m trying to work out is how to integrate commands such as iocage set allow_mlock=1 jailname in a generic way and not have to depend on a jail manager.
