Plugins, thoughts about which manager

H-minus 2yrs yet, I will most likely have to install and use some plugins for one need or another. I am contemplating installing and contributing to either BastilleBSD or BSDPot. Does anyone have thoughts on either. I am also wondering what happens when a configuration/template changes does that mean I need to redeploy or will either/any mangers reconfigure?

After initial reviews, I like the easy(er) templates in Bastille but like the apparent scalability in Pot. Bastille templates are stackable so a small wrapper can be made for the instances of redeployments but how are mount points handled?

UPDATE (my test server is up and running):
I’ve installed CBSD, BastilleBSD, and BSDPot on my “what-am-I-going-to-do-in-a-few-years-test-server” (you try and SSH into that a few times) and created a test jail with both. My test was simple: “which manager can I use to get a jail up and running, with networking, the easiest”.

  • CBSD: One command and networking didn’t work.
  • Bastille: One command and network didn’t work.
  • Pot: One command and everything worked.

Seems like Pot won so I will be using Pot.

Pot seems to be the most future-proof (hopefully it sticks around) with its containers concept–“export” jails for others to use (think: docker)-. Although, CBSD has similar container abilities, the documentation is tough to come by and seems to have a steeper learning curve. Pot lacks some documentation as well, but I think I have all the pieces I need.

I am now going to build a “Template” or two.

1 Like

Okay, template creation was stupid-easy!

cd /usr/local/etc/pot/flavours
mkdir -p mytemplates && cd mytemplates


Creating a template (intall neovim as a test):


ASSUME_ALWAYS_YES=yes pkg bootstrap

touch /etc/rc.conf
service sendmail onedisable

pkg install -y neovim

pkg clean -y

Use the template:

pot create -p testJail -b 14.1 -t single -f


BSDPot is creating the network but I obviously have some reading to do. When I created a pot with a specific IP and I created a more complicated template to 1. create user, 2. setup ssh, etc. and I tried to ssh into the jail from my laptop I ended up on my test server. So I “inherited” my server’s network stack instead of adding to it. I’m not so sure I actually know what I’m doing at this point (this stuff is confusing)!

Wish me luck.

I got it!! yay!

Using BSDPot I can create a jail with its own IP on my network. I tested by ssh’ing in with my laptop. This will be very cool; I can create a jail, destroy it, and recreate it with different release version and a different IP (I can even be a VM) with the same template whenever I want and be back up and running in seconds. I can share my template (aka flavor) with you guys, or even export my jail to a shareable container (like a ZFS snapshot).

Next steps:

  1. Write this stuff down so I can reproduce these results.
  2. Backup my config file(s).
  3. Create a simple jail like: Plex

You couldn’t get BastilleBSD to work with one command?
I haven’t had any issues…

I could have ran it wrong. I was going fast. I can look again in a bit though. I may need to pick your brain when I do.

I created a more complicated version of my template which is configured like the potluck versions (think: full on docker). This would allow me to launch my template with nomad but I cannot get a simple jail “working”. The jail will create, and everything seems fine but the rc.d script doesn’t seem to run on boot. I must have a setting messed up.

I will play more when I get more time.

Example Flavor (git server/jail):
potluck/git-nomad at master · bsdpot/potluck · GitHub

Another milestone!
My template is set up like this:

  1. Create the jail (nothing more)
  2. Create an RC.D script (in the jail).
  3. Once the jail is started, preform my operations with that RC.D script (set up: ssh, users, packages, etc).

I’m going to tweak my setup (like move package install to step 1) but for the most part, this is turning out kind of cool and I’m understanding how this jail setup works.

I’m still wrapping my head around the non-persistent part (where jails cand be automatically destroyed when no processes are running) but we’ll get there.

I am building a plexmediaserver template (well, the template is essentially built, i just need to test the deployment and configuration). I will let you know how it goes in the next few days.

This is going to be my first “real test”.

Templates should always allow a person to configure a mount point, and allow for installing the template overtop existing data IMO.

Check. Those kind of options are already in a config file and look something like:

mount-in -m /mnt/Movies -d <mylocation>/Media/Movies

M = jail location D = server files

The main thing I need to check–related to mounting–is permissions and what not. -i.e. I may want to include UID type stuff options in my template for the mountings.

For pled it’s that simple. But for things like vaultwarden and uptimekuma, more needs to be done because those specific services create data directories once you install them.

What I’ve done in the scripts I’ve created is add “if reinstall” (which is true upon detection of files in the POOL_PATH variable) statements to skip certain steps if it is true.

Bastillefile doesn’t really allow if statements though, and I wonder what the best way forward would be there.

But I suppose that you generally will run a template once and never really have to worry about it.

But for things like nextcloud, plex, and others it’s sometimes easier to just rerun the script instead of having to do a dependency update like php82>php83

I was typing up a reply to my reply where I was going to say that I thought I may have misunderstood you. But I guess it still pertains. Where I (re)thought you were going was to have the mountpoints handled with the /etc/rc.d script (dynamic) and mounted upon startup. As in: “Here is my Plex Jail/Container, start it like:”

pot set-env -p plex_1 -E MYMOVIES=value -E MYMUSIC=value
pot start -p plex_1

And I was going to say that this will not be the greatest idea because you may not have a “MUSIC” directory so the arguments will get messy and I am currently specifiying the IP instead of DHCP.

I am going to think about this more though because this is more than “just a template”. This is more like docker+. Using a very dumb/simple example: I could destroy these jails everynight and rebuild them every morning with a cron script.

But as to your example issue about “reinstall”, if I modify the template for nextcloud to use NINX instead of CADDY. Pot will skip over nextcloud and install the missing components. …but I wonder about the PHP example; it may just be easier to recreate the jail in that case.

…good thoughts though. I’m going to be thinking about these. Thanks.

rc scripts should only be to start and stop services running in the jail. Mount points should be a set and forget kind of thing. You set the mount (read only if you prefer) and leave it.

When the jail is deleted the mount goes away, but data stays. I want the mount to happen when the template is applied, not with the rc script, because most services have their own rc script that will be replaced anyway on updating the service.

I agree about the rc scripts only starting services. However, in my current setup, the first time the jail is started, I do some basic security setup, create an admin user and setup ssh. After that initial startup those steps are skipped.

Last night I created a “plex template” (aka: flavor) and I spun it up this morning and everything just worked. That may have been too easy of a test.

How I plan to use this (plex flavor) is: if I need to upgrade or whatever I can just allow pkg to figure out how to fix dependencies and just spin up a new version if I have to.
Initial: pot create -p plex_1 -b 13.2
Time passes…now, that’s old and not updating, so: pot destroy -p plex_1
Upgrade: pot create -p plex_1 -b 14.1

Like I said, the plex template just worked with zero fuss so I’m going to find a better test. There is a nextcloud template already so I’ll study that one a bit (nextcloud is a fickle beast).

On the statement I made about the NINGX v CADDY…that just doesn’t make sense; with these templates you’d just destroy the jail, change the template (or pull the updated version), and create another jail. Done.

So far, I’m just getting my feet wet, but I can already see how these templates (Bastille/BSDPot/CBSD/etc) can make life easier. Very cool stuff.

What does the output of ifconfig -a on the host look like with a jail up and running and networking established with pot?

right now (but I set this up as “auto” not “public-bridge”):

em0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	ether 00:23:24:c0:59:e5
	inet netmask 0xffffff00 broadcast
	inet netmask 0xffffffff broadcast
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
	inet netmask 0xff000000
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
	groups: lo
pflog0: flags=1000141<UP,RUNNING,PROMISC,LOWER_UP> metric 0 mtu 33152
	groups: pflog

I should adjust my config and see if I can’t get the bridge setup. BTW, what is 255?! Where did that come from?

So it’s not a VNET jail. It shares the IP stack with the host. I would not want that.