I’ve been contemplating on what to do since iX announced they’re killing off CORE and I realized I made a mistake in TrueNAS (specifically the apps portion). What I want is ZFS. A “Plex server” is nice, but I don’t need a file server to do that. So, what I’ve decided is that my TrueNAS machine should do zfs and that’s essentially it (I just want it to keep my data safe). TrueNAS CORE has done a great job as my file/media server for years now so I see no reason why it cannot continue to do that but in a much “simpler” state as a NFS server.
It doesn’t make much sense for me to stay on this roller coaster ride anymore. A file server should host and backup files. I can serve up files–like in the case of Plex–with separate, easy to setup/maintain machines.
And it’s completely fine! Hyperconvergence is neither the law not a sacred scripture!
That being said, lots of users run plex and other “apps” inside jails… might become an issue in the future when FreeBSD 13.3 will become EOL, but for now things work. Can understand wanting to avoid the apparently inevitable exodus.
Point is: as long as it works for you, that’s great.
And I totally understand your frustration about iX and CORE’s management… many of us do. When life gives you lemons…
I’d expect 13.4 jails to run under 13.3, just as 13.2 jails run under 13.1, but it’s certainly possible that a breaking change will occur.
But I’m coming to believe, whatever you might think of TrueCharts, that they have the right idea–move the apps off of TrueNAS entirely. Whether you want to use their TalOS/Kubernetes approach or something else, the less control iX has over it, the lower the likelihood that they’re going to leave you stranded.
CORE’s fate has a lot to do with it but, honestly it is more like the last straw (aka: kick in the pants).
Setting up a fresh FreeBSD machine wasn’t hard. Yes, I used to run OpenBSD on my desktop, laptop, and server a few years back but I have always struggled with networking; dhclient was my best friend most times but even I was eventually able to cobble some things together just reciently (and pmh helped make the final push over the wall for me). Once that wall was cleared, it was more smooth sailing setting up some “jail templates” (I built my “plex jail template” and now I can destroy and create as many times as I want). I set up some datasets for hosting files and even data which gets mounted (so to continue the example of ‘plex’, I have a dataset that has my ‘plexdata’ directory which would get mounted in my ‘plex jail’ every time I create a new instance).
Having my TrueNAS act as an NFS server has really cleaned up my stuff. I mount my home directory, so no matter which machine I ssh into my home is the same. Same Vim, same ZSH/CSH, same everything. Passwordless login throughout because my public key there. Very much simpler.
Right now, I’m just using jails (taking baby steps) but I want to get into using images, containers and dispatchers but, honestly, that’s a bit overkill right now. Jails work good right now.
You can run SCALE as a file server or a block storage device. The “Services” tab exists for a reason (turning off services), and K3s doesn’t even run if you don’t have a pool set.
I’m not going to sit hear and preach, because there plenty of validity surrounding much of the sentiment here. However, I will share my personal experiences.
On a professional note, I was a datacenter operations manager for a large school district. I also come from VMWare land. My ELA with VMware was insanely expensive. I had allllll the fancy bells and whistles, I drank the kool-aide.
Nutanix once sent me pies on Pi day to try and get me on the phone. I politely declined and enjoyed the pie anyway. . I wasn’t gonna tear it all down just cause someone said “oooo oooo hyperconverged!!”.
That being said, VMWare VSAN works. But I actually “undeployed” it. Living in real-life land where stupid problems happen all of the time, putting all of your eggs in one basket caused problems that otherwise wouldn’t have existed. Segregating storage from compute is a valid strategy, and at scale (no pun intended) is the RIGHT way to go.
I also refurbished some old EOL SANs and installed TrueNAS 12/13 on them. I had a conversation recently with my old director. He says all of the work I did was like a time capsule, standing still for the past 18 months. So clearly most of that work was not for naught if it’s been chugging along with very little attention.
But not everyone has to professionally manage 40-odd sites with compute resources all over a city. Even still, only 2 of those sites were proper datacenters. The use case at the other 38-sites was basic stuff, local domain controllers, DHCP, DNS, telephone systems, security cameras, access control, PA systems, log aggregation, etc. If a fiber got cut, which happened often enough, I ran as much as I could locally.
Today, if in that position again, I would have no qualms ripping the VMWare servers out at those 38 sites, and I would replace them with TrueNAS SCALE as a hypervisor.
I had two systems in each of those closets, a hypervisor and a storage box (cameras). Those closets were old custodial closets, vaults, classroom closets…the buildings were built before computers were “a thing” and the idea of thermally overloading a room never cross anyone’s minds. Reducing the heatload by going from 2 servers to 1 is a great idea.
By that same logic, I cancelled my VMUG subscription in 2021 when SCALE came out. At home, I have an electric bill that makes people blush. I have a closet that I’ve jerry rigged a portable AC unit into that exhausts out the side of the house, dedicated circuits, the works. Most people don’t have that, and even I struggle with the heat load in my closet at times. Moving from 3 servers (firewall, vmware, storage) to 1 has certainly helped. Virtualization and containerization in SCALE works well.
This deployment model has it’s problems for sure, don’t get me wrong. Race conditions aplenty, SMB can’t start if the domain controller it’s bound to is hosted on the SAME NAS. But you work around these things, or enjoy other alternate options.
Final thoughts, there’s a reason why I bet on the VM deployment strategy. Apps, whether it be on TrueNAS or Synology or even Tanzu are great for dev, but not so great for prod. In my opinion, production (non-cloud) services should be hosted in VMs. The failure domain is much more limited, and you (the sysadmin) have alot more control over whats going on. I’d even argue they are just as, if not more portable.
The fact I was able to move my existing VMWare VMs onto the very first release of SCALE with minimal effort speaks volumes to the maturity and stability of KVM. I’m still running the “same” Plex server I was running in 2018. It’s been on 2 differant VMWare servers and 3 differant SCALE servers. How many people running Plex as “an app” in TrueNAS or otherwise can say that?
How many of you have had to adopt a new Plex server and clear out the old one, while dealing with your non-IT relatives complaining that it’s down, even though it’s been up and running on the new server for weeks? VMs are tried-and-true. Apps have been historically (across vendors) far less stable. Too many changes, too much rapid development. With few exceptions, they are a toy.
The overhead of virtualization exists, but it’s greatly exaggerated IMO. Paravirtualized disks and NICs are more than adequate for most use cases. I’ve run production MS-SQL server clusters on VMWare. Don’t let people tell you you can’t run Plex in a VM on TrueNAS running on an old corporate desktop. VirtIO is plenty good, and you can.
I don’t know if this would be helpful or address any concerns, but, I recently completed a migration from CORE to SCALE and I have to be honest, I’m really enjoying SCALE.
I think moving to a linux based distro is long term a better move.
I was able to back up everything I have onto a 22TB drive and once the system was set up rebuild my applications and services. It took a little time, but SCALE has been very solid so far.
My only gripe is that the virtualization is less than desirable since the transition to the Spice viewer, but there’s some guides that describe ways around that limitation using containers etc.
I use my setup for Plex, Jellyfin, and Nextcloud currently. Hoping to follow Stux’s ( I think that’s his name) guide to setting up containers for virtualization.
That or I’ll just set up another machine and try out proxmox for virtualization.
That’s great. I’m happy for you. If you find a solution that works for you, it’s a nice feeling! I hope you’re right but, I unfortunately, do not think a Linux kernel is a good long term solution.
As it should be, but I was thinking a process or two on scale is riding dirty (core at 0% whereas scale 2-4%).
