~8-year-old Pool on FreeNAS 11.3... I want TrueNAS Scale - Should I just back up my pool and start from scratch?

[Strohminator]

Hi Folks

It’s been a while…

I’m running a reasonably sizeable FreeNAS box… media hoarding and Plex serving. I’ve recently upgraded some of the hardware (denoted by *):

Norco RPC-4224

• Intel S2600CW (from SuperMicro X9 SRi-F)
• Dual Intel Xeon E5-2660v4 (from Intel Xeon E5-2680v2)
• 512GB ECC RDIMM (from 128 GB ECC LRDIMM)
  Striped RAID-Z2 Pool consisting of 3x VDEV’s (512 sector size, 4K stripe):
• 8x 18TB Seagate Skyhawk in RAID-Z2 (Currently replacing from 8x 8TB WD Reds)
  8x 8TB Seagate Archive RAID-Z2 (Yes, I know, SMR, yuck… Will soon replace these with a mix of new WD Purples and healthy WD Reds from first VDEV)
  8x 10TB Seagate Skyhawk AI RAID-Z2
• Intel X520DA 10Gb NIC
  Corsair AX1200
  Sandisk 16GB Flash Drive for Boot

175TB Raw, ~129TB formatted capacity, and about 94% full. (This will be sorted in the next few days when I finish replacing & resilvering the 18TB’s)

Everything’s been running beautifully… I still need to tweak the 10Gb NIC to get higher throughputs, and at some point, I should probably shift my boot image to an SSD (this is news to me - only found this to be a revised best practice recently).

And yes, I know, I should’ve migrated to TrueNAS ages ago, but I’ve been gun-shy about upgrading again, ever since the move from FreeNAS 10 to 11 broke my Plex jails into a million pieces, and I had to spend a few days re-indexing the entire media collection… So, I adopted the mantra of “If it ain’t broke, don’t fix it”, and simply carried on.

That said, I’ve recently scored a bargain on an older AIC 36-bay server (basically a Supermicro SC846 clone) - I ripped out the old dual Opteron 6320 build, and installed the X9SRi/E5-2680v2 setup that came out of my media server… and I’ve been playing around with TrueNAS Scale Electric Eel on that with a boot SSD and a few 2TB’s I had laying around. More on this build later…

BUT… to say that I’m impressed with Scale, would be an understatement.

So, now, the meat and potatoes… I’m seriously considering switching to TrueNAS Scale… If I do proceed, and being fully cognizant that Plex will most likely break anyway somewhere along the line if I follow the upgrade/migration route, should I not simply back up the Plex data from the jail, nuke the entire pool, and start afresh with TrueNAS Scale?

Obviously, first prize would be to simply migrate through the staged update process (11.3 U5, 12, 13, Core, Scale, if I’m not mistaken?), but aside from the high likelihood of Plex breaking, I have other reservations:

1. Having to do a fresh install on an SSD and transfer across current the configuration to have FreeNAS partition the SSD correctly.
2. File fragmentation that’s occurred over the course of the last 7-8 years since I started this ZFS dataset.

I have 2x 12x4TB MD1200 Dell DAS units that I could configure for the backup, and enough spare drives (8x 8TB’s and some randoms) laying about to populate a 15-bay EMC DAS… So it would simply be a case of slotting my Dell 6gbps HBA into the server, creating new pools on the DAS units, and dragging the data across.

Recommendations, please… I’m resigned to having to shake things up a fair bit and potentially having a week or so of downtime while repopulating the dataset and configuring + indexing Plex… but would love some feedback on whether my thinking & intended methodology is sound?

PS — I doubt I’d need to resort to L2ARC or SLOG - way too much RAM as it is, and no databases writing huge stuff to the pool.

Similarly, I was intrigued at the notion of a fusion pool with a dedicated metadata mirror on SSD’s - but even that is gross overkill for what will essentially be little more than a Plex server with some other apps running (Home Assistant, a bittorrent client, and a few other randoms perhaps… VM’s? We’ll see, doubt it though)

*** On the other hand, if others share the school of thought of “If it ain’t broken, then don’t fix it”, I’m quite happy to leave it at that as well.
I could then simply leave Plex in place as-is on this server, and instead use the Scale install on the 36-bay server for all the other intended stuff.
I’d just be kicking myself for having upgraded the CPU/Mobo/RAM to a somewhat overkill spec, to have only Plex running on it.

Apologies for the lengthy tome… and thank you in advance!

[Strohminator]

Actually, I’ve just had a bit of a brain fart…

I could simply transfer the flash drive and 24x HDD’s to the 36-bay NAS, and resume as-is. It’ll simply pick up where it left off with that mobo/CPU/RAM setup.

Then I’ll have 12x free bays for future pool expansion and spares as well.

Maybe just chuck another 128GB of DDR3 LR-DIMMs in there, switch to an SSD for the boot drive, small incremental update to 11.3 U5… and call it a day?

Or, if I’m missing out on essential Plex functionality - update the OS as far as I can (ideally 13.3 Core), without breaking Plex (or stopping short of a point where I can’t roll back if I DO wind up breaking the Plex Jail) - whichever comes first)

Then the 2011-3 setup in the Norco RPC-4224 can be repurposed to the new Scale build, to run everything else…

Thoughts, suggestions & advice welcome?

Thanks again!

[Stux]

The pool should work fine. Yes, rebuilding and restoring would give you a fresh pool and allow you to adjust the layout if you want

It’s up to you if you’d prefer to upgrade through all the steps, or just install the latest and reconfigure.

I’d suspect that the plex meta data will get zapped as you most likely have no been keeping up with updates.

Personally (and I did) I’d upgrade to the latest core… and then scale.

[Strohminator]

Thanks for the feedback - much appreciated!

No layout adjustment required - I was more concerned about potential fragmentation and parity balancing issues on a pool this old.

So, then, just a couple of questions, if I may:

1. Am I to assume that relative age-related pool fragmentation doesn’t really matter? (or can be remedied in-situ with a re-balancing script or similar)

2. I’m assuming there’ll be a major mismatch in relative parity distribution between other ~94% full VDEV’s, and the one VDEV’s newly increased 8x 18TB Z2 capacity of around 50GB (up from 8x 8TB) – where these drives will now only be about 40% full… Can this similarly be remedied with a re-balancing script? And is it worth bothering?

3. Moving to an SSD for boot pool instead of Flash drive… I understand best practice to be not to clone it, as carrying the 9 Ashift over from the flash drive will cause excessive writes to the SSD. So it becomes a case of backup config, fresh install on SSD and import config…
   So:

3a) Do I do this migration now, installing a scratch FreeNAS 11.3 U2 on the SSD, importing config and then migrating through the updates to Scale on that installation?

Or 3b) Do I update the flash drive install all the way to Scale, and then rather do the config backup, fresh install & import to SSD once in Scale?

I’ll assume that my current FreeNAS 11.3 config can’t be directly imported into Scale… So I won’t even ponder asking a stupid 3c).

4. Given that my FreeNAS install (and pool) hails from all the way back to 9.3…
   Is there any historic performance-hampering update “bloat”, or bad settings / drivers that have been accumulated along the way / get lost in translation coming across through updates in FreeNAS / TrueNAS?

I’m pretty inexperienced in FreeBSD / Linux… and ask this purely from an anecdotal the best practice philosophy in Windows, to rather do a fresh Windows 11 install, than drag an older Windows 10 installation kicking and screaming through the upgrade-and-update process… the potential for bloat, performance hits and straight up driver shenanigans is simply too much of PITA in Windows.

Does FreeNAS/TrueNAS suffer something similar? Or am I simply not giving it enough credit as a well-designed somewhat niche OS?

Thanks in advance!

[Stux]

1. it depends on what you use it for. Adding a new VDev and rebalancing will make more free space on the older vdevs.

2. again, it depends on how much churn your pool sees. Rebalancing is basically forcing churn.

3. defiantly a good idea to reinstall to upgrade to an SSD. If you do keep the flash drives the upgrades will be slower, and I’d suggest mirroring as the upgrade process puts a lot of wear on the drive. Probably best to try the SSD upgrade sooner rather than later. You can always keep your existing flash drive on a shelf

4. a TrueNAS upgrade is effectively a clean install. The config is merely a database import. Each time you restart TrueNAS re-applies all the settings in the config to take the basic install to where you want it. Sort of. The cruft is in the database. And that’s why you need to upgrade in steps so the migrations can run. Read all the release notes for each upgrade. Make changes as advised.

Or just reinstall and reconfigure. Your choice. But my systems have made the journey from 9 to EE.