Moving from Ubuntu ZoL to Scale 24.10

TrueNAS General
1

I’ve been using ZoL for about 8 years on Ubuntu and now that Scale has Docker I think I am ready to rebuild, but want to make sure I don’t have any blind spots.

64gb ECC
Optane 900p 280gb boot/docker
Optane 900p 280gb
24TBx3 raidz1 (new)
2x 1tb SSD Stripe - Used just for processing, usually empty.

My questions.

  1. Should I setup my Optane in a mirror? Can I boot from a mirror?
  2. I process video files (usually less than 50gb) on the 2x1 stripe, then move the files to spinners to storage. Any benefit to using one of the optanes or a portion of the mirror as a ZIL or SLOG?
  3. Any docker weirdness I should know about? Can I just move over my data and run my compose files?
  4. I have a few shell scripts that run in Cron currently. I assume I can still do this on TrueNas?
2
  1. Yes, but Optane 900p is stupidly overspecced and power-hungry for an app pool, not to mention as boot drive. And booting out of mirrorred partitions is just going to be a maintenance pain. Best get a (sub-) $10 SSD for use as boot drive, and some regular SSD for app pool (backed to the HDD pool if not redundant).
  2. No. SLOG is only used for synchronous writes, and sync writes should be used only for mission-critical workloads. Sync writes are an order of magnitude slower than async; sync writes with a SLOG suck less, but are still much slower than async. Your scratch stripe pool should have sync writes disabled and no SLOG, for the sake of speed.

Raidz1 with such large drives is of questionable security. Have good backups!

1 Like
3

Ditto to what @etorix said. Plus SLOG is only of benefit when the SLOG device is significantly faster than the data device.

Unclear why you have a 2TB stripe to process a 50GB file. Unless you definitely need the space, you really really really shouldn’t stripe non-redundant vDevs.

4

Ok thanks both.
I have another 256GB NVMe drive laying around I can boot from instead of the optane drives.

So putting a SLOG in front of the spinners wont do me any good? Just looking for something to do with optane drives sitting on my desk.

Regarding networking, I have 10Gbe on the servers and another NAS for archive/backup, but part of adding the 3x24TB is to bring most of the storage into the same system and not ever traverse the network until its archived on the NAS permanently. Reads of this data will be over 1gbe from client devices.

24TB spinners are backed up to another NAS.

5

Boot drive needs to be (say) 32GB. The rest is wasted.

You can go unsupported by hacking the install script to make the boot pool use e.g. the first 32GB of your Optane/NVMe/SSD so that you can use the rest of the drive for e.g. an apps / VM pool. It works fine, and TrueNAS upgrades also work fine, but you will make your life significantly more difficult if you ever need to swap boot drive or rebuilt it.

SLOG in front of the spinners won’t do any good - but if you really need it you can use the Optanes as a mirrored special allocations vDev in front of the spinning RAIDZ1 HDDs and this will store the metadata and (if you want) small files below a configurable size. BUT… this vDev is an integral part of the pool, cannot later be removed without recreating the pool afresh, and if you lose the special vDev (i.e. because you lose more drives than the redundancy) you lose all your data in the same way that you would if you lost more drives than the RAIDZ redundancy- so you are adding additional and cumulative points of failure on the pool.

1 Like