Trying to undestand the specifics of the read-only rootfs

In particular, what sort of rw activity, besides during software updates or changes in settings, is happening on the drive TrueNAS is installed on?
What changes happen on the boot drive, besides when packages are updated or settings are changed? Do all settings live on the drive (including VMs, etc.), or just basic settings?
Aside from a slower boot and slower software updates, what sort of performance impact would I have to expect installing the OS e.g. on an eMMC drive instead of on an SSD.

To give some context:
The system I’m mapping out has an internal USB3 port, into which I could place a USB-to-eMMC adapter, and install the OS onto that (256GB eMMC is more than enough). This would free up an NVMe slot, allowing 7 NVMe SSDs in the storage pool rather than just 6, which would give me an extra 8TB storage.

The idea would be to simply image the entire boot device after relevant changes, so if there’s an issue, one can simply flash the image to another device, replace it, and continue…

Portions of the root filesystem are read-only (/, /usr, etc), other portions like logs and audit databases are read-write.

Gives some description of where we store state.

Not to step on any toes from iX, (like @awalkerix), but in general, you don’t want to perform simple boot-pool image as a backup method.

Now in theory, an off-line image of the boot-pool could be made. Or one from a recursive top level snapshot.

However, the “real” method to restore functionality when a boot-pool fails, (like due to single device), is to reload the current version of TrueNAS on to a new boot device. Then restore the saved configuration. (Of course, that implies you save your configuration off NAS after any changes… Some people even save their configurations to their data pool.)

Most of this is probably covered in the fine manual.

To reduce down time, you can do this:

  • Regularly save, or automate saving the configuration
  • Keep a copy of the install image, of the version used on your TrueNAS, already copied to a flash drive
  • Keep a spare boot-pool device available

With that, you can reduce any down time. Simply perform the stock recovery:

  1. Shutdown / power off server
  2. Replace boot device
  3. Put on install device
  4. Boot to the install device
  5. Re-install TrueNAS, (same version)
  6. Reboot, (and remove install device)
  7. Using the TrueNAS GUI, restore configuration
  8. Done.
2 Likes

No worries about stepping on my toes. I didn’t read the post particularly carefully :slight_smile:

I was trying to address what we store where, but didn’t look at bigger context. The gist there is that the boot device isn’t readonly (we can do quite a bit of writing there).

The correct way to preserve system config is to use our supported methods to export / import config. Doing otherwise is working against the design of TrueNAS and will probably give mixed / bad results.

What you use as a boot device is a matter of risk tolerance. If you regularly back up your config, then the primary risk would be system downtime on failure.

3 Likes

Downtime isn’t a major issue, it’s a personal device, as long as it’s reasonably widely spaced. So then the question is, provides an eMMC module sufficient endurance to not have to replace the boot pool all too often?

Backing up configs is really not a problem, I do that with other systems that offer such a mechanism, regularly, too.

How big are this config files typically?
Would it be possible to write a script whereby they are automatically e-mailed to an admin after any config changes?

Not very, but I don’t have a number for you. I would also guess that more complex configurations would be larger…

Check the forum for various scripts. In the past their was one that could be cron’ed, and save current configuration to a data pool. There is also the Multi-Report script that both mails status of your TrueNAS, and sends a copy of the configuration as well.

Just to chime in: my last config backup was 810KB in size

Any reason to go for USB-to-eMMC over more common USB-to-SATA (or USB-to-NVMe) and a cheap SSD?

Thank you for giving the OP a figure.

Back in the Jurassic era, that would not fit on a 256KB, 512KB or 720KB floppy. You would have to use one of the 1.44MB floppies for that size of file. But, we are so far beyond that. Even CPU L1 memory is generally in the mega-byte range.

My Fangtooth config file is 33KB, Core is 50KB. These files are compressed (.tar) which is easily opened on any platform. They really shrink down the 810KB. Inside my file is 752KB which supports @LarsR comment.

Multi-Report by default will email you them every Monday.

If all you want is the config file, look at the link below:

This is all it will do in case you do not desire a full report and Multi-Report to manage your SMART testing.

2 Likes

Cool, thanks, that’s helpful!

Just size. I have internal space the size of a USB stick. And eMMC is supposed to be more reliable than a regular USB stick. A USB-SATA solution would dangle outside the case.

Sorry for tagging along on this post with my own issue but it kinda relates to this post and I didn’t want to create another issue. I am a beginner with TrueNAS and I have an issue where my boot-pool right after installation is readonly. Now I am not sure if that is normal but I am pretty sure it shouldn’t be read only. Can somebody maybe expain why a new install would become ro straight from the get go? I did 2 days of troubleshooting by changing the usb drive, formats, filesystems and even cleared the NVRAM on my server… Help? Thank you.:grin:

@jonpoe - Welcome to the TrueNAS forums!

Are you saying your entire boot-pool became R/O, (Read Only)?

If you are attempting to use the boot-pool for general purpose work, like scripts and such, it may not work. TrueNAS CE, (formerly SCALE), is an Appliance OS not a general purpose Linux distro. As such, iX has been locking down changes and removing un-needed files to improve maintainability and increase reliability.

Thus, certain OS directory trees are now R/O, unlike what they would normally be in a Linux distro, (desktop or general purpose server).

Hi Arwen. My whole boot-pool is ro. I checked it by running mount | grep “ on / “ and it comes on as ro. I then mounted the EFI to /mnt/efi/EFI to check the contents and i have 3 folders there, debian, boot and Dell(i have a T430 server). I believe it has something to do with the bootloader but I can’t tell what is wrong.

Most of he boot pool is readonly. We only set r/w on specific datasets. like /audit /data and others where r/w is required.

So that would mean that my install is normal and not broken. I was under the impression that the root would have r/w for the boot-pool but I was wrong. I don’t need to edit anything anyway, I only wanted to mount a drive from another system that I didn’t export and it was in ro and that’s how it all started. I then went down the rabbit hole for 2 days :rofl: Thank you guys for your guidance. I’ll be back with more silly questions ASAP :+1:

1 Like