I have to waste an entire drive just for booting?

[iX_Resources]

This resource was originally created by user: jgreco on the TrueNAS Community Forums Archive. Please DM this account or comment in this thread to claim it.

With the advent of TrueNAS Scale, we seem to be getting a large number of people from the Linux community who are not quite grasping what TrueNAS is.

TrueNAS is not a “Linux distro” and it is not designed to try to meet your arbitrarily spec’d use cases. TrueNAS is an appliance firmware for storage, which happens to be built on top of FreeBSD and Linux as underlying operating systems. It is intended to be installed like firmware onto a storage server, which can then attach and manage a pool of storage drives.

ZFS was originally designed by Sun Microsystems for storage systems with dozens or hundreds of hard drives. iXsystems wanted to duplicate this but based on a free software platform, so ZFS on FreeBSD was the natural choice. At the same time, they used NanoBSD to boot from a 1GB USB thumb drive. Over time, the boot strategy and OS was changed to a more complete FreeBSD-on-ZFS install.

Some people, especially those wishing to build TrueNAS systems on very small systems, have asked that they be able to share the boot media with the data devices. This would be a common way to build a small server, if you were building a ground-up FreeBSD or Linux server. iXsystems has declined to do this for you, and is not interested in trying to create the abstractions to support such complexity.

The basic idea here is that iXsystems is building an enterprise appliance that they can sell to customers on hardware they provide. They provide a separate boot device, so that shelves of disks can be attached to a TrueNAS head unit and imported as a pool. Their software design is oriented towards their hardware design needs. They have data pool separate from boot pool, and this allows data or boot drives to be swapped easily when one fails. Having data and boot shared would introduce lots of new complexity and edge cases. Are these solvable? Sure! Absolutely! But at some significant dedication of developer time, testing, and support.

And here’s where it seems to go off the rails. Community members should please try to be aware that the hyperconverged storage market is filled with competition. iXsystems is trying to build an amazing product with a limited staff, and they need to be able to focus on the things that pay the bills. If they allow themselves to go off tilting at windmills like shared device support, they are taking limited developer resources away from more important things, and this will hurt iXsystems and this community in the long run. We need to be respectful that they’ve chosen to decline to try to support this, and, in fact, rather than getting demanding or insulting about it, all of us here in the community would be well advised to take a moment to be thankful for the generosity of iXsystems, who has offered you the ability to use the fruits of their labor on your own server, free of charge.

If you do not like this, you have options, which include, at a minimum:

1. Go use a product that works the way you prefer. TrueNAS is simply not aimed at your small scale, low port count NAS build. No one here is going to be offended if you decide that TrueNAS is the wrong product for your needs. You’re welcome to ask for suggestions for alternatives.

2. Fork TrueNAS and build the feature yourself. This is the power of open source software. You can disagree with directions and decisions of others. Write what you want.

3. Buy some high endurance USB SSD-quality thumb drives and boot from USB. This is known to work, but is relatively expensive. SSD’s with USB-to-SATA adapters are usually cheaper.

4. Manually partition the drive yourself, and then either don’t complain in the forums or endure the inevitable dressing down if and when something goes wrong and you need to ask for help.

There is no benefit to arguing for a feature that isn’t going to happen and has clear engineering reasoning behind that decision.

[Protopia]

Despite being someone who has used a single USB SSD for both boot pool and apps pool, thus breaking not one but two of ixSystems strong recommendations, I fully stand behind this statement.

My use is for a home media / backup server, and high-availability is not a requirement, but cost very much was a concern. So I took the decision to do this with the explicit knowledge that this was not a supported option.

So far, this decision has been OK - aside from a glitch where the GPT Primary Partition Table became corrupted (which could have been a USB thing), this has worked just fine. I have upgraded SCALE versions twice without any issues.

[farkingjeebus]

Ok option 4 it is!

[Aqua1ung]

Can any of you guys post a quick tutorial as to how to do the partitioning in TrueNAS CLI (“hot partitioning”)? Is that possible? If possible, I’d like to avoid the hassle of having to open up my TrueNAS box, yank out the drive, partition it elsewhere etc. etc. etc.

Many thanks!

[maranite]

Thank you so much for writing this post - very clear, thoughtful, helpful and enabling.

[Jatrabari]

I too agree 100 % with the first post. Focus is the key.

Thank you to iXsystems for continued development of a great OS for reliable storage. I have been using it for about 5 years and been very happy with it.

[BioRebel]

Is it too much to even ask for the ability to install TO a partition rather than a drive? I’m trying to create a test system before my actual disk drives arrive, working only with a single nvme but I guess thats just “too complex”.

[dan]

@BioRebel

Is it too much to even ask for the ability to install TO a partition rather than a drive?

Yes. It’s been this way for 15 years, it isn’t going to change because you can’t scrounge up a USB stick^[1] for your test system.

---

1. definitely not recommended as a boot device for production use ↩︎

[Drew1]

Option 1 it is.

As a private user the only conclusion I can make - TrueNAS is not meant for private use. Community Apps are confusing. Why would someone install Immich on an “enterprise appliance”?

[afrosheen]

Well said. Not everything is for everyone and that’s OK.

[dan]

@Drew1

Option 1 it is.

Glad you joined to post this.

[farkingjeebus]

I would like to say, ~1.8 years later I still do not regret my choice of option 4

I’ve gone 24.04 → 24.10 → 25.04 → 25.10 and had no issues along the way due to my supposed poor life choice.

In saying that I am not using the inbuilt app or container system, I’m using jlmkr and running my own jail independently so that reduces my risk quite a lot as TrueNAS doesn’t need to handle application upgrade on update like the transition from K8s to docker for example.

[treetoon]

The lack of sufficient PCIe/NVME slots is not an “arbitrarily spec’d use case” but a limitation artificially imposed by hardware manufacturers that unintentionally make a regular low-wattage home nas more cumbersome. There are really only two truenas caveats that I’ve found so far, artificially restricting the ability to partition the boot drive and not providing a decent way to install/replace drivers.

To me, resolving these two things is a no-brainer. People aren’t actively choosing these hardware limitations, it’s being imposed upon them. It’s unreasonable to expect the ix devs to support all kinds of “arbitrarily spec’d“ hardware, this is understandable, but what is unreasonable is to intentionally design the system not to allow for such use cases. I think meeting people half-way is perfectly reasonable given that it really isn’t much of a squeeze. These small changes don’t exactly warrant a whole new truenas fork.

I would ask the ix devs to reconsider, no OS should be designed to use the entire drive unless there is a very distinct and good reason for why it cannot keep track of partitions.

That being said, truenas is terrific and I appreciate it a lot.

[oxyde]

@treetoon

The lack of sufficient PCIe/NVME slots is not an “arbitrarily spec’d use case” but a limitation artificially imposed by hardware manufacturers that unintentionally make a regular low-wattage home nas more cumbersome

More than a conspiracy, it seems like the usual trade-off between cost, space, and market targeting.
And i personally totally respect this specific limitation, because pro are most then cons.
Grab an USB - SATA (or USB - NVME) adapter and move on

[etorix]

@farkingjeebus

I’ve gone 24.04 → 24.10 → 25.04 → 25.10 and had no issues along the way due to my supposed poor life choice.

It is not implied you’d have issues with installing and upgrading (though there’s still the possibility that an update may assume full ownership of the boot device). Issues are predicted if and when your boot-and-other device fails.
If you account for this, and keep both a copy of the TrueNAS configuration file and full configuration details for your jails/instances so as to be able to rebuild all on a different boot device, you should do fine.

@treetoon

I would ask the ix devs to reconsider, no OS should be designed to use the entire drive unless there is a very distinct and good reason for why it cannot keep track of partitions.

The original post (in its original form on the old forum even) does explain the very good reason and why your request will be rejected.
Suitable (small) boot devices can be found for less than $20 / 20€ (quite possibly half of that…) on local small ads. If you really have no port to dedicate for it, you have picked some really terrible hardware—and may still throw a USB adapter in the mix.

[pmh]

And if you are building for enterprise use, SATA-DOMs still exist.

[Fleshmauler]

To add on - it isn’t an issue of what won’t work, it is an issue of what has the least amount of headache when something goes wrong. Manually drawing outside the lines of what is recommended/acceptable is always a valid choice - like manually partitioning the boot drive. Original post was more to explain reasoning & provide options.

Lotta folks see recommendations/best practices as a personal affront to how they want to do things & show functional builds as proof that these practices shouldn’t exist; that is a fallacy. Best practices & creative workarounds both have their place as long as risks are understood.

[vincentlarkin]

As somebody who runs TrueNAS for production on a consumer-grade tower, personally I’ve found the only limitation would be the power supply, as you can add SATA or NVME slots via PCIE and have at minimum 2 unless you are using an ethernet card.
I support the initial decision as I can’t imagine having to deal with an extra partition setup wizard when installing it and the truth is home Linux users already have enough issues with booting and partitioning; there would probably be no good drop-in way to support this.

[Arwen]

I do agree that modern, low end CPU sockets, like AMD’s AM5, don’t include enough PCIe lanes directly. More can be had from a chipset, but that is clumsy.

In my opinion, their should be a step up socket. To use AMD as an example, their AM5 has 2 RAM channels, (ignoring the option to split each in half), and only 28 PCIe lanes. For slight step up, a new, modern, desktop or small server should have:

• 40 PCIe lanes, (with 4 probably used by the chipset…)
• Perhaps having the ability by the system board designer to change some PCIe lanes into SATA lanes. Or USB 3 lanes.
• 4 RAM channels, (not as much as Threadripper or Epyc, but a very good improvement). Plus, ECC pins… though depends on CPU, BIOS and system board.

Just those small improvements means that we can take up to 8 x 64GB, (512GB), in memory. Not to mention including more NVMe and potentially multiple >=10Gbit/ps Ethernet.

---

However, in regards to iX adding boot device partitioning, that would be an exclusively Non-Enterprise feature. I mean someone could use TrueNAS Community Edition in an Enterprise environment, even a fancy Data Center. But, iX makes no money from such builds. And all the iX supplied hardware includes designated slots for boot devices.

Even if iX were so inclined to add boot device partitioning, the amount of variety and user decisions would make it hard to develop something reliable. You might say that is false. Someone might want a simple piece from their Apps device(s). Yet another might want space from a 30TByte HDD, (forcing all the others in a RAID-Zx or dRAID-x to use smaller space).

There is no one perfect NAS solution.

[treetoon]

You can argue that, but at the end of the day it’s just another excuse - no matter how you frame it. Small cheap boot devices either boot slower or de facto waste slots, regardless of hardware choices. It’s an added cost in the final analysis, with virtually zero benefits in my use case. A simple partition flag would suffice. Let’s be honest here and say it like it is, ix wants everyone to accept “their way of life”. Everything else is an excuse in favor of that idea.

Reminds me of Apple