When I used BTRFS, (before 2015), I used Grub version <1.0, (I think it was 0.9x). That meant manual entries in Grub for booting. Just fine for me.
Later with OpenZFS, I had to use an initial RAM disk, and transitioned to Grub 2.0 and all of it’s horrors.
I probably can’t use ZFS Boot Menu because I need to alter specific things. Like in some cases I might have Grub entries that do this;
Kernel X.Y.Z with RAM disk 2 on ABE 20240714 InitRD update
Kernel X.Y.Z with RAM disk 1 on ABE 20240714 Distro update
Kernel X.Y.B with RAM disk 2 on ABE 20240701 InitRD update
Kernel X.Y.B with RAM disk 1 on ABE 20240701 Kernel update
Kernel X.Y.A with RAM disk 4 on ABE 20240701 Distro update
Kernel X.Y.A with RAM disk 4 on ABE 20240615 InitRD update
…
In essence, I might have 3 Grub entries for a single ABE. Depends on if I update the kernel. Or if a package, like OpenZFS, updates kernel modules. In which I need another Grub entry for that.
As I get 5 or more ABEs, (but NOT kernel or InitRD lines), I delete some ZFS ABEs, and any associated kernel or InitRD lines. Since I generally don’t go back more than 1 BE, I don’t care as much about keeping them clean. But they are somewhat a backup.
With my naming scheme, I know what each Grub line is about: InitRD, Kernel or distro update and what boot environment it is on.
@Arwen Can you shed some light on the Linux boot process for some ignoramus like myself? Why this ram disk thing at all? I really do not get it. My baseline is the FreeBSD boot loader which I consider straight forward and simple.
OpenZFS can not be easily built into the Linux kernel, (like BTRFS can be). Thus, the Initial RAM disk is used to supply the kernel with OpenZFS modules that allow root on ZFS.
With ZFS support via modules in the booted kernel, I can use Mirror or even RAID-Zx root configurations.
Further, even though Grub 2.x has some support for OpenZFS, you have to limit the feature set. Or Grub can’t use “/boot” on ZFS.
In my case, I use EXT4 on top of Linux MD-RAID Mirror for “/boot” to be extra safe. But, I have been considering migrating to “/boot” on ZFS. Or possibly not having a separate “/boot” and simply use it on the root FS, (which is already ZFS).
One last quirk. With the kernel modules for ZFS being loaded via Grub, an update of ZFS can cause a reasonably harmless version mis-match between the old ZFS modules and an updated ZFS userland. In the past before using static API interface configuration, that could lead to a non-bootable system. But, that was fixed 5 or so years ago.
However, because of that quirk, you should not attempt to use a new feature of ZFS without both regenerating the Initial RAM disk, and rebooting to use that new Initial RAM disk.
I do know that RHEL 7 uses InitRD and probably other Enterprise Linux releases do too.
One of the problems InitRD solves, is having to build in all the disk and network drivers into the kernel it’s self. See, if you don’t have the disk driver or network driver available, you can’t use it at boot time.
So it is common to build ALL disk and network, (as well as some other drivers), as loadable modules. Then include them into the InitRD for use at boot. Any modules, (disk, network, share protocol, whatever), that are not used, are simply not loaded.
Now for me, I build custom kernels for each of my home Linux computers. Anything that is used continuously is built into the kernel. Other optional things, like NFS, is a module. Exceptions are things like OpenZFS or NVidia which are always modules, because they are supplied externally.
With BTRFS tied into kernel source, it can be built into the kernel file. When I used BTRFS that meant I could continue to avoid using InitRDs at boot. Like I did when using EXT3/4.
One problem of building all the modules and including them into an InitRD, is size. On RHEL servers with a separate “/boot”, I routinely see 500MegaBytes used for “/boot” size. Originally at home I used 50MegaBytes for “/boot” and that was pretty generous.
For example, my test SCALE VM has 2 InitRDs, at 73MegaBytes and 75MegaBytes. The kernels associated with them are about 9MegaBytes each. Then some support files are 6MegaBytes per kernel. So my test SCALE VM uses about 170MegaBytes in “/boot”. (Which is part of “/” in SCALE.)
The “basket” of all the commonly needed modules (for a wide variety of different hardware) requires that much space?
Another thing I don’t understand (and maybe I’m not looking hard enough) is how come generating an initramfs on your distro (be it Arch, Ubuntu, Gentoo, etc) doesn’t automatically determine which modules are needed for the very specific computer it’s being run on?
I can understand a bloated initramfs for installing and booting up a Linux distro for the first time. But any subsequent generations of the initramfs should be very lean and small, and specifically based on your hardware. Why is the default behavior “let’s just throw the the entire kitchen into the initramfs every time it’s generated.”
I just checked my main admin server at work, (RHEL 8.10), and it has a 1GigaByte “/boot” and uses about 200MegaBytes.
That is 3 separate kernels, with support files, (InitRD, map and config), plus a separate recovery image. Pretty normal. Not sure why it has 1GigaByte assigned to “/boot”.
But, NO, Linux is not that smart. It does NOT restrict InitRDs to just what is needed. Now, I DO program mine to skip useless stuff, like BTRFS support or LVM support, since neither is in use. But that is mostly to prevent errors of “can’t load X, program does not exist”.
To be clear, the InitRD may have all the modules, (in RHEL and other distros), but they are NOT loaded into RAM at run time unless the requisite hardware is present. My Linux servers at work are mostly under VMWare, so they don’t need physical disk or network drivers.
Do I think this is a good scheme?
No.
But, it is what the industry has chosen to use as the “gold” standard for *nix in the Enterprise. (Aka, AIX, Solaris & HP-UX are fading away…)
In debian you can choose a generic or targeted initrd at install time. Post install you can change from MODULES=most to MODULES=dep. or vice-versa, in /etc/initramfs-tools/initramfs.conf
nmbl ( no more boot loader) might becoming to REHL, of course Ferdora get to be the lab rats first. The news is not universally welcome - Why though? [LWN.net]
