LXC created in 25.04.x won't start in 26.0.0-BETA.1

TrueNAS General
1

internal error: guest failed to start: Failure in libvirt_lxc startup: cannot find init path '/sbin/init' relative to container root: No such file or directory

2

I am fairly certain that the upgrade migration script destroyed the LXC. Booting back into 25.10.3, I see ā€œno containersā€. Switching to 26.0.0-BETA.1 again, the container shows up, but is missing files and canā€™t start.

Iā€™ll keep this container around for a bit in case you want more data. You have a debug. Happily it was a small test only, and having lost it is no big deal.

3

Iā€™m seeing the same issue after upgrading to 26.0.0-BETA.1.

An old Ubuntu container that worked before no longer starts. At first I got:
ā€œFailed to access ā€˜/mnt/crucial-4tb/4TB-Crucial/Ubuntu-3ā€™: No such file or directoryā€

I fixed the ACL/path access part, and now the error changed to:
ā€œcannot find init path ā€˜/sbin/initā€™ relative to container root: No such file or directoryā€

I checked the container dataset on disk, and it looks broken/incomplete. For example, under:

/mnt/.truenas_containers/truenas-pool-1/containers/Ubuntu-1/

I only see:
.oldroot
data
data2
dev
proc
sys

There is no /sbin/init in the container root, and no sbin directory inside data or data2 either.

So it looks like the container rootfs was not migrated correctly, or the container metadata now points to an incomplete root.

Did anyone find a recovery method, or is this a known migration bug in 26.0.0-BETA.1?

4

You should use Report a Bug at the upper, top, right of the forum. Be sure to attach the debug dump. Post a link or ticket number back to the forum after youā€™re done.

5
  • Choose the 25.10.3 boot entry, get back to that
  • Restore the migrated dataset from snapshot, it should be <pool>/.ixvirt/containers/<lxc-name>

If you donā€™t have a snapshot, youā€™re out of luck

6

Iā€™m hit with this issue and I rolled back, now I think its in a worse situation, so when back to the BETA. I also normally snapshot everything, so either I didnā€™t think about it, or the migration killed them. Iā€™m confused about that bug report, it looks closed, but there are recent comments.

Iā€™d love troubleshoot this myself, but get the following with virsh. Are truenas actively preventing this!

sudo virsh list
error: failed to connect to the hypervisor
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
7

If you go back to 26.0.0-BETA.1, there is a workaround

From shell

sudo zfs inherit canmount <pool>/.truenas_containers/containers/<container-name>
sudo zfs mount <pool>/.truenas_containers/containers/<container-name>

The first command might say itā€™s not inheritable. Thatā€™s fine.

After this try to start the container again, it should come up

In 25.10.3 it might be gone tbh, unless you have a snapshot of <pool>/.ixvirt

BETA.2 will have this fix included

1 Like
8

Thanks,

I tired that earlier and it made no change, but now it worksā€¦ I have been floating between versions though.

I now need to resolve what appears to be similar to @Evgenii issue:

internal error: guest failed to start: Failure in libvirt_lxc startup: Failed to bind mount directory /.oldroot/mnt/nvme1/some/dir to /mnt/nvme1/some/dir: Operation not permitted

Iā€™ve not rebooted, but I assume that fix will currently need to be applied at each boot, until BETA-2. Is that bug report now officially closed? It looks like it was erroneously closed by an automation.

9

The report is closed because the engineer recorded the versions this is fixed in.

1 Like