TrueNAS 26 Beta 1 LXC: Docker real images fail with “failed to register layer: operation not permitted”; /dev/fuse unavailable

Okay, I’ve found a very, very strange workaround. Create a container with ID Map Type = Privileged and Capabilities Policy = Allow All as before, then use virsh edit to manually add the following under <domain>…</domain>:

  <idmap>
    <uid start='0' target='0' count='2147483648'/>
    <gid start='0' target='0' count='2147483648'/>
  </idmap>

Then restart the container with virsh (not via TrueNAS GUI, or it’ll wipe out your changes).

After that, Docker works fine with the default storage driver! No need to pass /dev/fuse in and change to fuse-overlayfs. And since the idmap is basically 1:1, there’s no file permission conflicts vs. the host. A container run as UID 2999 should be able to read and write files on host mounts using that UID.

But… why?! It seems that with a new userns (even one that’s effectively a 1:1 mapping like I used above), Docker works fine. But I truly have no idea why it works, and unfortunately, there’s not many up-to-date docs about the libvirt variant of LXC (which it kind of seems like the rest of the world has moved on from?).

Anyway, TrueNAS will wipe out the custom container XML every time it touches the container (including, I think, on boot). So I think some TrueNAS fix is still needed here. But it’s… progress?