Linux Jails (containers/vms) with Incus

Apps and Virtualization
186

No, it’s just a file I touched in a dataset. I can write files without issue, but can’t remove them.

stat /mnt/media/plex/docker1 
  File: /mnt/media/plex/docker1
  Size: 0         	Blocks: 1          IO Block: 131072 regular empty file
Device: 0,107	Inode: 822         Links: 1
Access: (0644/-rw-r--r--)  Uid: (  568/    apps)   Gid: (  568/    apps)
Access: 2025-03-11 17:15:25.386386069 +0000
Modify: 2025-03-11 17:15:25.386386069 +0000
Change: 2025-03-11 17:17:35.255450682 +0000
 Birth: 2025-03-11 17:15:25.386386069 +0000
187

Oh. I see. You don’t have write permissions in the directory in which the file is located.

188

Here is the config for the UI created instance. Only thing changed was adding recursive: true to the media mount.

incus config show docker-ui
architecture: x86_64
config:
  boot.autostart: "false"
  image.architecture: amd64
  image.description: Debian bookworm amd64 (20250310_05:24)
  image.os: Debian
  image.release: bookworm
  image.serial: "20250310_05:24"
  image.type: squashfs
  image.variant: cloud
  raw.idmap: |-
    uid 568 568
    gid 568 568
  user.autostart: "true"
  volatile.base_image: e2861d7b6322249446c1ef7f3ff6820a0420ee32a1007e3313aee71c93748e1e
  volatile.cloud-init.instance-id: 33e463eb-7b8c-456f-b81e-6b4336a0802b
  volatile.eth0.host_name: vethd4992e40
  volatile.eth0.hwaddr: 00:16:3e:92:71:d0
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":2147000001,"Nsid":0,"Maprange":568},{"Isuid":true,"Isgid":false,"Hostid":568,"Nsid":568,"Maprange":1},{"Isuid":true,"Isgid":false,"Hostid":2147000570,"Nsid":569,"Maprange":458183},{"Isuid":false,"Isgid":true,"Hostid":2147000001,"Nsid":0,"Maprange":568},{"Isuid":false,"Isgid":true,"Hostid":568,"Nsid":568,"Maprange":1},{"Isuid":false,"Isgid":true,"Hostid":2147000570,"Nsid":569,"Maprange":458183}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":2147000001,"Nsid":0,"Maprange":568},{"Isuid":true,"Isgid":false,"Hostid":568,"Nsid":568,"Maprange":1},{"Isuid":true,"Isgid":false,"Hostid":2147000570,"Nsid":569,"Maprange":458183},{"Isuid":false,"Isgid":true,"Hostid":2147000001,"Nsid":0,"Maprange":568},{"Isuid":false,"Isgid":true,"Hostid":568,"Nsid":568,"Maprange":1},{"Isuid":false,"Isgid":true,"Hostid":2147000570,"Nsid":569,"Maprange":458183}]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
  volatile.last_state.ready: "false"
  volatile.uuid: b4e5d079-6c48-452c-bc15-396790602c19
  volatile.uuid.generation: b4e5d079-6c48-452c-bc15-396790602c19
devices:
  disk0:
    path: /mnt/data
    source: /mnt/sol/data/apps
    type: disk
  disk1:
    path: /mnt/db
    source: /mnt/sol/database
    type: disk
  disk2:
    path: /mnt/containers
    source: /mnt/sol/containers
    type: disk
  disk3:
    path: /mnt/media
    recursive: "true"
    source: /mnt/sol/media
    type: disk
  disk4:
    path: /opt/stacks
    source: /mnt/sol/data/stacks
    type: disk
  gpu0:
    gputype: physical
    pci: 0000:2b:00.0
    type: gpu
ephemeral: false
profiles:
- default
stateful: false
description: ""
189

Let me test something…

190

No. You don’t understand. Your test is invalid. It wouldn’t work on the host either because you don’t have write permissions to the directory where the file is located. This is a basic unix permissions problem.

191

Right, I see that and I do understand that. That is what I was correcting.

$ id
uid=568(apps) gid=568(apps) groups=568(apps)

drwxr-xr-x   2 apps   apps          2 Mar 11 18:09  docker1

incus exec docker-ui bash

cd /mnt/media/plex/docker1

su apps

$ cd docker1
$ touch docker 
$ ls
docker
$ ls -la
total 10
drwxr-xr-x  2 apps   apps     3 Mar 11 18:10 .
drwxr-xr-x 13 nobody nogroup 18 Mar 11 18:09 ..
-rw-r--r--  1 apps   apps     0 Mar 11 18:10 docker
$ rm docker

It’s working now.

So basically, I need to map any users/groups I want to use from the host in the container.

192

What’s the actual midclt required to set the idmap to direct? I’d prefer to use the existing users and groups setup with permissions rather than have to change everything to apps user.

I tried below but it doesn’t actually apply the map inside a container on rebooting the container.

midclt call user.update 74 ‘{“userns_idmap”: “DIRECT”}’

193

Or just map groups. They’re generally a much better choice for handling permissions, and in most places is considered best-practice. It’s easy to control access by just making a user a member of a group. Once a user no longer needs access to something you remove them from the group. This avoids the need to ever have to change permissions.

2 Likes
194

Do you know that 74 maps to the DB id of the user you’re interested in? midclt call user.get_instance 74

If it does, then did you restart through our UI / API or through the incus command? We only update the instance idmap on restart through our UI / API (because we don’t want to mess with running instances without admin interaction).

195

This is how I do mine currently and what I’m hoping to test.

I’ve got 3 users setup for owning the actual application configuration data and 2 groups setup for permissions to shared data between containers.

uid 300 gid 300
uid 301 gid 301
uid 302 gid 302
gid 1002
gid 1003

On the host and in containers the uids are proper members of the groups 1002/1003 if required.

ID 74 does match to the correct uid 301 in the list above.

root@truenas:~# midclt call user.update 74 '{"userns_idmap": "DIRECT"}' | jq
{
  "id": 74,
  "uid": 301,
  "username": "incus-media",
  "unixhash": "*",
  "smbhash": "*",
  "home": "/mnt/data1/homes/incus-media",
  "shell": "/usr/bin/bash",
  "full_name": "incus-media",
  "builtin": false,
  "smb": false,
  "userns_idmap": "DIRECT",
  "group": {
    "id": 118,
    "bsdgrp_gid": 301,
    "bsdgrp_group": "incus-media",
    "bsdgrp_builtin": false,
    "bsdgrp_sudo_commands": [],
    "bsdgrp_sudo_commands_nopasswd": [],
    "bsdgrp_smb": false,
    "bsdgrp_userns_idmap": 0
  },
  "groups": [
    46,
    47
  ],
  "password_disabled": true,
  "ssh_password_enabled": false,
  "sshpubkey": null,
  "locked": false,
  "sudo_commands": [],
  "sudo_commands_nopasswd": [],
  "email": null,
  "id_type_both": false,
  "local": true,
  "immutable": false,
  "twofactor_auth_configured": false,
  "sid": null,
  "roles": [],
  "api_keys": [],
  "password": null
}

Let me try again the instance I am testing with does not have shared datasets right now just a configuration dataset.

196

These APIs do not create users for you in the containers. They are about setting up idmaps only. As always, the onus is on the administrator to create appropriate users and groups in a container.

197

Right, makes sense. I should’ve said user/groups. Thank you for making things clearer. :slight_smile:

198

I tested with different instance this time that doesn’t have any shared datasets but it doesn’t look like it mapped the group as well in.

$ ls -la
total 61
drwx------  5 thelounge nogroup     9 Feb 26 13:31 .
drwxr-xr-x 78 root      root      152 Mar  5 06:49 ..
-rw-r--r--  1 thelounge nogroup 16874 Jan  1  2022 config.js
-rw-r--r--  1 thelounge nogroup 20047 Apr  6  2024 config.js.dpkg-dist
drwxr-xr-x  3 thelounge nogroup     4 Mar 11 13:23 logs
drwxr-xr-x  4 thelounge nogroup     6 Jan  1  2022 packages
-rw-r--r--  1 thelounge nogroup   212 Mar 11 11:37 sts-policies.json
drwxr-xr-x  2 thelounge nogroup     3 Mar 10 23:16 users
-rw-r--r--  1 thelounge nogroup   169 Jan  1  2022 vapid.json
$ id
uid=300(thelounge) gid=300(thelounge) groups=300(thelounge)
$

root@truenas:~# midclt call user.get_instance 73|jq
{
  "id": 73,
  "uid": 300,
  "username": "incus-general",
  "unixhash": "*",
  "smbhash": "*",
  "home": "/mnt/data1/homes/incus-general",
  "shell": "/usr/bin/bash",
  "full_name": "incus-general",
  "builtin": false,
  "smb": false,
  "userns_idmap": "DIRECT",
  "group": {
    "id": 117,
    "bsdgrp_gid": 300,
    "bsdgrp_group": "incus-general",
    "bsdgrp_builtin": false,
    "bsdgrp_sudo_commands": [],
    "bsdgrp_sudo_commands_nopasswd": [],
    "bsdgrp_smb": false,
    "bsdgrp_userns_idmap": 0
  },
  "groups": [],
  "password_disabled": true,
  "ssh_password_enabled": false,
  "sshpubkey": null,
  "locked": false,
  "sudo_commands": [],
  "sudo_commands_nopasswd": [],
  "email": null,
  "id_type_both": false,
  "local": true,
  "immutable": false,
  "twofactor_auth_configured": false,
  "sid": null,
  "roles": [],
  "api_keys": []
}
199

it doesn’t look like it mapped the group as well in.

It doesn’t look like you tried to map the group. Mapping a user via this does not automatically map its primary group.

200

I was just about to reply I see once updated the group setting as well it worked properly.

midclt call group.update 119 ‘{“userns_idmap”: “DIRECT”}’

Time for some more testing of the more complex instances but that is one instance no longer needing shift or privileged.

201

Here’s a simple script, not much work put into it. I’m sure it can be improved greatly.

Anyway, it will help you grab the id for the user that you want to use with idmap. Just copy and paste it into a file and save it as something like get-user-instances.sh

Script:

#!/bin/bash

for i in {1..100}
do
  USER=`midclt -q call user.get_instance $i` &>/dev/null 
  [[ -z $USER ]] && continue
  echo $USER
done

If you want to just want to dump all of that, you can do:

get-user-instances.sh

Prettier output:

get-user-instances.sh|jq

Search for specific users:

get-user-instances.sh|grep -E 'user1|user2|user3'

Search for specific users (prettier):

get-user-instances.sh|grep -E 'user1|user2|user3'|jq

Example output:

get-user-instances.sh|grep -E 'apps'|jq
{
  "id": 66,
  "uid": 568,
  "username": "apps",
  "unixhash": "*",
  "smbhash": "*",
  "home": "/var/empty",
  "shell": "/usr/sbin/nologin",
  "full_name": "Unprivileged Apps User",
  "builtin": true,
  "smb": false,
  "userns_idmap": "DIRECT",
  "group": {
    "id": 96,
    "bsdgrp_gid": 568,
    "bsdgrp_group": "apps",
    "bsdgrp_builtin": true,
    "bsdgrp_sudo_commands": [],
    "bsdgrp_sudo_commands_nopasswd": [],
    "bsdgrp_smb": false,
    "bsdgrp_userns_idmap": "DIRECT"
  },
  "groups": [],
  "password_disabled": false,
  "ssh_password_enabled": false,
  "sshpubkey": null,
  "locked": false,
  "sudo_commands": [],
  "sudo_commands_nopasswd": [],
  "email": null,
  "id_type_both": false,
  "local": true,
  "immutable": true,
  "twofactor_auth_configured": false,
  "sid": null,
  "roles": [],
  "api_keys": []
}

id only:

get-user-instances.sh|grep -E 'apps'|jq '.id'
66
202

try midclt call user.query | jq

1 Like
203

Well, crud… :joy:

204

also

midclt call user.query '[["username","=","apps"]]' | jq

205

and id:

midclt call user.query '[["username","=","apps"]]' | jq '.[].id'
66