Issues with App in TrueNAS Scale 24.10.1 and 24.10.2

Apps and Virtualization
, ,
1

Hello,

I recently upgraded my TrueNAS Scale Server from 24.04.x to 24.10.1 then to 24.10.2. I realized when I went to install Minio that I seem to be getting a repeated error coming from the applications installer. It reads that it failed to ‘down’ minio and to see /var/log/app_lifecycle.log. I can’t seem to figure out why I keep getting this user and group id integer issue. Any ideas?

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'

[2025/01/23 09:47:20] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'minio' app: 1 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'

[2025/01/26 22:16:50] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'photoprism' app: 1 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'

[2025/01/26 22:16:51] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'photoprism' app: 1 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'

[2025/01/26 22:22:53] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'photoprism' app: 1 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'

[2025/01/26 22:22:53] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'photoprism' app: 1 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'

[2025/01/26 22:30:19] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'scrutiny' app: 1 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'

[2025/01/26 22:30:19] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'scrutiny' app: 1 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'

[2025/01/28 14:04:46] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'home-assistant' app: 1 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'

[2025/01/28 14:04:46] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'home-assistant' app: 1 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'
2

Have you refreshed the apps catalog using the link in the UI?

3

Yes I have.

4

Just tried Portainer again after and I get this in the ui:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 509, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 556, in __run_body
    rv = await self.middleware.run_in_thread(self.method, *args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1367, in run_in_thread
    return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1364, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 268, in nf
    rv = func(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 55, in nf
    res = f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 183, in nf
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 203, in do_create
    return self.create_internal(job, app_name, version, data['values'], complete_app_details)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 248, in create_internal
    raise e from None
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 241, in create_internal
    compose_action(app_name, version, 'up', force_recreate=True, remove_orphans=True)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/compose_utils.py", line 36, in compose_action
    compose_action(app_name, app_version, 'down', remove_orphans=True)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/compose_utils.py", line 61, in compose_action
    raise CallError(err_msg)
middlewared.service_exception.CallError: [EFAULT] Failed 'down' action for 'portainer' app. Please check /var/log/app_lifecycle.log for more details

app_lifecycle.log:

[2025/01/28 14:45:42] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'portainer' app: 2 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'
* 'group_add[1]' expected type 'string', got unconvertible type 'int', value: '999'

[2025/01/28 14:45:42] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'portainer' app: 2 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'
* 'group_add[1]' expected type 'string', got unconvertible type 'int', value: '999'

I’ll add a couple of things I’ve tried too:

  • Unset and reset the pool and service
  • Checked Created users under /etc/passwd to make sure there is no conflicting uid or guid.
  • Reboot obviously
  • I have other hosts where this isn’t an issue.
5

This looks like you have uids in some configuration where it’s expecting strings.

6

Yes, where it should be…
Screenshot 2025-01-28 at 15.25.00

7

Same issue:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 509, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 556, in __run_body
    rv = await self.middleware.run_in_thread(self.method, *args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1367, in run_in_thread
    return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1364, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 268, in nf
    rv = func(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 55, in nf
    res = f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 183, in nf
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 203, in do_create
    return self.create_internal(job, app_name, version, data['values'], complete_app_details)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 248, in create_internal
    raise e from None
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 241, in create_internal
    compose_action(app_name, version, 'up', force_recreate=True, remove_orphans=True)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/compose_utils.py", line 36, in compose_action
    compose_action(app_name, app_version, 'down', remove_orphans=True)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/compose_utils.py", line 61, in compose_action
    raise CallError(err_msg)
middlewared.service_exception.CallError: [EFAULT] Failed 'down' action for 'minio' app. Please check /var/log/app_lifecycle.log for more details

app_lifecycle.log:

[2025/01/28 15:27:22] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'minio' app: 1 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'

[2025/01/28 15:27:23] (ERROR) app_lifecycle.compose_action():56 - Failed 'down' action for 'minio' app: 1 error(s) decoding:

* 'group_add[0]' expected type 'string', got unconvertible type 'int', value: '568'
8

Is that screenshot from another app? None of those errors are reporting 473

9

No both are minio. Where I started with this issue.

10

My guess is the error is referencing the app user…

apps:x:568:568:Unprivileged Apps User:/var/empty:/usr/sbin/nologin

11

It is, but I’m wondering where it is coming from in your config

12

Got me lol. Is a pretty basic box other wise. I literally just clicked initialize on creating the file structure in the pool and refreshed the app list and I get this with every app stable or community.

13

You lost me.

I just did a test install of Minio on 24.10.2 and it came up without issues.

14

I’ve tested on both 24.10.1 and 24.10.2 and neither worked

15

What is the output of docker info?

16 
Client: Docker Engine - Community
 Version:    27.1.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.23.3
    Path:     /root/.docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 27.1.1
 Storage Driver: overlay2
  Backing Filesystem: zfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.6.44-production+truenas
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 60.74GiB
 Name: masterassassin
 ID: 43d4db88-2f2f-4eec-bddf-af622b9b070c
 Docker Root Dir: /mnt/.ix-apps/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.16.0.0/12, Size: 24
17 
  compose: Docker Compose (Docker Inc.)
    Version:  v2.23.3
    Path:     /root/.docker/cli-plugins/docker-compose

This is a non-standard path for docker-compose binary. Have you modified the base install?

18

No I didn’t touch anything.

19

Were you using docker in 24.04 or otherwise experimenting with these things through shell in 24.04?

20

I don’t believe so.