Immich App Fails to Start After Upgrade (v1.7.47)

Apps and Virtualization
1

Was working fine until I attempted to upgrade to v1.7.47. Now the app fails to start with the below message. I also can’t rollback the app, as it needs to be running (who’s bright idea was that?!)

Any help or advice appreciated, thanks.

[EFAULT] Failed 'up' action for 'immich' app. Please check /var/log/app_lifecycle.log for more details

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 515, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 562, in __run_body
    rv = await self.middleware.run_in_thread(self.method, *args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 599, in run_in_thread
    return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 596, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 294, in nf
    rv = func(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/api/base/decorator.py", line 96, in wrapped
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 148, in do_create
    return self.create_internal(job, app_name, version, data['values'], complete_app_details)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 193, in create_internal
    raise e from None
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 186, in create_internal
    compose_action(app_name, version, 'up', force_recreate=True, remove_orphans=True)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/compose_utils.py", line 61, in compose_action
    raise CallError(err_msg)
middlewared.service_exception.CallError: [EFAULT] Failed 'up' action for 'immich' app. Please check /var/log/app_lifecycle.log for more details
2

Here’s some of the contents of /var/log/app_lifecycle.log >

Extracting [======>                                            ]  35.65MB/292.2MB\n adc7f3c741ae Download complete \nfailed to register layer: rename /mnt/.ix-apps/docker/image/overlay2/layerdb/tmp/write-set-2285767043 /mnt/.ix-apps/docker/image/overlay2/layerdb/sha256/af1c3c13c1706a442abdc9eb9347532b63fc2a2ebb4d83263356edd5739838a9: file exists\n
[2025/05/28 14:55:03] (ERROR) app_lifecycle.compose_action():56 - Failed 'up' action for 'immich' app:

Extracting [==============>                                    ]  84.12MB/292.2MB\n 2fae6551e81e Extracting [>                                                  ]  491.5kB/48.99MB\nfailed to register layer: rename /mnt/.ix-apps/docker/image/overlay2/layerdb/tmp/write-set-4071368185 /mnt/.ix-apps/docker/image/overlay2/layerdb/sha256/af1c3c13c1706a442abdc9eb9347532b63fc2a2ebb4d83263356edd5739838a9: file exists\n
3

See if you can shell in and delete that file, seems like it’s trying to pull something that’s written or not complete

rename /mnt/.ix-apps/docker/image/overlay2/layerdb/tmp/write-set-4071368185 /mnt/.ix-apps/docker/image/overlay2/layerdb/sha256/af1c3c13c1706a442abdc9eb9347532b63fc2a2ebb4d83263356edd5739838a9: file exists

4

Thanks Steven,

What file would I be attempting to delete?

/mnt/.ix-apps/docker/image/overlay2/layerdb/tmp/write-set-4071368185

or

/mnt/.ix-apps/docker/image/overlay2/layerdb/sha256/af1c3c13c1706a442abdc9eb9347532b63fc2a2ebb4d83263356edd5739838a9
5

The latter first, aka /mnt/.ix-apps/docker/image/overlay2/layerdb/sha256/af1c3c13c1706a442abdc9eb9347532b63fc2a2ebb4d83263356edd5739838a9, if not may have to do both

6

Thanks again Steve.

However, deleting that file didn’t work. I’m not sure I want to risk any more deletions. The other file looks to be in a temporary (in ‘/tmp/’ folder) anyway.

I’ve also raised an issue over on the GitHub, so hopefully between here and there I can get help to resolve.

7

Hooray, now resolved!

For anyone with this issue in the future, here’s the steps I made to fix.
Steve’s suggestion above was right, but I just didn’t take it far enough.
In the end I had to delete 5 folders under /mnt/.ix-apps/docker/image/overlay2/layerdb/sha256/

  1. Attempted to pull the latest image via shell.
sudo docker pull ghcr.io/immich-app/immich-server:commit-fd2c428faac7c956a78ffef8f0880f7a9ddbef61
  1. Noted which folder was reported to be the trouble maker and deleted it with
sudo rm -rf /mnt/.ix-apps/docker/image/overlay2/layerdb/sha256/af1c3c13c1706a442abdc9eb9347532b63fc2a2ebb4d83263356edd5739838a9
  1. Repeated step 1&2 until the image pulled without errors.

I had to delete 7 folders in all.

Then I was able to install the app without issue.

image
image877×457 39.9 KB

3 Likes