Apps and Datasets GUI Broken, Middleware Log Shows Same Error Every Time

Apps and Virtualization
1

I’ve got an issue where I cannot use the GUI any more for Datasets nor Apps. Every time I browse to those pages I get the following error:

Error: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/api/base/server/ws_handler/rpc.py", line 323, in process_method_call
    result = await method.call(app, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/api/base/server/method.py", line 52, in call
    result = await self.middleware.call_with_audit(self.name, self.serviceobj, methodobj, params, app)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 911, in call_with_audit
    result = await self._call(method, serviceobj, methodobj, params, app=app,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 731, in _call
    return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 624, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/api/base/decorator.py", line 101, in wrapped
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 72, in query
    apps = list_apps(available_apps_mapping, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/ix_apps/query.py", line 159, in list_apps
    **app_metadata | {'portals': normalize_portal_uris(app_metadata['portals'], host_ip)}
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/ix_apps/query.py", line 67, in normalize_portal_uris
    return {name: normalize_portal_uri(uri, host_ip) for name, uri in portals.items()}
                                                                      ^^^^^^^^^^^^^
AttributeError: 'list' object has no attribute 'items'

I know exactly what caused it. I manually edited a metadata.yaml file for a custom app I created. The edit I made was incorrect, and I restarted the app. Now all subsequent visits to those 2 pages fail with that error. I’ve since reverted the changes to that yaml file, but I cannot perform any related actions to revert it with midclt because it fails with the same error in the screenshot. So I can’t stop/start/create/delete etc. through the middleware. I’m guessing the issue is in the database?

I’m looking for guidance on how I can clear what’s in the db related to the app metadata. Or if I’m going down the wrong path, any guidance on the correct way to handle this. I’ve already restarted the server as last ditch effort, but as you can probably guess, that hasn’t helped.

2

Forgot to mention that I’m on 25.04.2.5

3

Nevermind. Figured it out. There’s a collective metadata file under /mnt/.ix-apps/metadata.yml. I’m guessing that gets rebuilt on every app change that the middleware picks up. I fixed the bad syntax in that and all is working again.

4

Dang where was this thread 2 weeks ago when I was in the trenches with this son of a b**** of an error.

Wish there was a rebuild metadata cronjob or something that could automatically fix these kinds of problems.

Nice job zeroing in on the root file. I don’t remember editing that file directly when it came up for me. I ended up finding a miracle snapshot that testing goldeye beta(or RC?) created on just the right date/time to be a reliable solution.

1 Like