I have a new install of TrueNAS SCALE version 24.10.
Everything seems to be working find. It’s a new NAS and I don’t even have data on it, plenty of space both on the OS installation drives as well as the data drives I have setup.
I installed the first app “netdata” without any issues.
When I tried installing prometheus I got this error:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 509, in run
await self.future
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 556, in __run_body
rv = await self.middleware.run_in_thread(self.method, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1367, in run_in_thread
return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1364, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 268, in nf
rv = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 55, in nf
res = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 183, in nf
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 203, in do_create
return self.create_internal(job, app_name, version, data['values'], complete_app_details)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 248, in create_internal
raise e from None
File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 241, in create_internal
compose_action(app_name, version, 'up', force_recreate=True, remove_orphans=True)
File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/compose_utils.py", line 61, in compose_action
raise CallError(err_msg)
middlewared.service_exception.CallError: [EFAULT] Failed 'up' action for 'prometheus' app. Please check /var/log/app_lifecycle.log for more details
When I look at the error in the /var/log/app_lifecycle.log logs I see this:
[2025/03/10 08:28:29] (ERROR) app_lifecycle.compose_action():56 - Failed 'up' action for 'prometheus' app: Network ix-prometheus_default Creating
Network ix-prometheus_default Created
Container ix-prometheus-init-1 Creating
time="2025-03-10T08:28:28-07:00" level=warning msg="config `uid`, `gid` and `mode` are not supported, they will be ignored"
Container ix-prometheus-init-1 Created
Container ix-prometheus-prometheus-1 Creating
Container ix-prometheus-prometheus-1 Created
Container ix-prometheus-init-1 Starting
Container ix-prometheus-init-1 Started
Container ix-prometheus-init-1 Waiting
Container ix-prometheus-init-1 service "init" didn't complete successfully: exit 1
service "init" didn't complete successfully: exit 1
I know from reading around that TrueNAS is now using docker compose (I think) to run these services. I took a look at the app files available for prometheus and found these:
/run/middleware/ix-apps/catalogs/ix-dev/stable/prometheus/app.yaml
/run/middleware/ix-apps/catalogs/ix-dev/stable/prometheus/item.yaml
/run/middleware/ix-apps/catalogs/ix-dev/stable/prometheus/ix_values.yaml
/run/middleware/ix-apps/catalogs/ix-dev/stable/prometheus/questions.yaml
/run/middleware/ix-apps/catalogs/ix-dev/stable/prometheus/templates/docker-compose.yaml
/run/middleware/ix-apps/catalogs/ix-dev/stable/prometheus/templates/test_values/basic-values.yaml
/run/middleware/ix-apps/catalogs/ix-dev/stable/prometheus/templates/test_values/hostnet-values.yaml
/run/middleware/ix-apps/catalogs/trains/stable/prometheus/1.2.12/app.yaml
/run/middleware/ix-apps/catalogs/trains/stable/prometheus/1.2.12/ix_values.yaml
/run/middleware/ix-apps/catalogs/trains/stable/prometheus/1.2.12/questions.yaml
/run/middleware/ix-apps/catalogs/trains/stable/prometheus/1.2.12/templates/docker-compose.yaml
/run/middleware/ix-apps/catalogs/trains/stable/prometheus/1.2.12/templates/test_values/basic-values.yaml
/run/middleware/ix-apps/catalogs/trains/stable/prometheus/1.2.12/templates/test_values/hostnet-values.yaml
/run/middleware/ix-apps/catalogs/trains/stable/prometheus/item.yaml
My guess is that there is something off in the docker compose configuration that is not letting this app start up. I took a look at some of these files but they seem to be heavily templated and customized docker compose files. Is there a good way to test them? or run them from the command line instead of the GUI? Is there a way to see all the docker containers running from the command line?
And finally, if anyone knows what this specific issue is with prometheus, any help would be appreciated.