TrueNAS SCALE Electric Eel "Call for App Testing"

E_B · August 28, 2024, 3:44pm

I thought the warning might need to be a “standard” requirement - all understood.

I’m having a go at submitting the query “bug”. It may not be eloquent …

NickF1227 · August 28, 2024, 9:33pm

Immich cleanly transitioned over for me without a hiccup.

Go_blin · August 29, 2024, 8:16am

[EFAULT] Failed to render compose templates: Traceback (most recent call last): File "/usr/bin/apps_render_app", line 33, in sys.exit(load_entry_point('apps-validation==0.1', 'console_scripts', 'apps_render_app')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/catalog_templating/scripts/render_compose.py", line 47, in main render_templates_from_path(args.path, args.values) File "/usr/lib/python3/dist-packages/catalog_templating/scripts/render_compose.py", line 19, in render_templates_from_path rendered_data = render_templates( ^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/catalog_templating/render.py", line 36, in render_templates ).render({'ix_lib': template_libs, 'values': test_values}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1301, in render self.environment.handle_exception() File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 936, in handle_exception raise rewrite_traceback_stack(source=source) File "/mnt/.ix-apps/app_configs/immich/versions/1.0.7/templates/docker-compose.yaml", line 36, in top-level template code {% if values.immich.enable_ml %} ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 485, in getattr return getattr(obj, attribute) ^^^^^^^^^^^^^^^^^^^^^^^ jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'immich'

Hello, I installed immich on 24.10 and the above error occurred. What’s the problem?

ABain · August 29, 2024, 12:22pm

Confirmed issue, fix is currently gong through final checks.

j0hnby · August 29, 2024, 1:10pm

I did both a test system, then moved my main over to EE (I know - risky, nothing I can’t face re-creating for the apps on there, I have it all backed up, and have had to re-create enough times when using other charts I have it down to an artform)

The update went flawlessly. My apps all converted (except the ones that are not currently supported for conversion or not “done” in the apps list - for those I had already backed up their config and taken screenshots of the settings so I can re-create them)

I only had two confusing moments.

getting apps to communicate to each other - previously I used ix-appname.ix-appname.cluster.local (or similar) - now it has to be the physical interface of the server. I think that’s because all of the apps have their own separate networks in Docker - so can’t speak to each other (but that’s only with a couple of hours of Google/YouTube and playing about kind of knowledge). I tried appname.networkname, and just appname (found via Portainer) but didn’t have any luck with DNS resolution or ping/connections.
My NVIDIA graphics card doesn’t show up in the Apps (I have enabled the drivers in Settings) - but there is already a Jira issue for that so will just wait.

Really good so far!

Edited: My average CPU usage is down too vs Dragonfish! Woop woop.

PackElend · August 31, 2024, 7:29pm

Hi there,
can I assign apps to certain network interfaces? I would like to do VLAN segregation.
Or do I still have to go with Jailmaker?
thx
stefan

Trudge · August 31, 2024, 7:47pm

Will Electric Eel implement … a first party application backup and restore system at a per app level?

Jorsher · August 31, 2024, 7:58pm

I wasn’t sure if this is an ‘issue’ or something I can correct myself, though I think future releases should increase the limits.

So far, the new Apps are working great. They do start up and shut down faster with a noticeably lower amount of overheard. I miss the network/cpu/memory stats from Dragonfish, but in github it looks like there is already work in progress to add the stats later.

The issue is that at around ~30 running apps, I’m no longer able to start more:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 469, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 513, in __run_body
    rv = await self.middleware.run_in_thread(self.method, *args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1356, in run_in_thread
    return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1353, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 183, in nf
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 55, in nf
    res = f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/app_scale.py", line 36, in start
    compose_action(app_name, app_config['version'], 'up', force_recreate=True, remove_orphans=True)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/compose_utils.py", line 52, in compose_action
    raise CallError(f'Failed {action!r} action for {app_name!r} app: {cp.stderr}')
middlewared.service_exception.CallError: [EFAULT] Failed 'up' action for 'qb-test' app:  Network ix-qb-test_default  Creating
 Network ix-qb-test_default  Error
failed to create network ix-qb-test_default: Error response from daemon: all predefined address pools have been fully subnetted

‘docker network inspect bridge’ shows a /16 so I’m not sure where the limitation is. I’m new to docker so may not be looking in the right place.

What’s curious to me is why inspecting each ‘app’ network, they each seem to be given either a /20 or a /16 and I’m not sure why they are given such a large subnet.

BjornRuud · September 5, 2024, 1:29pm

I upgraded to Electric Eel on my home NAS and there were no issues. Everything seems to be running as before. I had Jellyfin and Nextcloud apps installed and they were both migrated without issue.

I installed Nginx Proxy Manager and used the data and cert dirs from a previous installation. The app runs fine but it takes a very long time to go from “deploying” to “running”. The log indicates it’s the “setting ownership” phase that is slow:

2024-09-05 13:12:12.426107+00:00e[1;34m❯ e[1;36mConfiguring npm user ...e[0m
2024-09-05 13:12:12.703327+00:00useradd warning: npm's uid 568 outside of the UID_MIN 1000 and UID_MAX 60000 range.
2024-09-05 13:12:12.763053+00:00e[1;34m❯ e[1;36mConfiguring npm group ...e[0m
2024-09-05 13:12:12.799395+00:00e[1;34m❯ e[1;36mChecking paths ...e[0m
2024-09-05 13:12:12.816989+00:00e[1;34m❯ e[1;36mSetting ownership ...e[0m
2024-09-05 13:18:26.912666+00:00e[1;34m❯ e[1;36mDynamic resolvers ...e[0m
2024-09-05 13:18:26.916367+00:00e[1;34m❯ e[1;36mIPv6 ...e[0m

Stux · September 5, 2024, 6:59pm

This is true, but it does finish at least

Tsaukpaetra · September 6, 2024, 7:04am

Yeah, there’s an issue about that:

github.com/NginxProxyManager/nginx-proxy-manager

First time boot takes incredibly long due to chown -R "$PUID:$PGID" /opt/certbot

opened 11:20PM - 23 Aug 23 UTC

devedse

bug

**Checklist** - Have you pulled and found the error with `jc21/nginx-proxy-manager:latest` docker image? - Yes - Are you sure you're not using someone else's docker image? - Yes - Have you searched for similar issues (both open and closed)? - Yes **Describe the bug** When I start the container for the first time it's taking an incredibly long due to it setting ownership: ``` nginx-proxy-manager | ❯ Configuring npm user ... nginx-proxy-manager | ❯ Configuring npm group ... nginx-proxy-manager | ❯ Checking paths ... nginx-proxy-manager | ❯ Setting ownership ... ``` I did a bit of investigation and found out it's due to this command: ``` chown -R "$PUID:$PGID" /opt/certbot ``` This code was added in this commit: https://github.com/NginxProxyManager/nginx-proxy-manager/commit/05307aa253c073cf94237fc96d816ec2919f4d7f I ran this command myself inside the container and it took about 5 minutes to complete: ![image](https://github.com/NginxProxyManager/nginx-proxy-manager/assets/2350015/309149e4-73d3-4494-b1b1-7dcb87fca6d2) To me it feels a bit strange to have to execute this command on the /opt folder which isn't even mapped to my host system. So why not do this during the creation of the container rather then during boot? **Nginx Proxy Manager Version** 2.10.4 **To Reproduce** See above **Expected behavior** Boot quickly **Screenshots** N/A **Operating System** Docker inside a container on a Synology NAS **Additional context**

It’s apparently taking ownership of everything in /opt/certbot so that it doesn’t complain about not being root. ~~Shouldn’t do it after the first launch though~~.

Edit: No I guess it still does unless you apply a workaround. Disappointing lmao.

awalkerix · September 6, 2024, 1:30pm

This practice is unfortunately commonplace.

IIRC one needs to be particularly careful about anything provided by linuxserver.io since practice there is to recursively chown (possibly chmod) everything. It has been a while so I’m a little fuzzy on the details, but it was particularly bad with it borking user permissions.

Tsaukpaetra · September 6, 2024, 2:23pm

That’s distressing. I able have thought that brainstorm in Windows world would have stayed there, but I guess you can’t fix stupid…

I loathe every “App” that can’t figure out separation of program data and user data and this “solves” it by shoving everything into Application Data…

etorix · September 6, 2024, 3:40pm

Many Linux heads appear to think that if Windows does “it”, they should do the same to take the market over… “It” being any bad decision about user interface or program design.

RetroG · September 6, 2024, 9:39pm

believe it or not… one of the things Windows does is have ACLs by default, instead of the limited and fragile posix/linux permissions model and which the blind chown startup/install scripts that exist because of (and it’s not just linuxserver either).

how often do typically you have to change ownership of a file (or worry about it in the first place) on a windows box? a lot less… how often do ACLs and ownership conflict on a windows box? almost never unless you are messing with system files.

sure you get the odd legacy application that assumes program files is writable, but that’s not really comparable to the Linux ecosystem where similar legacy applications just don’t usually exist untouched.

awalkerix · September 7, 2024, 12:05am

If you use NFSv4 acltype on your datasets it’s a windows-equivalent ACL model. At the end of the day this recursive chown / chmod nonsense boils down to bad development practices IMHO.

Tsaukpaetra · September 7, 2024, 2:56am

Users don’t because Windows will just go “Hey, want to auto-sudo to continue?”, and programmers saw that as “well we don’t want to annoy users so let’s stuff everything where we don’t have to do it right and potentially annoy users”

But that’s neither here nor there, let’s test more apps!

Tsaukpaetra · September 7, 2024, 5:47am

Actual feedback:

A few times, I found it a bit annoying digging through the creation form when an error occurred (typically when a port was already used or a storage wasn’t specified in the case of filebrowser). It would be nice if the offending line item were marked invalid on the form when that happens during check.

BjornRuud · September 9, 2024, 3:42pm

Installation of the Joplin app on BETA.1 fails:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 469, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 513, in __run_body
    rv = await self.middleware.run_in_thread(self.method, *args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1356, in run_in_thread
    return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1353, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 268, in nf
    rv = func(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 55, in nf
    res = f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 183, in nf
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 195, in do_create
    return self.create_internal(job, app_name, version, data['values'], complete_app_details)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 244, in create_internal
    raise e from None
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 232, in create_internal
    compose_action(app_name, version, 'up', force_recreate=True, remove_orphans=True)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/compose_utils.py", line 52, in compose_action
    raise CallError(f'Failed {action!r} action for {app_name!r} app: {cp.stderr}')
middlewared.service_exception.CallError: [EFAULT] Failed 'up' action for 'joplin' app:  Network ix-joplin_default  Creating
 Network ix-joplin_default  Created
 Container ix-joplin-permissions-1  Creating
 Container ix-joplin-permissions-1  Created
 Container ix-joplin-postgres-1  Creating
 Container ix-joplin-postgres-1  Created
 Container ix-joplin-joplin-1  Creating
 Container ix-joplin-joplin-1  Created
 Container ix-joplin-permissions-1  Starting
 Container ix-joplin-permissions-1  Started
 Container ix-joplin-permissions-1  Waiting
 Container ix-joplin-permissions-1  Exited
 Container ix-joplin-postgres-1  Starting
 Container ix-joplin-postgres-1  Started
 Container ix-joplin-postgres-1  Waiting
 Container ix-joplin-permissions-1  Waiting
 Container ix-joplin-permissions-1  Exited
 Container ix-joplin-postgres-1  Error
dependency failed to start: container ix-joplin-postgres-1 is unhealthy

DjP-iX · September 9, 2024, 5:52pm

Can you show your configuration for the app? I just did a test install on BETA.1 and it deployed without issue.