Iconik Storage Gateway App

Apps and Virtualization
1

Hello! I’m working on upgrading my current CORE solution, and a few client systems and they all are running various forms of Iconik Storage Gateway. On my test scale setup I haven’t been able to use the community application installation - I can however create a custom app just fine (or run from dockge/portainer). I can’t quite figure out why the community application isn’t working, but my installation errors are as follows:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 509, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 556, in __run_body
    rv = await self.middleware.run_in_thread(self.method, *args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1367, in run_in_thread
    return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1364, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/service/crud_service.py", line 268, in nf
    rv = func(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 55, in nf
    res = f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 183, in nf
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 203, in do_create
    return self.create_internal(job, app_name, version, data['values'], complete_app_details)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 248, in create_internal
    raise e from None
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/crud.py", line 233, in create_internal
    update_app_config(app_name, version, new_values)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/ix_apps/lifecycle.py", line 59, in update_app_config
    render_compose_templates(
  File "/usr/lib/python3/dist-packages/middlewared/plugins/apps/ix_apps/lifecycle.py", line 50, in render_compose_templates
    raise CallError(f'Failed to render compose templates: {cp.stderr}')
middlewared.service_exception.CallError: [EFAULT] Failed to render compose templates: Traceback (most recent call last):
  File "/usr/bin/apps_render_app", line 33, in <module>
    sys.exit(load_entry_point('apps-validation==0.1', 'console_scripts', 'apps_render_app')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/catalog_templating/scripts/render_compose.py", line 47, in main
    render_templates_from_path(args.path, args.values)
  File "/usr/lib/python3/dist-packages/catalog_templating/scripts/render_compose.py", line 19, in render_templates_from_path
    rendered_data = render_templates(
                    ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/catalog_templating/render.py", line 36, in render_templates
    ).render({'ix_lib': template_libs, 'values': test_values})
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/mnt/.ix-apps/app_configs/iconik-storage-gateway-cmnty/versions/1.0.7/templates/docker-compose.yaml", line 18, in top-level template code
    {% do c1.environment.add_env("AUTH_TOKEN", values.iconik.auth_token) %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 485, in getattr
    return getattr(obj, attribute)
           ^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'iconik'

I have filled out the variables the exact same as my custom working app - but still get the same result when testing the community app. I know this is all community based, and I really don’t know my way around GIT or the charts process, I’m a little over my head when it comes to that stuff!

I don’t think I get far enough in the process to see any app_lifescycle.log files either.

Thanks for any help from anyone - I know it’s a very specific app so probably not to many folks using it!

2

Sorry I didn’t respond earlier.

Did you resolve?

3

Hey - Thanks for the reply! I have not - I actually was tinkering with it yesterday, went as far as to fork the App Catalog to see if I could change the line or two needed, but didn’t realize I couldn’t add custom catalogs just yet to my test server!

As far as I can tell - my very limited knowledge and a little language-model help (!) - key named iconik in the values dictionary, but it’s not present in the actual values passed to the template in the docker-compose.yaml file. I sort of know what that means, but honestly not nearly skilled enough to be of help.

I can run from the custom apps either by entering in proper fields and building the image locally - even with straight yaml it works just fine - but I like all the easy of the install from the actual app catalog and all the bells and whistles it comes with for future me, and clients as well :slight_smile:

Not sure if it helps at all, but an example of a working YAML custom app install that works fine is something like this (I have it up and running on a few servers):

services:
  iconik-storage-gateway:
    container_name: iconik-storage-gateway
    cpus: '32.0'
    environment:
      - ICONIK_URL=https://app.iconik.io
      - AUTH_TOKEN=
      - APP_ID=
      - STORAGE_ID=
      - ICON_URL=
    image: iconik-storage-gateway:latest
    mem_limit: 180g
    mem_reservation: 77g
    restart: unless-stopped
    volumes:
      - /volumes:/here
4

Great that the custom app is working for you.

The Community app should work. If you want to try it on a system and troubleshoot, we’d like to help.

5

I’d love some help if possible! I have a test system that I’ve been trying to install - but just to make sure it’s not just that system I’ve got a production machine I’ve tried the same thing with.

When I try to install the app doesn’t generate log info other than the error I pasted at the top comment - I’ve also tried to get it working without any host path mounts or anything - so as basic as possible of an install but still get the same error.

Is there another log file I can post that might be helpful? Usually the app_lifecycle helps but since it’s failing early on in the install I didn’t see much else! Thanks so much for your willingness to help - I appreciate it!

6

I’d suggest starting with an App issue here:

Post the link and if we don’t see any response, I will escalate.

7

All taken care of - love it - glad I know the process now. Apprecaite the help and everything is working great!

8

Can you describe how it was fixed… so that others can find the information.

Glad it worked.

9

Posted the problem to GitHub and was taken care of very quickly - sorry I’m not exactly sure what was changed, but I think it was adding the “iconik” to the questions.yaml file (line 24):

  • variable: iconik

Sorry if this is a dummy question - but looks like the app is pulling a funky images - app version says v 3.12.0 (from this page - ghcr.io/truenas/iconik-storage-gateway-docker:3.12.0 - I can pull 3.12.0 manually with the custom app) - but upon successfully installing with the latest fix it’s pulling 3.11.3 somehow, but labeled as 3.12.0.

Is this something I can also post on github to have someone look fix or look at? Sorry - just making sure I’m helping, and not being annoying :slight_smile:

10

For individual app issues, github is like “reporting a bug”

If you find a solution or can’t get an issue resolved, post back here with a link.