TrueNAS 24.10-RC.2 is Now Available!

Is there a way to manually restart the Kubernetes-to-Docker migration script? It failed on the initial run (see error below), and I can’t figure out how to kick off a retry.

[2024/10/06 01:05:09] (DEBUG) app_migrations.migrate():225 - Migration details for 'system-update--2024-10-06_05:01:35' backup on 'fast' pool
[2024/10/06 01:05:09] (DEBUG) app_migrations.migrate():231 - 'jellyfin' app failed to migrate successfully: 'Failed to migrate config: Failed to migrate config: Traceback (most recent call last):\n  File "/mnt/.ix-apps/truenas_catalog/trains/community/jellyfin/1.0.23/migrations/migrate_from_kubernetes", line 51, in <module>\n    print(yaml.dump(migrate(yaml.safe_load(f.read()))))\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/mnt/.ix-apps/truenas_catalog/trains/community/jellyfin/1.0.23/migrations/migrate_from_kubernetes", line 17, in migrate\n    "TZ": config["TZ"],\n          ~~~~~~^^^^^^\nKeyError: \'TZ\'\n'

See this:

I had the same issue.

My three Apps disappeared after update from RC.1 to RC.2, because the docker system was not initiated.

Booted back into RC.1 and everything is fine again.

Upgraded from 24.04.2.2 and all apps gone. Looks like migration script never ran.

Maybe that is because it is bugged for passphrase encrypted application pools? Only checks at boot but not after pool is unlocked? Just a guess

Edit: Reverted back to 24.04.2.2 and all apps are back.

Correct, it runs at boot once. If you have everything encrypted, you can unlock and manually trigger the migration with:

midclt call -job k8s_to_docker.migrate <poolname>

Some feedback from someone not using Scale, but using ZFS on Linux elsewhere, regarding the docker storage driver:

The performance difference between the old zfs driver and the overlay2 driver is monstrous. I have not done an apples-to-apples comparison, but deploying the Gitlab Omnibus image went from >5 minutes to <10 seconds (this is just the work that’s done after everything is downloaded and decompressed).
Now, big caveat, this is 2x2 NVMe mirrors versus a single mirror of SATA SSDs, so that’s going to account for some performance difference. But I’m still confident that overlay2 is going to be a much more pleasant experience.

So it doesn’t do funny things with datasets, snapshots and clones… Honestly, it’s probably better this way. The value that ZFS was adding with those features was marginal (the real value was not needing a dedicated partition for docker, or conceptually a zvol). At least zfs list isn’t completely polluted by endless docker datasets. I hate having to go back and always do zfs list -d3 just to get readable output.

This 100% ^^ We originally had the ZFS driver back in the early days of SCALE. But switching to overlayfs was a night and day difference.

Thanks. Appreciate the response.

Thanks!

I reported this issue a while back and glad someone took the time to submit the issue. Now to move everything from dockge back to TrueNAS.

How do updates work with custom apps? Will I get a notification when a newer image is available?

Were you ever able to diagnose or resolve this? I’m getting a nearly identical error where the storj migrator is trying to read an empty config.

[2024/10/06 13:44:07] (DEBUG) app_migrations.migrate():231 - 'storj' app failed to migrate successfully: 'Failed to migrate config: Failed to migrate config: Traceback (most recent call last):\n  File "/mnt/.ix-apps/truenas_catalog/trains/stable/storj/1.1.8/migrations/migrate_from_kubernetes", line 65, in <module>\n    print(yaml.dump(migrate(yaml.safe_load(f.read()))))\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/mnt/.ix-apps/truenas_catalog/trains/stable/storj/1.1.8/migrations/migrate_from_kubernetes", line 18, in migrate\n    old_wallets = config["storjConfig"].get("wallets", {})\n                  ~~~~~~^^^^^^^^^^^^^^^\nKeyError: \'storjConfig\'\n'

I’m wondering if there’s a way to increase verbosity or see the raw config that it’s reading in order to provide a more useful bug report.

EDIT: Tracked as [NAS-131627] - iXsystems TrueNAS Jira

Hi all,

I have upgraded my test machine to electric eel RC2 today. Basic functions seem to work as expected. However, I have two questions w.r.t. the new app system:

  1. How can I completely reset an app, such that all settings are deleted? E.g., when I install emby, configure it, then uninstall the app, and reinstall it, it is still configured. I.e., the admin account still exists and configurations I did in the initial install within the app are still there.
    Where is this data stored, and how can it be deleted?
  2. Specifically for jellyfin, I realize that “Host network” and the DLNA plugin do not seem to work properly. When I configure jellyfin with host network (which is needed for DLNA due to broadcasting), and then install the DLNA plugin, the server is not found by my media players in the LAN. Doing the same setup with emby, it works as expected.

Thank you for support and hints.

Really appreciate all the feedback. Those who are experiencing apps migration issues (or any Apps service issues) on RC2, it would be really helpful you can submit a bug (link in the TrueNAS UI) and provide a debugs.

2 Likes

Has anyone experimented with upgrading from Core 13.0 directly to Scale 24.10 yet?

My two TrueNAS systems have been on core for a long time. The addition of docker in 24.10 has sold me on upgrading to scale, though probably not until the release version, rather than RC.

I am seeing an oddity in the application list: if an application crashes, the status goes blank like so:

Also, unlike in Cobia/Dragonfish, there doesn’t seem to be a way to look at the logs for the failed container. Although running docker logs is pretty easy enough to do manually.

1 Like

No - its best to update via 24.04 at this stage.

Please submit a bug report and then let us know the NAS ticket.

https://ixsystems.atlassian.net/browse/NAS-131629

1 Like

Update went without a hitch … Plex is still working … so far, very uneventful :grinning:

This issue was fixed for me in RC2. Previously the Web GUI button for Portainer would open with 0.0.0.0 now it correctly uses the TrueNAS IP.

In case you want your APPs to update automatically, I’d use watchtower for that. In its config you can also only have it update specific apps.

I have been testing the custom apps feature in EE, but I ran into too many issues where FreeNAS does not tell me what actually went wrong. The error message it throws is totally useless and there is no way to access the docker log from the WebGUI.

So I just went with portainer (available as an APP) to manage my custom containers - which works like a charm.

2 Likes