TrueNAS SCALE Electric Eel "Call for App Testing"

TrueNAS Electric Eel (aka SCALE 24.10) introduces Docker Compose capabilities within Apps. With this release, industry-standard Docker Compose applications can now be easily integrated with TrueNAS and ZFS. Electric Eel nightly ISOs are available for testing and the BETA1 version is expected to be ready next week (ETA August 27th).

Electric Eel Will Automatically Convert Apps to Docker Compose

TrueNAS 24.10 automates the migration process for TrueNAS charts and standard Docker Apps from the current Kubernetes back-end to Docker Compose. Apps like NextCloud, Plex, and hundreds of others will reboot with the TrueNAS Electric Eel update in their new configuration, but their software, data, and networking will be preserved.

This is how it’s been planned out, and how it’s functioning in the TrueNAS test labs; however, we’d like to hear real-world feedback from our community.

If you’re a TrueNAS community user with the ability to build a test system - bare-metal, or in a VM - we would appreciate any reviews, bug reports, and general feedback. Even if your experience is perfectly seamless; please let us know! The Apps ecosystem is undergoing a large change and we want to present a well-vetted solution with the help of the community.

App Testing Needed

There are several types of scenarios where testing is needed:

App Deployment - New apps are deployed from the standard TrueNAS catalog

App Migration - Existing TrueNAS Apps (edit: but not Docker Custom Apps) are migrated during a software upgrade from Dragonfish

At a later date, we’ll ask for more complex migration scenarios, but we want to ensure the fundamentals are solid first. For each of these scenarios, we recommend different approaches.

App Deployment

The Electric Eel Apps available during BETA1 will be a subset of the Dragonfish Apps, as we continue to populate the new catalog. Before planning a new test system, check that your desired Apps are available by looking at the “added” column of our Apps repo on Github. If you’re missing a crucial App, you’ll have to wait until it’s been added - or perform your testing without it for the moment.

Download the new Electric Eel .iso file and install a new system. You can use bare-metal hardware or a VM - even a VM set up within an existing TrueNAS SCALE Dragonfish installation.

Set up your system as normal, creating a pool, datasets, and file shares.

After you have file shares created, you can deploy Apps and Docker Containers. Your testing should verify that each App deployed successfully. Please let us know which Apps succeeded and report a bug for each App that did not.

We expect new Docker-powered App deployments to be 2-3x faster than the previous Kubernetes installations. Let us know how your experience was - we don’t need a stopwatch, but if you’re willing to break one out, more power to you!

If you feel particularly adventurous, you can try the new Docker Compose management Apps Dockge and Portainer, and use them to hook into the new TrueNAS SCALE Apps framework.

App Migration

The previous testing assumed a fresh install of TrueNAS 24.10. While that’s an important test scenario for newcomers, our existing TrueNAS users will want to ensure a smooth migration of Apps from Dragonfish to Electric Eel.

While the software framework for the migration to Docker is complete, and many of the Apps in the catalogs have an associated migration script, some Apps will need to be completed closer to RELEASE. The list of completed Apps and their migration script status can be found on our Github here. If you’re currently using specific TrueNAS Apps, please don’t upgrade until they have been completed, as indicated by a checkbox in the “migrated” column.

When you are ready to test, review the performance of your current Apps and the load on your system for future comparison. Download the .iso or update bundle and perform a manual update. SCALE does have boot environments for easy downgrades, but you can also make a checkpoint of your entire pool or manually take snapshots of your primary datasets before the upgrade.

As part of the app migration process, Electric Eel takes a snapshot of the App and clones it during the upgrade from Dragonfish. The cloned dataset is then manipulated to convert the existing Kubernetes data to Docker Compose format. This migration operation is performed on one App at a time, based on migration information in the TrueNAS catalog. If the migration information is not present, the App will not be migrated and will not restart. All Apps that are migrated will successfully restart after the upgrade.

Because the Apps dataset is automatically snapshotted and cloned, Electric Eel maintains the ability of TrueNAS to rollback after an upgrade, including your Applications. This allows for an easy and complete restoration of your previous Dragonfish installation if desired. Even if your upgrade went smoothly, this is something that adventurous users can perform manually, just to help validate any potential edge cases.

If you have upgraded prior to all your Apps being migratable, there is a mechanism (documented in the BETA Release notes) to re-run the migration later. This will attempt to migrate any Apps that previously had no migration script.

Because each user has unique hardware, Apps, networking, and disk configurations, the TrueNAS QA team cannot cover every single test case; which is why we’re asking for additional community testing. Feedback on the various aspects of App migration would be greatly appreciated, including:

  • Apps that migrated successfully
  • Apps that migrated unsuccessfully or partially
  • Apps that did not migrate at all
  • Rollback success or failure
  • Reattempted Migration success or failure
  • App performance after upgrade
  • System utilization (CPU%) changes after the upgrade
  • How long did the migration process take?

Thanks to Community

Thanks in advance to all those users who help with this testing. We appreciate those with the interest, energy, and time to ensure that the upgrade process is smooth, easy, and non-disruptive.

3 Likes

Is this supposed to link to a download or something?

EDIT: The link is working now. Thank you

Bleh, still had an old URL in there. Fixed it up, it should point to https://download.sys.truenas.net/ now.

Note that nightly ISOs/updates are intended purely for testing - they’re even more “freshly baked” than BETA, and functionality can change from day to day. We’d recommend that only the most eager users jump on the Nightlies, but the BETA releasing next week should have broader appeal.

3 Likes

I’ve installed multiple apps just fine using the August 19th nightly.

4 Likes

I was gonna test this in VirtualBox - Which Debian version is EE based on?

Bookworm (12) x64

1 Like

I understand this is an early release, and much work is still to be done…

I have a few questions

Is there a way to load my own docker-compose file (now or planned for later)?

I understand the appliance aspect of TrueNAS - but I also know that some tasks can still be done using the command line (even if not recommended)

Can I safely use my own docker-compose file with the following run from the CLI?

docker compose -f /path/to/file up -d

Thanks

Yes, you can use docker command from the CLI. Additionally you can install a tool like dockge or potainer from the default app catalog if you wish a nice GUI method to manipulate docker compose things :slight_smile:

1 Like

I test the Dockge app, using it for compose and docker run, after migrating from a sandbox in this video.

It’s a very good way to run custom apps on TrueNAS

4 Likes

Just for grins, lets play a game to see if you can spot where I did the upgrade and converted from k8s to docker:

3 Likes

Wow.

1 Like

Note: This is on a system with a handful of Apps, and a HAOS VM running. So, not exactly “idle”, but the k3s overhead was unmistakable.

3 Likes

Good ridance!

are there any plans to implement “Docker Custom App” migration? or at the very least a way to view the settings as they existed?

a good number of my apps are this kind of deployment due to the general broken-ness of the ix charts or just weren’t available… it would be a giant pain to have to put all the values in from memory.

:point_up_2:

I’d be very pleased to join in too - a couple of days ago I got hold of an “EE” nightly release and stuck it in a virtualbox VM just to see what happened.

I’ll update it to today’s release and I’ll play around with docker and apps and dockge etc. during the first part of next week. I’ll try to provide germane feedback (noting that I am not very much of an expert at linux, docker or TrueNAS; merely a hobby user).

Exciting!

I would love to help testing, but I’m blocked with an early live-boot error using EE nightly isos on my server. Looks like it is trying/failing to find/load the md drivers:

find: /lib/modules/6.6.44-production+truenas/kernel/drivers/md: No such file or directory

This is followed by other secondary errors up the stack locating device-mapper and failed insmod, mknod calls.

This occurs on my system with both of the following builds:

  • MASTER-20240822-035851
  • MASTER-20240716-035108

…but does not occur on the same system with the latest stable:

  • 24.04.02

Any ideas? I am not actually using raid on this system, so I could enter my BIOS and make sure RAID is disabled, but it is curious that whatever the root cause is, that 24.04.02 is robust to it, and neither of the two most recent 24.10.* nightlies are.

EDIT: I think the problem was with Ventoy booting. Burning the latest 20240822 nightly iso directly onto a different usb stick with dd now boots. :-/

1 Like

FWIW, ElectricEel-24.10.0-MASTER-20240827-035855 is working well for me so far.

I’ve already noticed a few bug fixes I reported against 0816/0819 :wink:

(1) I upgraded my virtualbox installlation of EE from a week or so ago, using the TrueNAS UI (version is now ElectricEel-24.10.0-MASTER-20240827-035855) and then I

(2) installed dockge (from the built-in App catalogue) as my first port of call, which was painless. I noticed the fairly frightening warning note at the start of the app installation

iXsystems does not guarantee application safety or reliability, and such applications may not be covered by support contracts. Issues with core NAS functionality may be closed without further investigation if the same data or filesystems are accessed by these applications

and I wondered if iX’ dockge (and portainer?) app might become better assessed so that it is recognised as safe by iX-systems (whilst things which people then install via dockge/portainer are indeed at their own risk).

(3) Then I copied two different docker compose files from my native dockge installation and bunged them into the vbox hosted EE version with very little thought other than to give them unique ports (7777 and also 6124). Both applications (“MotionEye” and “Hamclock”) sprung into life without problem and gave me the feeling that I must know what I am doing. Amusing.

(4) I installed another iX app from the catalogue - Adguard home - but I decided to set the port to 6162. The installation completed but the UI took me through the five setup steps (username/password) and then tried to populate the full UI but failed. I left it a good fifteen minutes and then went back into the app’s UI and set the port to 30000 as recommended and restarted it. This time it worked properly, so I expect I chose “:6162” poorly.

It does seem very straightforward, even for duffers …

I can try a few other things (acknowledging that my vbox installation is quite resource limited) but I don’t know what else I could do which might be useful to iX/others. Please let me know!

[minor edits to reduce ambiguity]

2 Likes

This is a standard warning. The issue is any 3rd party app that you expose your data to has potentially full control over that data. We have to cover our bases, since we don’t want somebody blaming iX if the deploy an app that runs rm -rf on a directory either by mistake, or maliciously, that we have zero control over.

Nice! So glad to hear that it is behaving as expected :slight_smile:

That’s possibly a bug on our app catalog. You can make an issue here for us to investigate:

What you did test here is great, appreciate all the effort and keep it coming. Expecting BETA to be good, but RC1 already has a slew of fixes landing for it as well. Excited to get this to a .0 in a couple months :slight_smile:

2 Likes