Electric Eel Emerges

(This post is largely a copy of the TrueNAS blog posted on Thursday 22nd, but is reposted for easier comment and conversation.)

TrueNAS Electric Eel (aka SCALE 24.10) introduces Docker Compose capabilities within Apps. Industry-standard Docker Compose applications can now be easily deployed on TrueNAS to benefit from the stability, flexibility, and performance of ZFS. The Electric Eel Nightly images are now available for testing, and the BETA1 version is expected to be ready this coming week. Bug fixes, feature updates, and ongoing polishing will continue until the targeted release date in October 2024.

Electric Eel Automatically converts Existing Apps to Docker

We’ve previously revealed that the transition of the TrueNAS Apps system from Kubernetes to Docker will take place in Electric Eel. We’re working hard to ensure that the migration from Helm Charts & Kubernetes to Docker Compose will be as seamless as possible for our user base.

The straightforward configuration of Docker Compose has led it to become the industry-standard tool for application distribution. Combined with optional Docker management layers like Dockge and Portainer, it’s now even easier to configure complex applications and networks.

Early testing has shown that Docker Apps not only deploy up to three times faster but also have lower CPU overhead when idle, reducing overall system power consumption. With this change, your TrueNAS Apps experience will be both faster and more efficient.

The update to Electric Eel automates the entire Docker Compose transition process for official TrueNAS charts and standard Docker Apps deployed through the “Custom App” button. Apps like NextCloud, Plex, Syncthing, and hundreds of others will reboot with the TrueNAS Electric Eel update using Docker, but their software, data, and networking configuration will be preserved.

Because these Apps features and migration process need significant testing and validation, users with production workloads should refrain from deploying Electric Eel until after BETA and community feedback. A “Call for Testing” has been made to test either nightlies or the upcoming BETA.

In addition to expanding the available Apps list and improving performance, leveraging Docker Compose capability also provides the ability to create more complex App environments, such as multiple Apps, dependencies, and associated ingress, load-balancing, VPN, and gateway policies. Multiple Apps can be bundled into their own private network. Please review the documentation and give us feedback on what works or is missing.

Sandboxes will persist during the upgrade to Electric Eel, and the existing Jailmaker script can still be utilized. Custom sandboxes are a common approach for advanced users wishing to run a Kubernetes, Podman, or other stack for those who need specific capabilities beyond Docker, such as Kubernetes APIs and Helm charts.

@Stux created an awesome video on how to smoothly migrate from Sandboxes to Docker Compose using the Dockge tool.

Electric Eel delivers more features on top of Dragonfish

TrueNAS Dragonfish (aka SCALE 24.04) has delivered on its promises with both a rich set of new features and the achievement of Enterprise-grade quality with 24.04.2.

In addition to the Docker Compose transition, TrueNAS Electric Eel includes 400+ enhancements and bug fixes. Early release notes are available.

The most obvious of the new features are described in the following sections.

Major Web UI Overhaul

The web UI has its largest overhaul since SCALE was first introduced in 2022. While the SCALE WebUI was significantly more dynamic and easier to use, the overhaul adds modern features for even greater ease-of-use.

Global Search capability significantly reduces the time to complete a task. Search for an item, and you’ll be taken to the right screen, highlighting the element to examine.

Table searching and filtering improves manageability for systems with hundreds of datasets, shares, LUNs or drives. Find problems or perform tasks in much less time.

Dashboard widgets have been extended and become size configurable. Create your own dashboards and share your recommendations.

Here is a video demo of the new Web UI.

Cloud Backup

Cloud backup to iX-Storj has been enabled. Cloud Backup extends TrueNAS functionality to backup your data to the Cloud by providing integrated snapshots (LUNs and shares), deduplication, and simple restore capabilities.

OpenZFS 2.3

OpenZFS 2.3 is still in development, and TrueNAS is tracking the latest approved updates, with several highly anticipated additions.

RAIDZ Expansion allows RAIDZ (Parity) vdevs to be expanded by one drive at a time, ideal for small footprint systems looking to expand incrementally. This feature also permits 2-drive RAIDZ1 (not yet made available in API/Web UI) systems, ideal for home users who want to start small on a budget.

The re-engineered Fast Dedup feature allows for deduplication to have much less performance overhead than the current dedup algorithm. Please note that Fast Dedup is not fully complete or tested and is still considered “experimental” at this time, and should not be used on production systems.

System Integration Improvements

TrueNAS is designed to simplify integration with other systems with its fully capable API. Electric Eel adds even more system integration capabilities.

Improved Data Migration: When using Syncthing to migrate from another SMB storage system, the Alternative Data Streams (ADS) are now also migrated. This can be used to migrate data from any SMB server, such as Windows, Synology, QNAP, Netapp, Pure, or Dell systems.

Cloud Management: A general initiative enables TrueNAS to be more easily installed and operated by cloud management services. This doesn’t reduce the ability to use the WebUI but makes it easier to install and operate tens and thousands of systems. Electric Eel has hooks for API-driven installation and a globally unique system ID.

FreeIPA support: While Microsoft Active Directory dominates the market, TrueNAS now enables FreeIPA, an open-source alternative to traditional AD environments.

Modular NVIDIA Drivers: The binary drivers for NVIDIA GPUs are now decoupled from the TrueNAS installation, allowing for these drivers to be updated on separate cycles from the main product.

Improved Logging: The TrueNAS built-in auditing capabilities will now log a wider variety of important changes made to the system. This includes audit logs of all changes made to any share settings or permissions and all commands run via sudo or with elevated administrative permissions. These are important for NIST 800-209 compliance.

TrueNAS Enterprise Improvements

Every TrueNAS version includes improvements that enable us to build faster and better-managed TrueNAS Enterprise appliances. Electric Eel includes a few of those features. Enterprise users should wait until Electric Eel is battle-tested before using the new software.

Tri-Mode Support: Some TrueNAS platforms are being enabled with tri-mode storage capabilities, allowing for combinations of HDDs, SAS SSDs, or NVMe SSDs in the same slots. This improves the cost, performance, and power of edge systems.

NVMe-oF Support: TrueNAS F-Series is stacked with ES24N (all-NVMe) expansion shelves for very large flash systems. This NVMe-oF architecture enables high-performing 5PB all-NVMe systems with similar management capabilities to SAS expansion shelves (JBODs).

Improved Enclosure Management: We’ve improved this popular tool for remotely managing the TrueNAS appliances. It allows the admin to visualize systems remotely and provide clear instructions for repair. The improvements include more accurate pictures of the enclosures and much faster rendering of the visual depiction, especially for very large systems.

When Should I Migrate?

If you are deploying a new TrueNAS system, we recommend TrueNAS SCALE 24.04.2 for added functionality, vastly broader hardware support, an expanded App catalog, better performance on most workloads, and an improved Web UI, all of which make managing TrueNAS easier than ever. TrueNAS 24.04 “Dragonfish” includes support for Sandboxes, which provide jail-like capabilities using systemd nspawn containers.

TrueNAS 13.0 users looking for the new capabilities outlined above can sidegrade to TrueNAS SCALE anytime, preserving data and essential NAS functionality such as SMB, NFS, iSCSI, and VMs - with the primary exception being Jails. TrueNAS 13.3 allows updates to FreeBSD jails.

As discussed here, TrueNAS 24.10 “Electric Eel” also provides a native Docker Compose environment that vastly improves running applications with lower overheads and opens the door to more complex network setups (similar to Jails). However, we do not recommend updating to Electric Eel until it is more tested and mature. For current software recommendations, always review the Software Status page for recommendations based on your profile.

11 Likes

We often get asked what are the most popular versions of TrueNAS?

The answer is that it’s constantly changing. 24.04.2 is leading the pack right now.
Some old versions are stubbornly hanging around - if they work, why change?

Based on previous cycles, 24.10 will probably become significant in November and the leader in February. In the meantime 24.04.2 will be the “conservative” version to use.

4 Likes

Am I the only one for which this page displays really badly on iPhone?

Will nvmeof of support be added in 24.10?

I still have a couple of TrueCharts apps installed that I haven’t migrated to custom apps yet (and a couple that I can’t seem to delete, although I suspect the latter is due to the former for some non-obvious reason). I assume that I should focus on migrating/removing them before trying out Electric Eel?

Best way is to migrate to docker in a sandbox in Dragonfish before upgrading to Eel,

and then that is trivial to migrate to docker on electric eel at your leisure

Looks like CORE is the winner anyway. U6.1 and U6.2 still equals more than 24.04.2. :smirk:

5 Likes

From the graph it looks like SCALE is equivalent (or greater) than CORE.

Will be interesting to see the data in a few months, maybe after EE lands.

2 Likes

Yes, 13.0 has been rock solid for a couple of years now and has accumulated a large base. All-SCALE is larger than 13.0, but not specifically 24.04.2

2 Likes

Can anyone running the nightlies confirm or deny if the nvidia runtime for docker is installed by default?

1 Like

With Electric Eel, will we be limited to a special iX maintained docker repo, or will we be able to pull docker images from anywhere?

There will a catalog but you will also have the ability to have custom apps. I suggest checking out the Call for Testing post Captain_Morgan had in the initial post on this thread, lots of great info on that thread about the new setup.

3 Likes

Regarding the modular nvidia drivers:
Will it be possible to reenable support for older GPUs with the 400 drivers or does it just refer to a potentially faster update cycle for the current 500 drivers?

Its possible, but needs someone to test. We don’t have a library of old GPUs to make that specific claim.

2 Likes

The Electric Eel 24.10 BETA was released today as expected. We’re looking forward to the feedback.

1 Like

Awesome and thanks! Dumb question time - does it contain everything up to the current nightly or was it cut off from contributions on a specific date?

Edit: Nighty to nightly - two very different things.

BETA.1 is a bit behind the EE nightly. The EE nightly is what will become RC.1 next month, so a handful of fixes and features already landing in that branch.

1 Like

Most of my apps are “custom”. Is there any capability for custom apps (that can be configured through the Web UI) that the EE Docker Compose version will not support? And will the migration handle custom apps transparently, or do we need to do manual work (and what)?

Cheers
– perry

I had AD directory corruption again on Dragonfish… It happened two days in a row (the first day I deleted ACLs and reapplied presets as I have I have done before) and the next day when it happened again I also cleared the AD cache…

I know that is not much to go on but that seems to point to there may be some AD cache corruption occurring. Previously when this occurred to me the SIDs were getting mapped all incorrectly.

I’ll try to provide some more data on this next time it occurs I sent in a report the last time this occurred. (I can’t have shares in a broken state in production). Hopefully Electric Eel updates may correct whatever is occurring.

I usually test on my home NAS before updating the one at work but its not on an AD.

Electric Eel or Dragonfish?

Can you start a new thread with details… perhaps post the link here.