The self-hosting community is currently facing a hardware gap for AI inference workloads. Users of popular applications like Frigate NVR, Immich, and LocalAI essentially have only two choices:
Google Coral TPU: Efficient but often underpowered (4 TOPS), bottlenecking high-resolution object detection or LLM tasks.
NVIDIA GPUs: High performance but expensive, large, and power-hungry, often unsuitable for compact NAS builds.
The Axera AX8850 (found in the M5Stack LLM module) solves this by offering 24 TOPS of performance in a standard M.2 form factor with low power consumption.
However, TrueNAS SCALE currently lacks the native kernel modules (ax_pci.ko) for this chipset. Because TrueNAS is an appliance OS, users cannot simply install these drivers themselves without breaking the system or update train. This renders the hardware completely unusable on the host system, preventing native Kubernetes or Docker apps from accessing the accelerator.
Impact
Adding this driver would significantly benefit the growing segment of TrueNAS users running home AI labs.
Benefits:
Performance: Unlocks 6x the inference speed of a Coral TPU, enabling real-time LLMs and high-frame-rate detection on the NAS itself.
Efficiency: Allows high-performance AI without the power draw of a discrete GPU.
Simplicity: Aligns with the TrueNAS SCALE philosophy by allowing Apps to consume hardware resources natively.
Compatibility: The drivers are already maintained for Debian/Ubuntu, minimizing the friction of integration into the SCALE base OS.
Disadvantages:
Slight increase in base OS image size to include the kernel modules.
User Story
Current State: A user installs the M.2 accelerator into their TrueNAS server hoping to upgrade their AI performance. Upon booting, the device is listed in lspci as a generic multimedia controller, but no driver is loaded. The user checks their Apps (Frigate/Immich) and finds no option to select the Axera NPU. The hardware sits idle and useless.
Desired State:
User installs the Axera/M5Stack M.2 card into their TrueNAS server.
User boots TrueNAS SCALE. The OS automatically loads the ax_pci module.
User goes to Apps > Frigate (or Immich).
In the GPU/Device Configuration section, the user selects “Axera NPU” (similar to how “Intel iGPU” or “NVIDIA” are currently selected).
The container launches with direct access to /dev/ax_npu, providing immediate, high-performance acceleration.
That looks like very custom hardware, intended for SBCs. I suppose that its has some competition/alternatives in this space, so there could be a lot of additional drivers to package (and keep updated!) to generally add support for such AI accelerators.
Thanks for the feedback. While a VM is a functional workaround, it fundamentally undermines the TrueNAS “Appliance” model for one major reason, Using a VM forces a “split-brain” architecture. I would have to manage standard apps via the TrueNAS GUI but “exile” my AI capable apps (Frigate, Immich, Nextcloud, Ollama and jellyfin can also benefit with transcoding) into a manual Linux VM just to access the hardware. This forces me to abandon the official App ecosystem for my most important services.
Regarding the hardware: While these chips started in SBCs, M.2 accelerators are becoming standard for x86 NVR builds. I have already verified the drivers work perfectly on standard Debian 12 (x86_64), so support is not limited to ARM/SBCs.
I do not question the support on x86, I question how many of these accelerators are out there (just looking up what the AX8850 was returned comparisons with other products) and at which rate their drivers are updated (assuming fast, because the field is changing quickly, and TrueNAS may not keep up with that).
Given that many old-timers are wary of the official app ecosystem and prefer to deploy their own stacks through Dockge, the loss may be smaller than you fear.
Coral TPU is a good example of that. Products come and go too quickly. Look at the complaints with NVIDIA and their video card drivers. UPS people wanting NUT updated but we follow what is in Debian and it’s considered and OLD driver. People can’t have the latest, greatest, bleeding edge.
Items need to be stable and around for quite a while to make it in.
While it would be nice to support many types of hardware, I have to agree with @SmallBarky in that the devices need to be both common and around a bit of time.
TrueNAS is intended to be solidly stable for the Enterprise Data Center. Adding random drivers, applications and additional software means that people would think they are supported items. Then want them fixed when they break.
Case example: The developers of TrueNAS removed the DM-Crypt software because TrueNAS was no longer going to use it. But, some people wanted it back because THEY used it, and now they think it should be supported forever.
Their really is no easy answer.
I had hoped that perhaps HexOS, (based on TrueNAS), might pick up some of the more consumer level hardware support. Like the requested Axera AX8850, or perhaps more current NUT UPS drivers. But, it does not appear that HexOS is going to do that.
Perhaps someone else should or will fork TrueNAS, and make it easier to update packages, as well as add new packages.
I understand the concern regarding maintenance burden and bloat. However, the binary choice between ‘Official Support’ and ‘Zero Support’ (VMs) limits the platform’s potential.
I propose a third option (I’m still a truenas newbie but loving the truenas ecosystem and it potential it has. Also please correct me if we already have this feature): A User-Installable Driver Framework
Instead of iXsystems maintaining niche drivers, TrueNAS could provide a mechanism to load community maintained Kernel Modules (DKMS) similar to Apps and user can on click add and remove those thirdparty drivers/modules
Decoupled Maintenance: The community or hardware vendor maintains the driver package, not iXsystems. But they will be verified by iXsystems for safety and security reasons.
No Bloat: The base OS remains lean. Only users with the specific hardware install the module
Preserves Ecosystem: This allows ‘Edge’ hardware (NPU, TPU) to be exposed to the Host, keeping the official Apps/Charts ecosystem viable for users who want appliance-like simplicity without being forced into manual VM management
While an interesting idea to have community supported modules, it is not just kernel modules. People want whole packages updated, like NUT for new UPSes. Or NVIDIA package changes to support either really old GPUs, or really new GPUs.
Again while nice in concept, I doubt it will work. Even if iX tried to verify new packages, or the kernel modules, without the exact hardware to test against, I am not sure any malware could be fully detected.
The current developer mode is not great, but does what some people need. (Others that have reduced Linux skills may want to update a package or kernel module, but don’t have knowledge to perform the task…)
This kinda goes back to what I said years ago when TrueNAS started supporting Linux:
Linux may bring more users to use TrueNAS. However, some of those users are going to want specialized hardware support that is outside commonly used on NASes, (and especially not Enterprise NASes). Or additional software loaded by default.
Those users assume that if it is supported by Linux, it must be supported by TrueNAS CE / SCALE. Or assume that TrueNAS CE is a Linux distro with a NAS focus. Neither is true.
As I said, no easy answer. In the past, when TrueNAS was based on FreeBSD, if FreeBSD did not support some hardware, then the user(s) were SOL. That was just the way it was.
Whilst we typically keep feature requests open for sometime, in this case the lack of support for this from the Linux LTS and the need for custom hardware for us to test and support this means we will not consider this at this time for TrueNAS. Thank you to the community members who have contributed on this request.
