Hi community,
Thought I’d provide a bit of a write up (and AI assisted write up) of using Hermes Agent to start migration from K8S era Truenas to a new system running v27 and consolidation of my AI platform.
I’ve got legacy Unifi and Homeassistant and various other tooling that can be classed as Tech Debt. Like most folk, Im time poor and the thought of upgrading was a weekend task I didnt want to do. So, enter my AI harness of choice - Hermes Agent (running on a 16GB Pi5/Hailo10H for local AI and backing off to Openrouter.AI for the complex stuff).
All development is using Gitlab-CE using full gitflow/SecureSDLC + full agile ceremonies and Goal sought. PRD/Specsheets created 
So, phase 1 was to prove the initial truenas install and get platform/silicon optimizations in place and get that quantified.
Over to the AI write up:
I’m consolidating an ageing TrueNAS NAS and a Raspberry Pi automation server onto a GMKtec NucBox K6. Hermes Agent, the open-source agent framework from Nous Research, is doing the research and much of the hands-on engineering.[7] That means authenticated root SSH and middleware access to the TrueNAS rig, not just a chatbot suggesting commands for me to paste. It has installed the OS, built the lab storage and container environment, benchmarked both accelerators, changed boot tuning and automated a vendor BIOS flash. I supply physical access and intervene at critical approval points.
The work is tracked in GitLab: issue, research, plan, implementation, tests, merge request, then verification against the running machine or published artifact. I’ll include the numbers and the awkward findings, because those are more useful than a claim that an agent “managed a NAS”. I’d be interested in comparisons with anyone attempting something similar; I don’t have evidence that we’re the first.
This is a lab-stage account, not a completed production migration or an unattended-upgrade recommendation. Hermes is the orchestration framework; the local models being benchmarked are a separate part of the project. The migration agent is not already running entirely on the new local inference stack.
What is moving, and which versions?
The source NAS runs TrueNAS 24.04.2.5 Dragonfish with Kubernetes-era apps. The Pi runs our Hermes automation stack, while a separate household NAS on 25.10.4 stays as a secondary host.
The K6 lab rig runs 27.0.0-RC.1: Ryzen 7 7840HS, Radeon 780M integrated graphics, XDNA1 NPU and 32 GB RAM, with a matched 64 GB kit planned. TrueNAS 27 is the renamed 26 development line.[1] We’re testing the RC without moving the production pools onto it; production cutover waits for stable and our own acceptance checks. This is a fresh-target migration, not a claim that a direct in-place jump from Dragonfish to 27 is supported.
Applications in scope include Jellyfin, Tdarr, Sonarr, Radarr, NZBGet, PhotoPrism, Pi-hole and Tailscale, alongside GitLab/Pages, databases and scheduled automation. The planned layout keeps TrueNAS on bare metal, media and inference in apps, and Hermes, Home Assistant OS and UniFi OS Server in separate VMs. Those guests and the combined production workload have not yet passed acceptance.
What the agent actually controls
With the access I’ve granted, Hermes can inspect and change the rig through key-based SSH and the authenticated WebSocket middleware API. It queries actual schemas rather than assuming an older API still applies. It can create pools, configure Apps, build containers, register Init/Shutdown scripts, inspect firmware and collect before/after evidence. I remain responsible for deciding what is allowed, approving risky steps and being available when a machine cannot recover over the network.
The main migration issue is #595, with separate workstreams for storage, hardware, GPU inference, NPU experiments, API changes and recovery. Findings feed back into the plan. One example: storage inspection corrected our initial assumption that the six-bay enclosure needed replacing. Another: a faster PyTorch build overturned an apparent NPU performance win. The issue record keeps those corrections instead of quietly replacing the earlier verdict.
Code changes go through branches and merge requests. CI is scoped to the files changed: a site-only MR runs frontend tests and a build, not every Python test in the workspace. A prior boot-audit MR did pass a 1,707-test CI run with 36 skips. A later 1,716-test full local run had two existing host-side registration/coverage failures and 20 skips; those failures are tracked, not counted as green. We also use delivery contracts and read-back checks so a scheduled job firing or an API returning success is not mistaken for a working result.
Building TrueNAS and reclaiming the boot disk
The lab’s single 512 GB NVMe was being swallowed by the default boot layout, leaving nowhere useful to put Apps. Rather than following an old shell-installer recipe, the agent downloaded and SHA256-checked the actual 27-RC1 ISO and read its Python installer source.
Its partitioner used -n3:0:0, assigning all remaining space to the boot partition. A one-line change to -n3:0:+32768M in the live installer environment produced a 1 MiB BIOS-boot partition, 512 MiB ESP, 32 GiB boot-pool and about 444 GiB free. There was an initial local-console step to patch and restart the installer; Hermes drove the subsequent installation through its WebSocket API. The recorded payload-install phase took 42 seconds, not 42 seconds for downloading, booting and configuring the whole machine.
Afterward, it created the fourth partition, built an ashift=12 ZFS data pool, exported it and imported it through middleware so TrueNAS adopted it. Apps/Docker then had a real data pool. A subsequent boot returned with pools imported, ix-apps mounted, Docker running and the GPU pin restored in roughly 114 seconds, without manual repair.
This was a destructive fresh install on the disposable rig, not a live shrink of an existing ZFS boot pool. It’s a nonstandard lab layout, not our production boot-device recommendation. Production gets dedicated boot storage. The achievement is understanding and adapting the real installer, then proving the resulting system boots and runs Apps, rather than claiming an unsupported partition trick is universally safe.
Some useful 27-RC1 discoveries:
pool.import_poolwanted a GUID, not just a pool name.- Docker was not available through the old
service.queryassumption. In our recovery path,docker.state.start_serviceworked once the ix-apps dataset was correctly mounted at/mnt/.ix-apps. - The root filesystem is read-only. A command trying to create a default pool mountpoint under
/failed; software that assumes a writable appliance root needs review.
Lab records: #595, notes 8700 and 8704.
A remote UEFI BIOS-flash pipeline
The vendor kit looked Windows-centric, but it also contained GMKtec’s EFI-shell tooling. Hermes inspected that kit and built a pipeline using the vendor’s AMI AFU executable and flags, rather than another model’s BIOS or an improvised programmer.
First came a read-only rehearsal through the same boot route. AFU /S (system ROM ID), /U (file ROM ID) and /D (verification) all returned 0x0; the verify output reported ROM File Verification Status ..... Pass.
The authorised write run then used a one-shot UEFI boot entry on the internal ESP, entered the shell, auto-ran startup.nsh, invoked the vendor flag set /P /B /N /R /K /X /l, and returned to TrueNAS. The deliberate change was /shutdown to /REBOOT, so the machine came back without someone pressing its power button. The flash path itself required no Windows, removable USB stick or hand-entered EFI commands once armed.
The durable receipt was BIOS 102 (05/10/2024) → 103 (04/29/2026), followed by healthy pools. Armed at 08:28:50 and back online at 08:37:36: ~9 minutes for the whole cycle. The issue’s estimate of 2–3 minutes programming is an estimate, not a separate captured AFU timing trace.
Getting there exposed a genuine UEFI-shell trap. Auto-run can start without a selected filesystem/current directory; a relative output redirect then aborts the script with Unable to redirect file and the misleading Invalid Parameter (line N). Even cd can fail before interpreting its path. We fixed this by using volume-qualified paths or explicitly selecting the filesystem first. In our flash run, redirected AFU output was empty after AFU’s own reset; the version change is stronger evidence than that missing log. The buffering explanation is an inference, not a measured firmware trace.
A successful official flash is not recovery from a failed POST. It does not make arbitrary NVRAM edits safe. Lab record: #595, note 8746.
GPU memory and boot-persistence work
Out of the box, the 780M reported an 800 MHz active clock. Setting power_dpm_force_performance_level to high produced 2700 MHz, and an API-created POSTINIT script restored it after restart. We verified the post-boot sysfs values rather than assuming the saved script had run. That establishes the clock setting; it does not by itself measure an inference speedup.
The memory inventory also mattered: both channels were populated, but the mixed 16 GB modules ran at 4800 MT/s because one DIMM was rated slower. Our production BOM calls for a matched 2×32 GB DDR5-5600 kit, followed by a live speed/channel check. A faster module’s label isn’t proof of the installed memory’s operating speed.
For larger models, we initially applied amdgpu.gttsize=24576 through system.advanced.kernel_extra_options. The 6.18-series kernel reported that knob deprecated. Hermes researched the replacement, changed it to ttm.pages_limit=6291456, and verified a new boot ID and matching runtime TTM/GTT values of 25,769,803,776 bytes, or 24 GiB. The separate TTM page-pool cache stayed at its default.
That is a shared-memory allocation/eviction ceiling, not reserved VRAM and not permission to load a 24 GiB model beside ZFS and three VMs on a 32 GB system. The original successful inference benchmarks below used the earlier GTT setting; the later TTM migration proves equivalent reported capacity and clean scoped boot checks, not a new benchmark win.
The agent also built a read-only boot audit covering the expected parameters, runtime agreement, IOMMU/KVM, GPU/NPU settings and pool health. It failed against the captured original state and passed after restart. A small GRUB token-filter workaround removed invalid vendor-default IOMMU arguments while preserving other options. That appliance-file workaround has an explicit upgrade/boot-environment caveat; it is not presented as a permanent supported configuration interface. Other boot errors remain separately tracked under #611.
Lab records: #597, notes 8693 and 8695; #609, notes 8780 and 8794.
Local AI benchmarks on the Radeon 780M
The practical aim is a much larger local model than the Pi’s 1.7B model, while retaining enough memory and compute for the NAS. These are observed llama.cpp server results on our rig, using the Vulkan backend in Docker with /dev/dri, GPU offload -ngl 999, context -c 8192 and flash attention -fa on.
| Model and quantisation | File size | Prefill | Generation | Observed load |
|---|---|---|---|---|
| Qwen3-4B Q4_K_M smoke test | 2.4 GB | 67.8 tok/s | 26.8 tok/s | ~3 s |
| Gemma 4 26B-A4B QAT q4_0, MoE | 13.8 GB | 415.9 tok/s | 24.3–25.5 tok/s | 12–15 s |
The Gemma prefill figure came from a 595-token prompt. The acceptance targets were at least 20 tok/s generation and 40 tok/s prefill, so that run passed both. A 26B MoE has roughly 4B active parameters per token; this is not the same compute workload as a dense 26B model. Longer agent prompts make prefill important, but one prompt is not a long-context scaling curve.
Memory pressure was a major part of getting stable launches. On the test rig we clamped ZFS ARC to 4 GiB after a much larger ARC interfered with loading. Page-cache pre-reading was also used. The load figures therefore aren’t a controlled disk-cold comparison, and the results do not establish performance under simultaneous media transcoding and three active guests. We still need pinned-runtime repeat runs and the planned quantisation/context sweep.
The model answered the “all but nine” sheep question and a time-arithmetic question correctly, and its returned CRC-16/XMODEM code was inspected against the expected polynomial and loop. Those are useful smoke checks, not a broad reasoning benchmark or an executed coding-test pass.
An open failure is just as important: two larger files, 16.2 GB and 17.7 GB, stalled during loading with a single core busy and GTT usage around 14–15 GiB. A RADV/allocation boundary is a working hypothesis, not an established root cause. Different load modes, reduced contexts, pinned llama.cpp versions and another runtime remain experiments. Raising the GTT ceiling did not magically solve model fit.
Lab record: #601, note 8725.
XDNA1: real NPU execution, and a corrected performance verdict
The K6 also has a Phoenix/XDNA1 NPU. We proved container access through /dev/accel/accel0, installed XRT and its amdxdna plugin from open Arch packages, and ran an int16 256×256×256 matrix multiply in 2.1 ms, matching NumPy exactly. That is an accelerator execution/correctness test, not an embedding-service latency number.
Hermes then reconstructed the open IRON/Peano build route for bf16 embedding kernels. MiniLM returned 384-dimensional vectors through an OpenAI-compatible /v1/embeddings endpoint, with cosine similarity 0.999307 against the CPU output. HTTP shape and vector correctness were tested separately from speed.
The first single-column kernels were slow. Rebuilding them through the four-column whole_array flow changed the fused-QKV kernel from 40.63 ms to 4.39 ms, or 9.3×, with a reported 825.7 GFLOP/s. MiniLM’s batch-64 forward improved from about 1013 ms to a best observed 184 ms. That kernel speedup is not a 9.3× improvement in the entire service.
Then we found an unfair CPU baseline. Replacing Arch’s generic PyTorch package with the official CPU wheel changed CPU MiniLM from 344 to 76 ms/batch. The fairer batch-64, bf16 comparison was:
| Backend | MiniLM forward time, batch 64 | Interpretation |
|---|---|---|
| Official PyTorch CPU build | ~76 ms | Fastest measured throughput path |
| Four-column NPU, performance mode | ~205–242 ms steady-state | ~184 ms best observed; offloads GEMMs |
In that comparison the CPU is about 2.7–3.2× faster. Later NPU profiling attributed 190.5 ms of a 230.3 ms forward to GEMM calls, about 83%; this was the later pipeline, not the original single-column configuration. Thermal/DVFS and test order affected session results, so we distinguish best observed from steady-state ranges instead of inventing a single universal latency.
We also ran Qwen3-Embedding-4B through the embedding engine: about 2240 ms per one-text forward, with 18 GB peak RAM. That was CPU-backed, not a 4B model running on the NPU. Nor do we have a working LLM-generation deployment on XDNA1: the LLM overlay route investigated targeted a different XDNA generation. That is a limitation of the route tested, not proof that every future XDNA1 implementation is impossible.
The NPU can offload work, but lower energy use has not been measured. TOPS/W and sustained mixed-load tests remain on the list. For wall-clock embedding throughput on this box, the corrected CPU backend wins. This is precisely why we keep the failed hypotheses and before/after measurements.
Lab records: #607, note 8707 for first execution, note 8717 for embeddings, and #607, notes 8720–8722 for kernel rebuilds, corrected baselines and profiling.
Our test approach
We separate several questions that are easy to collapse into one optimistic verdict:
- Can the device execute anything? Check the driver, firmware, device node and container access, then run a small known-answer operation.
- Is the result correct? Compare against a CPU/NumPy reference, inspect vectors and API responses, and use small reasoning/code smoke cases. A plausible response alone is insufficient.
- Is it faster under stated conditions? Record model/quantisation, backend, context, prompt length where known, batch size, memory settings and CPU baseline. Separate a kernel timer, model forward pass and end-to-end service latency. Repeat when thermals or DVFS move the result.
- Does it survive restart? Read a changed boot ID, settings, service state and pool health. A config write is not persistence evidence.
- Does delivery work? Run focused offline regression tests, pass the appropriate MR pipeline, then read the actual live output back. Scoped audits only prove their scoped checks.
What we haven’t proved yet includes broad model-quality evaluations, the full 8k/32k/64k context sweep, simultaneous AI/transcoding/VM loads, sustained power efficiency and production rollback timing. Those stay pending even when an individual smoke test is green.
Storage findings beyond “move the disks”
All source data disks turned out to be USB-attached. The main enclosure is a TerraMaster D6-320, with Realtek RTS5420 uplink hubs and ASM1153-class bay bridges using UAS. Later inspection corrected the original five-gigabit-only enclosure assumption: it is a 10 Gbps-class enclosure currently negotiating at 5 Gbps, with the host/cable path needing a controlled test.
Scrub history gave approximate baselines of 280 MB/s for the main pool and 120 MB/s for the separate single-disk enclosure on usb-storage. These are inferred scrub rates under their existing conditions, not clean fio or application benchmarks. A faster negotiated link might help, but no doubling of real throughput has been measured.
The storage audit checked vdev topology, properties, scrubs and snapshots. It found that the SSD special-vdev bridges advertised no discard support: autotrim=on was not proof that TRIM reached the disks. It also changed the media datasets’ recordsize from 128K to 1M for new writes; existing files are not retroactively rewritten. Backup/replication findings still need their own restore and policy checks.
USB storage is an existing homelab constraint, not something I’m recommending over a proper HBA/direct-attached design. Shared links, resets, power and bridge behaviour matter particularly during resilvering; the community’s warnings are relevant here.[9] Moving the same enclosures doesn’t remove those risks. Export/import rehearsal, load tests and backups are acceptance requirements, and source data won’t be deleted just because an import succeeded.
Lab records: #599, notes 8614, 8659 and 8673.
Read-only firmware archaeology
After the official update, Hermes extracted and decoded the exact BIOS 103 image and compared its setup forms with live NVRAM fields. It identified the Setup Item Hide Control gate suppressing the Advanced/Chipset menus, then mapped AMD CBS/PBS power, thermal and UMA settings. One trap: the displayed “30W” and “65W” choices both encoded as the same enum value. Guessing at offsets or copying another NucBox’s settings would be a poor plan.
A tested read-only firmware audit checks the expected stores, lengths and selected fields, skipping the efivarfs attribute header. Store hashes remained unchanged across the tested restart. We have not unlocked the menus, changed those power controls or booted an assumed-compatible third-party firmware browser. Even the visibility-bit experiment waits for a demonstrated board-specific recovery path. This is analysis completed, not a successful unlock being implied.
Lab record: #610, note 8784.
Why UniFi, Home Assistant and automation need separate migrations
UniFi is more than copying a container. Our estate includes legacy Network 9.5.21 and Kubernetes-era hosting. The plan is UniFi OS Server in an Ubuntu VM, with a fresh backup, isolated restore rehearsal and a deliberate device-inform handover. A restored controller database doesn’t prove the access points moved. DNS indirection before cutover is part of our plan, not a claim that the handover has already passed.[2]
Our old Home Assistant chart is stopped. We chose HAOS in a VM for Supervisor and add-ons.[3] That doesn’t mean Home Assistant Container is deprecated: upstream retired the Core and Supervised installation methods while retaining HAOS and Container.[5]
TrueNAS 27 also removes the REST API.[4] We saw REST v2 returning 404 on the RC and used authenticated WebSockets for the rig. Porting the rest of our automation and rehearsing a config restore remain cutover gates. Installed versioned middleware schemas helped research calls; an attempted system.reboot call failed validation despite a reason string, so we recorded the anomaly and used an OS-level fallback rather than treating the submitted job as a reboot. That is our observation, not an upstream bug diagnosis.
What still has to happen
The production pools need a rehearsed export/import; app configurations need restoring and checking service by service; the three planned guests need building and exercising. We need the matched RAM, a controlled USB link/storage test, larger-model load diagnosis, mixed-load and thermal/power tests, and a timed cutover/rollback rehearsal. Tested restores come before decommissioning either source system.
I’d particularly welcome corrections or comparable measurements from people running a 7840HS with TrueNAS, HAOS/UOS guests or large Vulkan models. Our detailed GitLab records are private, so the issue/note numbers above are audit references rather than public links. We can share sanitised commands and benchmark excerpts for a specific result; the ongoing Pages build log stays local to the homelab. Nothing has been posted to this forum automatically.