TrueNAS on ARM - Now Available

I was waiting for this, TrueNAS should make this official.

Unofficial. Feel free to test and give feedback. Full hardware details, etc.

Given TrueNAS Scale is based on Debian and there is Armbian for ages now it should not be impossible to bring this to U-Boot devices (and such every ARM board out there), other then UEFI/Grub as such the bootloader is there anything speaking against it?

Hi, I’m a developer from Cix Company. The CPU on the Radxa O6 is our product.

I’m thrilled to see TrueNAS running on ARM. I’m currently trying to get the hardware drivers for the Radxa O6 working on TrueNAS, but the system has disabled package management tools like apt and dpkg by default. I’d like to know how to lift these restrictions so I can proceed with testing. Thank you!

Hey @ClayStan

Since you’re doing development work here, the fastest method is to use the install-dev-tools script (as sudo or root) which will re-enable those.

Do be cautious and review what packages will be added or removed when using apt or dpkg as we’ve seen some seemingly innocent commands do things like “will remove middlewared” (which is kind of crucial to TrueNAS itself remaining functional!)

2 Likes

Hi, Thank you very much for your response!

Please don’t worry about these issues—I’m a Debian maintainer and very familiar with Debian’s package management.

3 Likes

Love seeing this progress.

Personally I would love to build a mini ARM box to house maybe just 2 drives, but I also need something that can do 10Gb SFP+, so i often get stuck with options only being a mITX board and case, which I would love to keep something smaller than my current QNAP 4 bay as my final immutable backup destination.

mini arm box with 2 drives? do you consider NVME drives ? has a PCIE slot so you can drop a SFP+ nic in it. MINISFORUM MS-R1 Workstation

Its power consumption is a bit disappointing, but it’s 17W so i mean it’s not terrible but compared to Apple M4 at 1/4 that it’s not great.

That said, good sign we’re almost there

1 Like

NVMe drives may count BUT you’re not going very far with just two M.2. For the sake of storage, this PCIe slot would best be used for a HBA. Of course, the on-board 10G NIC is Realtek 8127 :roll_eyes:
So, no, this is NOT going to be a good platform for an ARM NAS. Realistically, this is a cheap desktop for an ARM developer; it’s not for users yet. If you’re optimistic, this is one step towards (non-Apple) general purpose ARM computers. But these remain a very aspirational long-term goal.

I’m wondering if it would be (eventually) possible to run this in vm on a Mac mini, while passing through the PCIe HBA (such as Broadcom 9500), connected via thunderbolt PCIE enclosure.

This would tick all boxes:

  • provide reliability of TrueNAS software with native access to Native zfs/hba/sas/hdds
  • more than enough cpu raw power
  • 10G Ethernet built in — or another thunderbolt adapter.
  • Power efficiency and stability

Mac mini: No ECC RAM, Aquantia NIC. That’s two fails in my book for reliability before we even get to discuss whether running a pool on a Thunderbolt adapter is a good idea.

Some more benchmarks here and in my linked GitHub issue: Minisforum stuffs an entire Arm Homelab in the MS-R1 | Jeff Geerling

This box does have two M.2 slots (1 x4 Gen 4 for main NVMe SSD, 1 x2 Gen 4 (I think) for WiFi, but comes with E-key to U.2 and M.2 adapters. Then there’s a x16 (x8 connected) slot which fits half-height, single slot cards.

It’s similar to their other MS-series boxes, this could make a fun little NAS, especially if Cix ever figures out their own chip and gets idle power draw down.

Probably - anyone with Apple Silicon want to try it out via UTM?

1 Like

Well, we are not taking about high availability enterprise environment.

On point:

  • ECC is desirable, but not required. Its importance is overblown. Only risk is slightly elevated risk for user data corruption at write (one flipped bit every 6–12 months on average per 32GB of ram, but distribution is not uniform, most users will never see it). Either way, it’s acceptable for home users. Zfs itself is protected by own checksumming.
  • nic chipset: irrelevant. It runs Apple drivers and Apple firmware, stability is ensured by Apple. TrueNAS will see virtual nic, also pretty stable drivers.
  • pool on a thunderbolt adapter: what are your concerns? I have been using such adapters quite successfully for years, under different circumstances. The only limitation I encounter is bandwidth - which is not an issue here. I did not see any stability issues. Again, maybe some crappy HBA are sensitive to extra latency but that would be an HBA bug that would eventually get addressed.

Interestingly, this minisforum unit claims that it can do ECC as long as you’re ok with locking 10% of memory capacity. I consider ECC a must have for a ZFS NAS but they’re not being super detailed re: what type of ECC is in play here.

For me, the x8 slot would go to an HBA like a 9400-e16 that would allow an external HDD tower to be attached. That would leave me with a pretty compact NAS that also happens to be on the lower end of the power consumption limits.

Given the published power consumption for the board alone, I’m not even close to considering it. When all is said and done, I’d be highly surprised if a AM4 or AM5 based motherboard wouldn’t consume less power.

I agree on the ECC RAM, it’s a shame that with the demise of the intel MacPro, there’s no Mac in the lineup with ECC, or at least not transparently so. For all we know Apple might expose some aspects of DDR5’s internal error checking to their OS internals in their unified memory architecture and SoC platform, but nothing is known or software accessible, so for all intents and purposes it’s absent even if it were hypothetically used internally somehow.

What’s the issue with the Aquantia NIC? Haven’t heard anyone complaining about perfomance issues, so far…

Not sure UTM would be the way to go. Apple’s own virtualization is much closer to the hardware. That can be used “experimentally” under UTM (last I checked), or Apple has sample code. Creating a virtualization app is just a few lines of code.

Apple even has a contrainer framework now, so given that TrueNAS only needs a web interface, it could probably be running in a container with proper hardware passthrough, right?

You might be able to get pretty close to metal performance on a Mac using Parallels. The key here would be to make the various bits of hardware available to the NAS VM and keep them dedicated to it (Parallels allows that). Spinning up a Debian / Ubuntu / whatever Linux VM is trivial, support for PCIe tunneling may not be.

If I were to do this with a Mini, I’d first experiment inside Parallels, hook up an external enclosure for an HBA, then go from there. Four lanes of PCIe 3.0 isn’t going to break speed records but it’ll likely be enough. Then use the second Thunderbolt port to host a QNAP external SFP+ to Thunderbolt adapter.

Could make for a neat, relatively low-power NAS as long as you’re OK with having multiple enclosures, lots of chicken-wire and band-aids to hold it together. And… living with the fear that either Apple, Parallels, or iXsystems does something to bork the enchilada.

1 Like

well purely in theory here

TrueNAS doesn’t need to understand PCIe tunnelling here, because the host Apple/MacOS will be responsible for handing that. The VM would just see “PCIe device present” - although we’re reliant on the device politely transitioning in and out of the right sleep/power states.

PCIe 3.0 x4 is ~4GB/s of bandwidth, the same as PCIe 2.0 x8 which is what many older HBAs were designed around, so I wouldn’t necessarily consider it a bottleneck for the average home user - and certainly not if the external network link is only 10Gbps over another adapter. You’ll lose some of that bandwidth to redundancy since you’re having to write to multiple devices on the other end, but you’ve got a fair bit of headroom to play with.

Virtual TrueNAS already has its own weird failure modes, which we do our best to avoid deliberately triggering, but it’s definitely going to have a lot more moving parts. I don’t anticipate Apple deciding to (re?)-introduce ZFS to MacOS, so a double-mount situation is unlikely, but there’s likely other opportunities for shenanigans.

1 Like

Actually 12.5%

*ECC enabled: capacity reduced by 1/8

which suggests that it takes 8 RAM chips and runs “real ECC” with 7 data + 1 parity.
Interesting, if done right.

That still leaves the mini-PC form factor without dedicated cooling for the PCIe slot—fine for desktop use with a half-height GPU with blower fan for AI, not a good form factor for a NAS. And the relatively high idle power: Obviously unoptimised, and since the CPU does not use the latest ARM cores it’s quite possibly not manufactured on the latest nodes either.
The MS-R1 is an intriguing step towards the non-Apple ARM desktop; it is not (yet) an ARM desktop for the masses—and, just like its MS-01/A2/S1 siblings it is not designed to be used as a NAS.

edit: link to STH review

1 Like