TrueNAS could do much on 64 core and 128 core arm boards with 128 PCIe lanes AsRock Rack Ampere | Newegg.com
@joespeed If you take the plunge, Senior Vice-President Palpatine has already welcomed you to try an ARM build of SCALE…
1 Like
Yes, those do look like nice system boards. A bit costly, though expected with server style boards and likely lower volume of sales.
However, the issue with ARM64 boards, is the lack of choices. With x64, you don’t just get 1 CPU vendor with perhaps a dozen well known system board vendors. But, their are 2 well known CPU vendors, with hundreds of CPUs available, and hundreds of boards actively available.
AND, to hit hard, x64 includes options for lower power, lesser cost, smaller size, or extreme amount of builtin storage.
Again, while I wish ARM64 well, it is just not yet suitable for general purpose NAS based on TrueNAS.
1 Like
I concur with this in general. Beyond the Rapberry-pi community and the big hyperscalers who write their own custom stacks, I’ve seen very little effort from a software development standpoint for ARM.
Even in consumer facing devices, it’s been a very hard sell until recently. In a previous life I bought tens of thousands of laptops for students, COVID was a fun time for me. 3/4 years ago I demoed some ARM Chromebooks. They were garbage, the build quality was awful because they were all cost-down models and the performance was abysmal. I selected to buy fewer, but better, X86 Windows laptops instead.
This was around the same time M1 Apple silicon came out, and for the limited few Macs we bought, we bought the last of the Intel stock at a slightly discounted price. Today’s consumer market is changing, and I’m more interested in ARM in that space now than I was then, particularly in Apple devices, but even Micrsosoft’s got some stuff going on now.
Windows on Arm finally has legs - The Verge
It’s a bit topsy-turvey. Alot of new technology comes from the “enterprise” and trickles down to consumers as the technology evolves and costs get reduced. The ARM universe started from the bottom (phones, embedded devices) and is working it’s way up. I think it’ll be many years still before I would consider ARM to be a viable X86 competitor across the segments, but I’ve been wrong before.
Intel and AMD stand the most to lose here. If we use them as a compass, you’ll find both companies had ARM projects that never took off. AMD in particular had Jim Keller design an entire ARM platform concurrently with first gen Ryzen, and they more-or-less abandoned the project.
AMD K12 - Wikipedia
3 Likes
radxa ROCK 5 ITX
$120-$240 based on memory choice
CPU: Rockchip RK3588
GPU: Arm® Mali™ G610MC4
NPU: INT4 / INT8 / INT16 / FP16 / BF16 and TF32 acceleration
RAM: LPDDR5 4GB, 8GB, 16GB, 32GB
Multimedia: Most popular codecs
Storage
1x Onboard eMMC for ROOBI OS
1x microSD Card Slot
1x M.2 M Key Connector for M.2 NVMe SSD(PCIe 3.0 2-lane)
4x SATA Ports with Power Header
1x SPI Flash for Bootloader
USB
2x USB 2.0 Type-A HOST Ports
4x USB 3.0 Type-A HOST Ports
1x USB 3.0 OTG / HOST Type-C Port
2x USB 2.0 Interface via Front USB header
Ethernet 2x 2.5G Ethernet Ports both with PoE support(Additional PoE Module Required)
Wireless 1x M.2 E Key Connector for WiFi 6
Other
1x PWM fan connector
1x RTC Socket(CR1220 battery)
1x Front Panel header for Power Button / Reset Button / Status LED
Power
1x 12V DC Power Jack(55x25mm)
1x Standard 24-Pin ATX Power Supply Interface
1x PoE Header(Additional PoE Module Required)
Nice ARM board, with a decent CPU.
However, with limited storage connections, and no real PCIe slots, it is limited for NAS purposes.
A great many ARM boards are designed for embedded use, small desktop or small server use. We have yet to see real NAS server boards with ARM processors. Even the expensive ARM server board was not designed for NAS, though could be used as such.
With only 1 x M.2 slot, (which is only PCIe 3.0 2-lane), & 4 SATA means small NAS at most, unless you use some M.2 to PCIe adapter thing for a SATA / SAS expansion card. (USB attached storage may be fine for some people. But whence you get to either larger installations or TrueNAS, USB storage is much less of a real option.)
I am not trying to bring ARM down. Just pointing out that NAS, Network Attached Storage, is a specialized use case. Generic computers, or servers, may not be suitable for many NAS applications.
It is like a network firewall & routing board. While a generic computer board can be made into such, having specialized network chips, (aka ones with TCP and other off-load capabilities), can make the difference in usability for purpose.
1 Like
Well I’m not Jeff or Wendell so Ampere isn’t going to give me a multi-thousand dollar workstation for free (-; , my thought was that it’s good enough to do porting work.
Good point for those interested.
However, my point was that porting of TrueNAS SCALE would be useless until their were actual suitable ARM64 servers that offer reasonable features for a NAS.
I have a soft spot for RISC-V since it is an open CPU architecture. It takes a different approach for vector instructions than other CPU designs. Intel & AMD, (plus probably ARM), use specialized instructions that work on vectors based on data size. RISC-V uses variable width data, (if I understand it correctly), so does not have dozens of instruction sets for that purpose.
However, RISC-V boards is also not suitable for most tasks. Not even simple desktops, (yet). Even the company producing a RISC-V tablet states it is for developers only.
I think you and some of the others in this forum are bordering on gatekeeping. What is a “proper” NAS and what are “proper” features? What is a “real” NAS board. What makes a generic computer or server not suitable for simple file serving and an app or two?
Synology, QNAP, Buffalo, etc. all seem to be doing just fine selling small NAS boxes, with app support, that have boards that are two to three times slower and less feature filled than this board.
4 x 20 TB → 51 TIB raidz1
2 x 2 mirrored 20 TB → 35 TiB
Have we really reached the point where if you’re not running a fire breathing Supermicro Xeon/Epyc bazillion GB ECC RAM Petabyte rack mounted system that we’re not good enough for this community or TrueNAS?
I think the biggest issue with ARM boards is lack of the official ARM SystemReady support. SBC boards usually have custom boot loaders, running patched (and outdated) Linux kernels. Most of them can’t run a standard Debian ISO installation OOTB.
That means, IMO, separate TrueNAS Aarch64 images for UEFI (like Ampere), Raspberry Pi, Rockchip, etc…
It could be potentially also be an overlay for Armbian, instead of a whole OS: Download - Armbian
1 Like
What’s the point of porting work if there’s no suitable hardware to deploy for production?
The multi-thousand Ampere One/Altra systems are suitable for large deployment. Fruits-Pi are not suitable for use as ZFS NAS. And there’s not much in between which would make a decent home lab ZFS NAS—“decent” as in “ECC and enough available PCIe lanes”, “home” as in “not the $$$$ workstation”.
1 Like
Why is this a bad thing?
Boo hoo. TrueNAS has always had substantial hardware requirements and (in all likelihood) always will. Emotive language like “not good enough” and “gatekeeping” contributes nothing to the discussion.
Door’s open to anyone who’s interested. The fact that nobody really has invested their own time does tend to suggest that maybe it really isn’t good enough to be worth the work involved.
In someways you are right.
Occasionally we get people running very low end hardware and have trouble. For example, one current forum thread is about a 8GB RAM TrueNAS server running out of memory which seems to prevent data pool import. Now their may be something else at stake, the issue is still being investigated. But, 8GBs is the bare minimum of memory. If that user could add more memory, the problem might just go away.
My points are to help people make reliable, somewhat expandable NAS that can run the TrueNAS software, (Core or SCALE). At present, in my opinion, their are not enough suitable ARM64 boards for worthwhile porting of TrueNAS SCALE.
Now that does not prevent anyone from making their own NAS server with ZFS. I have a miniature PC that I use as a media server, which uses ZFS, both for OS pool and data pool. It works fine. As does my very old, (2014, but using 2012 technology), laptop with only 2GBs of RAM. It too uses ZFS, both for OS pool and separate dumping ground pool.
Back to the subject of porting TrueNAS SCALE, (Core is being deprecated), to ARM64.
How many ARM64 boards are suitable for this purpose, right now?
Since USB attached storage is not recommended with TrueNAS, (their have been far too many problems with permanently attached USB storage devices), how many of those boards have enough SATA, SAS or NVMe ports?
With x64 we have hundreds of boards, and hundreds of CPU choices. Both, 5 or so years old and new. With ARM64, how many choices? And does that justify major software porting work?
I am not trying to prevent an ARM64 TrueNAS SCALE version. Just want someone to walk into the project with their eyes wide open and know that the amount of users that want both ARM64 and whatever amount of storage ports are available is much smaller, today, than x64.
So, imagine an ARM64 port of TrueNAS SCALE only being used by dozens, (or hundreds), of people, (to start with), because of the lack of storage ports, PCIe slots or 10Gbit/ps Ethernet. Then imagine some upon expanding they find that they have to replace their ARM64 boards with either larger ARM64 boards. Or just migrate to x64.
Steering people to reliable, somewhat expandable hardware then makes some sense, (today).
2 Likes
Chicken and egg…
Fruits-Pi are not suitable for use as ZFS NAS
Why? There are multiple SBCs with adequate capabilities for simple home and SMB file sharing and an app or two.
Please name just ONE SBC with at least 6 SATA ports, 10 Gb/s NIC or the PCIe connectivity to add such a NIC, and ECC RAM.
1 Like
What part of “simple home and SMB file sharing and an app or two.” didn’t you understand?
What part of “ZFS” didn’t you understand?
2 Likes
Get building, then–the code is open-source, and you’ll have a killer product.
2 Likes
Even if an ARM64 board did not have what @etorix suggested, can you name ONE ARM64 board with something like this?
- 2 or more SATA ports, with reasonable performance. Meaning not a SATA 1.x, (aka 1.5Gbits/ps), and especially not using a SATA port multiplier, (which even Intel says are less than reliable or performant).
- Or even multiple NVMe ports?
- Decent network, (this is about a NETWORK Attach Storage server after all). So a USB attached Ethernet chip, (via on-board USB switch like the original Raspberry Pi or externally), is likely not going to be reliable. Maybe suitable for low home use, but that asks the question why are they trying to use TrueNAS?
- Acceptable boot device. Small board’s eMMC or SD Cards don’t qualify because TrueNAS uses ACTIVE ZFS boot pools, which could eat them up yearly.
- Minimum of 8GB of memory, without sharing any of that with GPU. Meaning if there is a GPU and requires a noticeable amount of CPU RAM, then more than 8GBs will possibly be needed to account for the “lost” memory due to the GPU.
Remember, using USB attached disks for permanent part of a data pool has simply shown it to be unreliable in the long term. PLENTY of small or tiny home NASes use USB attached disks, but I doubt most or any of them use ZFS. ZFS can cause tremendous I/O loads during data scrubs, that can cause USB interface chips to over-heat, or find corner case bugs that cause problems. And turning off ZFS scrubs simply asks the question, “Then why use ZFS? Or TrueNAS?”.
Please note that I am not trying to be a nay sayer. I simple don't follow the ARM64 single board computers that much, so you may have more recent or complete information that me, (or the rest of us).
All in all, as I have said before:
There is no one NAS to rule them all
TrueNAS was simply not written for the very low end of hardware. It may work, and work fine for that user. But, when problems develop, like using less than 8GBs of memory, the answer might just be “Get more / better hardware”, (like add more memory).
2 Likes