I’ve had a variety of personal systems in my home lab over the past 15-20 years. I had rack mounts with all sorts of official ‘iX’ gear (TrueNAS, FreeBSD poudriere builders, PC-BSD hosting, etc etc).
Until last year I had a rack in my home office, with nearly 100T usable and somewhere around 100 drives spread across many systems. It was way overkill for what I needed and far too noisy for my home office.
This past year I downsized into desktop chassis and went all Flash (spinning needs to die), one system with 4x NVMe M.2 devices, the other using a 9500 Tri-Mode HBA so I can use 4x U.2 NVMe drives. Both systems are using the same 12th Gen Intel(R) Core(TM) i5-12600 CPUs, with onboard GPU for some apps, and one of the boxes has an ARC A380, for testing and Apps as well. One system always runs latest release of SCALE, the other I keep on nightly images for dev work and testing. I like to re-build my TrueNAS systems every 3-4 years, so I can get the fun of “system building” while also trying out new hardware combos first-hand.
Maybe in a future podcast I’ll go into more detail on the home rigs if anybody is curious. To answer your question of tech “worker” vs “enthusiast”, I’d say I’m a mix at this stage. I build systems for fun, have RPI devices aplenty, contribute the occasional code and App to TrueNAS. My last “for fun” tech experiment was installing TrueNAS on a Radxa x4, which worked surprisingly well
The n100 in my mini lab also works surprisingly well for the needs of the lab. Running Scale because the 10gb adapter isn’t supported in Core (and scale is really the way forward). Only 6 SATA spinning drives right now, didn’t want to spend the money on SSD yet.
Wish I had more time to throw into figuring out how to build Truenas on Arm.
Looks like this is going to be tough to do on a new platform. make checkout works fine. But then make packages, which is supposed to build all of the packages from the source that just got checked out, immediately makes a truenas scale chroot to build in. Which of course only exists on x86 right now.
The README says to use a debian chroot to build it and then it just makes its own environment? What the heck? Hopefully it won’t be too hard to have it create a debian chroot from the debian repos and build from there. But from the looks of it, one can’t simply build the os, one would need to setup repos as well.
@bobpaul - If I remember correctly, one of the iX people said that if someone found things to change, that help ARM64 builds, to make a Pull request.
It is completely understandable that a company developing software for a limited hardware set, did not take into account building on other Archs. But, they do seem receptive of making general purpose changes that allow building on ARM64.
Does it, though? How important is caching in an all-NVMe system?
Obviously it isn’t going to run TrueNAS, and if it did, it’d be missing the switch management aspect. But it really seems like they’d be better off working to integrate an existing NAS OS than trying to shoehorn those features into RouterOS.
ECC RAM, after all, every last bit goes through the RAM, whether it is network packet or NVMe block
It mentions 2 x M.2 SATA slots, can those be used for a larger OS? And booting? Or would the NAND need to be a pre-boot code location?
The U.2 storage is through 16x PCIe 3.0 and multiple PCIe switches, limiting each U.2 device to 2 lanes of PCIe 3.0
The SFF-8644 shows 4x PCIe 3.0 lanes, so their is some expansion.
CPU is wired to the Ethernet switch via 4 x 25Gbit/ps Ethernet. That limits individual streams.
The 1Gbit/ps Ethernet port is not part of the switch chip, it truly is out of band management.
However, all those questions / comments aside, for the price, it looks good. (Though for specialized hardware like that, it either needs Open Source software support or compelling vendor software.)
Fair point…
I guess it depends what is required of the NAS. But if we were to put ZFS on it, I guess ZFS would at least want all of its metadata cached in RAM. With U.2 drives now exceeding 100 TB apiece and 20 U.2 drives in total that could be A LOT of metadata.
Of course, there are some “hammer and nail” issues lurking around. Mikrotik has its own RouterOS, so they want to start from it and add a NAS component. We are a TrueNAS forum here, so we tend to put/want/dream ZFS everywhere.
I genuinely have no idea whether it would be easier to bring OpenZFS to RouterOS—and then decent NFS, SMB and iSCSI capabilities—or to bring all the niceties that Mikrotik invested into RouterOS into TrueNAS. I guess, however, that neither Mikrotik nor iXsystems are eager to go for such a merger
Probably not, though HexOS suggests iX is interested in working with others.
You’re right, of course, that this device isn’t ideal hardware for TrueNAS, even leaving aside ARM vs. x86–but it’s among the closest I’ve seen to good hardware for it, and to hardware that might start driving a port of TrueNAS to ARM. Something like this, but without the switch, would border on “compelling.” Give it expandable ECC RAM, well…
Run for the hills. The SMB file sharing presently built into routerOS can only be described as primitive. I have no doubt that Mikrotik is working on something better but I’d give them at least a year or two before the biggest issues have been uncovered and perhaps resolved.
I have a lot of respect for Mikrotik but even something as seemingly-simple as hosting a iTunes repository for a Sonos 1 system was beyond the capabilities of a HexS. Apple solved that problem with far less capable CPUs over 20 years ago. Speeding up the back end by deploying lots of NVME won’t solve the atrocious UI nor the wonky behavior.
Good point: RAM is ECC.
Bad points: RAM is soldered. U.2 bays are 7 mm, no hot-swap (powering off the switch to replace a drive? oops!). And storage configuration, including creating RAID arrays and shares is through the CLI.
“Badly” and “primitive” are hereby nominated for the Understatement Of The Year award…
If Patrick at STH comes away confuddled re: how this thing is supposed to work as a NAS, good luck to me or anyone else not extremely conversant re: MikroTik / routerOS terminal speak to make it happen.
I agree that this is an interesting gateway / router to help run a decent sized business. It has tons of connectivity! But using this as a NAS for hundreds of users will likely cause multiple admins to quit in protest.
I would hope that if you have hundreds of users, you will buy/build a real NAS. This Mikrotik device is designed for a handful of users, getting past about 10 and I would be looking for a different solution.
