The reason behind that is the cost of such solution,
It’s as easy as just electrically wiring more than one socket, each with one pcie lane. No extra switch chip needed, just enough power to run such setup (nvme requires more than other m.2 cards) and kernel support (bifurcation). No BIOS option is needed like in x86.
In CM3588 board they have decided to go with 4x * 1x pcie3.0. As You mentioned it should be ok for 2.5G, but You can also add 10G m.2 board to one of slots (they are 2x pcie 3.0, so it will be limited to about 8Gbit, still more than 2.5G). It should be also possible to connect 5Gbit.
Is this setup ok? For some use cases it may be fine. Personally I think that 2x2 division is better as in ROCK 5B+ SBC. One slot alone with 4x pcie was just too fast, having two gives more options and some redundancy if You build NAS. Also there are many cards that uses just one or two lanes so You will not get more than this, with two slots You can use two different cards.
BTW: also it should be possible to use PLX switch chip with RK3588, it should divide lane bandwidth automatically. I saw cards that has two m.2 slots and two 10G ethernet ports. The main problem is that broadcom chip is just expensive, such card cost about the price of whole SBC with 32GB of RAM and uses quit much power, produces significant heat… and it’s larger than most SBC’s.
Yes, all this is far away from most x86 build with ton of pcie lanes ready to use. In ARM world it’s nothing strange to use cards limited by it’s port. You can get it’s full bandwidth, just use it wisely For CM3588 I would attach 10G to first slot, and three ASM1064 m.2 cards (each is 1x pcie 3.0 to 4x sata-III ports) or ASM1166 (2x pcie with 6 ports, but You don’t need to use all ports, and with 3.5 inch hdds it’s still not reaching limit). This makes quite good and cheap NAS with good network connectivity It just don’t have to be 4x nvme board, but that also have it’s own features (redundancy, small size).
I just wish there were better storage operating systems for things like that CM3588 kit, I don’t think Openmediavault is fully up to the task.
Now if that CM3588 kit supported 10gbe, I would have bought it. I’d gladdly live without the two USB3 ports, give me a single USB2 port for local keyboard and mouse. I’d even go without the HDMI input if it would give me the networking I want.
If it goes on sale cheap enough, I’ll grab one and just convert my lab over to 2.5gbe. But I am also thinking about the 10gbe option card with SATA drives through one of the other nvme sockets.
With this promising product ARM is back in the game with rich i/o and fast, modern SOC… still huge hole for anything else than OMV for this arch to build nice NAS.
There additional two in m.2 A+E slot,
Total 14x pcie 4.0
This makes really reasonable space for NAS build. Of course it’s not as much as You can find on most servers or big X86 motherboards, but quite much to build 10G multi disk setup.
Well it can take a HBA and an expander can run off molex so there isn’t really an upper limit on the HDD capacity and it can take A+E slot coral TPU so it can run frigate. If it doesn’t do transcoding it can run Jellyfin, and Nextcloud isn’t exactly demanding.
Personally I also don’t have IPMI on my TrueNAS systems here either. Only time I ever had a monitor/keyboard connected was during the initial system install, but again handy if it is available!
Yes, and your question was “what else could someone want”. Likewise for ECC. I don’t claim that either is essential, but they’re both, at least, nice to have. A $25 NanoKVM does “well enough” as a substitute for IPMI, but it of course doesn’t have the integration into the system that IPMI would–it can’t cycle power, measure power consumption, report chassis intrusion, etc.
It’s a rhetorical question! I forgot I’m on a tech forum.
etorix, it has dual 5g ethernet sockets so it shouldn’t need a 10g NIC.
I think it’s a far point that these developments open up more choices for consumers and that is a good thing. They are not suitable for everyone. Even if iX decide this is not a viable path for them as a company, there is still Ubuntu server with ZFS available on arm64 for the home user.
This is what I continue to struggle with. I won’t deny that there may be a use case for low-drive-count, low-power applications, like let’s say a raspberry pi media server. Having ZFS run on those with its checksums is definitely a step up from EXTFS or whatever.
But why bother with TrueNAS when a better use case is installing ZFS for ARM (which I believe exists) and running that efficiently close to metal instead? Yes, it’s all command line but it can be done today without gobs of work cross compiling on a different platform.
I don’t even want to think about how difficult it will be to get everything to behave as intended, never mind drivers for all that hardware out there.
It is interesting and might be an OK ARM dev board for working the bugs out of Truenas Scale on ARM, might be a better choice than the cm3588 NAS board above since you can start working with real professional storage cards. But I don’t see this specific model being great for anything but development, same as the cm3588 board above.
That said, the server world has “been going to ARM” for what, a decade now? And we can see how well this has gone. Even AMD has been lacking in this area for years, and there are some real advantages to using AMD processors with tens of PCI lanes (hoping to buy AMD and NVME for my next hypervisor storage).
Also most of the ARM processors should not need a Coral module, they almost all have some NPU on the chip, so Frigate should be OK.
And all that said, yes I’d like to see Truenas on ARM, but I’m not smart enough or have enough time to work on this for myself. If I see that cm3588 NAS on sale cheap enough, I’m buying one to work with. Probably just OMV and Frigate and Home Assistant work for that board. Money is the object here, I just spent over $1000 on my “home” lab in the goal of reducing heat and noise and power draw while expanding the networking (24x10gbps and 2x40gbps). Tip of the week, DO NOT keep track of expenses on “hobby” items (or home lab), it only leads to grief about the amount you sink into these things. And I’m far from the most expensive home lab rig that I see, also not the cheapest.
ARM cores, licensed stuff has been floating all over for years. For example, Apple’s silicon is a licensed derivative of ARM and has made them independent from Intel (x86), Motorola (PowerPC & 68K), etc. thanks to the rise of independent fabs, Apple’s scale, and in-house expertise.
That strategically-important independence is one of the reasons that Apple is a leader in the computer / tablet / phone / etc. space. Scale + independent fab foundries enabled it. Gone are the days that Intel could reportedly afford to hire away every competent fab engineer at Motorola and thereby sabotage the PowerPC alliance.
Between Intel and AMD, I have a pretty hard time understanding what hole ARM would plug in the server world. Arguably, the bigger issue is how few server motherboards seem to be designed for HDD file server applications these days. SM seems to be coasting on the X10SDV series for that role and doesn’t have anything close to those D-15xx boards in AMD versions as far as I can tell.
More generally, SATA is being phased out. Consumer motherboards have less ports, and server boards possibly none at all. EPYC (the real ones, not 4004) still have flexible I/O lanes in the SoC which could be used as either SATA or PCIe, but EPYC 8004 “Siena” is not quite what one would want for a home NAS.
X11SDV and X12SDV were going for more compute, and more power, compared with X10SDV. And X13SDV will likely not have SATA at all, like the non-embedded Xeon 6.
I do not see a successor to Xeon D-1500 or Atom C3000 from either Intel or AMD.
And ARM is not going to help either if one has to pair these CPUs with a 10-15 year old SAS HBA for use as low-power home NAS.
For CM3588 I would attach 10G to first slot, and three ASM1064 m.2 cards (each is 1x pcie 3.0 to 4x sata-III ports) or ASM1166 (2x pcie with 6 ports, but You don’t need to use all ports, and with 3.5 inch hdds it’s still not reaching limit). This makes quite good and cheap NAS with good network connectivity 