Support plans related to RoCE in storage protocols (iSER, NVMe-oF, SMB Direct, NFSoRDMA)

TrueNAS General
, , ,
21

Edit: TL/DR good reason is at the bottom.

I would take you up on the tester license. I’m not that smart, but I already implemented RDMA and iSER using scstadmin, Connectx-3, with a Debian initiator, I just can’t use the GUI or else it overwrites the scst.conf and I lose the FC target config.

Now, I will feel smart if I can make iSER work with my ESXi version, lol. Try having no vendor support there.
I do this stuff for the fun and glory of learning how.

I don’t expect any IX systems direct support on enterprise features, nor am I advocating for all of the Enterprise features (that businesses should pay for) to be available freely in a manner that lets them circumvent paying for the r&d it takes to implement solid products. I’m just on an eBay budget.

Heck, as you pointed out, modern hardware can beat these things we’re taking about all day long. I still use 4gb FC, ConnectX-3, and HP Gen 8, cause that’s real businesses trash on eBay, which I love to buy :slight_smile:

Anyway, I’m still planning to migrate from Core to Scale so I can maybe switch from FC to iSER for a little while.

Sign me up for your NFR/Test license though!?

Also, seconding the use case Rand mentioned. For me also, TN being shared storage for hypervisor (s) like VMware until I finally stop clinging to the past and just get on Proxmox, using old hardware for education and testing.

An actually good reason: RDMA seems to be becoming hugely important to the industry and we’ll need to be up on how losses networks work.

Also @Rand , I wish I had some of that NMVe money :money_mouth_face:

22

100Gbe NICs and RoCE capable switches are becoming quite affordable. Mikrotik just enabled RoCE support on their switches and there are plenty of capable NICs our there for cheap.

I think there is an appetite for a paid license for enterprise features. Many organizations like the one I work for have a list of approved hardware vendors and iX systems isn’t on it. A software license on the other hand is very doable.

1 Like
23

Thanks for the description of the problem. I do think we will need to provide hardware if HA and enterprise-grade support is required. Please ask them to approve us :slight_smile:

24

Hello, it’s me, unique.

Where do I get my license? :grin:

3 Likes
25

My home network is 100 GbE. I now have to migrate away from TrueNAS and use Windows Server with OpenZFS to get RDMA / RoCE.

1 Like
26

As a long-time TrueNAS SCALE user (last on version 22.10), I’ve noticed recent changes with some features moving behind a paywall, which I understand is necessary to support development. However, I see no mention of a dedicated Homelab Edition. Many homelab enthusiasts, like myself, are using 25Gb or 100Gb NICs, which are now quite affordable—recent models can be found on eBay for around $250 if you shop carefully. Are there plans to introduce a Homelab Edition license tailored for such users, or is this unlikely to happen?

2 Likes
27

As someone who basically has a datacenter in his basement, I can say that I was generally unaware of the “general availability” of 25Gb/100Gb switches on places like eBay at those prices. I would love some 25Gb hardware, but I don’t think its quite out there yet.

Lets follow Broadcom here, cause that’s probably the best way to figure out pricing.

2 years ago I bought a Cisco Nexus N9K-C9396-PX for around that $250 price point, and it’s the last generation(ish) 10/40G hardware (Broadcom Trident II).

In Cisco Land, the switch I have was “soft” replaced by the 93128-TX (10/40G but “Broadcom Trident II+”) and hard replaced by the Nexus 93180YC-EX (1/10/25 and 40/100). The “EX” line is barellllyyy outside of it’s last ship date, and is therefore extremely expensive in the grey market still.
https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/nexus-c93180yc-fx-c93108tc-fx-fixed-switches-eol.pdf#:~:text=The%20last%20day%20to%20order%20the%20affected%20product(s),shown%20in%20Table%201%20of%20the%20EoL%20bulletin.

Similar situations can be found with HP/Aruba, as an example, since they are my personal “number 2” switch vendor I check for deals on. However, Arista looks like they had something thats viable https://www.arista.com/assets/data/pdf/Datasheets/7060X_7260X_DS.pdf.

There’s a generation of hardware—now nearly nine years old—built around Broadcom Tomahawk 1 for 100G networking. Most of it runs ONIE, is out of service and out of warranty, and was originally designed for hyperscalers rather than traditional datacenters. So yes, they do exist, but they are a bit “weird” to someone who prefers tradational datacenter networking.

Many of us are waiting for the good enterprise gear to trickle down still…but it’s all still a little bit too “new”. 10/40G is still very much alive.

28

I see they support this now: NFS over RDMA support this is something i would like to test with useing direct connect to two other host’s and this feature.

And search for this card: OEM Intel 100GbE E810-CQDA2 2Port QSFP28 Eth Network Adapter E810CQDA2G2P5

Also mikrotik has affordable switches that support 100gbps and rdma

29

Yea, we’ve been asked about it enough now that I think it has to happen before too long :slight_smile:

5 Likes
30

Thanks for your response Kris for the record im building out a new homelab and testing diffrent hypervisors etc. is there any chance to get a NFR license or a trial cant find anything about it.

31

We don’t have the capability to hand those out… Yet… Stay tuned.

6 Likes
32

Any idea when that could be like a month or 6 months ?

33

Kris,

Like the others, we’ve spoken about this too. I’m interested also. I remember one of the concerns being legitimate businesses that should be doing enterprise support trying to circumvent the costs of the product (TrueNAS) that enables them to make money. That’s lame.

I mention the above because I assume you don’t want them to just use a lab/hobby version of an enterprise license, which would need to be considerably cheaper than a real enterprise license (otherwise we couldn’t afford it).

Maybe the community can give ideas on what capabilities they need like speed, number of interfaces, protocols, capacity, pools, etc that would allow them to use a hobby license.

It doesn’t do the community any good if the technical limitations of their cheap hobby license doesn’t give them the features/capacity they need. It does the entire community and iX no good if businesses try licensing their enterprise features with hobby licenses and not provide the funding back to IX to support the solution that TN/iX provides to their business.

I imagine all professional hobbyists and tech geeks like the people here understand this and really want everyone to win.

As an example, I am using three physical hosts running VMware’s ESXi and wanted to try doing RDMAoE with old Mellanox 40gbps (Likely to change after the Broadcom douchery). I have a couple of pools: One SSD for VMs, one HDD for VMs, and one for NAS files.

So, if hobby licenses are wanted for RDMA or Fibre channel(?), would everyone be OK with restricting the pools to XTBs?, number of pools?, number of initiators/hosts to subscribe storage? Just questions.

1 Like
34

@corycandia I am glad you wrote this. I’ve been trying to formulate something analogous (and far less eloquent).

If TrueNAS sold a “Home”/“Non-Commercial” license, I’d buy one on the spot. I would expect that it unlocks features in exchange for the money, but I wouldn’t be expecting 24/7/365 on-call support. It would be nice if iX could find a way to justify shifting the license revenue recognition. In this case it would, presumably, be a much higher margin than anything sold to enterprise customers.

[Edit: this paragraph may be orthogonal, but there are seemingly multiple discussions going on concurrently on the topic of licensing, so I’m adding my general thoughts here.] I think storage protocols would be top of my list personally, but I’d also expect the niceties that go along with a “storage protocol pack” license. In other words, include .zfs snapshotdir support in NFS, iSCSI fast block cloning, the SMB fast clone support for Veeam, and so on. Speaking to that desire for myself, I use Veeam’s non-commercial licenses at home because it’s fantastic software, and I’m on the verge of buying a community license for Proxmox for a new build, which I’d like to partially back with storage from my TrueNAS server.

I think it would be a tremendous mistake to use a license to enforce limits that aren’t there without a license, generally. Especially if those limits applied to something you weren’t purchasing from iX, like drive capacity. If limitations must be added (which shouldn’t be assumed) then those limitations should only be on the “thing” being purchased (e.g., here it would be protocol support). I think anything else risks alienating the more diehard installations/users, which even if not revenue-generating by default, are immensely valuable to iX.

Proxmox has the right idea on community licensing IMHO.

[sorry for a million edits. I haven’t finished my coffee and shouldn’t be allowed near the keyboard yet.]

1 Like
35

Regarding the community license, I’d like to remind everyone that TrueNAS is currently advertised as an open source system. Even the license services are open source (currently). You can find the relevant information on GitHub.

Because of this, if you want to avoid these licensing restrictions, anyone can modify the middleware or view the code base of the relevant license services to understand how it works.

2 Likes
36

Of course, if ixsystems wants to generate this revenue, I’m not opposed to a reasonably priced community license.

But whether this revenue can improve the development of TrueNAS is something ixsystems needs to consider.

37

TrueNAS is advertised as open source, yes. However not all the code in the repos is open source licensed. Personally, I don’t like to cross that line. I’d rather buy a license.

1 Like
38

So its been a few months - any news ?

In the planning phase for homelab redesign (VMWare) and wonder which role Truenas will be able to play …

39

Count me in as well. I was about to buy 100 gigabit gear now that they are affordable for us mere mortals. The new Ubiquiti switch is calling my name given its price point and 100 gb NICs are obtainable for a few hundred dollars. Would be nice to actually be able to leverage the available performance from the hardware side without the software limitations.

My system is a 2nd gen Epyc machine so it’s not overly new or anything, but is still more than plenty for this use case. I know it’s tricky to balance the licensing model appropriately. Thank you for anything you’re doing to address this so we can leverage things like NFSoRDMA, NVMe-oF, etc.

40

@Solinus

…I so hate you right now… (sarcasm/jealousy)