Problem/Justification
I want to backup/mirror my NAS to another NAS just to be sure that data is kept when major upgrades are due
Impact
TrueNas Core 13.06 (BSD based) supports the Mellanox Connectix3 40 GBit Ethernet out of the box, migrating to TrueNAS scale shows that Mellanox 40GBit Ethernet is not supported anymore.
User Story
The User wants to backup a whole NAS with 10+TB to another NAS the Bottleneck is Network in most cases, with 40GBit Ethernet and a fast CPU the Bottleneck will be HDD which shows that everything else is as fast as can be. Also NVME over Ethernet makes sense with 40 GBit Ethernet.
Mellanox which was bought by nvidia provides drivers for their products here:
”network dot nvidia dot com/products/ethernet-drivers/linux/mlnx_en/”
So please bundle the drivers into the Debianish Linux you distribute.
Mounting the /usr +rw and chmod +x for dpkg is a pain. As every manual installation will be overwritten by the next update as there are images to be installed not packages as common using Linux.
Thanks a lot for reading.
I need support for Mellanox connectix-3 and newer connectix-4, 5 boards.
So I managed to install the mellanox software on the truenas, but I had to do the following:
mount -o remount,rw ‘boot-pool/ROOT/25.10.0.1/usr’ /usr
then move all fake apt tools in /usr/local/bin to _apt-…
chmod +x /usr/bin/apt-…
important is apt-key because otherwhise one gets the warning about repository keys as there is a fake tool for it in /usr/local/bin.
So then I installed the debian packages of nvidia / mellanox by hand as nvidia installer does not recognize the debian distro.
Important is to install the dkms package to get a kernel module and here one needs to install all dependencies to get dkms, gcc, and so on.
This all went quite successful.
One also needs the melanox firmware tools mft to check that both interfaces are configured for ethernet
mstconfig -d 01:00.0 query
the number is the pci-bus-id you get using lspci
I wrote a one liner and have put it into /root so every time I want to change something I call it:
root@truenas[~]# cat rw.sh
#!/bin/bash
mount -o remount,rw ‘boot-pool/ROOT/25.10.0.1/usr’ /usr
A long travel just because truenas does not support the gbit ethernet out of the box.
So please make it a short travel for all, so one does not need to “void the warranty” by using apt.
The software by nvidia/mellanox provides support for Mellanox connectix-3 and newer connectix-4, 5 boards with 40Gbit, 100GBit Ethernet.
