Chelsio Communications Inc T580-LP-CR Unified Wire Storage Controller is not seen any truenas scale. 24.10 or 24.04
Hey @kspare 
Does the card show in lspci at all?
it does, but it shows a firmware issue and wont initialize. works fine under core
can you post the output of the dmesg/lspci so we can ID?
Purely theorycrafting here; the Linux Chelsio driver might be new enough that it wants to selectively load firmware modules for things like FCoE, so if it isn’t present in SCALE that might be the tripping point.
83:00.6 Fibre Channel: Chelsio Communications Inc T580-LP-CR Unified Wire Storage Controller
Can you lspci -vv the entire relevant section around the card? That’ll show driver/capabilities/etc.
83:00.6 Fibre Channel: Chelsio Communications Inc T580-LP-CR Unified Wire Storage Controller
Subsystem: Chelsio Communications Inc T580-LP-CR Unified Wire Storage Controller
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin C routed to IRQ 5
NUMA node: 1
IOMMU group: 14
Region 0: Memory at fa100000 (64-bit, non-prefetchable) [size=512K]
Region 2: Memory at fa080000 (64-bit, non-prefetchable) [size=512K]
Region 4: Memory at fac00000 (64-bit, non-prefetchable) [size=8K]
Expansion ROM at fa000000 [disabled] [size=512K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/32 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 2048 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM L0s L1, Exit Latency L0s <256ns, L1 <64us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 8GT/s, Width x8
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp+ ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [b0] MSI-X: Enable- Count=40 Masked-
Vector table: BAR=4 offset=00000000
PBA: BAR=4 offset=00001000
Capabilities: [d0] Vital Product Data
Product Name: T580-LP-CR
Read-only fields:
[PN] Part number: 110116450C1
[EC] Engineering changes: 0000000000000000
[SN] Serial number: PT43140225
[RV] Reserved: checksum good, 165 byte(s) reserved
Read/write fields:
[RW] Read-write area: 249 byte(s) free
End
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [170 v1] Device Serial Number 00-00-00-00-00-00-00-00
Capabilities: [190 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 7
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [1a0 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Capabilities: [1c0 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration- 10BitTagReq- Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy- 10BitTagReq-
IOVSta: Migration-
Initial VFs: 0, Total VFs: 0, Number of VFs: 0, Function Dependency Link: 00
VF offset: 8, stride: 4, Device ID: 5810
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 0000000000000000 (64-bit, non-prefetchable)
Region 2: Memory at 0000000000000000 (64-bit, non-prefetchable)
Region 4: Memory at 0000000000000000 (64-bit, non-prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [200 v1] Transaction Processing Hints
Interrupt vector mode supported
Steering table in MSI-X table
Oct 3 09:08:37 truenas kernel: cxgb4 0000:83:00.4: Successfully configured using Firmware Configuration File “On FLASH”, version 0x1, computed checksum 0x30b6a157
Oct 3 09:08:37 truenas kernel: cxgb4: probe of 0000:83:00.4 failed with error -22
ct 3 09:08:37 truenas kernel: cxgb4 0000:83:00.4: Successfully configured using Firmware Configuration File “On FLASH”, version 0x1, computed checksum 0x30b6a157
Oct 3 09:08:37 truenas kernel: cxgb4: probe of 0000:83:00.4 failed with error -22
Oct 3 09:08:37 truenas kernel: cxgb4 0000:83:00.4: Direct firmware load for cxgb4/t5-config.txt failed with error -2
Oct 3 09:08:37 truenas kernel: cxgb4 0000:83:00.4: Coming up as MASTER: Initializing adapter
Let me know if theres anything else youd like to try!
It seems to be picking this up in FCoE mode rather than regular Ethernet. I don’t have one offhand to check but you might need to push a config to the card’s EEPROM (possibly from a non-TrueNAS OS) to get it to behave like a regular network interface and therefore bind to the cxgb4 driver.
1 Like
how would I do that? it works perfectly fine under core?
@HoneyBadger so I checked on another box…
83:00.5 SCSI storage controller: Chelsio Communications Inc T580-SO-CR Unified Wire Storage Controller
83:00.6 Fibre Channel: Chelsio Communications Inc T580-SO-CR Unified Wire Storage Controller
Shows that no matter what. so I don’t think thats the problem.
This was under a CORE machine or a different Linux system - it was working as an Ethernet controller there?
That lspci output shows it registering as an FC HBA so it won’t pop up as a network card.
Yes sir, works fine. I haven’t seen an option to change it?
Have you submitted a bug report in to the Jira instance with the details? The only Chelsio card I have on hand is a T320 so it’s too old to use the current driver set.
No, @Captain_Morgan asked me to put a post here. Im just ordering new cards as I want this up and running with the same as my other servers.