TrueNAS Scale + HPe Nimble ES1 drive chassis fan speed/noise

[mike548141]

Hi

More of a hardware question but I thought someone here may have come across a fix/hack…
I would like to reduce the noise from the ES1 drive chassis fans, they are defaulting to a high RPM because they don’t receive any fan control messages. I was thinking there may be a command in the SAS/SES/SMP/SCSI protocol that may reduce fan speed, or a hardware hack to fix them to a lower speed.

I am using,

• An older generic PC: ASUS Z390 motherboard, i7 CPU, 64GB RAM, dual 10Gbe SFP+ network card etc. Nothing too special.
• A 2nd hand HPe Nimble ES1 (ES1-H65B) chassis with dual power supplies, dual controllers, and each controller has a dual port SAS card (2 x SFF-8088 SAS ports).
• A LSI SAS9207-8e host bus adapter (small form factor as its in a rack case), a couple of 1m SFF-8088 SAS cables connect the HBA to the ES1.
• 5 refurbished MDD 20TB 3.5" 6Gbs SATA drives, and some other smaller drives.
• Environment is is about 10-14° celsius, no obstructions to airflow, drives are 20-30°C and only have light workload, fans operational i.e. no sounds like ball bearings wearing etc.
• TrueNAS Scale 24.10.2.2

If I power up the ES1 without any drives or SAS cables connected the fans work as expected. They run at a low speed during the boot, then ramp up to full speed gradualy to test the fans, and then slowly reduce to the their lowest speed which is barely audible.

The ES1 does not have a management port (network or serial), there could be something on the circuit board (I’ve not opened the ES1 up).

Cheers,

MC

[mike548141]

Some info on SES for context

RajendraDivecha_SCSI_SES.pdf

MC

[LarsR]

Someone wrote a small docker image for fancontrol, search for truefan on GitHub, maybe it can help you

[mike548141]

Thanks @LarsR I will check that out.

I’ve since opened up the ES1 and found its really a re-badged SuperMicro chassis (case, circuit boards, and PSU’s).
Each controller has 6 fans installed in pairs using PWM for fan speed, and each PSU has at least 2 fans installed as a pair. The fans in the controllers are connected using standard molex connectors so it would be very easy to simply disconnect n pairs per controller to reduce the noise but I’m confident I can control them using SES commands to the controllers which is obviously preferable…

Also of note is a port that I assume is for a ethernet network jack labelled LAN on the circuit board, I need to see what type of interface this is… Might be able to add a management interface to each of the controllers?..
The chips above it is a WE-MIDCOM 7090-37 which is a 10/100 Base-T enternet controller

Pictures

[mike548141]

Sharing in case its useful. Next step will be to use sg_ses to send control commands to each of the fans. If that works I will drop it into a script and share.

lsscsi shows not only the drives but the enclosure

lsscsi -g
[0:0:0:0]    enclosu LSI      SAS2X36          0724  -          /dev/sg1 
[0:0:1:0]    disk    ATA      ST2000DM001-1CH1 CC27  /dev/sdc   /dev/sg2 
[0:0:2:0]    disk    ATA      ST31500341AS     CC1H  /dev/sdd   /dev/sg3 
[0:0:3:0]    disk    ATA      ST32000542AS     CC34  /dev/sdb   /dev/sg4 
[0:0:4:0]    disk    ATA      ST32000542AS     CC34  /dev/sda   /dev/sg5 
[0:0:5:0]    disk    ATA      ST6000VN0021-1Z8 SC60  /dev/sde   /dev/sg6 
[0:0:6:0]    disk    ATA      ST6000VN0021-1ZA SC61  /dev/sdg   /dev/sg7 
[0:0:7:0]    disk    ATA      ST6000VN0021-1ZA SC61  /dev/sdi   /dev/sg8 
[0:0:8:0]    disk    ATA      ST6000VN0021-1ZA SC61  /dev/sdh   /dev/sg9 
[0:0:9:0]    disk    ATA      ST6000VN0021-1Z8 SC60  /dev/sdj   /dev/sg10
[0:0:10:0]   disk    ATA      OOS20000G        OOS1  /dev/sdf   /dev/sg11
[0:0:11:0]   disk    ATA      OOS20000G        OOS1  /dev/sdk   /dev/sg12
[0:0:12:0]   disk    ATA      OOS20000G        OOS1  /dev/sdl   /dev/sg13
[0:0:13:0]   disk    ATA      OOS20000G        OOS1  /dev/sdm   /dev/sg14
[0:0:14:0]   disk    ATA      OOS20000G        OOS1  /dev/sdn   /dev/sg15
[0:0:15:0]   enclosu LSI      SAS2X36          0724  -          /dev/sg16
[1:0:0:0]    disk    ATA      Netac SSD 128GB  330   /dev/sdo   /dev/sg17
[2:0:0:0]    disk    ATA      Netac SSD 128GB  09BA  /dev/sdp   /dev/sg18
[3:0:0:0]    cd/dvd  HL-DT-ST BD-RE  BH16NS55  1.05  /dev/sr0   /dev/sg19
[7:0:0:0]    enclosu AHCI     SGPIO Enclosure  2.00  -          /dev/sg0 
[N:0:1:1]    disk    CT1000P1SSD8__1                            /dev/nvme0n1  -        
[N:1:0:1]    disk    WD_BLACK SN770 500GB__1                    /dev/nvme1n1  - 

So I queried the generic SCSI devices in Linux /dev using the SES protocol, that showed all sorts from voltage and temperature sensors to fans.

sudo sg_ses --all /dev/sg16
  LSI       SAS2X36           0724
  Primary enclosure logical identifier (hex): 500304801e6b997f
Drive Slots [0,-1]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
Slot 01 [0,0]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 0
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b994c
      phy identifier: 0x0
Slot 02 [0,1]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 1
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b994d
      phy identifier: 0x0
Slot 03 [0,2]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 2
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b994e
      phy identifier: 0x0
Slot 04 [0,3]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 3
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b994f
      phy identifier: 0x0
Slot 05 [0,4]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 4
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 06 [0,5]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 5
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 07 [0,6]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 6
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9952
      phy identifier: 0x0
Slot 08 [0,7]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 7
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9953
      phy identifier: 0x0
Slot 09 [0,8]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 8
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9954
      phy identifier: 0x0
Slot 10 [0,9]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 9
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9955
      phy identifier: 0x0
Slot 11 [0,10]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 10
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9956
      phy identifier: 0x0
Slot 12 [0,11]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 11
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9957
      phy identifier: 0x0
Slot 13 [0,12]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 12
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9958
      phy identifier: 0x0
Slot 14 [0,13]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 13
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9959
      phy identifier: 0x0
Slot 15 [0,14]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 14
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b995a
      phy identifier: 0x0
Slot 16 [0,15]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 15
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b995b
      phy identifier: 0x0
Temperature Sensors [1,-1]  Element type: Temperature sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, OT failure=0, OT warning=0, UT failure=0
    UT warning=0
    Temperature: <reserved>
P_Temperature [1,0]  Element type: Temperature sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0, OT failure=0, OT warning=0, UT failure=0
    UT warning=0
    Temperature=28 C
S_Temperature [1,1]  Element type: Temperature sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0, OT failure=0, OT warning=0, UT failure=0
    UT warning=0
    Temperature=36 C
Voltage Sensors [2,-1]  Element type: Voltage sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0,  Warn Over=0, Warn Under=0, Crit Over=0
    Crit Under=0
    Voltage: 0.00 volts
5V [2,0]  Element type: Voltage sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0,  Warn Over=0, Warn Under=0, Crit Over=0
    Crit Under=0
    Voltage: 5.03 volts
12V [2,1]  Element type: Voltage sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0,  Warn Over=0, Warn Under=0, Crit Over=0
    Crit Under=0
    Voltage: 12.14 volts
Power Supplies [3,-1]  Element type: Power supply
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Do not remove=0, DC overvoltage=0, DC undervoltage=0
    DC overcurrent=0, Hot swap=0, Fail=0, Requested on=0, Off=0
    Overtmp fail=0, Temperature warn=0, AC fail=0, DC fail=0
Power Supply 1 [3,0]  Element type: Power supply
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, DC overvoltage=0, DC undervoltage=0
    DC overcurrent=0, Hot swap=0, Fail=0, Requested on=0, Off=0
    Overtmp fail=0, Temperature warn=0, AC fail=0, DC fail=0
Power Supply 2 [3,1]  Element type: Power supply
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, DC overvoltage=0, DC undervoltage=0
    DC overcurrent=0, Hot swap=0, Fail=0, Requested on=0, Off=0
    Overtmp fail=0, Temperature warn=0, AC fail=0, DC fail=0
Fans [4,-1]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
P_Fan1 [4,0]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9480 rpm, Fan at third highest speed
P_Fan2 [4,1]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9120 rpm, Fan at third highest speed
P_Fan3 [4,2]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9360 rpm, Fan at third highest speed
S_Fan1 [4,3]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9240 rpm, Fan at third highest speed
S_Fan2 [4,4]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9840 rpm, Fan at third highest speed
S_Fan3 [4,5]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9600 rpm, Fan at third highest speed
T_Fan1 [4,6]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
T_Fan2 [4,7]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
T_Fan3 [4,8]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
Midplane SEEPROM [5,-1]  Element type: Enclosure
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Time until power cycle=0, Failure indication=0
    Warning indication=0, Requested power off duration=0
    Failure requested=0, Warning requested=0
1LFM27A6S12146526833625 AF-116883                   0xf9081fb3   [5,0]  Element type: Enclosure
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Time until power cycle=0, Failure indication=0
    Warning indication=0, Requested power off duration=0
    Failure requested=0, Warning requested=0
SAS Expanders [6,-1]  Element type: SAS expander
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0
RHS Expander [6,0]  Element type: SAS expander
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 28
    SAS address: 0x800403507f996b1e
    Attached connector; other_element pairs:
      [0] Mini SAS 4x, pl=255 [2]
      [1] Mini SAS 4x, pl=255 [2]
      [2] Mini SAS 4x, pl=255 [2]
      [3] Mini SAS 4x, pl=255 [2]
      [4] Mini SAS 4i, pl=255 [0]
      [5] Mini SAS 4i, pl=255 [0]
      [6] Mini SAS 4i, pl=255 [0]
      [7] Mini SAS 4i, pl=255 [0]
      [8] Mini SAS 4i, pl=255 [1]
      [9] Mini SAS 4i, pl=255 [1]
      [10] Mini SAS 4i, pl=255 [1]
      [11] Mini SAS 4i, pl=255 [1]
      [12] SAS Drive backplane, pl=0 [6]
      [13] SAS Drive backplane, pl=0 [7]
      [14] SAS Drive backplane, pl=0 [8]
      [15] SAS Drive backplane, pl=0 [9]
      [16] SAS Drive backplane, pl=0 [10]
      [17] SAS Drive backplane, pl=0 [11]
      [18] SAS Drive backplane, pl=0 [12]
      [19] SAS Drive backplane, pl=0 [13]
      [20] SAS Drive backplane, pl=0 [14]
      [21] SAS Drive backplane, pl=0 [15]
      [22] SAS Drive backplane, pl=0 [16]
      [23] SAS Drive backplane, pl=0 [17]
      [24] SAS Drive backplane, pl=0 [18]
      [25] SAS Drive backplane, pl=0 [19]
      [26] SAS Drive backplane, pl=0 [20]
      [27] SAS Drive backplane, pl=0 [21]
LHS Expander [6,1]  Element type: SAS expander
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 28
    SAS address: 0x800403507f996b1e
    Attached connector; other_element pairs:
      [0] Mini SAS 4x, pl=255 [5]
      [1] Mini SAS 4x, pl=255 [5]
      [2] Mini SAS 4x, pl=255 [5]
      [3] Mini SAS 4x, pl=255 [5]
      [4] Mini SAS 4i, pl=255 [3]
      [5] Mini SAS 4i, pl=255 [3]
      [6] Mini SAS 4i, pl=255 [3]
      [7] Mini SAS 4i, pl=255 [3]
      [8] Mini SAS 4i, pl=255 [4]
      [9] Mini SAS 4i, pl=255 [4]
      [10] Mini SAS 4i, pl=255 [4]
      [11] Mini SAS 4i, pl=255 [4]
      [12] SAS Drive backplane, pl=0 [6]
      [13] SAS Drive backplane, pl=0 [7]
      [14] SAS Drive backplane, pl=0 [8]
      [15] SAS Drive backplane, pl=0 [9]
      [16] SAS Drive backplane, pl=0 [10]
      [17] SAS Drive backplane, pl=0 [11]
      [18] SAS Drive backplane, pl=0 [12]
      [19] SAS Drive backplane, pl=0 [13]
      [20] SAS Drive backplane, pl=0 [14]
      [21] SAS Drive backplane, pl=0 [15]
      [22] SAS Drive backplane, pl=0 [16]
      [23] SAS Drive backplane, pl=0 [17]
      [24] SAS Drive backplane, pl=0 [18]
      [25] SAS Drive backplane, pl=0 [19]
      [26] SAS Drive backplane, pl=0 [20]
      [27] SAS Drive backplane, pl=0 [21]
SAS Connectors [7,-1]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, No information
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Upstream Connector 1 (RHS) [7,0]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Upstream Connector 2 (RHS) [7,1]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Downstream Connector (RHS) [7,2]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4x receptacle (SFF-8088) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Upstream Connector 1 (LHS) [7,3]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Upstream Connector 2 (LHS) [7,4]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Downstream Connector (LHS) [7,5]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4x receptacle (SFF-8088) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Drive Connector 00 [7,6]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 01 [7,7]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 02 [7,8]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 03 [7,9]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 04 [7,10]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 05 [7,11]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 06 [7,12]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 07 [7,13]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 08 [7,14]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 09 [7,15]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 10 [7,16]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 11 [7,17]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 12 [7,18]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 13 [7,19]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 14 [7,20]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 15 [7,21]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Ethernet ports [8,-1]  Element type: Communication port
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Disabled=0
Ethernet_port_1 [8,0]  Element type: Communication port
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Disabled=0
Ethernet_port_2 [8,1]  Element type: Communication port
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Disabled=0
Local SEEPROM [9,-1]  Element type: Enclosure services controller electronics
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Do not remove=0, Report=0, Hot swap=0
???????????????????????????????????????????????????????????????? [9,0]  Element type: Enclosure services controller electronics
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0, Do not remove=0, Report=0, Hot swap=0

And I tested that I could send a command via SES by turning the identifier LED on & off on the left most drive

# Turn the LED on
sudo sg_ses --dev-slot-num=15 --set=ident /dev/sg16
sleep 10
# Turn the LED off
sudo sg_ses --dsn=15 --clear=ident /dev/sg16

[mike548141]

No luck so far. I can retrieve the fan speed, I can make the fans surge to top speed but I’ve not been able to slow them down yet. I’m thinking maybe I could tweak the microcode on the fan controller to change the default speed to idle at?

Prints out the cooling detail for fan 0 from enclosure SG16

sudo sg_ses --index=coo,0 /dev/sg16
  LSI       SAS2X36           0724
Enclosure Status diagnostic page:
  INVOP=0, INFO=1, NON-CRIT=0, CRIT=0, UNRECOV=0
  generation code: 0x0
  status descriptor list
      Element 0 descriptor:
        Predicted failure=0, Disabled=0, Swap=0, status: OK
        Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
        Off=0, Actual speed=9120 rpm, Fan at third highest speed

Prints out the speed for fan 0 from enclosure SG16 as an integer. That integer can vary by make/model/firmware but for this controller it defaults to 5 which is about 8,500-9,500 RPM. Max speed is 7 which is about 13,000 RPM

sudo sg_ses /dev/sg16 --index=coo,0 --get=speed_code
5

Changes the fan speed for fan 0 from enclosure SG16. But no matter what I set it to the fans rev up to 7 and then gradually slow back down over the next ~20-40 seconds.
I’ve tried setting it to whole numbers from -4 to 7 and get the same response.

sudo sg_ses /dev/sg16 --index=coo,0 --set=speed_code=7
sudo sg_ses --index=coo,0 /dev/sg16                 
  LSI       SAS2X36           0724
Enclosure Status diagnostic page:
  INVOP=0, INFO=1, NON-CRIT=0, CRIT=0, UNRECOV=0
  generation code: 0x0
  status descriptor list
      Element 0 descriptor:
        Predicted failure=0, Disabled=0, Swap=0, status: OK
        Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
        Off=0, Actual speed=13320 rpm, Fan at highest speed

Similar command to the one above but 3:2:3 seems to work instead of speed_code

sudo sg_ses /dev/sg16 --index=coo,0 --set=3:2:3=7

Dump of all info from both controllers in the ES1 enclosure

sudo sg_ses --all /dev/sg1                  
  LSI       SAS2X36           0724
  Primary enclosure logical identifier (hex): 500304801e6ccdff
Drive Slots [0,-1]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
Slot 01 [0,0]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 0
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 02 [0,1]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 1
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 03 [0,2]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 2
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 04 [0,3]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 3
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 05 [0,4]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 4
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 06 [0,5]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 5
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 07 [0,6]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 6
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 08 [0,7]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 7
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 09 [0,8]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 8
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 10 [0,9]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 9
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 11 [0,10]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 10
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 12 [0,11]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 11
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 13 [0,12]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 12
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 14 [0,13]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 13
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 15 [0,14]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 14
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 16 [0,15]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 15
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Temperature Sensors [1,-1]  Element type: Temperature sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, OT failure=0, OT warning=0, UT failure=0
    UT warning=0
    Temperature: <reserved>
P_Temperature [1,0]  Element type: Temperature sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Fail=0, OT failure=0, OT warning=0, UT failure=0
    UT warning=0
    Temperature=0 C
S_Temperature [1,1]  Element type: Temperature sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Fail=0, OT failure=0, OT warning=0, UT failure=0
    UT warning=0
    Temperature=0 C
Voltage Sensors [2,-1]  Element type: Voltage sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0,  Warn Over=0, Warn Under=0, Crit Over=0
    Crit Under=0
    Voltage: 0.00 volts
5V [2,0]  Element type: Voltage sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0,  Warn Over=0, Warn Under=0, Crit Over=0
    Crit Under=0
    Voltage: 5.03 volts
12V [2,1]  Element type: Voltage sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0,  Warn Over=0, Warn Under=0, Crit Over=0
    Crit Under=0
    Voltage: 12.03 volts
Power Supplies [3,-1]  Element type: Power supply
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Do not remove=0, DC overvoltage=0, DC undervoltage=0
    DC overcurrent=0, Hot swap=0, Fail=0, Requested on=0, Off=0
    Overtmp fail=0, Temperature warn=0, AC fail=0, DC fail=0
Power Supply 1 [3,0]  Element type: Power supply
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    Ident=0, Do not remove=0, DC overvoltage=0, DC undervoltage=0
    DC overcurrent=0, Hot swap=0, Fail=0, Requested on=0, Off=0
    Overtmp fail=0, Temperature warn=0, AC fail=0, DC fail=0
Power Supply 2 [3,1]  Element type: Power supply
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    Ident=0, Do not remove=0, DC overvoltage=0, DC undervoltage=0
    DC overcurrent=0, Hot swap=0, Fail=0, Requested on=0, Off=0
    Overtmp fail=0, Temperature warn=0, AC fail=0, DC fail=0
Fans [4,-1]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
P_Fan1 [4,0]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
P_Fan2 [4,1]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
P_Fan3 [4,2]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
S_Fan1 [4,3]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
S_Fan2 [4,4]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
S_Fan3 [4,5]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
T_Fan1 [4,6]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
T_Fan2 [4,7]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
T_Fan3 [4,8]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
Midplane SEEPROM [5,-1]  Element type: Enclosure
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Time until power cycle=0, Failure indication=0
    Warning indication=0, Requested power off duration=0
    Failure requested=0, Warning requested=0
Error: Read/Write SEEPROM FAILED [5,0]  Element type: Enclosure
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Time until power cycle=0, Failure indication=0
    Warning indication=0, Requested power off duration=0
    Failure requested=0, Warning requested=0
SAS Expanders [6,-1]  Element type: SAS expander
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0
RHS Expander [6,0]  Element type: SAS expander
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 28
    SAS address: 0x80040350ffcd6c1e
    Attached connector; other_element pairs:
      [0] Mini SAS 4x, pl=255 [2]
      [1] Mini SAS 4x, pl=255 [2]
      [2] Mini SAS 4x, pl=255 [2]
      [3] Mini SAS 4x, pl=255 [2]
      [4] Mini SAS 4i, pl=255 [0]
      [5] Mini SAS 4i, pl=255 [0]
      [6] Mini SAS 4i, pl=255 [0]
      [7] Mini SAS 4i, pl=255 [0]
      [8] Mini SAS 4i, pl=255 [1]
      [9] Mini SAS 4i, pl=255 [1]
      [10] Mini SAS 4i, pl=255 [1]
      [11] Mini SAS 4i, pl=255 [1]
      [12] SAS Drive backplane, pl=0 [6]
      [13] SAS Drive backplane, pl=0 [7]
      [14] SAS Drive backplane, pl=0 [8]
      [15] SAS Drive backplane, pl=0 [9]
      [16] SAS Drive backplane, pl=0 [10]
      [17] SAS Drive backplane, pl=0 [11]
      [18] SAS Drive backplane, pl=0 [12]
      [19] SAS Drive backplane, pl=0 [13]
      [20] SAS Drive backplane, pl=0 [14]
      [21] SAS Drive backplane, pl=0 [15]
      [22] SAS Drive backplane, pl=0 [16]
      [23] SAS Drive backplane, pl=0 [17]
      [24] SAS Drive backplane, pl=0 [18]
      [25] SAS Drive backplane, pl=0 [19]
      [26] SAS Drive backplane, pl=0 [20]
      [27] SAS Drive backplane, pl=0 [21]
LHS Expander [6,1]  Element type: SAS expander
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 28
    SAS address: 0x80040350ffcd6c1e
    Attached connector; other_element pairs:
      [0] Mini SAS 4x, pl=255 [5]
      [1] Mini SAS 4x, pl=255 [5]
      [2] Mini SAS 4x, pl=255 [5]
      [3] Mini SAS 4x, pl=255 [5]
      [4] Mini SAS 4i, pl=255 [3]
      [5] Mini SAS 4i, pl=255 [3]
      [6] Mini SAS 4i, pl=255 [3]
      [7] Mini SAS 4i, pl=255 [3]
      [8] Mini SAS 4i, pl=255 [4]
      [9] Mini SAS 4i, pl=255 [4]
      [10] Mini SAS 4i, pl=255 [4]
      [11] Mini SAS 4i, pl=255 [4]
      [12] SAS Drive backplane, pl=0 [6]
      [13] SAS Drive backplane, pl=0 [7]
      [14] SAS Drive backplane, pl=0 [8]
      [15] SAS Drive backplane, pl=0 [9]
      [16] SAS Drive backplane, pl=0 [10]
      [17] SAS Drive backplane, pl=0 [11]
      [18] SAS Drive backplane, pl=0 [12]
      [19] SAS Drive backplane, pl=0 [13]
      [20] SAS Drive backplane, pl=0 [14]
      [21] SAS Drive backplane, pl=0 [15]
      [22] SAS Drive backplane, pl=0 [16]
      [23] SAS Drive backplane, pl=0 [17]
      [24] SAS Drive backplane, pl=0 [18]
      [25] SAS Drive backplane, pl=0 [19]
      [26] SAS Drive backplane, pl=0 [20]
      [27] SAS Drive backplane, pl=0 [21]
SAS Connectors [7,-1]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, No information
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Upstream Connector 1 (RHS) [7,0]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Upstream Connector 2 (RHS) [7,1]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Downstream Connector (RHS) [7,2]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4x receptacle (SFF-8088) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Upstream Connector 1 (LHS) [7,3]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Upstream Connector 2 (LHS) [7,4]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Downstream Connector (LHS) [7,5]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4x receptacle (SFF-8088) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Drive Connector 00 [7,6]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 01 [7,7]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 02 [7,8]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 03 [7,9]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 04 [7,10]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 05 [7,11]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 06 [7,12]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 07 [7,13]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 08 [7,14]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 09 [7,15]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 10 [7,16]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 11 [7,17]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 12 [7,18]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 13 [7,19]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 14 [7,20]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 15 [7,21]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Ethernet ports [8,-1]  Element type: Communication port
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Disabled=0
Ethernet_port_1 [8,0]  Element type: Communication port
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Disabled=0
Ethernet_port_2 [8,1]  Element type: Communication port
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Disabled=0
Local SEEPROM [9,-1]  Element type: Enclosure services controller electronics
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Do not remove=0, Report=0, Hot swap=0
???????????????????????????????????????????????????????????????? [9,0]  Element type: Enclosure services controller electronics
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0, Do not remove=0, Report=0, Hot swap=0
sudo sg_ses --all /dev/sg16
  LSI       SAS2X36           0724
  Primary enclosure logical identifier (hex): 500304801e6b997f
Drive Slots [0,-1]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
Slot 01 [0,0]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 0
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b994c
      phy identifier: 0x0
Slot 02 [0,1]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 1
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b994d
      phy identifier: 0x0
Slot 03 [0,2]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 2
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b994e
      phy identifier: 0x0
Slot 04 [0,3]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 3
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b994f
      phy identifier: 0x0
Slot 05 [0,4]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 4
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 06 [0,5]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Not installed
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 5
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for:
      attached SAS address: 0x0
      SAS address: 0x0
      phy identifier: 0x0
Slot 07 [0,6]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 6
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9952
      phy identifier: 0x0
Slot 08 [0,7]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 7
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9953
      phy identifier: 0x0
Slot 09 [0,8]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 8
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9954
      phy identifier: 0x0
Slot 10 [0,9]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 9
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9955
      phy identifier: 0x0
Slot 11 [0,10]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 10
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9956
      phy identifier: 0x0
Slot 12 [0,11]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 11
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9957
      phy identifier: 0x0
Slot 13 [0,12]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 12
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9958
      phy identifier: 0x0
Slot 14 [0,13]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 13
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b9959
      phy identifier: 0x0
Slot 15 [0,14]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 14
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b995a
      phy identifier: 0x0
Slot 16 [0,15]  Element type: Array device slot
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    OK=0, Reserved device=0, Hot spare=0, Cons check=0
    In crit array=0, In failed array=0, Rebuild/remap=0, R/R abort=0
    App client bypass A=0, Do not remove=0, Enc bypass A=0, Enc bypass B=0
    Ready to insert=0, RMV=0, Ident=0, Report=0
    App client bypass B=0, Fault sensed=0, Fault reqstd=0, Device off=0
    Bypassed A=0, Bypassed B=0, Dev bypassed A=0, Dev bypassed B=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 1, not all phys: 0, device slot number: 15
    phy index: 0
      SAS device type: no SAS device attached
      initiator port for:
      target port for: SATA_device
      attached SAS address: 0x500304801e6b997f
      SAS address: 0x500304801e6b995b
      phy identifier: 0x0
Temperature Sensors [1,-1]  Element type: Temperature sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, OT failure=0, OT warning=0, UT failure=0
    UT warning=0
    Temperature: <reserved>
P_Temperature [1,0]  Element type: Temperature sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0, OT failure=0, OT warning=0, UT failure=0
    UT warning=0
    Temperature=28 C
S_Temperature [1,1]  Element type: Temperature sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0, OT failure=0, OT warning=0, UT failure=0
    UT warning=0
    Temperature=35 C
Voltage Sensors [2,-1]  Element type: Voltage sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0,  Warn Over=0, Warn Under=0, Crit Over=0
    Crit Under=0
    Voltage: 0.00 volts
5V [2,0]  Element type: Voltage sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0,  Warn Over=0, Warn Under=0, Crit Over=0
    Crit Under=0
    Voltage: 5.03 volts
12V [2,1]  Element type: Voltage sensor
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0,  Warn Over=0, Warn Under=0, Crit Over=0
    Crit Under=0
    Voltage: 12.13 volts
Power Supplies [3,-1]  Element type: Power supply
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Do not remove=0, DC overvoltage=0, DC undervoltage=0
    DC overcurrent=0, Hot swap=0, Fail=0, Requested on=0, Off=0
    Overtmp fail=0, Temperature warn=0, AC fail=0, DC fail=0
Power Supply 1 [3,0]  Element type: Power supply
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, DC overvoltage=0, DC undervoltage=0
    DC overcurrent=0, Hot swap=0, Fail=0, Requested on=0, Off=0
    Overtmp fail=0, Temperature warn=0, AC fail=0, DC fail=0
Power Supply 2 [3,1]  Element type: Power supply
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, DC overvoltage=0, DC undervoltage=0
    DC overcurrent=0, Hot swap=0, Fail=0, Requested on=0, Off=0
    Overtmp fail=0, Temperature warn=0, AC fail=0, DC fail=0
Fans [4,-1]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
P_Fan1 [4,0]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9120 rpm, Fan at third highest speed
P_Fan2 [4,1]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9000 rpm, Fan at third highest speed
P_Fan3 [4,2]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9120 rpm, Fan at third highest speed
S_Fan1 [4,3]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9240 rpm, Fan at third highest speed
S_Fan2 [4,4]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9840 rpm, Fan at third highest speed
S_Fan3 [4,5]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=9480 rpm, Fan at third highest speed
T_Fan1 [4,6]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
T_Fan2 [4,7]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
T_Fan3 [4,8]  Element type: Cooling
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unknown
    Ident=0, Do not remove=0, Hot swap=0, Fail=0, Requested on=0
    Off=0, Actual speed=0 rpm, Fan stopped
Midplane SEEPROM [5,-1]  Element type: Enclosure
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Time until power cycle=0, Failure indication=0
    Warning indication=0, Requested power off duration=0
    Failure requested=0, Warning requested=0
1LFM27A6S12146526833625 AF-116883                   0xf9081fb3   [5,0]  Element type: Enclosure
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Time until power cycle=0, Failure indication=0
    Warning indication=0, Requested power off duration=0
    Failure requested=0, Warning requested=0
SAS Expanders [6,-1]  Element type: SAS expander
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0
RHS Expander [6,0]  Element type: SAS expander
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 28
    SAS address: 0x800403507f996b1e
    Attached connector; other_element pairs:
      [0] Mini SAS 4x, pl=255 [2]
      [1] Mini SAS 4x, pl=255 [2]
      [2] Mini SAS 4x, pl=255 [2]
      [3] Mini SAS 4x, pl=255 [2]
      [4] Mini SAS 4i, pl=255 [0]
      [5] Mini SAS 4i, pl=255 [0]
      [6] Mini SAS 4i, pl=255 [0]
      [7] Mini SAS 4i, pl=255 [0]
      [8] Mini SAS 4i, pl=255 [1]
      [9] Mini SAS 4i, pl=255 [1]
      [10] Mini SAS 4i, pl=255 [1]
      [11] Mini SAS 4i, pl=255 [1]
      [12] SAS Drive backplane, pl=0 [6]
      [13] SAS Drive backplane, pl=0 [7]
      [14] SAS Drive backplane, pl=0 [8]
      [15] SAS Drive backplane, pl=0 [9]
      [16] SAS Drive backplane, pl=0 [10]
      [17] SAS Drive backplane, pl=0 [11]
      [18] SAS Drive backplane, pl=0 [12]
      [19] SAS Drive backplane, pl=0 [13]
      [20] SAS Drive backplane, pl=0 [14]
      [21] SAS Drive backplane, pl=0 [15]
      [22] SAS Drive backplane, pl=0 [16]
      [23] SAS Drive backplane, pl=0 [17]
      [24] SAS Drive backplane, pl=0 [18]
      [25] SAS Drive backplane, pl=0 [19]
      [26] SAS Drive backplane, pl=0 [20]
      [27] SAS Drive backplane, pl=0 [21]
LHS Expander [6,1]  Element type: SAS expander
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0
  Additional Element Status:
    Transport protocol: SAS
    number of phys: 28
    SAS address: 0x800403507f996b1e
    Attached connector; other_element pairs:
      [0] Mini SAS 4x, pl=255 [5]
      [1] Mini SAS 4x, pl=255 [5]
      [2] Mini SAS 4x, pl=255 [5]
      [3] Mini SAS 4x, pl=255 [5]
      [4] Mini SAS 4i, pl=255 [3]
      [5] Mini SAS 4i, pl=255 [3]
      [6] Mini SAS 4i, pl=255 [3]
      [7] Mini SAS 4i, pl=255 [3]
      [8] Mini SAS 4i, pl=255 [4]
      [9] Mini SAS 4i, pl=255 [4]
      [10] Mini SAS 4i, pl=255 [4]
      [11] Mini SAS 4i, pl=255 [4]
      [12] SAS Drive backplane, pl=0 [6]
      [13] SAS Drive backplane, pl=0 [7]
      [14] SAS Drive backplane, pl=0 [8]
      [15] SAS Drive backplane, pl=0 [9]
      [16] SAS Drive backplane, pl=0 [10]
      [17] SAS Drive backplane, pl=0 [11]
      [18] SAS Drive backplane, pl=0 [12]
      [19] SAS Drive backplane, pl=0 [13]
      [20] SAS Drive backplane, pl=0 [14]
      [21] SAS Drive backplane, pl=0 [15]
      [22] SAS Drive backplane, pl=0 [16]
      [23] SAS Drive backplane, pl=0 [17]
      [24] SAS Drive backplane, pl=0 [18]
      [25] SAS Drive backplane, pl=0 [19]
      [26] SAS Drive backplane, pl=0 [20]
      [27] SAS Drive backplane, pl=0 [21]
SAS Connectors [7,-1]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, No information
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Upstream Connector 1 (RHS) [7,0]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Upstream Connector 2 (RHS) [7,1]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Downstream Connector (RHS) [7,2]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4x receptacle (SFF-8088) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Upstream Connector 1 (LHS) [7,3]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Upstream Connector 2 (LHS) [7,4]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4i receptacle (SFF-8087) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Downstream Connector (LHS) [7,5]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Mini SAS 4x receptacle (SFF-8088) [max 4 phys]
    Connector physical link=0xff, Mated=0, Fail=0, OC=0
Drive Connector 00 [7,6]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 01 [7,7]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 02 [7,8]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 03 [7,9]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 04 [7,10]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 05 [7,11]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 06 [7,12]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 07 [7,13]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 08 [7,14]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 09 [7,15]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 10 [7,16]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 11 [7,17]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 12 [7,18]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 13 [7,19]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 14 [7,20]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Drive Connector 15 [7,21]  Element type: SAS connector
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, SAS Drive backplane receptacle (SFF-8482) [max 2 phys]
    Connector physical link=0x0, Mated=0, Fail=0, OC=0
Ethernet ports [8,-1]  Element type: Communication port
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Disabled=0
Ethernet_port_1 [8,0]  Element type: Communication port
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Disabled=0
Ethernet_port_2 [8,1]  Element type: Communication port
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Disabled=0
Local SEEPROM [9,-1]  Element type: Enclosure services controller electronics
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: Unsupported
    Ident=0, Fail=0, Do not remove=0, Report=0, Hot swap=0
???????????????????????????????????????????????????????????????? [9,0]  Element type: Enclosure services controller electronics
  Enclosure Status:
    Predicted failure=0, Disabled=0, Swap=0, status: OK
    Ident=0, Fail=0, Do not remove=0, Report=0, Hot swap=0