I’m doing some renovations on my oldest TrueNAS system, requiring moving some drives in and out of the system. As suggested above, I’ve historically been able to move a pool’s drives between SATA ports on the motherboard and an LSI SAS3008 HBA in a PCIe slot.
For some reason, one pool “TANK_SATA” – built on top of newer Seagate 24TB 4Ke SED drives – only works when attached to the motherboard ports. When I try to relocate it to the HBA, the drives show up, but the partitions on the drive don’t, and thus the pool doesn’t load. When I move the pool’s drives back to the motherboard ports, everything works fine again.
In the past, another pool on the system “ZFS_20TB” – built on top of older WD 10TB drives (except for one failure replacement) – moved between motherboard and HBA SATA connections without issue.
Perhaps, complicating the matter is the newer drives on the TANK_SATA pool are set to use 4Ke logical sectors as well as to use SED encryption. I do have TrueNAS configured to unlock the SED drives, which works fine on the motherboard SATA connection – and the settings for which which even still shows up in the TrueNAS drive configuration settings when connected via HBA. However, oddly enough, the newer drives on the HBA don’t even get recognized by sedutil-cli as SED Enterprise drives, like they do when connected to the motherboard.
Should I be able to move the TANK_SATA drives from the motherboard SATA interface to the HBA? (As I have in the past for other drives). Is this as expected? Any troubleshooting ideas?
Here’s some of my system details if it helps call attention to anything significant:
My TrueNAS, running 25.10.3, is actually running virtualized in an ESXi 7 host, running on a SuperMicro X11SCA-F, with a number of PCIe device being passed-through – the main focus here is:
- the motherboard SATA (Cannon Lake PCH SATA AHCI Controller)
- the HBA (Broadcom / LSI SAS3008 PCI-Express)
root@truenas02[/mnt/TANK_NVME/ek/APPS/Docker]# lspci |grep -E -i "ata|sas|nvme|graphics"
02:03.0 SATA controller: VMware SATA AHCI controller
05:00.0 Non-Volatile memory controller: Micron Technology Inc 7450 MAX NVMe SSD (rev 01)
0b:00.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics P630]
0c:00.0 Non-Volatile memory controller: VMware NVMe SSD Controller
13:00.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)
14:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 (rev 02)
1c:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)
Attached to these controllers, I have two sets of 4 SATA drives, making up two zpools:
- ZFS_20TB (an older pool made up largely of older WD 10TB drives)
and - TANK_SATA (a newer pool made up of Seagate 24TB SED drives, with 4K logical sectors)
root@truenas02[/mnt/TANK_NVME/ek/APPS/Docker]# lsblk -o NAME,MODEL,SIZE,LABEL,FSROOTS,TRAN,PHY-SEC,LOG-SEC,fstype,fsver
NAME MODEL SIZE LABEL FSROOTS TRAN PHY-SEC LOG-SEC FSTYPE FSVER
sda Virtual disk 16G 512 512
├─sda1 260M EFI 512 512 vfat FAT32
└─sda2 15.7G freenas-boot 512 512 zfs_member 5000
sdb Virtual disk 16G 512 512
├─sdb1 260M EFI 512 512 vfat FAT32
└─sdb2 15.7G freenas-boot 512 512 zfs_member 5000
sdc WDC WD100EFAX-68LHPN0 9.1T sas 4096 512
├─sdc1 2G freenas02:swap0 4096 512 linux_raid_member 1.2
└─sdc2 9.1T ZFS_20TB 4096 512 zfs_member 5000
sdd WDC WD100EFAX-68LHPN0 9.1T sas 4096 512
├─sdd1 2G freenas02:swap1 4096 512 linux_raid_member 1.2
└─sdd2 9.1T ZFS_20TB 4096 512 zfs_member 5000
sde WDC WD100EFAX-68LHPN0 9.1T sas 4096 512
├─sde1 2G freenas02:swap1 4096 512 linux_raid_member 1.2
└─sde2 9.1T ZFS_20TB 4096 512 zfs_member 5000
sdf ST18000NM000J-2TV103 16.4T sas 4096 512
├─sdf1 2G freenas02:swap0 4096 512 linux_raid_member 1.2
└─sdf2 16.4T ZFS_20TB 4096 512 zfs_member 5000
sdg ST24000NM001H-3KS113 21.8T sata 4096 4096
└─sdg1 21.8T TANK_SATA 4096 4096 zfs_member 5000
sdh ST24000NM001H-3KS113 21.8T sata 4096 4096
└─sdh1 21.8T TANK_SATA 4096 4096 zfs_member 5000
sdi ST24000NM001H-3KS113 21.8T sata 4096 4096
└─sdi1 21.8T TANK_SATA 4096 4096 zfs_member 5000
sdj ST24000NM001H-3KS113 21.8T sata 4096 4096
└─sdj1 21.8T TANK_SATA 4096 4096 zfs_member 5000
sr0 VMware Virtual SATA CDRW Drive 1.9G Ubuntu 18.04.3 LTS amd64 sata 2048 2048 iso9660 Joliet Extension
nvme0n1 Micron_7450_MTFDKCC3T2TFS 2.9T nvme 4096 4096
└─nvme0n1p1 2.9T TANK_NVME nvme 4096 4096 zfs_member 5000
root@truenas02[/mnt/TANK_NVME/ek/APPS/Docker]# zpool status ZFS_20TB
pool: ZFS_20TB
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0B in 20:07:22 with 0 errors on Sun Apr 12 20:07:24 2026
config:
NAME STATE READ WRITE CKSUM
ZFS_20TB ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ata-WDC_WD100EFAX-68LHPN0_7JH34DSC-part2 ONLINE 0 0 0
sdf2 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
ata-WDC_WD100EFAX-68LHPN0_7JH6YWGC-part2 ONLINE 0 0 0
ata-WDC_WD100EFAX-68LHPN0_7JHB9RWC-part2 ONLINE 0 0 0
errors: No known data errors
root@truenas02[/mnt/TANK_NVME/ek/APPS/Docker]# zpool status TANK_SATA
pool: TANK_SATA
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 1M in 08:23:39 with 0 errors on Thu Apr 16 02:40:46 2026
config:
NAME STATE READ WRITE CKSUM
TANK_SATA ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c4ec3173-5bdc-4992-b299-566a994f7756 ONLINE 0 0 0
068f10fd-6e65-44ab-9206-f8bfc62f7599 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
9bbd6841-dab3-45da-8d73-fbb6f6a20c6d ONLINE 0 0 0
0daa796e-9040-4bab-9608-225beed63b70 ONLINE 0 0 0
errors: No known data errors
My HBA is running recent firmware:
root@truenas02[/mnt/TANK_NVME/ek/APPS/Docker]# sas3flash -list
Avago Technologies SAS3 Flash Utility
Version 16.00.00.00 (2017.05.02)
Copyright 2008-2017 Avago Technologies. All rights reserved.
Adapter Selected is a Avago SAS: SAS3008(C0)
Controller Number : 0
Controller : SAS3008(C0)
PCI Address : 00:14:00:00
SAS Address : 500605b-0-0c22-e330
NVDATA Version (Default) : 0e.01.00.08
NVDATA Version (Persistent) : 0e.01.00.08
Firmware Product ID : 0x2221 (IT)
Firmware Version : 16.00.10.00
NVDATA Vendor : LSI
NVDATA Product ID : SAS9300-8e
BIOS Version : 08.37.00.00
UEFI BSD Version : 10.00.00.00
FCODE Version : N/A
Board Name : SAS9300-8e
Board Assembly : 03-25718-00B
Board Tracer Number : SP63314371
Finished Processing Commands Successfully.
Exiting SAS3Flash.
When connected via the motherboard SATA, the 24TB drives are recognized as SED:
root@truenas02[/mnt/TANK_NVME/ek/APPS/Docker]# sedutil-cli --scan
Scanning for Opal compliant disks
/dev/nvme0 2 Micron_7450_MTFDKCC3T2TFS E2MU200
NVME Security Command Error:1
/dev/nvme1 No VMware Virtual NVMe Disk 1.0
/dev/sda No Virtual disk 2.0
/dev/sdb No Virtual disk 2.0
/dev/sdc No
/dev/sdd No
/dev/sde No
/dev/sdf No
/dev/sdg E ST24000NM001H-3KS113 SS07
/dev/sdh E ST24000NM001H-3KS113 SS07
/dev/sdi E ST24000NM001H-3KS113 SS07
/dev/sdj E ST24000NM001H-3KS113 SS07
No more disks present ending scan
When connected via the HBA, the 24TB drives are NOT recognized as SED – so I can’t even try to manually unlock, in case auto unlock isn’t working:
root@truenas02[/mnt/TANK_NVME/ek/APPS/Docker]# sedutil-cli --scan
Scanning for Opal compliant disks
/dev/nvme0 2 Micron_7450_MTFDKCC3T2TFS E2MU200
NVME Security Command Error:1
/dev/nvme1 No VMware Virtual NVMe Disk 1.0
/dev/sda No Virtual disk 2.0
/dev/sdb No Virtual disk 2.0
/dev/sdc No ST24000NM001H-3KS113 SS07
/dev/sdd No ST24000NM001H-3KS113 SS07
/dev/sde No ST24000NM001H-3KS113 SS07
/dev/sdf No ST24000NM001H-3KS113 SS07
/dev/sdg No
/dev/sdh No
/dev/sdi No
/dev/sdj No
No more disks present ending scan
And likewise, or perhaps consequently, the partitions didn’t show:
root@truenas02[/mnt/TANK_NVME/ek/APPS/Docker]# lsblk -o NAME,VENDOR,MODEL,SERIAL,UUID,FSTYPE,WWN,TRAN
NAME VENDOR MODEL SERIAL UUID FSTYPE WWN TRAN
sda VMware Virtual disk 6000c29670c855c11671914611423854 0x6000c29670c855c11671914611423854
├─sda1 44B7-30B3 vfat 0x6000c29670c855c11671914611423854
└─sda2 7890944875863301110 zfs_member 0x6000c29670c855c11671914611423854
sdb VMware Virtual disk 6000c29b52d6202c11eb5765e97c3844 0x6000c29b52d6202c11eb5765e97c3844
├─sdb1 44BB-3A10 vfat 0x6000c29b52d6202c11eb5765e97c3844
└─sdb2 7890944875863301110 zfs_member 0x6000c29b52d6202c11eb5765e97c3844
sdc ATA ST24000NM001H-3KS113 ZYD185XX 0x5000c500e94ad15e sas
sdd ATA ST24000NM001H-3KS113 ZYD1QC54 0x5000c500e95ba470 sas
sde ATA ST24000NM001H-3KS113 ZYD1PS61 0x5000c500e958b4ec sas
sdf ATA ST24000NM001H-3KS113 ZYD0PEL2 0x5000c500e91c8955 sas
sdg ATA WDC WD100EFAX-68LHPN0 7JHB9RWC 0x5000cca266d33c24 sas
├─sdg1 f44780d9-4aea-e6c4-47ff-c51dcf538340 linux_raid_member 0x5000cca266d33c24
└─sdg2 16900213071200577067 zfs_member 0x5000cca266d33c24
sdh ATA WDC WD100EFAX-68LHPN0 7JH6YWGC 0x5000cca266d141eb sas
├─sdh1 083482a1-5110-f5bf-3389-007cb9ef0387 linux_raid_member 0x5000cca266d141eb
└─sdh2 16900213071200577067 zfs_member 0x5000cca266d141eb
sdi ATA WDC WD100EFAX-68LHPN0 7JH34DSC 0x5000cca266cf852d sas
├─sdi1 083482a1-5110-f5bf-3389-007cb9ef0387 linux_raid_member 0x5000cca266cf852d
└─sdi2 16900213071200577067 zfs_member 0x5000cca266cf852d
sdj ATA ST18000NM000J-2TV103 ZR52VKZ8 0x5000c500dbe50412 sas
├─sdj1 f44780d9-4aea-e6c4-47ff-c51dcf538340 linux_raid_member 0x5000c500dbe50412
└─sdj2 16900213071200577067 zfs_member 0x5000c500dbe50412
sr0 NECVMWar VMware Virtual SATA CDRW Drive 00000000000000000001 2019-08-05-19-29-01-00 iso9660 sata
nvme0n1 Micron_7450_MTFDKCC3T2TFS 22353CF2C223 eui.000000000000000100a075223cf2c223 nvme
└─nvme0n1p1
The drives also are running current firmware, and here’s full details of one of them:
root@truenas02[/mnt/TANK_NVME/ek/APPS/Docker]# /root/SeaChestUtilities/SeaChestUtilities/SeaChestUtilities/Linux/Non-RAID/x86_64/SeaChest_Info --scan
==========================================================================================
SeaChest_Info - Seagate drive utilities - NVMe Enabled
Copyright (c) 2014-2024 Seagate Technology LLC and/or its Affiliates, All Rights Reserved
SeaChest_Info Version: 2.7.0-8_0_1 X86_64
Build Date: Sep 26 2024
Today: 20260424T143439 User: root
==========================================================================================
nvme_ioctl_id: Not a tty
Vendor Handle Model Number Serial Number FwRev
VMware /dev/sg0 Virtual disk 6000c29670c855c11671 2.0
VMware /dev/sg1 Virtual disk 6000c29b52d6202c11eb 2.0
ATA /dev/sg2 WDC WD100EFAX-68LHPN0 7JHB9RWC 83.H0A83
ATA /dev/sg3 WDC WD100EFAX-68LHPN0 7JH6YWGC 83.H0A83
ATA /dev/sg4 WDC WD100EFAX-68LHPN0 7JH34DSC 83.H0A83
ATA /dev/sg5 ST18000NM000J-2TV103 ZR52VKZ8 SN06
AHCI /dev/sg6 SGPIO Enclosure 2.00
NECVMWar /dev/sg7 VMware SATA CD00 1.00
ATA /dev/sg8 ST24000NM001H-3KS113 ZYD0PEL2 SS07
ATA /dev/sg9 ST24000NM001H-3KS113 ZYD185XX SS07
ATA /dev/sg10 ST24000NM001H-3KS113 ZYD1QC54 SS07
ATA /dev/sg11 ST24000NM001H-3KS113 ZYD1PS61 SS07
NVMe /dev/nvme0n1 Micron_7450_MTFDKCC3T2TFS 22353CF2C223 E2MU200
root@truenas02[/mnt/TANK_NVME/ek/APPS/Docker]# /root/SeaChestUtilities/SeaChestUtilities/SeaChestUtilities/Linux/Non-RAID/x86_64/SeaChest_Info -i --device /dev/sg8 |grep -v Serial
==========================================================================================
SeaChest_Info - Seagate drive utilities - NVMe Enabled
Copyright (c) 2014-2024 Seagate Technology LLC and/or its Affiliates, All Rights Reserved
SeaChest_Info Version: 2.7.0-8_0_1 X86_64
Build Date: Sep 26 2024
Today: 20260424T143443 User: root
==========================================================================================
/dev/sg8 - ST24000NM001H-3KS113 - ZYD0PEL2 - SS07 - ATA
Model Number: ST24000NM001H-3KS113
Firmware Revision: SS07
World Wide Name: 5000C500E91C8955
Date Of Manufacture: Week 41, 2024
Drive Capacity (TB/TiB): 24.00/21.83
Native Drive Capacity (TB/TiB): 24.00/21.83
Temperature Data:
Current Temperature (C): 40
Highest Temperature (C): 52
Lowest Temperature (C): 28
Power On Time: 1 year 129 days 20 hours
Power On Hours: 11876.00
MaxLBA: 5859442687
Native MaxLBA: 5859442687
Logical Sector Size (B): 4096
Physical Sector Size (B): 4096
Sector Alignment: 0
Rotation Rate (RPM): 7200
Form Factor: 3.5"
Last DST information:
Time since last DST (hours): 214.00
DST Status/Result: 0x0
DST Test run: 0x1
Long Drive Self Test Time: 1 day 8 hours 33 minutes
Interface speed:
Max Speed (Gb/s): 6.0
Negotiated Speed (Gb/s): 6.0
Annualized Workload Rate (TB/yr): 22.38
Total Bytes Read (TB): 21.02
Total Bytes Written (TB): 9.32
Encryption Support: Full Disk Encryption
Cache Size (MiB): 512.00
Read Look-Ahead: Enabled
Write Cache: Enabled
Low Current Spinup: Disabled
SMART Status: Good
ATA Security Information: Not Supported
Firmware Download Support: Full, Segmented, Deferred
Specifications Supported:
ACS-5
ACS-4
ACS-3
ACS-2
ATA8-ACS
ATA/ATAPI-7
ATA/ATAPI-6
ATA/ATAPI-5
SATA 3.3
SATA 3.2
SATA 3.1
SATA 3.0
SATA 2.6
SATA 2.5
SATA II: Extensions
SATA 1.0a
ATA8-AST
Features Supported:
Read/Write Multiple
Sanitize
SATA NCQ [QD=32]
SATA Software Settings Preservation [Enabled]
SATA Device Initiated Power Management
Power Management
SMART [Enabled]
48bit Address
Set Max Security Extension
PUIS
GPL
Streaming
SMART Self-Test
SMART Error Logging
DSN
AMAC
EPC [Enabled]
Sense Data Reporting [Enabled]
Write-Read-Verify
SCT Write Same
SCT Error Recovery Control
SCT Feature Control
SCT Data Tables
Host Logging
Set Sector Configuration
Storage Element Depopulation + Restore
Field Accessible Reliability Metrics (FARM)
Seagate In Drive Diagnostics (IDD)
Adapter Information:
Adapter Type: PCI
Vendor ID: 8086h
Product ID: A352h
Revision: 0010h