hi i created a raidz1 pool with 4 identical 1 tb drive but the pool show as
1 x RAIDZ1 | 4 wide | Mixed Capacity
Do you have any data on this pool or is it empty? I am guessing you have other partitions on these drives and it makes the sizes slightly different.
I think this will show what may be wrong. Please run this in a Shell window and paste the results back using Preformatted text (</>) on toolbar above where you type replies.
sudo lsblk -o NAME,TYPE,UUID,PARTUUID,LABEL
2 Likes
NAME TYPE UUID PARTUUID LABEL
sda disk
├─sda1
│ part 128a7531-a49a-4adc-a0db-8bd913917da9
├─sda2
│ part 9298-8273 22483028-3c20-49de-96a7-f372f391bbe7 EFI
├─sda3
│ part 7411931415704018049 6dc29bb5-c2fd-4284-9806-eac686dcbb64 boot-pool
└─sda4
part b002203e-3ef4-4bf0-a1c1-558025fc8dc9
sdb disk
├─sdb1
│ part bd6bb883-af6f-4a37-a458-a500f1c3863f
├─sdb2
│ part 30C4-4953 602eaea3-da31-4a30-ae0b-207b6519d5d9 EFI
└─sdb3
part 7411931415704018049 7415de91-9a74-4d18-9685-94ad64a53959 boot-pool
sdc disk
└─sdc1
part 1855689867576217875 9a88f69b-adbf-4164-a262-b09af6d3f328 Titan
sdd disk
└─sdd1
part 1855689867576217875 462e806c-b170-4ac2-a4b5-f89c154d7292 Titan
sde disk
└─sde1
part 9778504802808583896 d6773804-5027-46c4-bb1d-3155eb868c0d Mars
sdf disk
└─sdf1
part 9778504802808583896 b9215220-a237-4870-91dd-8f72b9aba4ac Mars
sdg disk
└─sdg1
part 1855689867576217875 e8dcf9fa-6cac-4b32-828b-bd79affb3b3f Titan
sdh disk
└─sdh1
part 9778504802808583896 c07a1098-f477-468b-8c32-886075a87c0e Mars
sdi disk
└─sdi1
part 1855689867576217875 8698d538-80ae-4676-93b3-07f00ec6b58e Titan
Titan is the meant pool i made sure to wipe them clean so iam sure there was no partiton on any of them
It may be showing that because of different pools and vdev sizes?
Try this and post back like you did
sudo zpool status -v
pool: Mars
state: ONLINE
scan: scrub in progress since Sat Nov 30 17:12:42 2024
1.10T / 1.10T scanned, 676G / 1.10T issued at 253M/s
0B repaired, 60.26% done, 00:30:05 to go
config:
NAME STATE READ WRITE CKSUM
Mars ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
d6773804-5027-46c4-bb1d-3155eb868c0d ONLINE 0 0 0
b9215220-a237-4870-91dd-8f72b9aba4ac ONLINE 0 0 0
c07a1098-f477-468b-8c32-886075a87c0e ONLINE 0 0 0
errors: No known data errors
pool: Titan
state: ONLINE
scan: scrub repaired 0B in 00:22:25 with 0 errors on Fri Nov 29 20:14:16 2024
expand: expanded raidz1-0 copied 566G in 01:06:45, on Fri Nov 29 19:51:51 2024
config:
NAME STATE READ WRITE CKSUM
Titan ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
462e806c-b170-4ac2-a4b5-f89c154d7292 ONLINE 0 0 0
8698d538-80ae-4676-93b3-07f00ec6b58e ONLINE 0 0 0
e8dcf9fa-6cac-4b32-828b-bd79affb3b3f ONLINE 0 0 0
9a88f69b-adbf-4164-a262-b09af6d3f328 ONLINE 0 0 0
errors: No known data errors
pool: boot-pool
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0B in 00:01:29 with 0 errors on Fri Nov 29 03:46:31 2024
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
sda3 ONLINE 0 0 0
sdb3 ONLINE 0 0 0
errors: No known data errors
I don’t have a Scale system to check what the windows look like. Do you have more than one Topology window, one for each pool?
We can wait to see if anyone else comments. Would you add the SCALE tag to your first post. Are all four disk the same brand and model of HD? Wondering if they are just slightly different in capacity but close enough that they report the same in the Topology window with rounding of values.
1 Like
Please run the following enhanced lsblk command
lsblk -bo NAME,MODEL,ROTA,PTTYPE,TYPE,START,SIZE,PARTTYPENAME,PARTUUIDso we can see the exact size and model of each drive.
2 Likes
so iam runing lsi card when i entered the setting for the card i saw one hard has a 1 megabyte different is there any fix for that any correction can be done ?
Did you flash the LSI card to IT mode ?
sas2flash -list
sas3flash -list
will tell you that. If not I suggest to do that before you do anything else troubleshooting.
1 Like
Please run the commands and post the results using Preformatted text that @Protopia and @Farout asked for. Only one of the commands Farout asked for will return results depending on your LSI card model.
1 Like
Doesn’t look like it, looking at that screenshot…
1 Like
Yeah…thats what i was thinking aswell …
2 Likes
@m0pegasus DO NOT PUT ANY DATA ON YOUR NEWLY CREATED POOL because you have created it wrong and this may lead to losing your data!!!
The screen shot you posted is a BIOS shot that indicates that your LSI card is running in RAID mode and not IT mode. You have essentially understood that the disks need to be passed through as JBOD to TrueNAS, however if you do this in the RAID card settings rather than by flashing IT mode this can lead to data corruption.
You need to:
- Destroy the pool.
- Run the
sas2flash -listandsas3flash -listcommands and see what LSI card you have and then run the appropriate command to flash new firmware. - Power cycle your server.
- Create the pool again.
2 Likes
Step 0. Backup any data that is already in the pool.
1 Like
i changed the hba to hba 9400 8i since the oone i was used is 9361 which cannot be flashed to it mode and i have to use storcli command instead of sas3flah the problem still
@Farout @SmallBarky @neofusion
sudo storcli /c0 show
CLI Version = 007.2807.0000.0000 Dec 22, 2023
Operating system = Linux 6.6.44-production+truenas
Controller = 0
Status = Success
Description = None
Product Name = HBA 9400-8i
Serial Number = RRW8000000
SAS Address = 00000000r00g00k000
PCI Address = 00:00:10:00
System Time = 12/02/2024 00:33:08
FW Package Build = 24.00.00.00
FW Version = 24.00.00.00
BIOS Version = 09.47.00.00_24.00.00.00
NVDATA Version = 24.00.00.24
PSOC FW Version = 0x0001
PSOC Part Number = 05689
Driver Name = mpt3sas
Driver Version = 43.100.00.00
Bus Number = 0
Device Number = 16
Function Number = 0
Domain ID = 0
Vendor Id = 0x1000
Device Id = 0xAF
SubVendor Id = 0x1000
SubDevice Id = 0x3010
Board Name = HBA 9400-8i
Board Assembly = 00-00008-0000000
Board Tracer Number = 0000000
Security Protocol = None
Physical Drives = 7
PD LIST :
=======
---------------------------------------------------------------------------
EID:Slt DID State DG Size Intf Med SED PI SeSz Model Sp
---------------------------------------------------------------------------
0:1 3 JBOD - 931.512 GB SATA HDD - - 512B ST1000DM003-1SB102 -
0:2 2 JBOD - 698.636 GB SATA HDD - - 512B TOSHIBA MQ01ABD075 -
0:3 7 JBOD - 931.511 GB SATA HDD - - 512B ST1000DM003-1SB102 -
0:4 1 JBOD - 931.512 GB SATA HDD - - 512B ST31000524NS -
0:5 4 JBOD - 931.512 GB SATA HDD - - 512B ST1000DM010-2EP102 -
0:6 5 JBOD - 698.636 GB SATA HDD - - 512B TOSHIBA MQ01ABD075 -
0:7 6 JBOD - 698.636 GB SATA HDD - - 512B TOSHIBA MQ01ABD075 -
---------------------------------------------------------------------------
EID-Enclosure Device ID|Slt-Slot No|DID-Device ID|DG-DriveGroup
UGood-Unconfigured Good|UBad-Unconfigured Bad|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition
Requested Boot Drive = Not Set
Okay, unfortunately you still have the drives in JBOD mode.
What the practical differences between JBOD mode and HBA are exactly is outside my level of understanding.
At least it’s using the mpt3sas driver.
yeah you right but from what i can found it actually runing in the IT mode that is how the hba 9400 model work
Even when a RAID controller is in JBOD mode…
-
ZFS relies on writes being made to disk in a specific sequence for its integrity. RAID controllers can do things like resequencing I/Os.
-
ZFS relies on reading the serial numbers from the disks. RAID controllers can hide this and confuse ZFS.
-
ZFS relies on knowing the exact details of the hardware. RAID controllers can hide this and confuse ZFS.
-
ZFS has commands to repair redundant vDevs. With RAID controllers, these may not work.
Please run sas2flash -list and sas3flash -list and post the output here so that we can confirm this for you.
1 Like
He tried running those above and had to use a different tool?
1 Like
It’s a 9400-series, I’ll be surprised if it responds to sas2flash or sas3flash commands.
I am curious what the output of storcli /c0 show personality is.
1 Like