Can't add disk to mirror


I have a TrueNAS Core with three data disks. All three disks are the same.

My storage pool is made up of two of them and the third was a hot spare. I’ve removed the hot spare and want to add it to the mirror as a third disk, but I keep getting the following error:

Could anyone shed some light on what is happening here? The disk is present, I don’t know why I can’t add it. In testing, I created a new, one-disk pool with the disk and that worked just fine…I just can’t add it to this existing mirror.


Why do the disks show as “multipath/disk#”? Multipath support has been removed.

Don’t know…this is how it was setup when we got it from iX several years ago. It must have been supported then, because nothing special was done to setup multipath.

Having said that, it seems to be working fine now?

root@nas:~ # gmultipath status
           Name   Status  Components
multipath/disk1  OPTIMAL  da3 (ACTIVE)
                          da0 (PASSIVE)
multipath/disk2  OPTIMAL  da4 (ACTIVE)
                          da1 (PASSIVE)
multipath/disk3  OPTIMAL  da2 (ACTIVE)
                          da5 (PASSIVE)
root@nas:~ #

If it isn’t supported any more, is there a way to keep my pool and disable multipath?


Hey @kgmoney

Support for creating new pools with multipath devices was removed, but existing pools will continue to function on CORE. There’s no support for SAS multipath in SCALE, so you wouldn’t be able to migrate directly.

There’s a reference article on the change here as it relates to the older “FreeNAS Certified” hardware:

Was this an iXsystems unit?

1 Like

Ok, thanks for that. I don’t plan to leave core at this time. I’ll review the post you linked to. If there are no real problems with multipath, I’m happy to leave it alone, I just need to figure out how to modify my pool…in this case just add a disk to my mirror vdev.

Yes, this is a FNC device from iX, and I’ve reached out to them as well.

Yes, I found your case # :slight_smile: however I’m not sure on what the system topology of that hardware is.

If you’re able to (and comfortable with) the physical process of changing your SAS cabling from multipath to wide-port, then there are software commands you can issue against an exported pool to remove the multipath configuration. Because ZFS stores information about the pool on the member disks, you would be able to rescan and reattach the pool.

Operations against existing pools (including adding new disks) should continue to work; I’m not sure why it threw the error about being unable to find da5 initially since it should have been using multipath/disk3 but it appears to have picked it up now. If this was a fairly recent CORE 13 build it might be worth submitting a bug report with a debug attached so we can look through the trace logs.

Perfect, thanks!

Long term, I would like to move to wide-port since that’s the official direction now, but for the immediate term, I’d like to leave it as if it’s still a supported/safe configuration.

I believe my CORE 13 version is maybe one update behind, I’d have to check. I can do the update and try adding the disk again.

Thanks for your help!

Well, I did the update…so now I’m at version TrueNAS-13.0-U6.1…but I still get the same error when trying to extend the pool.

@HoneyBadger …where would I find info on the recabling and software commands you referred to? Also, where would I go to submit a bug report?

@kgmoney Sorry for the delay on the reply here.

:warning: Before running these commands, you will definitely need to export your pool from the UI. If you’re also planning to change the cables, a full shutdown might be better to avoid accidental hiccups from the live-disconnect of an internal cable (external SFF is designed for this, internal isn’t)

Since you only have three multipath disks, you can run it as a one-liner:

for i in `seq 1 3`; do gmultipath destroy disk$i; done

Or three individual gmultipath destroy disk1 commands, replacing with disk2 and disk3 on the next lines.

For changing the cabling, it would be specific to the design of your system; but it would involve either removing the secondary SAS cable from your controller to the drive backplane, and if possible connecting it to the “second port of the primary expander” as opposed to the “primary port of the secondary expander.” These are often labeled as ports A1, A2 (primary expander, ports 1 and 2) B1, B2 (secondary expander, ports 1 and 2) - if your cabling is presently “A1 + B1” then switch it to “A1 + A2” - but if your drive backplane only has “A1 + B1” as options, then you would disconnect “B1” and leave only “A1” connected. If you’ve got questions, please share some system build information including chassis and motherboard models and I’ll see if I can assist further.

Bug reports can be filed on our JIRA here:

1 Like