Expand Pool fails with OSError: [Errno 22] Invalid argument

TrueNAS General
1

I try to expand and pool which was initially created with Truenas CORE as as 6x4TB RAIDZ2
Over the time I replaced 2 disks with 12TB, and later on 2 with 16TB, now replaced the last 2 also with 16TB disks. Inbeween I also migrated(new install and import the pool) from CORE to SCALE.

Autoexpand didn’t work as expected even it’s on.
If I press the Expand button, I get an error

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 515, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 560, in __run_body
    rv = await self.method(*args)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 174, in nf
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 48, in nf
    res = await f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/pool_/expand.py", line 61, in expand
    await self.expand_partition(part_data)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/pool_/expand.py", line 94, in expand_partition
    await self.middleware.run_in_thread(wipe_label)
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 622, in run_in_thread
    return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 619, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/pool_/expand.py", line 91, in wipe_label
    f.seek(wipe_start)
OSError: [Errno 22] Invalid argument

It also appears that the partitions are in a disorganized state.

NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda           8:0    0  10.9T  0 disk 
├─sda1        8:1    0     2G  0 part 
└─sda2        8:2    0  10.9T  0 part 
sdb           8:16   0  14.6T  0 disk 
└─sdb1        8:17   0   3.6T  0 part 
sdc           8:32   0  14.6T  0 disk 
├─sdc1        8:33   0     2G  0 part 
└─sdc2        8:34   0   3.6T  0 part 
sdd           8:48   0  14.6T  0 disk 
└─sdd1        8:49   0  14.6T  0 part 
sde           8:64   0  14.6T  0 disk 
├─sde1        8:65   0     2G  0 part 
└─sde2        8:66   0   3.6T  0 part 
sdf           8:80   0  10.9T  0 disk 
└─sdf1        8:81   0   3.6T  0 part

I’ on the latest version 25.04.1.

sda and sdf was the first replacement done under CORE.
sdc and sde are done later, not sure, but i think already been done under SCALE
sdb and sdd are now the last one done with latest SCALE version.

Any hint, to get the pool expanded to the 12TB devices?

2

Just got the same error after replacing 12HDD’s in my raidz2 pool.
I have opened a ticket and i hope this issue will be fixed soon.

3

iX has idiotically decided that the pool auto-expansion that’s been around since at least FreeNAS 8 has to go, and broken things in the process. Go to the shell, as root, and run
parted /dev/sdb resizepart 1 100%. Then parted /dev/sdc resizepart 2 100%. Then parted /dev/sde resizepart 2 100%. Finally, parted /dev/sdf resizepart 1 100%. Share any errors.

It’s likely to be the same underlying cause, but the specifics will be different. If you can share the output of lsblk, as preformatted text (the image button in the editor), we can work out specific commands to fix it.

1 Like
4

@dan here’s the information you asked (it’s a long list)

NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda       8:0    0   5.5T  0 disk
├─sda1    8:1    0     2G  0 part
└─sda2    8:2    0   5.5T  0 part
sdb       8:16   0   3.6T  0 disk
├─sdb1    8:17   0     2G  0 part
└─sdb2    8:18   0   3.6T  0 part
sdc       8:32   0   5.5T  0 disk
├─sdc1    8:33   0     2G  0 part
└─sdc2    8:34   0   2.7T  0 part
sdd       8:48   0   5.5T  0 disk
├─sdd1    8:49   0     2G  0 part
└─sdd2    8:50   0   5.5T  0 part
sde       8:64   0   5.5T  0 disk
├─sde1    8:65   0     2G  0 part
└─sde2    8:66   0   5.5T  0 part
sdf       8:80   0   3.6T  0 disk
└─sdf1    8:81   0   3.6T  0 part
sdg       8:96   0   5.5T  0 disk
├─sdg1    8:97   0     2G  0 part
└─sdg2    8:98   0   5.5T  0 part
sdh       8:112  0   5.5T  0 disk
├─sdh1    8:113  0     2G  0 part
└─sdh2    8:114  0   5.5T  0 part
sdi       8:128  0   5.5T  0 disk
├─sdi1    8:129  0     2G  0 part
└─sdi2    8:130  0   2.7T  0 part
sdj       8:144  0   5.5T  0 disk
├─sdj1    8:145  0     2G  0 part
└─sdj2    8:146  0   5.5T  0 part
sdk       8:160  0   5.5T  0 disk
├─sdk1    8:161  0     2G  0 part
└─sdk2    8:162  0   5.5T  0 part
sdl       8:176  0   5.5T  0 disk
├─sdl1    8:177  0     2G  0 part
└─sdl2    8:178  0   2.7T  0 part
sdm       8:192  0   5.5T  0 disk
└─sdm1    8:193  0   2.7T  0 part
sdn       8:208  0   5.5T  0 disk
├─sdn1    8:209  0     2G  0 part
└─sdn2    8:210  0   2.7T  0 part
sdo       8:224  0   5.5T  0 disk
└─sdo1    8:225  0   5.5T  0 part
sdp       8:240  0   5.5T  0 disk
sdq      65:0    0   5.5T  0 disk
├─sdq1   65:1    0     2G  0 part
└─sdq2   65:2    0   5.5T  0 part
sdr      65:16   0   5.5T  0 disk
├─sdr1   65:17   0     2G  0 part
└─sdr2   65:18   0   5.5T  0 part
sds      65:32   0   5.5T  0 disk
├─sds1   65:33   0     2G  0 part
└─sds2   65:34   0   5.5T  0 part
sdt      65:48   0   5.5T  0 disk
├─sdt1   65:49   0     2G  0 part
└─sdt2   65:50   0   5.5T  0 part
sdu      65:64   0   5.5T  0 disk
└─sdu1   65:65   0   5.5T  0 part
sdv      65:80   0   3.6T  0 disk
├─sdv1   65:81   0     2G  0 part
└─sdv2   65:82   0   3.6T  0 part
sdw      65:96   0   5.5T  0 disk
└─sdw1   65:97   0   5.5T  0 part
sdx      65:112  0   5.5T  0 disk
├─sdx1   65:113  0     2G  0 part
└─sdx2   65:114  0   5.5T  0 part
sdy      65:128  0   3.6T  0 disk
├─sdy1   65:129  0     2G  0 part
└─sdy2   65:130  0   3.6T  0 part
sdz      65:144  0   5.5T  0 disk
├─sdz1   65:145  0     2G  0 part
└─sdz2   65:146  0   5.5T  0 part
sdaa     65:160  0   3.6T  0 disk
└─sdaa1  65:161  0   3.6T  0 part
sdab     65:176  0   3.6T  0 disk
└─sdab1  65:177  0   3.6T  0 part
sdac     65:192  0   2.7T  0 disk
├─sdac1  65:193  0     2G  0 part
└─sdac2  65:194  0   2.7T  0 part
sdad     65:208  0   5.5T  0 disk
├─sdad1  65:209  0     2G  0 part
└─sdad2  65:210  0   2.7T  0 part
sdae     65:224  0   2.7T  0 disk
└─sdae1  65:225  0   2.7T  0 part
sdaf     65:240  0   2.7T  0 disk
├─sdaf1  65:241  0     2G  0 part
└─sdaf2  65:242  0   2.7T  0 part
sdag     66:0    0   5.5T  0 disk
└─sdag1  66:1    0   5.5T  0 part
sdah     66:16   0   2.7T  0 disk
├─sdah1  66:17   0     2G  0 part
└─sdah2  66:18   0   2.7T  0 part
sdai     66:32   0   2.7T  0 disk
├─sdai1  66:33   0     2G  0 part
└─sdai2  66:34   0   2.7T  0 part
sdaj     66:48   0   3.6T  0 disk
├─sdaj1  66:49   0     2G  0 part
└─sdaj2  66:50   0   3.6T  0 part
sdak     66:64   0   3.6T  0 disk
├─sdak1  66:65   0     2G  0 part
└─sdak2  66:66   0   3.6T  0 part
sdal     66:80   0   2.7T  0 disk
├─sdal1  66:81   0     2G  0 part
└─sdal2  66:82   0   2.7T  0 part
sdam     66:96   0   2.7T  0 disk
├─sdam1  66:97   0     2G  0 part
└─sdam2  66:98   0   2.7T  0 part
sdan     66:112  0   2.7T  0 disk
└─sdan1  66:113  0   2.7T  0 part
sdao     66:128  0   2.7T  0 disk
└─sdao1  66:129  0   2.7T  0 part
sdap     66:144  0   2.7T  0 disk
├─sdap1  66:145  0     2G  0 part
└─sdap2  66:146  0   2.7T  0 part
sdaq     66:160  0   5.5T  0 disk
sdar     66:176  0   5.5T  0 disk
├─sdar1  66:177  0     2G  0 part
└─sdar2  66:178  0   5.5T  0 part
sdas     66:192  0   2.7T  0 disk
├─sdas1  66:193  0     2G  0 part
└─sdas2  66:194  0   2.7T  0 part
sdat     66:208  0   2.7T  0 disk
└─sdat1  66:209  0   2.7T  0 part
sdau     66:224  0   2.7T  0 disk
├─sdau1  66:225  0     2G  0 part
└─sdau2  66:226  0   2.7T  0 part
sdav     66:240  0 111.8G  0 disk
├─sdav1  66:241  0   260M  0 part
└─sdav2  66:242  0 111.5G  0 part
sdaw     67:0    0 111.8G  0 disk
├─sdaw1  67:1    0   260M  0 part
└─sdaw2  67:2    0 111.5G  0 part
sdax     67:16   0   1.8T  0 disk
├─sdax1  67:17   0     2G  0 part
└─sdax2  67:18   0   1.8T  0 part
sday     67:32   0   1.8T  0 disk
├─sday1  67:33   0     2G  0 part
└─sday2  67:34   0   1.8T  0 part
5

It is indeed. It looks like sdc, sdi, sdl, sdn, and sdad have data partitions on partition 2 that are considerably smaller than the disk size, with those as partition 2. To fix that, run (as root) parted /dev/<whatever> resizepart 2 100%. sdm looks to be the same way, except the data partition is partition 1; to fix that one, parted /dev/sdm resizepart 1 100%. For some reason, sdp doesn’t seem to have a partition table at all–is it part of your pool?

1 Like
6

Thanks for the information. I followed the steps and now my pool is finally expanded.
I do have 2 6Tb drives that are not part of any pool (I keep them as spares just in case).

1 Like
7

Hello all,

My apologies if I was supposed to start a new thread, but I’m having the same problem.

I went from a six-drive wide 4TB pool wayyyyy back in the freenas 7 days, occasionally adding a new 8TB drive, to recently putting in the final 8TB drive in my Z1 pool. The auto-expand didn’t work, and when I press the “Expand” button, I get “[Errno 22] Invalid argument” and the following:

Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/middlewared/job.py”, line 515, in run
await self.future
File “/usr/lib/python3/dist-packages/middlewared/job.py”, line 560, in run_body
rv = await self.method(*args)
^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/schema/processor.py”, line 174, in nf
return await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/schema/processor.py”, line 48, in nf
res = await f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/middlewared/plugins/pool/expand.py", line 61, in expand
await self.expand_partition(part_data)
File "/usr/lib/python3/dist-packages/middlewared/plugins/pool/expand.py", line 94, in expand_partition
await self.middleware.run_in_thread(wipe_label)
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 627, in run_in_thread
return await self.run_in_executor(io_thread_pool_executor, method, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/main.py”, line 624, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3.11/concurrent/futures/thread.py”, line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/middlewared/plugins/pool_/expand.py”, line 91, in wipe_label
f.seek(wipe_start)
OSError: [Errno 22] Invalid argument

lsblk command gives me:

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 7.3T 0 disk
└─sda1 8:1 0 7.3T 0 part
sdb 8:16 0 7.3T 0 disk
├─sdb1 8:17 0 2G 0 part
└─sdb2 8:18 0 3.6T 0 part
sdc 8:32 0 7.3T 0 disk
├─sdc1 8:33 0 2G 0 part
└─sdc2 8:34 0 7.3T 0 part
sdd 8:48 0 7.3T 0 disk
├─sdd1 8:49 0 2G 0 part
└─sdd2 8:50 0 7.3T 0 part
sde 8:64 0 7.3T 0 disk
├─sde1 8:65 0 2G 0 part
└─sde2 8:66 0 3.6T 0 part
sdf 8:80 0 447.1G 0 disk
├─sdf1 8:81 0 1M 0 part
├─sdf2 8:82 0 512M 0 part
└─sdf3 8:83 0 446.6G 0 part
sdg 8:96 0 7.3T 0 disk
├─sdg1 8:97 0 2G 0 part
└─sdg2 8:98 0 3.6T 0 part
zd0 230:0 0 100G 0 disk
nvme0n1 259:0 0 119.2G 0 disk
├─nvme0n1p1 259:1 0 512K 0 part
└─nvme0n1p2 259:2 0 119.2G 0 part

I’m running Truenas 25.04.2.3. Thank you for any help!