VM related Disk Sector size. I am confused and not happy!

In favor of a VM a ZVOL is needed. Best way IMHO is to create that ZVOL before defining the VM.

When creating a ZVOL you can choose the sector size which is based on ZVOL-size, file size, disktype etc. Perhaps 16K is a good value.

When defining the VM you select the ZVOL and …. you select one of the following sector sizes:

  • Default (the ^?^ explains that default equals the ZFS sector size 16K in this example
  • 4K
  • 512 byte

you also have to choose between:

  • VirtIO or
  • AHCI

Where VirtIO seems to be prefered (at least for Linux)

Where I can understand that it is an advantage if ZFS sectors size matches, and I can also understand 4K which is the sector size of all modern Harddisks, I can not really understand 512 byte nowadays …

What ever, in my situation:

  • the pool is based on an NVME-SSD
  • the ZVOL has a sector size of 16K
  • the IO type chosen is VirtIO
  • the sector size chosen is Default
  • the VM OS is the actual Ubuntu LTS
  • the ZVOL size is 750G (btfrs formatted)

And what do I see Ubuntu is telling me that the physical sector size is ……. 512 byte
(lsblk -o KNAME,NAME,FSTYPE,SIZE,MOUNTPOINT,PHY-SE)

And my feeling is that a ^physical^ sector size of 512 byte ….. is not a good idea !!!

So I am completely lost in relation to the setup of sector sizes !!!
And I also wonder what would be the setting ???

So I hope someone can explain and advice :face_blowing_a_kiss:

Zvol will use the block size of the zfs.

The selection you are making is the emulated sector size and I think default is 512.

Remember it is a VM, stuff is emulated.

The sectors do align correctly with the zfs file system to allow smooth running.

512 bytes allow legacy file systems and legacy OS, eg DOS

It is NOT OK at least not in line with the documentation. A modern OS which Ubuntu is should see 4K-blocks.

To be sure I did check the zvol block size ‘zfs get volblocksize <zvol_path>’ => 16K

I also noted this Jira. Described there

When creating a virt VM instance, we have a key root_disk_io_bus which has been introduced.
Valid choices for it are `‘NVME’, ‘VIRTIO-BLK’, ‘VIRTIO-SCSI’,`.
We should expose this attribute on create and update of virt VM instances only

I do not see any of that.

And I really wonder if I should have chosen 4K and not default at vm disk definitiions. What ever I can not imagine that 512byte is OK

  • 512 and 4096 will work fine for Ubuntu but you should use 4096/4K when a OS supports it.

  • Having a larger sector reduces disks calls for the same amount of data.

  • The reason 4096 is now the standard is that most files are of a size where it does not make sense to use such small segments as 512.

  • 512 sector sizes is only for old OS and compatibility with them and of course old file systems.

  • I use VirtIO with sector size 4096

  • To be sure I did check the zvol block size ‘zfs get volblocksize <zvol_path>’ => 16K

    • This makes no difference for the sector size within the ZVol.
  • Selecting default sets sector size of 512 (i think this is correct, but might of changed since I did these instructions). There are only 2 options, 512 and 4096.

Look at the section in my article: Setting up a Virtual Machine (Worked Example / Virtualmin) on how I setup my VM

I did choose 4K for other VM’s I did forgot that in this case. The big problem is that I have to redo the compleet VM install due to this ‘error’.

And reading the “? “ mark at the setup, that does in no way match what I am seeing / what is happening !!

And if it is true that you should select 4K, that is is damn stupid that the default is still 512byte in 2026 !!

Next to that the big question is: should the zvol volblocksize match the sector size !!??

So would the best option be to choose 4K when creating the zvol !!??

Last point is purely GUI related …… I would very much appreciate if the zvol volblocksize and the chosen sector size would stay visible in the GUI after the initial configuration phase, even if they are not changeable any more!!!

Depending on what you TrueNAS version is tehre are several different ways of imaging a harddisk. But essential create and image of the 512 disk, create a new 4096 disk, now image the image file to the 4096. This can be tricky so just decide which is more work.

If you can check this for me, it would be appreciated.

I dont like the default option as it is confusing,it should just be 512 and 4096 as must always select one.

It is, sort of, edit the disk device and you can see it there. Remember you are setting the physical hardware sectors, not the file system as that is secondary, but usually they do follow those sector size.

For info my actual TrueNas version is 25.10.2 - Goldeye

this has a disk import export thin if i am not mistake.

export the disk, use disk software and change the sector size, re-import to a new 4096 drive on truenas… or something along this line.

They did talk about that import export function in TrueNas Tech Talk, but not that I can find it :frowning: (apart from the question if I am going that way !)

Perhaps it is related to apps etc, which I do not use. Note that TrueNas is my data store, where things like apps do not belong IMHO from the security stand point.

I do use VM’s on TrueNas since thery are IMHO far more secure that app’s etc. And VM’s do support VLAN’s as it should (apps etc do not!)

Look on my article for alternative ways. I looked into this before the option was available.

I just created a ‘test4K VM’ and the way to force 4k is even worse than I remembered. There is only one moment in time you can do that!!

That moment is, after you did create the ZVOL, defined the VM and BEFORE the first start of the VM !! BIZAR !!!

So before the first start, goto edit select the disk and modify there default to 4K!! And after doing so start the VM and begin the installation.

As said BIZAR !!

However it works. See below

louisb@test4k:~$ lsblk -o KNAME,NAME,FSTYPE,SIZE,MOUNTPOINT,PHY-SEC,LOG-SEC
KNAME NAME FSTYPE SIZE MOUNTPOINT PHY-SEC LOG-SEC
loop0 loop0 squashfs 48.1M /snap/snapd/25935 512 512
loop1 loop1 squashfs 74M /snap/core22/2292 512 512
loop2 loop2 squashfs 13M /snap/canonical-livepatch/378 512 512
loop3 loop3 squashfs 76.6M /snap/powershell/333 512 512
vda vda 100G 4096 4096
vda1 ├─vda1 vfat 1G /boot/efi 4096 4096
vda2 ├─vda2 ext4 2G /boot 4096 4096
vda3 └─vda3 LVM2_member 96.9G 4096 4096
dm-0 └─ubuntu–vg-ubuntu–lv btrfs 96.9G / 4096 4096
louisb@test4k:~$

You can changes the physical cluster size of a ZVOL at any time, however the data that is stored on the drive will probably get corrupt of put in an unreadable mode. but practically at any point before you install anything on the virtual drive you can make the swap.

Last time I set up a VM, I am sure I had to make the disk first to be able to set 4k, otherwise the wizard used the 512/default setting.

@louis I hope you got sorted.

I did decide to configure a 4K version of the VM from zero. I am now on the same point I was before.

this stupid thing just did cost me 1,5 days !!

I really hope they will repair this!

Without a bug report or feature request i doubt it will get fixed. So if you want it fixed create either a jira ticket or a feature request on the forum…