[NVMe over TCP] Adding a Namespace: zVol or File? What is a "File?"

TrueNAS General
, ,
1

I’m working on getting my first NVME-over-TCP share set up using the official docs and the excellent guide by @imvalgo over at Getting Started with NVMe over TCP ; It didn’t take me long to run into something I don’t understand. :stuck_out_tongue:

This is the window I’m looking at:

image
image1084×1510 93.5 KB

I’m used to using zVols as backing storage for iSCSI, but what is a “[New | Existing] File?” I assume these are .raw files, whose performance is tied to recordsize instead of volbocksize, but that’s completely a guess. I’m aware that raw files can be used as a backing storage for VMs, but I’ve never actually seen it implemented anywhere.

Is there any written guidance anywhere on when I should use a zVol vs a “file” for NVME over TCP?

Thanks!

2

The NVMe-oF file is similar to zvol with some exception. You would created it as usual, give it a location, give it a name space, binding ip:port, and host exclusion if necessary.

You would query it and connect to it as if you would a zvol though they do not show up in your datasets (a zvol will show in your dataset).

You would format it and use it just as you would a zvol. While you can modify the properties of a zvol from your dataset, you must go through the NVMe-oF Share to make any modification to the file while it is not in use.

Another note, when deleting a zvol NVMe-oF, you have to do it in the NVMe-oF share and the dataset for the zvol where the file can just be done from the NVMe-oF share.

1 Like
3

Thanks.

Is there anything in the docs about the underlying “file,” and when to choose it over a zVol? In some of my reading, I’ve seen that using raw files in datasets as backing storage for block storage used by VMs, for example, can be more performant than using zVols, but if that’s the case, it seems like using the raw file storage method should be the default.

I also suspect that, as a file, its behavior on disk is governed by recordsize instead of volbocksize, but I haven’t found a way to confirm that yet.

4

I’ve always defaulted to zvols & have asked myself the same thing regarding zvol versus file storage.

Here’s a thread from Lvl1techs you might find useful - ZVOL vs. file as VM backing - huge performance difference on nvme-based zpool - #15 by Janos - Operating Systems & Open Source - Level1Techs Forums

1 Like
5

Awesome! Thanks. I’ll check this out. :slight_smile: