iSCSI - unable to add multiple targets to one exten

TrueNAS General
,
1

I currently have the latest version of TrueNAS Scale 24. In the iSCSI service I have a problem with adding several “terget” to the same “extent” (built on zvol) - the message “Extent is already in use” appears. I did not have such problems before, even on the same TrueNAS I currently have several “target” added to another “extent”, which I added quite a long time ago. Is this some kind of TrueNAS bug? I searched a bit, like here:

but I did not find a solution.
I would be grateful for any tips.

2

I tested create multiple targets to one extent on TrueNAS-SCALE-22.12.4.2 and it didin’t work. The same results I got on newer TrueNAS SCALE versions. Only for TrueNAS-SCALE-22.02.4 version it worked (and earlier). So, it is possible to enable to allow to add multiple targets to one extent?

3

Hi,
Try the following patch which comments out the safety check preventing the second target to extent mapping. The steps below assume running latest TrueNAS scale. Adjust as appropriate.

  • Save the following patch on the TrueNAS Scale box as ~admin/patch-target_to_extent.diff
--- target_to_extent.py.orig     2025-01-02 20:29:44.209411834 +1100
+++ target_to_extent.py  2025-01-02 20:31:22.256565684 +1100
@@ -238,10 +238,11 @@
         # a single target (at a single LUN) at a time.  Failure to
         # do so would result in a mechanism to avoid any SCSI based
         # locking, and therefore could result in data corruption.
-        if old_extent != extent and await self.query([
-            ('extent', '=', extent)
-        ], {'force_sql_filters': True}):
-            verrors.add(
-                f'{schema_name}.extent',
-                'Extent is already in use.'
-            )
+        # Commented out below as it blocked clustered iSCSI LUN use.
+        #if old_extent != extent and await self.query([
+            #('extent', '=', extent)
+        #], {'force_sql_filters': True}):
+            #verrors.add(
+                #f'{schema_name}.extent',
+                #'Extent is already in use.'
+            #)
  • Apply the patch using the following steps.
zfs set readonly=off boot-pool/ROOT/24.10.1/usr
cd /usr/lib/python3/dist-packages/middlewared/plugins/iscsi_
patch < ~admin/patch-target_to_extent.diff
zfs set readonly=on boot-pool/ROOT/24.10.1/usr

Any future TrueNAS updates will erase this patch. But should not affect already created mappings. Do test it well before relying on it. No idea if there is any issues lurking behind that safety check.

4

I would suggest that you do not bypass the check, it is there for a reason.

I’ll quote from text from NAS-133359

An extent should only be accessed via a single target. Multiple initiators can login to that target, but this should only be done if they implement some form for clustering (e.g. VMFS).

Typically any clustering will issue some form of SCSI locking primitive (e.g. persistent reservations) when interacting with the iSCSI target. These are implemented in the TrueNAS SCSI target stack (SCST). Furthermore, SCST also implements many VAAI operations for better performance when working with VMware initiators.

If a ZVOL were exposed via multiple targets then there would be no way to tie the locking together, rending it useless. This is why we have code to prevent exposing a ZVOL via multiple targets. Hacking to bypass this check will likely result in data corruption!

The one exception to this is in the TrueNAS Enterprise HA product, which has ALUA support for iSCSI. In ALUA a target can be accessed via either node in a HA pair, but elaborate steps are taken within TrueNAS to correctly implement this (SCST configured to use the kernel distributed lock manager, so that the locking for a target:lun on each node is consistent).

@m4rek11 wrote:

I currently have several “target” added to another “extent”, which I added quite a long time ago. Is this some kind of TrueNAS bug?

Yes. You should not have an extent exposed via more than one target. It was a bug if you were ever allowed to configure that.

5

Well, I did say there could be issues lurking behind that safety check.

Then the terminology used in the TrueNAS iSCSI administration page is really poorly worded and the lack of clear docs with examples for this type of deployment hurts. The TrueNAS approach is map hosts to zVols whereas industry norm is map zVols to hosts.

Referencing only the TrueNAS iSCSI sharing page, let’s have a go at mapping TrueNAS terminology to concepts and see if it gives the desired result.

Initiator Groups (IG)
Treat Initiator groups as a host ACL for accessing one or more zVols.

Create an IG for each initiator host IQN. This is used for zVols limited to that host.

Create an IG group containing all host’s IQN needing access to a shared zVol. This is used when multiple hosts need access to one zVol.

Useful Tip: Set the Description to the either the host name or the group of hosts collective (when sharing access)

Targets
Targets is a virtualised endpoint representing a combination of ACLs. This is where network addresses, iSCSI Portal, authentication, and IG ACLs are combined. An initiator logs into the TrueNAS IQN host, but think of the target being the ACL path used to reach an exposed iSCSI disk.

Useful Tip: Set the target name to hostname-purpose. E.g. host01-boot, or host01-localdata. Treat Target Alias as a description.

Note: The GUI layout is broken as the Initiator group ID shows the ID and all members IQNs and no description. This causes the left side Basic Info column to be squished up significantly resulting in networks only having 1 octave shown. Also makes one needing to work harder remembering what an IG ID is for. Be far better if the Initiator Group ID and Description (rather than all members IQNs) is displayed in the selection pop down with a width limit and text wrapping.

Extents
Extents are a map between a zVol (actual representation of block storage) to a name and NAA giving a “disk” ready for exposure to initiators. Think of each extent being an iSCSI disk.

Create an extent for each zVol. Name the extent representing the use of the zvol. Maybe a naming scheme like fqdn-purpose for individual host specific disks, and type-purpose for group shared disks.

Associated Targets
This is mapping the Target (ACL) to the extent (iSCSI disk representing a zVol). This is where who (initiators) can access a zVol is defined. This is also where the Disk LUN is defined. Note, the LUN is forced to be the same to each host.

This is where concepts greatly differ to other storage units. In many other implementations a disk or LUN is defined when attaching a disk to a host definition. In TrueNAS, the act is attaching an initiator host(s) definition to a disk.

The LUN limitation means it is not possible to map a shared group disk to different hosts using different LUNs.

Conclusion
By treating TrueNAS iSCSI as map hosts to zVols, and targets are in fact an initiator(s) ACL, it is then possible to map a zVol to multiple hosts. But the logic is really topsy turvy with very unique naming involved for those not immersed in TrueNAS ecosystem.

@m4rek11 hope the above helps and results in a working setup for you.

1 Like
6

Hello again,

thank you for reply @Brian_M, @jtwe. using multiple targets to one extens of course can lead to data corruption, but in my case I’m using this for a Hyper-V cluster using CSV.