LDAP health check seems broken with TrueNAS Scale + Google Secure LDAP

TrueNAS General
1

We’re trying to connect our TrueNAS SCALE device to Google Secure LDAP, which uses a client TLS certificate for authentication.

I would link to the documentation for Google Secure LDAP here, but I get an error trying to include a link in a post.

I’ve gotten everything configured, and when I click “Save” in the UI it appears to be working; it shows a message about ldap.update and says it’s synced ~60 users and ~10 groups from the LDAP server. But it immediately fails afterwards, with the following error:

{'msgtype': 101, 'msgid': 2, 'result': 50, 'desc': 'Insufficient access', 'ctrls': []}

Full traceback below

It appears to me that what is happening is that sssd is getting configured correctly to do certificate authentication, and starts up and synchronizes users, but that the health check does not pick up the certificate correctly and fails. This is pretty frustrating because the underlying software appears to all be fine, and it’s just the health check that’s broken.

Has anyone been able to successfully connect TrueNAS SCALE to Google Secure LDAP?

Here’s the full traceback:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/plugins/directoryservices_/ldap_health_mixin.py", line 44, in _health_check_ldap
    self.middleware.call_sync('ldap.get_root_DSE')
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1654, in call_sync
    return self.run_coroutine(methodobj(*prepared_call.args))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1694, in run_coroutine
    return fut.result()
           ^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/lib/python3/dist-packages/middlewared/plugins/ldap.py", line 784, in get_root_DSE
    return await self.middleware.call('ldapclient.get_root_dse', {"ldap-configuration": client_conf})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1629, in call
    return await self._call(
           ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1471, in _call
    return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1364, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 183, in nf
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/ldap.py", line 143, in get_root_dse
    results = LdapClient.search(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/ldap_/ldap_client.py", line 14, in inner
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/ldap_/ldap_client.py", line 189, in search
    (rtype, rdata, rmsgid, serverctrls) = self._handle.result3(
                                          ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/ldap/ldapobject.py", line 543, in result3
    resp_type, resp_data, resp_msgid, decoded_resp_ctrls, retoid, retval = self.result4(
                                                                           ^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/ldap/ldapobject.py", line 553, in result4
    ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/ldap/ldapobject.py", line 128, in _ldap_call
    result = func(*args,**kwargs)
             ^^^^^^^^^^^^^^^^^^^^
ldap.INSUFFICIENT_ACCESS: {'msgtype': 101, 'msgid': 2, 'result': 50, 'desc': 'Insufficient access', 'ctrls': []}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 509, in run
    await self.future
  File "/usr/lib/python3/dist-packages/middlewared/job.py", line 554, in __run_body
    rv = await self.method(*args)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 49, in nf
    res = await f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/schema/processor.py", line 179, in nf
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/ldap.py", line 682, in do_update
    await self.__start(job, ds_type)
  File "/usr/lib/python3/dist-packages/middlewared/plugins/ldap.py", line 981, in __start
    await self.middleware.call('directoryservices.health.check')
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1629, in call
    return await self._call(
           ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1471, in _call
    return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1364, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/middlewared/plugins/directoryservices_/health.py", line 117, in check
    self._health_check_ldap()
  File "/usr/lib/python3/dist-packages/middlewared/plugins/directoryservices_/ldap_health_mixin.py", line 47, in _health_check_ldap
    raise LDAPHealthError(
middlewared.utils.directoryservices.health.LDAPHealthError: {'msgtype': 101, 'msgid': 2, 'result': 50, 'desc': 'Insufficient access', 'ctrls': []}
2

The health check tries to use your credentials to connect to the root DSE of the LDAP server. Generally, LDAP servers don’t restrict access to this (in fact, the RFC for LDAP says it’s required to be accessible). RFC 4512: Lightweight Directory Access Protocol (LDAP): Directory Information Models

If they’re making root DSE access unreliable then I don’t really see a way to improve the health check. You’ll probably just have to disable it and hope for the best.

3

I don’t have any trouble querying the root DSE with ldapsearch:

$ LDAPTLS_CERT=Downloads/Google_2028_04_30_75757.crt LDAPTLS_KEY=Downloads/Google_2028_04_30_75757.key ldapsearch -H ldaps://ldap.google.com -b "" -s base "(objectClass=*)"
SASL/EXTERNAL authentication started
SASL username: st=California,c=US,ou=Google Workspace,cn=LDAP Client,l=Mountain View,o=Google Inc.
SASL SSF: 0
# extended LDIF
#
# LDAPv3
# base <> with scope baseObject
# filter: (objectClass=*)
# requesting: ALL
#

#
dn:
objectClass: top

# search result
search: 3
result: 0 Success

# numResponses: 2
# numEntries: 1
4

This sounds like it would get me up and running; how do I disable the health check?

5

It’s in the alerts UI. Do note that there are general limitations to how much you can accomplish on a NAS using google secure LDAP. For instance, its accounts cannot be used for SMB authentication.

6

That’s fine, all I need are UIDs and GIDs for NFS

7

That didn’t seem to help; I set all these to NEVER:

(screenshot omitted because it won’t let me post after uploading one)

…but it still seems to be trying to do the health check as part of saving the LDAP configuration, and won’t let me save it because of the root DSE error.

8

Here it is from the CLI; it’s definitely getting those users, and then it just gives up:

[lax-storage-01]> directory_service ldap update enable=true anonbind=true
[0%] Preparing to configure LDAP directory service....
[15%] Filling cache...
[40%] Preparing to add users to cache...
[50%] lab: adding user to cache. User count: 0...
[50%] vaheed: adding user to cache. User count: 10...
[50%] rick: adding user to cache. User count: 20...
[50%] elena: adding user to cache. User count: 30...
[50%] jamie: adding user to cache. User count: 40...
[50%] brian.schmid: adding user to cache. User count: 50...
[50%] nick: adding user to cache. User count: 60...
[70%] Preparing to add groups to cache...
[80%] apps.eng: adding group to cache. Group count: 0...
[80%] uhoh: adding group to cache. Group count: 10...
[100%] Cached 65 users and 20 groups....
Error: LDAPHealthError(<LDAPHealthCheckFailReason.LDAP_BIND_FAILED: 1>, "{'msgtype': 101, 'msgid': 2, 'result': 50, 'desc': 'Insufficient access', 'ctrls': []}")

[lax-storage-01]>
9

I think the bug is here: middleware/src/middlewared/middlewared/plugins/ldap.py at 9b439c7680d1cc0f649995e330980e13dd997a1c · truenas/middleware · GitHub

        if data['anonbind']:
            client_config['bind_type'] = 'ANONYMOUS'
        elif data['cert_name']:
            client_config['bind_type'] = 'EXTERNAL'
        elif data['kerberos_realm']:
            client_config['bind_type'] = 'GSSAPI'
        else:
            client_config['bind_type'] = 'PLAIN'
            client_config['credentials'] = {
                'binddn': data['binddn'],
                'bindpw': data['bindpw']
            }

…should be…

        if data['cert_name']:
            client_config['bind_type'] = 'EXTERNAL'
        elif data['anonbind']:
            client_config['bind_type'] = 'ANONYMOUS'
        elif data['kerberos_realm']:
            client_config['bind_type'] = 'GSSAPI'
        else:
            client_config['bind_type'] = 'PLAIN'
            client_config['credentials'] = {
                'binddn': data['binddn'],
                'bindpw': data['bindpw']
            }
10

Are you specifying anonymous bind and a certificate? If you’re using client certificate then you’re not anonymous.

11

Yes, it won’t let me leave the Bind DN blank unless I click the Anonymous box, even if I specify a certificate.

12

I guess an alternative would be to change this: middleware/src/middlewared/middlewared/plugins/ldap.py at 9b439c7680d1cc0f649995e330980e13dd997a1c · truenas/middleware · GitHub

        if not new["bindpw"] and not new["kerberos_principal"] and not new["anonbind"]:

…to this:

        if not new["bindpw"] and not new["kerberos_principal"] and not new["anonbind"] and not new["certificate"]:
13

Ah. I see. I’ll make the change for 25.04.1.

1 Like
14

Ok, I made a PR for my first approach here (and I’m currently building an upgrade package with that to test): Prefer SASL EXTERNAL over ANONYMOUS when using an LDAP client cert by jordemort · Pull Request #16414 · truenas/middleware · GitHub

No offense if you’d rather close that and fix it some other way though. Happy to test anything you’d like tested!

15

Your second alternative is the correct fix. If you want to redo your PR with the correct fix I’m happy to merge it in.

16

Sure, I can do that; getting pretty close to the start of my weekend though, so it might be Monday.