LDAP can't bind "certificate verify error"

chakkerz · 04-14-2010, 07:02 PM

Hello there

The internet is full of these questions and i just can not spot what the issue is.

Firstly, this is not on the master node, but rather the node that is being replicated to. The problem occurs when i query using ldapsearch or an `getent passwd`

EG ldapsearch:

Code:

[root@cakeslave ~]# ldapsearch -x -b 'cn=Christian Unger,ou=People,dc=example,dc=org' -D "cn=replica,dc=example,dc=org" -H ldaps://cakeslave.example.org -w cakewalk
ldap_bind: Can't contact LDAP server (-1)
	additional info: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

with the ldap log contains a TLS failure:

Code:

Apr 15 09:39:50 cakeslave slapd[13069]: daemon: activity on 1 descriptor 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: activity on:
Apr 15 09:39:50 cakeslave slapd[13069]:  
Apr 15 09:39:50 cakeslave slapd[13069]: >>> slap_listener(ldaps:///) 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: listen=10, new connection on 15 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: added 15r (active) listener=(nil) 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=7 active_threads=0 tvp=zero 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=8 active_threads=0 tvp=zero 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=9 active_threads=0 tvp=zero 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=10 active_threads=0 tvp=zero 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: activity on 1 descriptor 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: activity on:
Apr 15 09:39:50 cakeslave slapd[13069]:  15r
Apr 15 09:39:50 cakeslave slapd[13069]:  
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: read active on 15 
Apr 15 09:39:50 cakeslave slapd[13069]: connection_get(15) 
Apr 15 09:39:50 cakeslave slapd[13069]: connection_get(15): got connid=1 
Apr 15 09:39:50 cakeslave slapd[13069]: connection_read(15): checking for input on id=1 
Apr 15 09:39:50 cakeslave slapd[13069]: connection_read(15): TLS accept failure error=-1 id=1, closing 
Apr 15 09:39:50 cakeslave slapd[13069]: connection_closing: readying conn=1 sd=15 for close 
Apr 15 09:39:50 cakeslave slapd[13069]: connection_close: conn=1 sd=-1 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: removing 15 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=7 active_threads=0 tvp=zero 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=8 active_threads=0 tvp=zero 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=9 active_threads=0 tvp=zero 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=10 active_threads=0 tvp=zero 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: activity on 1 descriptor 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: activity on:
Apr 15 09:39:50 cakeslave slapd[13069]:  
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=7 active_threads=0 tvp=zero 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=8 active_threads=0 tvp=zero 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=9 active_threads=0 tvp=zero 
Apr 15 09:39:50 cakeslave slapd[13069]: daemon: epoll: listen=10 active_threads=0 tvp=zero 
~

and the nss_ldap lookup results in an error in /var/log/messages:

Code:

Apr 15 09:47:40 cakeslave getent: nss_ldap: failed to bind to LDAP server ldaps://cakeslave.example.org/: Can't contact LDAP server
Apr 15 09:47:40 cakeslave getent: nss_ldap: could not search LDAP server - Server is unavailable

my slapd.conf is as follows:

Code:

include         /etc/openldap/schema/core.schema
include         /etc/openldap/schema/cosine.schema
include         /etc/openldap/schema/inetorgperson.schema
include         /etc/openldap/schema/nis.schema

## custom includes
include         /etc/openldap/custom/solaris.schema
include         /etc/openldap/custom/openssh-lpk_openldap.schema

allow bind_v2
pidfile         /var/run/openldap/slapd.pid
argsfile        /var/run/openldap/slapd.args

access to attrs=userPassword,sshPrivateKey,sshPublicKey
        by self write
	by dn="cn=admin,dc=example,dc=org" read
	by dn="cn=authenticated_LDAP,dc=example,dc=org" read
        by anonymous auth
        by * none

access to *
	by self write
	by dn="cn=admin,dc=example,dc=org" read
	by dn="cn=unprivuser,dc=example,dc=org" read
	by dn="cn=authenticated_LDAP,dc=example,dc=org" read
	by users auth
	by anonymous auth

TLSCipherSuite HIGH:MEDIUM:+SSLv2
TLSCACertificateFile /etc/openldap/certificate/cacert.pem
TLSCertificateFile /etc/openldap/certificate/cakeslavecert.pem
TLSCertificateKeyFile /etc/openldap/certificate/cakeslavekey.pem
TLSVerifyClient never

database        bdb
suffix          "dc=example,dc=org"
rootdn          "cn=replica,dc=example,dc=org"
rootpw          {SSHA}WSEae1GsFDN0aOnxHdslw1RaUuWb65gw

directory       /var/lib/ldap
loglevel	127

index objectClass                       eq,pres
index ou,cn,mail,surname,givenname      eq,pres,sub
index uidNumber,gidNumber,loginShell    eq,pres
index uid,memberUid                     eq,pres,sub
index nisMapName,nisMapEntry            eq,pres,sub

syncrepl rid=123
    provider=ldaps://cakewalk.example.org:636/
    type=refreshOnly
    interval=00:00:05:00
    searchbase="dc=example,dc=org"
    filter="(objectClass=*)"
    scope=sub
    attrs=""
    schemachecking=off
    bindmethod=simple
    binddn="cn=admin,dc=example,dc=org"
    credentials=cakewalk

and my /etc/openldap/ldap.conf is:

Code:

TIMEOUT=10
NETWORK_TIMEOUT=10

URI ldaps://cakewalk.example.org/

#TLSCipherSuite HIGH:MEDIUM:+SSLv2
TLS_CACERT /etc/openldap/certificate/cacert.pem 
TLS_REQCERT demand

bind_policy soft

and the nss_ldap /etc/ldap.conf is:

Code:

uri ldaps://cakeslave.example.org/
base dc=example,dc=org
rootbinddn cn=replica,dc=example,dc=org

timelimit 30
bind_timelimit 30
idle_timelimit 3600
bind_policy soft

nss_initgroups_ignoreusers root,ldap,named,avahi,haldaemon,dbus,radvd,tomcat,radiusd,news,mailman,nscd,gdm

ssl yes
tls_checkpeer yes
tls_cacertfile /etc/ssl/certs/cacert.pem
tls_ciphers AES256-SHA

pam_password md5

The `getent passwd` results in a very similar error in the ldap log, which again indicates a TLS failure.

The cacert.pem in /etc/ssl/certs and /etc/openldap/certificate are identical (check using md5sum). I have done an strace and found that it looks at /etc/pki/tls/cert.pem . I don't know why it does that, since i did not reference it anywhere, and i'm guessing it might be a red herring.

I'm hoping it's an obvious n00b error ...

EDIT: oh yeah, if i tell it to use ldap (instead of ldaps) it works fine, but i would prefer to use ssl.

kbp · 04-14-2010, 08:28 PM

I don't think "TLS_REQCERT demand" works, even though it should. Try:

"TLS_REQCERT allow"

If you run 'tcpdump -i eth0 -n port 389 or port 636' you should only see ldaps traffic

hth

chakkerz · 04-14-2010, 11:15 PM

but TLS_REQCERT allow doesn't do what i want (see below for the difference).

Also, tcpdump only verifies communication, i'm running the server with debugging so i can see the interaction happening.

Code:

       TLS_REQCERT <level>
              Specifies what checks to perform on server certificates in a TLS
              session, if any. The <level> can be specified as one of the fol-
              lowing keywords:

              allow  The server certificate is requested. If no certificate is
                     provided, the session proceeds normally. If  a  bad  cer-
                     tificate  is provided, it will be ignored and the session
                     proceeds normally.

              demand | hard
                     These  keywords are equivalent. The server certificate is
                     requested. If no certificate is provided, or a  bad  cer-
                     tificate  is  provided, the session is immediately termi-
                     nated. This is the default setting.

chakkerz · 04-14-2010, 11:45 PM

I just regenerated the server key and certs, and now it's happy.

kbp · 04-15-2010, 09:54 PM

Thanks chakkerz, forced me to resolve an issue I'd noticed but was too lazy to look at

chakkerz · 04-15-2010, 10:01 PM

glad i could help

chakkerz · 05-10-2011, 11:34 PM

So, today my production LDAP infrastructure face planted... Turns out my CA ran out.

I guess the message to take away from this is: pay attention to when the certificates and ca run out because if you don't you're stuck scratching your head wondering why did it all the ldap clients start having issues... nothing changed.

Also sharing your issues with others can be quite handy, especially when you read the post and go ... "wow that guy is having the exact same problem ... the writing style seems familiar ... oh ... i posted that".