The simple svnserve doesn't handle LDAP IIRC. You need to access SVN through Apache using WebDAV. This is a bit of a pain but I've gotten it working on one of my servers. If you Google for Apache DAV subversion ldap or something like that you should get a number of guides like this one
that will explain how to do it. The procedure is roughly equivalent for different distros (but obviously the config files will be in different places for different distros and there might be slightly different syntax for different Apache versions).