Mac OS X 10.9 Server authentication using OpenLDAP not working

I have a Debian server that is running OpenLDAP version 2.4.31, and now I am trying to connect my Mac OS X 10.9 Server, so that I can authenticate existing LDAP users. I successfully connected them using the directory utility and mapped the users record to the values returned by the server. I did the following mappings:

User: inetOrgPerson
    EmailAddress: mail
    FirstName: givenName
    LastName: sn
    Password: userPassword
    PrimaryGroupID: #5000
    RealName: cn
    RecordName: uid
    UniqueID: uidNumber
    GeneratedUID: mail

I can successfully authenticate using the Directory Utility and services that run on the local network, like Time Machine. However, when I try to access any service that is accessible from the Internet, like the Wiki Server or the Xcode Server, then authentication fails. Here's a log from my OS X Server when I tried to log into the Wiki:

Nov 29 21:06:55 osx.example.org collabd[437]: [CSAuthService.m:315 11d8d000 +269ms] Digest did not validate
Nov 29 21:06:55 osx.example.org collabd[437]: [CSServiceDispatcher.m:260 11d8d000 +0ms] Caught exception "Invalid Credentials" [CSAuthBadDigest] executing [http]Request{AuthService.validateUsernameAndPasswordDigest:remember:(<<scrubbed>>)}

And here's a second log from when I tried to log into Xcode Server:

Nov 29 21:23:27 osx.example.org collabd[437]: [CSAuthService.m:315 11b5a000 +95ms] Digest did not validate
Nov 29 21:23:27 osx.example.org collabd[437]: [CSServiceDispatcher.m:260 11b5a000 +0ms] Caught exception "Invalid Credentials" [CSAuthBadDigest] executing [http]Request{AuthService.sessionForDigest:remember:(Digest username="redacted",realm="osx.example.org",nonce="****",nc="*****",cnonce="***",qop="auth",algorithm=md5-sess,uri="/",response="***",0)}

It seems like it is connecting to the LDAP server, but what for? Anyway, here's the LDAP log:

filter: (&(|(objectClass=inetOrgPerson))(|(uid=redacted)(cn=redacted)))
ber_scanf fmt ({M}}) ber:
...
bdb_dn2entry("dc=example,dc=org")
search_candidates: base="dc=example,dc=org" (0x00000001) scope=2
...
5298f51f => send_search_entry: conn 1001 dn="uid=redacted,ou=users,dc=example,dc=org"
ber_flush2: 183 bytes to sd 12
ldap_write: want=183, written=183
...
5298f51f <= send_search_entry: conn 1001 exit.
5298f51f send_ldap_result: conn=1001 op=3 p=3
5298f51f send_ldap_result: err=0 matched="" text=""
5298f51f send_ldap_response: msgid=4 tag=101 err=0
ber_flush2: 14 bytes to sd 12
ldap_write: want=14, written=14
  0000:  30 0c 02 01 04 65 07 0a  01 00 04 00 04 00         0....e........

From what I can tell, the OS X server asks the LDAP server for the user records, but then something on the OS X side isn't working out right. Can someone tell me what I have to do so that authentication works? Could the problem be that the "realm" seems to be wrong? The ldap server's hostname is "example.org", but the osx server's hostname is "osx.example.org". If the realm is indeed the problem, how do I change it from osx.example.org to example.org?

I realize this is an old question, but I had the same issue and hit this post in my travels. If it helps you (or others), and for posterity, this is what I had to do to fix it:

On the server, open the WebConfigProperties.plist file and change the attribute RealmAuthType from Digest to Basic. By default, the configuration files should be located at /Library/Server/Web/Config/apache2/.

Hope this helps!