Premissions to access LDAP from a windows service

I'm building a windows service that polls a windows file share location for a file containing members that may be found in LDAP. The windows service is running as a specific user on the domain to have access to the file share. This user is in the Administrator group both on the server running the service and where LDAP is found.

The problem I have is that fetching the files works fine but then when accessing LDAP I get the following exception:

LDAPService.Program - Exception: An operations error occurred. InnerEx:

=== STACKTRACE ===
LDAPService.Program - at System.DirectoryServices.DirectoryEntry.Bind(Boolean throwIfFail)
at System.DirectoryServices.DirectoryEntry.Bind() at System.DirectoryServices.DirectoryEntry.RefreshCache()

=== BASE EXC ===
LDAPService.Program - System.DirectoryServices.DirectoryServicesCOMException (0x80072020): An operations error occurred.

Searching for this error there is a lot about ASP.NET and using Impersonate(). But that didn't help.

If I change the windows service to logon as Local System Account then I have no problems accessing LDAP (but I can't access the windows file share).

What other settings should I look at?

I was using two different LDAP directories and forgot to provide the user/pass for the second one. Had only given it for the first one.

A classic example of what we call SBS here in Sweden (Skit Bakom Spakarna).