Dealing with a 3rd party library who's function call causes heap corruptions when P/Invoking

0

I have created a library wrapper which invokes FlexNet Publisher. It is written in C. I am trying to P/Invoke it:

[DllImport("lmgr11.dll")]
public static extern void free_job();

Its implementation is fairly simple:

void WINAPI free_job()
{
    if (jobPtr != NULL)
        lc_free_job(jobPtr);
    jobPtr = NULL;
}

The documentation says that lc_free_job should free a job as well as all resources. Calling lc_free_job works just fine from native code (I made an ATL object expose a wrapper to it through a COM object and I can consume this method all day long from a Visual C++ console application, so I know it must work).

However, from C#, when I try to P/Invoke this method, I get the following error which crashes my application:

Unhandled exception at 0x00007FFA39358283 (ntdll.dll) in CerberusTestHarness.exe: 0xC0000374: A heap has been corrupted (parameters: 0x00007FFA393AF6B0).

Why is this? Can I catch it or circumvent it in any way? I can't seem to catch the exception if I wrap the call in a try-catch because its not throwing a .NET exception. Note that I don't have the source code for lc_free_job so I can't inspect it or view its source code, unfortunately. Its written in C.

c#
.net
c
exception
pinvoke
asked on Stack Overflow Mar 30, 2017 by Alexandru

1 Answer

0

The issue was due to my C# P/Invoke code from a previous invocation. Hopefully this helps anyone else who comes across the same issue.

I had defined this P/Invoke to get an error string back from a function which returns char *:

[DllImport("lmgr11.dll")]
public static extern string errstring();

I was calling it every time something failed in my 3rd party library as above. This is not the correct way to P/Invoke a method which returns char *. Otherwise, when freeing the error from the native side, it will cause a heap corruption due to the way this string was marshaled.

It needed to be defined like this:

[DllImport("lmgr11.dll")]
public static extern IntPtr errstring();

And called as follows:

var errorMessage = Marshal.PtrToStringAnsi(errstring());
answered on Stack Overflow Mar 31, 2017 by Alexandru • edited Mar 31, 2017 by Alexandru

User contributions licensed under CC BY-SA 3.0