Calling DLL from VB.net causes heap exception

2

I made a DLL with gcc that contains only the following function:

#include <windows.h>

BSTR __declspec(dllexport) testfunc(void)
{
     return SysAllocString(L"Hello");
}

which is based on the code at the end of this answer. The build command is gcc -shared -o testfunc.dll main.c -Os -s -loleaut32.

In Visual Basic using VS 2017 Community my code is:

Imports System.Runtime.InteropServices
Imports Microsoft.VisualBasic
Imports System
Imports System.Text

Module Module1
    <DllImport("testfunc.dll", CallingConvention:=CallingConvention.Cdecl
               )>
    Private Function testfunc() As String
    End Function

    Sub Main()
    Dim Ret = testfunc()
    Console.WriteLine(Ret)
    End Sub
End Module

However, executing the program causes an exception upon returning from testfunc. Execution never reaches the Console.WriteLine line. The exception is:

The program '[15188] ConsoleApp1.exe' has exited with code -1073740940 (0xc0000374).

which indicates heap corruption. What am I doing wrong?


Things I tried that didn't help:

  • Changing to __stdcall and declaring the function with Declare Auto Function testfunc Lib "testfunc.dll" Alias "testfunc@0" () As String instead of <DllImport...>

Things that did work correctly:

  • Changing the function to return an integer; but then I can't access my string of course.

Note: I'm aware that I could try "returning" the string via a ByRef StringBuilder parameter as suggested on the thread I linked, but that seems like quite a lot of work on the client end and I would like to make it as simple as possible for the client, i.e. see if I can get this approach working.

c
string
windows
vb.net
dll
asked on Stack Overflow Sep 25, 2018 by M.M

1 Answer

2

In order for data to be passed between managed and unmanaged code it has to be properly mashalled. Since the runtime can not know what your testfunc() returns, you have to tell it by providing a declaration of it, which you did by

<DllImport("testfunc.dll")>
Private Function testfunc() As String

But the information that the return type is a String is ambiguous since there are many way of string representation. Use the MarshalAs-Attribute to tell the runtime how to handle the returned value:

<DllImport("testfunc.dll")>
Private Function testfunc() As <MarshalAs(UnmanagedType.BStr)> String

Read more about Interop Marshaling and Passing strings between managed and unmanaged code.

answered on Stack Overflow Sep 25, 2018 by Swordfish • edited Sep 25, 2018 by Visual Vincent

User contributions licensed under CC BY-SA 3.0