Error invoking Windows DLL function from Python

0

I have a Windows DLL that exposes a function. Let's call it Fn. It takes two CStrings as parameters and returns a CString:

CString Fn(CString a, CString b)

The C++ code you see below successfully loads the DLL and invokes Fn. A side effect of Fn is that it prints the value of parameter a to stdtout. So I see:

Parameter a has value "A"

However, when I do the same from Python, I get:

Parameter a has value "B"

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: exception: access violation reading 0x00000005

So Fn somehow receives parameter b instead of parameter a.

My question is: Why do my function parameters get mangled when calling the DLL from Python, but not from C++?

In fact, this problem does not only occur when I call Fn from Python, but also when I call it from NSIS via:

System::Call "My::Fn(m 'A', m 'B') m .r0"

I thought that it might be a problem with the calling convention. So I changed windll.LoadLibrary to cdll.LoadLibrary. Unfortunately, this produces the exact same output.

Python code (not working)

from ctypes import *
f = windll.LoadLibrary('My.dll').Fn
f.argtypes = [c_char_p, c_char_p]
f.restype = c_char_p
f(b"A", b"B")

C++ Code (working)

#include <atlstr.h>

typedef CString (*Fn)(CString, CString);

int main( void ) {
    HINSTANCE dll = LoadLibrary("My.dll");
    Fn f = (Fn) GetProcAddress(dll, "Fn");
    f("A", "B");
    FreeLibrary(dll);
    return 0;
}
python
c++
windows
dll
ctypes

1 Answer

0

Using the comments to my question, I managed to find a solution.

The problem seems to have been that CString is an MFC concept that neither Python nor NSIS know about. Changing Fn's signature to

LPCTSTR Fn(LPCTSTR, LPCTSTR)

solved the issue.

answered on Stack Overflow Oct 7, 2019 by Michael Herrmann

User contributions licensed under CC BY-SA 3.0