Replacing a DLL got 0xC0000005: Access violation executing location 0xFFFFFFFF

2

I am trying to bypass a no longer needed DLL but we don't have the source because it's from a third party company of which they no longer maintain the code. We have an alternative solution to that DLL. However, unfortunately, it's a huge compiling work for our project so it's almost impossible for me now to re-compile the entire project in a such short period. So I am thinking I might be able to make a fake DLL and put my own algorithm in the critical function. But now I am getting the 0XC000005 violation and I couldn't resolve it.

The DLL was loaded and unloaded this way. It's in the source and I didn't write it. I just want to make my DLL work-around it.

DWORD WINAPI UnloadDLL(LPVOID Param)
{
    typedef void (WINAPI *WaitForFinish)();

    //  Param should be the address of the dllhandle, passed in from CheckAuthorization()
    HINSTANCE *dllhandle = (HINSTANCE*)Param;
    WaitForFinish WaitForFinish_ptr = 0;

    //  Attempt to load the function WaitForFinish().  If we cannot get this function,
    //  then this is a version of the DLL without any threads to wait on, so there's
    //  no need to return an error if it's missing.  We can just proceed to unload the
    //  DLL
    if ( (WaitForFinish_ptr = (WaitForFinish)GetProcAddress(*dllhandle, "WaitForFinish")) )
    {
        //  If it is there, then call WaitForFinish() to let all threads from
        //  CheckLicense() finish before we unload the DLL
        WaitForFinish_ptr();
    }

    //  Finally, unload the DLL
    FreeLibrary(*dllhandle);
    *dllhandle = 0;
    return 0;
}

int run() {
    int  UnloadDLLThreadID;
    HANDLE unloadhandle;
    HINSTANCE dllhandle;

    /* Loading library and run algorithm */
    typedef int (WINAPI *CheckPoint1)(long *ParmValue);
    CheckPoint1 CheckPoint1_ptr = 0;
    if (!(dllhandle = LoadLibrary("mydll.dll")))
    {
        MessageBox(0,"\n\n\n\n     mydll.dll is missing.               \n\n\n","ERROR",MB_OK|MB_ICONSTOP);
        ExitProcess(1);
    }

    if (!(CheckPoint1_ptr = (CheckPoint1)GetProcAddress(dllhandle, "CheckPoint1")))
    {
        FreeLibrary(dllhandle);
        dllhandle = 0;
        MessageBox(0,"\n\n\n\n      mydll.dll is missing or the wrong version.             \n\n\n","ERROR",MB_OK|MB_ICONSTOP);
        ExitProcess(1);
    }

    result = CheckPoint1_ptr(&ParmValue);

    /*
    Create a separate thread to call the function UnloadDLL, above.
    CheckPoint1 can spawn threads of its own, so we have to wait
    on those to finish before we can unload the DLL--otherwise the
    threads won't finish, and in worst case the entire program can
    hang.  To allow the rest of the program to run while we wait,
    we handle unloading the DLL in a separate thread.
    */
    unloadhandle = CreateThread(NULL, 0, UnloadDLL, &dllhandle, 0, &UnloadDLLThreadID);
    if( unloadhandle ){ CloseHandle(unloadhandle); }
}

So in the hacked DLL

In mydll.h

#ifdef MYDLL_EXPORTS
#define MYDLL_API __declspec(dllexport)
#else
#define MYDLL_API __declspec(dllimport)
#endif

#include <windows.h>
#include <stdio.h>
#include <stdlib.h>

extern "C" {
    
    MYDLL_API int CheckPoint1(long *parmValue); 
    MYDLL_API void WaitForFinish();
}

And the mydll.c

#include "mydll.h"

extern "C" {    
    int CheckPoint1(long *parmValue)
    {   
        // Try to see if it's loaded
        char szMessage[250];
        strcpy_s(szMessage, "Loaded...");
        MessageBoxA(0, szMessage, "INFORMATION", MB_OK | MB_ICONINFORMATION);
        int result = 10000;
        return result;
    }

    void WaitForFinish() {
    // Try to see if it's loaded
        char szMessage[250];
        strcpy_s(szMessage, "Waiting...");
        MessageBoxA(0, szMessage, "INFORMATION", MB_OK | MB_ICONINFORMATION);
    }
}

The above implementation in the DLL is my implementation for testing this solution. It compiled without any error. Then I replaced the old dll with this new one. The dll was loaded successfully, and I was able to see my algorithm function ran from the pop-up dialog. Then the program crashed(after I clicked "OK" from the pop-up). I used the debugger in VC2017 and it tells me I have

Exception thrown at 0xFFFFFFFF in mindcontrol.exe: 0xC0000005: Access violation executing location 0xFFFFFFFF.

The debug log is as follows

'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\ntdll.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\kernel32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\KernelBase.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\apphelp.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\AcLayers.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\msvcrt.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\user32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\win32u.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\gdi32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\gdi32full.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\msvcp_win.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\ucrtbase.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\shell32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\shlwapi.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\oleaut32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\combase.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\rpcrt4.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\setupapi.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\cfgmgr32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\bcrypt.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\mpr.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\sfc.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\winspool.drv'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\sfc_os.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\AcGenral.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\sechost.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\ole32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\advapi32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\uxtheme.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\winmm.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\samcli.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\msacm32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\version.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\userenv.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\dwmapi.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\urlmon.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\sspicli.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\winmmbase.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\iertutil.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\SHCore.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\imm32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\comdlg32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\msimg32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\oledlg.dll'. Symbols loaded.
'MindControl.exe' (Win32): Unloaded 'C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_6.0.19041.1_none_fd031af45b0106f2\comctl32.dll'
'MindControl.exe' (Win32): Loaded 'C:\Windows\WinSxS\x86_microsoft.windows.gdiplus_6595b64144ccf1df_1.1.19041.388_none_429cd6de8a9002ce\GdiPlus.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\oleacc.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\kernel.appcore.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\bcryptprimitives.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\riched32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\riched20.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\usp10.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\msls31.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Program Files (x86)\MindControl\mydll.dll'. Symbols loaded.
The thread 0x7fec has exited with code 0 (0x0).
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\msctf.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\TextInputFramework.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\CoreUIComponents.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\CoreMessaging.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\ws2_32.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\ntmarta.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\WinTypes.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\WinTypes.dll'. Symbols loaded.
'MindControl.exe' (Win32): Unloaded 'C:\Windows\SysWOW64\WinTypes.dll'
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\TextShaping.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\clbcatq.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\DataExchange.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\d3d11.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\dcomp.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\dxgi.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\twinapi.appcore.dll'. Cannot find or open the PDB file.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\WindowsCodecs.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\ExplorerFrame.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\windows.storage.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\wldp.dll'. Symbols loaded.
'MindControl.exe' (Win32): Loaded 'C:\Windows\SysWOW64\propsys.dll'. Symbols loaded.
Exception thrown at 0xFFFFFFFF in MindControl.exe: 0xC0000005: Access violation executing location 0xFFFFFFFF.

The program '[32632] MindControl.exe' has exited with code 0 (0x0).

I am guessing it's because WaitForFinish() wasn't loaded successfully because I didn't see the pop-up window. Other reason might be FreeLibary() call in the thread. But either way I don't know how to solve it. Can anyone help to identify where I did wrong and maybe a suggestion of resolving it?

Thank you for helping.

c
asked on Stack Overflow Aug 3, 2020 by Y. Chang • edited Aug 3, 2020 by Y. Chang

1 Answer

2

I'm going to put what I said in my comment down here, because after looking over it more, I am about 90% sure that's the problem.

The library you created has your exports defined as __cdecl.

One of the calls to your DLL (CheckPoint1) should work fine, because you defined it as so:

typedef int (*CheckPoint1)(long *ParmValue);

You didn't specify calling convention in that definition, so it defaults to __cdecl.

However, in the other export WaitForFinish, you defined it as:

typedef void (WINAPI *WaitForFinish)();

Here, you are specifying the calling convention as WINAPI (which is just __stdcall).

So to fix this, you could simply change it to:

typedef void (*WaitForFinish)();

or

typedef void (__cdecl *WaitForFinish)();

Both mean the same thing.

If you are interested in what these calling conventions do, take a look at this article.

ETA: This may be an issue, it may not:

unloadhandle = CreateThread(NULL, 0, UnloadDLL, &dllhandle, 0, &UnloadDLLThreadID);
if( unloadhandle ){ CloseHandle(unloadhandle); }

Typically, you should wait for the thread to finish before closing the handle to it. You can do this by using WaitForSingleObject().

answered on Stack Overflow Aug 3, 2020 by Andy • edited Aug 3, 2020 by ikegami

User contributions licensed under CC BY-SA 3.0