Meaning of "RtlpAnalyzeHeapFailure" in the watch-window of a dump

0

I'm working on following access violation:

Unhandled exception at 0x77DB2A10 (ntdll.dll) in <Process>.exe.dmp: 
0xC0000005: Access violation reading location 0x184487B8. occurred

The source code looks as follows:

Result CParameter::SetValue(..., ..., <internal_Class>* pBlock, ...)
{
  ...
  <internal_Class>* pStore = nullptr;
  if (!pBlock)
  {
    pStore = &m_Data; // m_Data is a global variable
  }
  else pStore = pBlock;

  if (pStore->pbData)
  {
    pStore->Clear(); // here we have the crash
  }

The call stack looks as follows:

   ntdll.dll!_RtlpCoalesceFreeBlocks@16()  Unknown Non-user code. Symbols loaded.
   ntdll.dll!@RtlpFreeHeap@16()    Unknown Non-user code. Symbols loaded.
   ntdll.dll!_RtlFreeHeap@12() Unknown Non-user code. Symbols loaded.
   ole32.dll!CRetailMalloc_Free(IMalloc * pThis=0x777476bc, void * pv=0x1842de40) Line 687  C++ Non-user code. Symbols loaded.
   ole32.dll!CoTaskMemFree(void * pv=0x1842de40) Line 475   C++ Non-user code. Symbols loaded.
=> <Process>.exe!CParameter::SetValue(..., <internal_Class> * pBlock=0x00000000, ...) Line 5528 C++ Symbols loaded.

Within the watch-window, I see the following value for &m_Data:

0x77e4f9ae {Inside ntdll.dll!_RtlpAnalyzeHeapFailure@12()} {pbData=0xd2b70f3d <Error reading characters of string.> ...}

More information:

  • The value of pBlock is NULL
  • The value of pStore is unknown by the debugger

Questions:

  • Where is the function "Clear()" in the call stack?
  • What about the value "RtlpAnalyzeHeapFailure" in the watch-window? Does it mean that my dump is that much corrupted that I can't get any useful information out of it?

Thanks in advance

c++
windows
visual-studio
crash
dump
asked on Stack Overflow Oct 24, 2017 by Dominique

1 Answer

1

Where is the function "Clear()" in the call stack?

This looks like it was inlined.

The value of pStore is unknown by the debugger

Isn't it still from m_Data?

If you look at the code near the crash point, you can see which register it has been in recently, and if it is preserved, then you should be able to see it saved at some point.

The error is due to the memory system detecting that the memory is being incorrectly freed. This may be that m_Data held a floating value, or had already been deleted.

I have produced a similar function.

__declspec(noinline) void Type::Clear()
{
    delete pbData;
    pbData = nullptr;
}

__declspec(noinline) void SetValue(Type * pBlock)
{
    Type * pStore = nullptr;
    if (!pBlock)
    {
        pStore = &m_Data; // m_Data is a global variable
    }
    else pStore = pBlock;

    if (pStore->pbData)
    {
        pStore->Clear(); // here we have the crash
    }
}

Its disassembly (from windbg) is :-

0:000:x86> uf debugging2!SetValue
debugging2!Type::Clear [c:\source\example\debugging2\debugging2.cpp @ 17]:
   17 01041020 56              push    esi
   17 01041021 8bf1            mov     esi,ecx
   18 01041023 6a00            push    0
   18 01041025 ff36            push    dword ptr [esi]
   18 01041027 e85c000000      call    debugging2!operator delete (01041088)
   18 0104102c 83c408          add     esp,8
   19 0104102f c70600000000    mov     dword ptr [esi],0
   19 01041035 5e              pop     esi
   20 01041036 c3              ret

debugging2!SetValue [c:\source\example\debugging2\debugging2.cpp @ 23]:
   23 01041040 833df833040100  cmp     dword ptr [debugging2!m_Data (010433f8)],0
   31 01041047 740a            je      debugging2!SetValue+0x13 (01041053)  
Branch

debugging2!SetValue+0x9 [c:\source\example\debugging2\debugging2.cpp @ 33]:
   33 01041049 b9f8330401      mov     ecx,offset debugging2!m_Data (010433f8)
   33 0104104e e9cdffffff      jmp     debugging2!Type::Clear (01041020)  Branch

debugging2!SetValue+0x13 [c:\source\example\debugging2\debugging2.cpp @ 35]:
   35 01041053 c3              ret  Branch

This shows (in my case) the call to Clear() has been replaced by a jump, hiding it from the stack with an optimization. This also shows that at address 01041049 ecx is loaded with the value for the call. MSDN : x86 calling conventions Ecx is not a preserved value, so we would not be able to find what value it held (other than it seems to be &m_Data from your comments).

But we can look at the functions up the stack...

In the ::Clear function ecx is moved to esi. So esi (which is preserved) now has the same value.

Looking at the next function (operator delete),

debugging2!operator delete 
 [f:\dd\vctools\crt\vcstartup\src\heap\delete_scalar_size.cpp @ 14]:
   14 01041088 55              push    ebp
   14 01041089 8bec            mov     ebp,esp
   15 0104108b ff7508          push    dword ptr [ebp+8]
   15 0104108e e890030000      call    debugging2!operator delete (01041423)
   15 01041093 59              pop     ecx
   16 01041094 5d              pop     ebp
   16 01041095 c3              ret

We see the esi is not changed, nor saved. So we look at the next function on the stack....

0:000:x86> uf 01041423
Flow analysis was incomplete, some code may be missing
debugging2!operator delete 
[f:\dd\vctools\crt\vcstartup\src\heap\delete_scalar.cpp @ 15]:
   15 01041423 e982090000      jmp     debugging2!free (01041daa)  Branch

In each case we are looking for esi to be stored somewhere on the stack, so we can find it...

answered on Stack Overflow Oct 24, 2017 by mksteve • edited Jan 19, 2021 by mksteve

User contributions licensed under CC BY-SA 3.0