Debugging heap corruption in windbg (glEnableVertexAttribArray gives access violation)

0
void TE::Render::APIVertexBufferLayout::Enable()
{
    if (m_hasPosition)
    {
        glEnableVertexAttribArray(0);
        glVertexAttribPointer(0, m_positionValueCount, m_positionValueType, GL_F    ALSE, m_stride, (void*)m_positionOffset);
    }
    if (m_hasNormals)
    {
        glEnableVertexAttribArray(1);
        glVertexAttribPointer(1, m_normalValueCount, m_normalValueType, GL_FALSE, 0, (    void*)m_normalOffset);
    }
    for(unsigned i = 0; i < m_textureCount; ++i)
    {
        glEnableVertexAttribArray(2+i);
        glVertexAttribPointer(2+i, m_textureValueCount[i], m_textureValueType[i], GL_FALSE, m_stride, (void*)m_textureOffset[i]);
    }
}

I was getting a access violation on the call glEnableVertexAttribArray(0) and after talking to someone in the #opengl irc channel it seemed like my heap was corrupted.

So I enable full pagehead with application verifier and launched my application through the debuger (visual studio 2012):

Microsoft (R) Windows Debugger Version 6.2.9200.16384 X86
Copyright (c) Microsoft Corporation. All rights reserved.

T-PC\T (npipe WinIDE_01CD7D9B3FF423A9) connected at Sun Aug 19 01:43:21 2012

Microsoft (R) Windows Debugger Version 6.2.9200.16384 X86
Copyright (c) Microsoft Corporation. All rights reserved.

CommandLine: "D:\coding\tengine\Executables\DebugOpenGL\TEEngineTest.exe" 
Symbol search path is: srv*
Executable search path is: 
ModLoad: 01000000 011f3000   TEEngineTest.exe
ModLoad: 77560000 776e0000   ntdll.dll
Page heap: pid 0x1534: page heap enabled with flags 0x3.
AVRF: TEEngineTest.exe: pid 0x1534: flags 0x80000001: application verifier enabled
ModLoad: 535c0000 53620000   C:\Windows\syswow64\verifier.dll
Page heap: pid 0x1534: page heap enabled with flags 0x3.
AVRF: TEEngineTest.exe: pid 0x1534: flags 0x80000001: application verifier enabled
ModLoad: 53b70000 53b9f000   C:\Windows\SysWOW64\vrfcore.dll
ModLoad: 75f90000 760a0000   C:\Windows\syswow64\kernel32.dll
ModLoad: 74eb0000 74ef6000   C:\Windows\syswow64\KERNELBASE.dll
ModLoad: 777a0000 78000000   d:\Progs\NVIDIA Corporation\Cg\bin\cg.dll
ModLoad: 58900000 5895b000   d:\Progs\NVIDIA Corporation\Cg\bin\cgGL.dll
ModLoad: 10000000 1046d000   D:\coding\tengine\Executables\DebugOpenGL\Assimp32.dll
ModLoad: 57a90000 57b1e000   C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.6161_none_50934f2ebcb7eb57\MSVCP90.dll
ModLoad: 74540000 745e3000   C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.6161_none_50934f2ebcb7eb57\MSVCR90.dll
ModLoad: 588a0000 588f2000   D:\coding\tengine\Executables\DebugOpenGL\glew32.dll
ModLoad: 6b4a0000 6b568000   C:\Windows\SysWOW64\OPENGL32.dll
ModLoad: 76560000 7660c000   C:\Windows\syswow64\msvcrt.dll
ModLoad: 76a40000 76ae0000   C:\Windows\syswow64\ADVAPI32.dll
ModLoad: 74e90000 74ea9000   C:\Windows\SysWOW64\sechost.dll
ModLoad: 74d60000 74e50000   C:\Windows\syswow64\RPCRT4.dll
ModLoad: 74c40000 74ca0000   C:\Windows\syswow64\SspiCli.dll
ModLoad: 74c30000 74c3c000   C:\Windows\syswow64\CRYPTBASE.dll
ModLoad: 74cd0000 74d60000   C:\Windows\syswow64\GDI32.dll
ModLoad: 763f0000 764f0000   C:\Windows\syswow64\USER32.dll
ModLoad: 760a0000 760aa000   C:\Windows\syswow64\LPK.dll
ModLoad: 760b0000 7614d000   C:\Windows\syswow64\USP10.dll
ModLoad: 6b470000 6b492000   C:\Windows\SysWOW64\GLU32.dll
ModLoad: 6b380000 6b467000   C:\Windows\SysWOW64\DDRAW.dll
ModLoad: 6b1b0000 6b1b6000   C:\Windows\SysWOW64\DCIMAN32.dll
ModLoad: 74fa0000 7513d000   C:\Windows\syswow64\SETUPAPI.dll
ModLoad: 74e50000 74e77000   C:\Windows\syswow64\CFGMGR32.dll
ModLoad: 766d0000 7675f000   C:\Windows\syswow64\OLEAUT32.dll
ModLoad: 761c0000 7631c000   C:\Windows\syswow64\ole32.dll
ModLoad: 764f0000 76502000   C:\Windows\syswow64\DEVOBJ.dll
ModLoad: 71980000 71993000   C:\Windows\SysWOW64\dwmapi.dll
ModLoad: 538c0000 5398b000   C:\Windows\SysWOW64\MSVCP110D.dll
ModLoad: 53720000 538bb000   C:\Windows\SysWOW64\MSVCR110D.dll
ModLoad: 76760000 76795000   C:\Windows\syswow64\WS2_32.dll
ModLoad: 77530000 77536000   C:\Windows\syswow64\NSI.dll
ModLoad: 58bc0000 58bee000   D:\coding\tengine\Executables\DebugOpenGL\libpng15.dll
ModLoad: 55510000 555cf000   C:\Windows\SysWOW64\MSVCR100.dll
(1534.1760): Break instruction exception - code 80000003 (first chance)
eax=00000000 ebx=00000000 ecx=1cf80000 edx=0009dc28 esi=fffffffe edi=00000000
eip=77600fab esp=00b3fae4 ebp=00b3fb10 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
ntdll!LdrpDoDebuggerBreak+0x2c:
77600fab cc              int     3
0:000> .symfix
0:000> .reload
Reloading current modules
......................................
0:000> g
ModLoad: 76150000 761b0000   C:\Windows\SysWOW64\IMM32.DLL
ModLoad: 76320000 763ec000   C:\Windows\syswow64\MSCTF.dll
ModLoad: 719a0000 71a20000   C:\Windows\SysWOW64\uxtheme.dll
ModLoad: 0bff0000 0c14c000   C:\Windows\SysWOW64\ole32.dll
ModLoad: 68f20000 6a1f9000   C:\Windows\SysWOW64\nvoglv32.DLL
ModLoad: 75340000 75f8a000   C:\Windows\syswow64\SHELL32.dll
ModLoad: 751c0000 75217000   C:\Windows\syswow64\SHLWAPI.dll
ModLoad: 729e0000 729e9000   C:\Windows\SysWOW64\VERSION.dll
ModLoad: 73d70000 73d91000   C:\Windows\SysWOW64\ntmarta.dll
ModLoad: 76510000 76555000   C:\Windows\syswow64\WLDAP32.dll
ModLoad: 72200000 72225000   C:\Windows\SysWOW64\POWRPROF.DLL
(1534.1234): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=00000000 ebx=092c9c40 ecx=00000001 edx=588e6d3c esi=0ae0f3d8 edi=0ae0f478
eip=69527d36 esp=0ae0f3d0 ebp=0ae0f3e4 iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
nvoglv32+0x607d36:
69527d36 ffa048080000    jmp     dword ptr [eax+848h] ds:002b:00000848=????????
0:002> k
ChildEBP RetAddr  
WARNING: Stack unwind information not available. Following frames may be wrong.
0ae0f3e4 01095f1c nvoglv32+0x607d36
0ae0f484 0108cfff TEEngineTest!TE::Render::Renderer::Enable+0x2cc [d:\coding\tengine\code\tegraphics\renderer\terenderer.cpp @ 155]
0ae0f4d0 0108ce6b TEEngineTest!TE::SceneGraph::SceneManager::DrawRenderable+0x10f [d:\coding\tengine\code\tegraphics\scenegraph\tescenemanager.cpp @ 50]
0ae0f538 010cda3b TEEngineTest!TE::SceneGraph::SceneManager::DrawScene+0xdb [d:\coding\tengine\code\tegraphics\scenegraph\tescenemanager.cpp @ 29]
0ae0f544 0104431d TEEngineTest!TE::Graphics::GraphicsTask::Update+0x2b [d:\coding\tengine\code\tegraphics\engine\tegraphicstask.cpp @ 21]
0ae0f554 01047195 TEEngineTest!TE::Engine::ThreadedTask::Execute+0x1d [d:\coding\tengine\code\tengine\engine\tetaskmanager.cpp @ 18]
0ae0f564 0104e9be TEEngineTest!srutil::delegate0<void>::method_stub<TE::Engine::ThreadedTask,&TE::Engine::ThreadedTask::Execute>+0x25 [d:\coding\tengine\externals\srdelegates\include\srutil\delegate\detail\delegate_template.hpp @ 109]
0ae0f574 010459eb TEEngineTest!srutil::delegate0<void>::operator()+0x1e [d:\coding\tengine\externals\srdelegates\include\srutil\delegate\detail\delegate_template.hpp @ 70]
0ae0f57c 0104710f TEEngineTest!boost::asio::asio_handler_invoke<srutil::delegate0<void> >+0xb [d:\coding\tengine\externals\boost_1_50_0\boost\asio\handler_invoke_hook.hpp @ 65]
0ae0f590 01052597 TEEngineTest!boost_asio_handler_invoke_helpers::invoke<srutil::delegate0<void>,srutil::delegate0<void> >+0x1f [d:\coding\tengine\externals\boost_1_50_0\boost\asio\detail\handler_invoke_helpers.hpp @ 39]
0ae0f5e8 01051e69 TEEngineTest!boost::asio::detail::completion_handler<srutil::delegate0<void> >::do_complete+0xa7 [d:\coding\tengine\externals\boost_1_50_0\boost\asio\detail\completion_handler.hpp @ 67]
0ae0f608 010528bf TEEngineTest!boost::asio::detail::task_io_service_operation::complete+0x29 [d:\coding\tengine\externals\boost_1_50_0\boost\asio\detail\task_io_service_operation.hpp @ 37]
0ae0f674 01056d2b TEEngineTest!boost::asio::detail::task_io_service::do_run_one+0x19f [d:\coding\tengine\externals\boost_1_50_0\boost\asio\detail\impl\task_io_service.ipp @ 396]
0ae0f708 01056737 TEEngineTest!boost::asio::detail::task_io_service::run+0xeb [d:\coding\tengine\externals\boost_1_50_0\boost\asio\detail\impl\task_io_service.ipp @ 146]
0ae0f72c 010471d5 TEEngineTest!boost::asio::io_service::run+0x37 [d:\coding\tengine\externals\boost_1_50_0\boost\asio\impl\io_service.ipp @ 59]
0ae0f73c 0104e97e TEEngineTest!srutil::delegate0<unsigned int>::method_stub<boost::asio::io_service,&boost::asio::io_service::run>+0x25 [d:\coding\tengine\externals\srdelegates\include\srutil\delegate\detail\delegate_template.hpp @ 109]
0ae0f74c 010566e9 TEEngineTest!srutil::delegate0<unsigned int>::operator()+0x1e [d:\coding\tengine\externals\srdelegates\include\srutil\delegate\detail\delegate_template.hpp @ 70]
0ae0f758 0110dad0 TEEngineTest!boost::detail::thread_data<srutil::delegate0<unsigned int> >::run+0x19 [d:\coding\tengine\externals\boost_1_50_0\boost\thread\detail\thread.hpp @ 75]
0ae0f784 5377b811 TEEngineTest!boost::`anonymous namespace'::thread_start_function+0x50 [d:\coding\tengine\externals\boost_1_50_0\libs\thread\src\win32\thread.cpp @ 191]
0ae0f7c0 5377b9e1 MSVCR110D!beginthreadex+0x1a1
0ae0f7cc 75fa339a MSVCR110D!endthreadex+0x171
0ae0f7d8 77599ef2 kernel32!BaseThreadInitThunk+0xe
0ae0f818 77599ec5 ntdll!__RtlUserThreadStart+0x70
0ae0f830 00000000 ntdll!_RtlUserThreadStart+0x1b

When I launched the application visual studio shows me a break point on the line m_apiVertexBuffers[&mesh]->Enable() in the below function.. but that is actualy after where I got the breakpoint earlier.. m_apiVertexBufferLayouts[&mesh]->Enable() is the call to the function in the paste above so it breaks in there..

void TE::Render::Renderer::Enable( const Mesh& mesh )
{
    assert(m_apiVertexBufferLayouts.find(&mesh) != m_apiVertexBufferLayouts.end() && "VertexBufferLayout not loaded");
    assert(m_apiVertexBuffers.find(&mesh) != m_apiVertexBuffers.end() && "VertexBuffer not loaded");
    assert(m_apiIndexBuffers.find(&mesh) != m_apiIndexBuffers.end() && "IndexBuffer not loaded");

    m_apiVertexBufferLayouts[&mesh]->Enable();
    m_apiVertexBuffers[&mesh]->Enable();
    m_apiIndexBuffers[&mesh]->Enable();
}

How can I debug this further? This is my first time in a debugger so I'm abit confused..

c++
opengl
heap
windbg
asked on Stack Overflow Aug 18, 2012 by bitgregor

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0