WinDbg does not show full stack trace for some minidump-files

Question

WinDbg does not show full stack trace for some minidump-files

I'm trying to set up a crashdump system to better be able to debug errors that I can't replicate easily on my own system

Here's my test program (Compiled with Release x64 Configuration with Visual Studio 2017):

#include "pch.h"
#include <iostream>
#include <Windows.h>
#include <DbgHelp.h>

#pragma comment(lib,"Dbghelp.lib")

BOOL MiniDumpWriteDump( 
  HANDLE hProcess, 
  DWORD ProcessId, 
  HANDLE hFile, 
  MINIDUMP_TYPE DumpType, 
  PMINIDUMP_EXCEPTION_INFORMATION ExceptionParam, 
  PMINIDUMP_USER_STREAM_INFORMATION UserStreamParam, 
  PMINIDUMP_CALLBACK_INFORMATION CallbackParam
);

LONG CreateMiniDump( struct _EXCEPTION_POINTERS *pExceptionInfo )//EXCEPTION_POINTERS* pep ) 
{
  // Open the file 

  HANDLE hFile = CreateFile( "MiniDump.dmp", GENERIC_READ | GENERIC_WRITE, 
    0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL ); 

  if( ( hFile != NULL ) && ( hFile != INVALID_HANDLE_VALUE ) ) 
  {
    // Create the minidump 

    MINIDUMP_EXCEPTION_INFORMATION mdei; 

    mdei.ThreadId           = GetCurrentThreadId(); 
    mdei.ExceptionPointers  = pExceptionInfo; 
    mdei.ClientPointers     = FALSE; 

    MINIDUMP_TYPE mdt       = MiniDumpNormal; 

    BOOL rv = MiniDumpWriteDump( GetCurrentProcess(), GetCurrentProcessId(), 
      hFile, mdt, (pExceptionInfo != 0) ? &mdei : 0, 0, 0 ); 

    if( !rv ) 
        std::cout<<"MiniDumpWriteDump failed. Error: "<<GetLastError()<<std::endl;
    else 
      std::cout<<"Minidump created."<<std::endl;

    // Close the file 

    CloseHandle( hFile ); 

  }
  else 
  {
      std::cout<<"CreateFile failed. Error: "<<GetLastError()<<std::endl;
  }
  return EXCEPTION_EXECUTE_HANDLER;

}

static void ftest3()
{
    throw std::runtime_error("Error");
}

static void ftest2()
{
    ftest3();
}

static void ftest1()
{
    ftest2();
}

int main()
{
    SetUnhandledExceptionFilter(&CreateMiniDump);
    ftest1(); 
}

(Source: http://www.debuginfo.com/articles/effminidumps2.html)

If I run it on my primary system (on which I compiled the program), and then load the .dmp-file in Visual Studio or WinDbg, the full stack trace is displayed correctly and I can see where the error has occurred:

*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP          RetAddr           Call Site
00 000000fd`3bbbf6f0 00007ff8`a72142cd KERNELBASE!RaiseException+0x68
01 000000fd`3bbbf7d0 00007ff7`7b4f130f VCRUNTIME140!_CxxThrowException+0xad [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp @ 133] 
02 000000fd`3bbbf840 00007ff7`7b4f1319 test_minidump!ftest3+0x1f [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp @ 63] 
03 000000fd`3bbbf890 00007ff7`7b4f1329 test_minidump!ftest2+0x9 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp @ 68] 
04 000000fd`3bbbf8c0 00007ff7`7b4f1346 test_minidump!ftest1+0x9 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp @ 73] 
05 000000fd`3bbbf8f0 00007ff7`7b4f1838 test_minidump!main+0x16 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp @ 79] 
06 (Inline Function) --------`-------- test_minidump!invoke_main+0x22 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78] 
07 000000fd`3bbbf920 00007ff8`b45d3034 test_minidump!__scrt_common_main_seh+0x10c [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 
08 000000fd`3bbbf960 00007ff8`b71a1461 kernel32!BaseThreadInitThunk+0x14
09 000000fd`3bbbf990 00000000`00000000 ntdll!RtlUserThreadStart+0x21

However, if I run the program on a different system, and then try to analyze the dump on my own system, all I get is this:

*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP          RetAddr           Call Site
00 00000021`ccbef870 00007ffe`8cfd4722 KERNELBASE!RaiseException+0x68
01 00000021`ccbef950 00000021`ccbef9f0 VCRUNTIME140!_CxxThrowException+0xc2 [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp @ 136] 
02 00000021`ccbef958 00007ffe`00000000 0x00000021`ccbef9f0
03 00000021`ccbef960 000001ee`2e25ff30 0x00007ffe`00000000
04 00000021`ccbef968 00000000`00000000 0x000001ee`2e25ff30

Which isn't exactly helpful. I've loaded the dump-file in WinDbg and then used the following commands:

.symfix
.sympath+ C:\Users\Florian\Documents\Projects\test_minidump\x64\Release
.srcpath C:\Users\Florian\Documents\Projects\test_minidump\test_minidump
!analyze -v
.ecxr
k

Here's the full output from WinDbg:

Microsoft (R) Windows Debugger Version 10.0.17134.12 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\Users\Florian\Desktop\tmp\MiniDump.dmp]
User Mini Dump File: Only registers, stack and portions of memory are available

Symbol search path is: srv*
Executable search path is: 
Windows 10 Version 17134 MP (4 procs) Free x64
Product: WinNt, suite: SingleUserTS
17134.1.amd64fre.rs4_release.180410-1804
Machine Name:
Debug session time: Sun Sep 23 11:56:45.000 2018 (UTC + 2:00)
System Uptime: not available
Process Uptime: not available
..............
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(169c.1b78): C++ EH exception - code e06d7363 (first/second chance not available)
ntdll!NtGetContextThread+0x14:
00007ffe`9abfbc44 c3              ret
0:000> .symfix
0:000> .sympath+ C:\Users\Florian\Documents\Projects\test_minidump\x64\Release
Symbol search path is: srv*;C:\Users\Florian\Documents\Projects\test_minidump\x64\Release
Expanded Symbol search path is: cache*;SRV*https://msdl.microsoft.com/download/symbols;c:\users\florian\documents\projects\test_minidump\x64\release

************* Path validation summary **************
Response                         Time (ms)     Location
Deferred                                       srv*
OK                                             C:\Users\Florian\Documents\Projects\test_minidump\x64\Release
0:000> .srcpath C:\Users\Florian\Documents\Projects\test_minidump\test_minidump
Source search path is: C:\Users\Florian\Documents\Projects\test_minidump\test_minidump

************* Path validation summary **************
Response                         Time (ms)     Location
OK                                             C:\Users\Florian\Documents\Projects\test_minidump\test_minidump
0:000> !analyze -v
*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************

KEY_VALUES_STRING: 1


TIMELINE_ANALYSIS: 1

Timeline: !analyze.Start
    Name: <blank>
    Time: 2018-09-24T10:08:32.944Z
    Diff: 87107944 mSec

Timeline: Dump.Current
    Name: <blank>
    Time: 2018-09-23T09:56:45.0Z
    Diff: 0 mSec


DUMP_CLASS: 2

DUMP_QUALIFIER: 400

CONTEXT:  (.ecxr)
rax=0000000000000000 rbx=00007ffe973da570 rcx=0000000000000000
rdx=0000000000000030 rsi=00007ff798203cd0 rdi=000001ee2e25c750
rip=00007ffe97bba388 rsp=00000021ccbef870 rbp=00000021ccbef9b0
 r8=0000000000000000  r9=0000000000000000 r10=000001ee2e0f0000
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=00000021ccbef9f0 r15=0000000000000000
iopl=0         nv up ei pl nz na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206
KERNELBASE!RaiseException+0x68:
00007ffe`97bba388 488b8c24c0000000 mov     rcx,qword ptr [rsp+0C0h] ss:00000021`ccbef930=000052660787015b
Resetting default scope

FAULTING_IP: 
KERNELBASE!RaiseException+68
00007ffe`97bba388 488b8c24c0000000 mov     rcx,qword ptr [rsp+0C0h]

EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 00007ffe97bba388 (KERNELBASE!RaiseException+0x0000000000000068)
   ExceptionCode: e06d7363 (C++ EH exception)
  ExceptionFlags: 00000001
NumberParameters: 4
   Parameter[0]: 0000000019930520
   Parameter[1]: 00000021ccbef9f0
   Parameter[2]: 00007ff798203cd0
   Parameter[3]: 00007ff798200000

BUGCHECK_STR:  CPP_EXCEPTION

DEFAULT_BUCKET_ID:  CPP_EXCEPTION

PROCESS_NAME:  test_minidump.exe

ERROR_CODE: (NTSTATUS) 0xe06d7363 - <Unable to get error code text>

EXCEPTION_CODE: (NTSTATUS) 0xe06d7363 - <Unable to get error code text>

EXCEPTION_CODE_STR:  e06d7363

EXCEPTION_PARAMETER1:  0000000019930520

EXCEPTION_PARAMETER2:  00000021ccbef9f0

EXCEPTION_PARAMETER3:  00007ff798203cd0

EXCEPTION_PARAMETER4: 7ff798200000

WATSON_BKT_PROCSTAMP:  5ba8b478

WATSON_BKT_MODULE:  KERNELBASE.dll

WATSON_BKT_MODSTAMP:  b0bb231d

WATSON_BKT_MODOFFSET:  3a388

WATSON_BKT_MODVER:  6.2.17134.165

MODULE_VER_PRODUCT:  Microsoft® Windows® Operating System

BUILD_VERSION_STRING:  17134.1.amd64fre.rs4_release.180410-1804

MODLIST_WITH_TSCHKSUM_HASH:  636d437315fdc5447e163e64094b1f6a26123a83

MODLIST_SHA1_HASH:  74dec0c40d229cd1e109927f1f5e794fa53165ac

DUMP_FLAGS:  0

DUMP_TYPE:  2

ANALYSIS_SESSION_HOST:  DESKTOP-VFOALG8

ANALYSIS_SESSION_TIME:  09-24-2018 12:08:32.0944

ANALYSIS_VERSION: 10.0.17134.12 amd64fre

THREAD_ATTRIBUTES: 
PROBLEM_CLASSES: 

    ID:     [0n317]
    Type:   [@APPLICATION_FAULT_STRING]
    Class:  Primary
    Scope:  DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
            BUCKET_ID
    Name:   Omit
    Data:   Add
            String: [CPP_EXCEPTION]
    PID:    [Unspecified]
    TID:    [Unspecified]
    Frame:  [0]

PRIMARY_PROBLEM_CLASS:  CPP_EXCEPTION

LAST_CONTROL_TRANSFER:  from 00007ffe8cfd4722 to 00007ffe97bba388

STACK_TEXT:  
00000021`ccbef870 00007ffe`8cfd4722 : 00000021`ccbef9f0 00007ffe`00000000 000001ee`2e25ff30 00000000`00000000 : KERNELBASE!RaiseException+0x68
00000021`ccbef950 00000021`ccbef9f0 : 00007ffe`00000000 000001ee`2e25ff30 00000000`00000000 00000001`e06d7363 : VCRUNTIME140!_CxxThrowException+0xc2
00000021`ccbef958 00007ffe`00000000 : 000001ee`2e25ff30 00000000`00000000 00000001`e06d7363 00000000`00000000 : 0x00000021`ccbef9f0
00000021`ccbef960 000001ee`2e25ff30 : 00000000`00000000 00000001`e06d7363 00000000`00000000 00000000`00000000 : 0x00007ffe`00000000
00000021`ccbef968 00000000`00000000 : 00000001`e06d7363 00000000`00000000 00000000`00000000 00000000`00000004 : 0x000001ee`2e25ff30


THREAD_SHA1_HASH_MOD_FUNC:  74c55fd19222dcb5ad415427110811b213f4c078

THREAD_SHA1_HASH_MOD_FUNC_OFFSET:  7592380e0f749a34b2d654a7c3b829a033ee79fa

THREAD_SHA1_HASH_MOD:  ad69e1f2190edbb9a7df71b4b6403437528645f2

FOLLOWUP_IP: 
KERNELBASE!RaiseException+68
00007ffe`97bba388 488b8c24c0000000 mov     rcx,qword ptr [rsp+0C0h]

FAULT_INSTR_CODE:  248c8b48

SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  KERNELBASE!RaiseException+68

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: KERNELBASE

IMAGE_NAME:  KERNELBASE.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  0

STACK_COMMAND:  ~0s ; .ecxr ; kb

BUCKET_ID:  CPP_EXCEPTION_KERNELBASE!RaiseException+68

FAILURE_EXCEPTION_CODE:  e06d7363

FAILURE_IMAGE_NAME:  KERNELBASE.dll

BUCKET_ID_IMAGE_STR:  KERNELBASE.dll

FAILURE_MODULE_NAME:  KERNELBASE

BUCKET_ID_MODULE_STR:  KERNELBASE

FAILURE_FUNCTION_NAME:  RaiseException

BUCKET_ID_FUNCTION_STR:  RaiseException

BUCKET_ID_OFFSET:  68

BUCKET_ID_MODTIMEDATESTAMP:  0

BUCKET_ID_MODCHECKSUM:  27e598

BUCKET_ID_MODVER_STR:  6.2.17134.165

BUCKET_ID_PREFIX_STR:  CPP_EXCEPTION_

FAILURE_PROBLEM_CLASS:  CPP_EXCEPTION

FAILURE_SYMBOL_NAME:  KERNELBASE.dll!RaiseException

FAILURE_BUCKET_ID:  CPP_EXCEPTION_e06d7363_KERNELBASE.dll!RaiseException

TARGET_TIME:  2018-09-23T09:56:45.000Z

OSBUILD:  17134

OSSERVICEPACK:  1

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK:  256

PRODUCT_TYPE:  1

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

OSEDITION:  Windows 10 WinNt SingleUserTS

OS_LOCALE:  

USER_LCID:  0

OSBUILD_TIMESTAMP:  2020-08-28 06:38:41

BUILDDATESTAMP_STR:  180410-1804

BUILDLAB_STR:  rs4_release

BUILDOSVER_STR:  10.0.17134.1.amd64fre.rs4_release.180410-1804

ANALYSIS_SESSION_ELAPSED_TIME:  a63

ANALYSIS_SOURCE:  UM

FAILURE_ID_HASH_STRING:  um:cpp_exception_e06d7363_kernelbase.dll!raiseexception

FAILURE_ID_HASH:  {1253aecc-520d-655b-58e3-5eb61e209188}

Followup:     MachineOwner
---------

0:000> .ecxr
rax=0000000000000000 rbx=00007ffe973da570 rcx=0000000000000000
rdx=0000000000000030 rsi=00007ff798203cd0 rdi=000001ee2e25c750
rip=00007ffe97bba388 rsp=00000021ccbef870 rbp=00000021ccbef9b0
 r8=0000000000000000  r9=0000000000000000 r10=000001ee2e0f0000
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=00000021ccbef9f0 r15=0000000000000000
iopl=0         nv up ei pl nz na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206
KERNELBASE!RaiseException+0x68:
00007ffe`97bba388 488b8c24c0000000 mov     rcx,qword ptr [rsp+0C0h] ss:00000021`ccbef930=000052660787015b
0:000> k
  *** Stack trace for last set context - .thread/.cxr resets it
 # Child-SP          RetAddr           Call Site
00 00000021`ccbef870 00007ffe`8cfd4722 KERNELBASE!RaiseException+0x68
01 00000021`ccbef950 00000021`ccbef9f0 VCRUNTIME140!_CxxThrowException+0xc2 [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp @ 136] 
02 00000021`ccbef958 00007ffe`00000000 0x00000021`ccbef9f0
03 00000021`ccbef960 000001ee`2e25ff30 0x00007ffe`00000000
04 00000021`ccbef968 00000000`00000000 0x000001ee`2e25ff30

There are no warnings or errors from what I can tell, so why am I not getting the full stack trace information?

c++

windows-10

64-bit

windbg

minidump

asked on Stack Overflow Sep 23, 2018 by

Silverlan • edited Sep 24, 2018 by

Silverlan

2 Answers

VCRUNTIME140!_CxxThrowException+0xad [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp @ 133]

versus

VCRUNTIME140!_CxxThrowException+0xc2 [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp @ 136]

is a bit worrying. It suggests that the minor version of the MSVC runtime doesn't match between the two machines (because otherwise, why would they invoke RaiseException from different source lines). If your WinDbg session doesn't have symbols for the remote machine's VCRUNTIME140 -- or, worse yet, if it's using the wrong PDB for it -- backtracing is going to become problematic at that point. WinDbg should automatically download and load symbols for whichever version is referenced from the minidump, but I suspect this isn't happening properly.

Use lm v to show the module listing for the minidump, and pay close attention to the checksum information. I suspect you'll see a different checksum for vcruntime140.dll between the two machines. If you do, you'll need to figure out why you're not getting the right symbols for the dll.

answered on Stack Overflow Sep 23, 2018 by

Sneftel • edited Sep 23, 2018 by

Sneftel

Thanks to the tips by users Sneftel and Sean Cline, I found the solution. Running the command !itoldyouso vcruntime140 gives me the following output:

sig MISMATCH: vcruntime140.amd64.pdb and VCRUNTIME140.dll

The problem was that my system which I use for debugging runs on an AMD CPU, while the system that generated the crashdump runs on an intel CPU. For some reason the vcruntime140.dll is system-specific and differs between the two. (Also, on AMD systems the pdb is called "vcruntime140.amd64.pdb", on intel it's "vcruntime140.i386.pdb".)

I've copied the vcruntime140.dll, vcruntime140.i386.pdb and msvcp140.dll from my intel system and added them to the symbol path for WinDbg. Then I ran the same commands as before, and now it outputs the full stacktrace.

*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP          RetAddr           Call Site
00 00000021`ccbef870 00007ffe`8cfd4722 KERNELBASE!RaiseException+0x68
01 00000021`ccbef950 00007ff7`9820130f VCRUNTIME140!_CxxThrowException+0xc2 [f:\dd\vctools\crt\vcruntime\src\eh\throw.cpp @ 136] 
02 00000021`ccbef9d0 00007ff7`98201319 test_minidump!ftest3+0x1f [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp @ 63] 
03 00000021`ccbefa20 00007ff7`98201329 test_minidump!ftest2+0x9 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp @ 68] 
04 00000021`ccbefa50 00007ff7`98201346 test_minidump!ftest1+0x9 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp @ 73] 
05 00000021`ccbefa80 00007ff7`98201838 test_minidump!main+0x16 [c:\users\florian\documents\projects\test_minidump\test_minidump\test_minidump.cpp @ 79] 
06 (Inline Function) --------`-------- test_minidump!invoke_main+0x22 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 78] 
07 00000021`ccbefab0 00007ffe`98983034 test_minidump!__scrt_common_main_seh+0x10c [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 
08 00000021`ccbefaf0 00007ffe`9abd1461 kernel32!BaseThreadInitThunk+0x14
09 00000021`ccbefb20 00000000`00000000 ntdll!RtlUserThreadStart+0x21

answered on Stack Overflow Sep 24, 2018 by

Silverlan

User contributions licensed under CC BY-SA 3.0