IIS Application Pool randomly crash cause by stack overflow

0

We have a .net web api application run on IIS 7.5 and Windows 2008 R2 SP1. Since this month our site randomly crash and automatic restart, and it always consecutive crash three times. There is no CLR Exception caughted by Application_Error in Global.ascs, only a Application Error with exception code 0xc00000fd in windows event viewer.

We tried to use DebugDiag1.2 to log the call stack, There is a obvious infinite loops but we don't know what reason. Here is the log content:

 WARNING - DebugDiag was not able to locate debug symbols for clr.dll, so the information below may be incomplete.

 w3wp__WeiXinService__PID__3688__Date__10_20_2016__Time_10_47_28AM__341__Second_Chance_Exception_C00000FD.dmp the assembly instruction at clr!CreateHistoryReader+1694f8 in C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll has caused a stack overflow exception (0xC00000FD) when trying to write to memory location 0x10992f10 on thread 32

Here is .NET Call Stack


    Function     Arg 1     Arg 2     Arg 3     Arg 4   Source 
    clr!CreateHistoryReader+1694f8     b4231457     10993fa0     00000000     00000000    
    clr!CreateHistoryReader+141f2     b42314e3     007c8ef8     77224744     007c8f00    
    clr!GetMetaDataInternalInterfaceFromPublic+abc7     10993fa0     00000001     10994058     10994058    
    ntdll!RtlGetProcessHeaps+49d     007c8ef8     109940a8     00000000     10994040    
    ntdll!RtlCaptureContext+187     10994058     109940a8     109945c8     00000001    
    ntdll!RtlMoveMemory+416     10994058     109940a8     10994058     109940a8    
    ntdll!KiUserExceptionDispatcher+f     e0434352     00000001     00000005     109945c8    
    clr!CopyPDBs+15478     00000000     b4236dbb     028de0c0     00000011    
    clr!CopyPDBs+160b6     028dcae8     028dc9c8     6f1d5643     01eed478    
    System_Xml_ni+42422e     028ddff0     00000471     028dc9c8     1099473c    
    System_Xml_ni+42587f     00000471     000004cb     028dcc78     0000005d    
    System_Xml_ni+63ee52     10994758     1099475c     0000045a     00000035    
    System_Xml_ni+d78cf     028dc9c8     028dc86c     109947b4     6ee835bc    
    System_Xml_ni+d55b7     028da224     028ddfd4     028cdabc     109947ac    
    System_Xml_ni+d35bc     109947e0     6ee844d6     00000000     028cdabc    
    System_Xml_ni+10123a     028dc998     028dc86c     00000000     00000000    
    System_Xml_ni+e2eb4     00000001     028dbec8     028cde94     028da248    
    System_Xml_ni+100c09     028cde94     028cdabc     028cac60     109948ec    
    mscorlib_ni+2f704d     028cdb64     028cdb58     015de390     00000000    
    0x0e7df9eb     0ff00dd0     028cac60     00000000     00000000    
    0x0ff5b196     028ca6d0     028caa50     05f7facc     10994a2c    
    0x0ff5bac2     028ba4a4     028ba0cc     028b7270     10994a78    
    mscorlib_ni+2f704d     028ba174     028ba168     015de390     00000000    
    0x0e7df9eb     0ff00dd0     028b7270     00000000     00000000    
    0x0ff5b196     028b6ce0     028b7060     05f7facc     10994bb8    
    0x0ff5bac2     028a6ab4     028a66dc     028a3880     10994c04    
    mscorlib_ni+2f704d     028a6784     028a6778     015de390     00000000    
    0x0e7df9eb     0ff00dd0     028a3880     00000000     00000000    
    0x0ff5b196     028a32f0     028a3670     05f7facc     10994d44    
    0x0ff5bac2     028930c4     02892cec     0288fe90     10994d90    
    mscorlib_ni+2f704d     02892d94     02892d88     015de390     00000000    
    ...
    ...
    and this repeat several hundred times
    ...
    ... 
    mscorlib_ni+2f704d     0628182c     06281820     015de390     00000000    
    0x0e7df9eb     0ff00dd0     0627c91c     00000000     00000000    
    0x0ff5b196     0627c38c     0627c70c     05f81618     109ce104    
    0x0ff5bac2     06269c18     05fcbb00     0625e45c     109ce150    
    mscorlib_ni+2f704d     062698e8     062698dc     015de390     00000000    
    0x0e7df9eb     0ff00dd0     0625e45c     00000000     00000000    
    0x0ff5b196     0625be94     0625e24c     05f81618     109ce290    
    0x0ff5bac2     0622ffb4     05fc9b4c     06201454     109ce2dc    
    mscorlib_ni+2f704d     0622fc04     0622fbf8     015de390     00000000    
    0x0e7df9eb     0ff00dd0     06201454     00000000     00000000    
    0x0ff5b196     061f7b64     109ce518     109ce528     0ff5a455    
    0x0ff5af35     109cf388     72b8a75d     109cf40c     00c0139c    
    0x0ff5a455     00000000     00000000     00000000     00000000    
    0x0ff59a36     00000000     00000000     00000000     00000000    
    0x0ff52fe3     05fb3920     05f480bc     0ff50bc5     05fb38a4    
    0x0e8602c1     05fb38a4     0fc0fd31     05fb38a4     00000000    
    0x0ff50a07     05fa0afc     0605d4a0     00000000     00000000    
    0x0fc0f8a4     01f4bf54     00000000     00000000     01f4bf54    
    0x0fc0f7d7     00000000     00000000     00000000     00000000    
    0x0fc0f767     05fa0afc     0fc0f690     109ceba8     0fc09096    
    0x0fc0f6a5     05fa0afc     109cec20     05fa3338     05f18394    
    0x0fc09096     01f4bf54     00000000     00000000     01f4bf54    
    0x0fc07d8f     00000000     00000000     00000000     00000000    
    0x0fc07d1f     05fa0afc     7cfc31f3     0606cd6c     0606cbfc    
    0x0fc048d2     05fa0afc     109ced6c     00000000     00000000    
    0x0fc02108     05fa0afc     6aee2dcc     060322f0     0602e8ac    
    System_Net_Http_ni+1d328     05fa0afc     109ceec4     109cee3c     109cee4c    
    System_Net_Http_ni+1d4da     05fa0afc     00000000     109ceec4     00000000    
    0x0fc001d5     05fa0afc     6aed5938     109cefd8     109cefd8    
    System_Net_Http_ni+1d328     6aed5938     109cefd8     109cefd8     109cef60    
    0x05fa0afc     109cefd8     109cefd8     109cef60     0eccf7d1    
    System_Net_Http_ni+15938     109cefd8     109cef60     0eccf7d1     05fa0afc    
    0x109cefd8     109cef60     0eccf7d1     05fa0afc     109cefd8    
    0x109cefd8     109cf040     6fdd1d74     0603221c     06048d40    
    System_Web_ni+8b1d47     0603221c     06048d40     01f88e50     109cf058    
    System_Web_ni+8b1d74     00000000     05f9be18     01f4b5a8     01f4f05c    
    System_Web_ni+8b1cfe     00000000     01f4f07c     01f8c08c     06048d10    
    System_Web_ni+b20c7a     6fd728d5     6f610744     01f94f48     109cf0d4    
    System_Web_ni+1c977c     109cf190     01f904c4     00000000     01f4f280    
    System_Web_ni+1db817     00000000     00000000     109cf220     109cf22c    
    System_Web_ni+1c99c0     01f09500     109cf214     109cf2e4     7cfc31f3    
    System_Web_ni+1c649b     01f4b5a8     7cfc31f3     72a3acd8     109cf40c    
    System_Web_ni+1cb931     00000000     0000000b     b426d8e3     00000000    
    System_Web_ni+1cb63f     0eef9ee0     109cf3cc     72b8e8cd     109cf524    
    clr+26f0     109cf524     00000010     0085afe0     01cb1cb8    
    clr!CopyPDBs+1ad6a     109cf4b8     b426dffb     01cb1cce     00000002    
    clr!StrongNameFreeBuffer+1af95     00000002     72b8e85c     109cf4b8     01000001    
    clr!CopyPDBs+1acd8     00bf1184     0154a12c     0000000b     00000000    
    webengine4!W3_MGD_HANDLER::ProcessNotification+5a     7306aa00     7417a2f4     109cf598     72b8147a    
    webengine4!ProcessNotificationCallback+32     0154a12c     b426de0b     109cf5c5     109cf5c7    
    clr!CopyPDBs+d917     109cf5c5     109cf5c7     00020002     0eef9ee0    
    clr!GetMetaDataInternalInterfaceFromPublic+29800     00000000     109cf74c     10072d30     b4728db1    
    clr!GetMetaDataInternalInterfaceFromPublic+1b084     0eeeeac0     109cfa7c     771592e2     0eeeeac0    
    kernel32!BaseThreadInitThunk+e     0eeeeac0     bff8c0ad     00000000     00000000    
    ntdll!RtlInitializeExceptionChain+63     72bca8c0     0eeeeac0     ffffffff     771e7707    
    ntdll!RtlInitializeExceptionChain+36     72bca8c0     0eeeeac0     00000000     00000000 

.net
iis
crash
stack-overflow
asked on Stack Overflow Oct 28, 2016 by Richard.R

1 Answer

0

Thanks very much for your comments, Finally we found the problem and fixed it. Unfortunately we are in mainland china that can not access Google directly. Your comments helped us a lot.

Use WinDbg to load crash dump can show managed .Net stack trace.

answered on Stack Overflow Oct 28, 2016 by Richard.R • edited Oct 28, 2016 by Richard.R

User contributions licensed under CC BY-SA 3.0