Analyzing this Procdump .dmp file - from Apache httpd.exe

1

Recently my server's Apache httpd.exe has been crashing like crazy (posted to server fault thread here https://serverfault.com/questions/998227/windows-server-2008-r2-apache-2-4-constant-crashing-with-faulting-module-nam).

I tried all solutions I found on the web but it still happen. Finally, I used Procdump to monitor httpd.exe process and get this dump file when it crashes. But how to analyze the result is out of my knowledge. I need help on this.

Opening the .dmp file using WinDbg:

Microsoft (R) Windows Debugger Version 6.12.0002.633 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\Users\Zenn\Desktop\httpd.exe_200111_125801.dmp]
User Mini Dump File: Only registers, stack and portions of memory are available

Comment: '
*** procdump64  -t -e 7052
*** Unhandled exception: C0000005.ACCESS_VIOLATION'
Symbol search path is: *** Invalid ***
****************************************************************************
* Symbol loading may be unreliable without a symbol search path.           *
* Use .symfix to have the debugger choose a symbol path.                   *
* After setting your symbol path, use .reload to refresh symbol locations. *
****************************************************************************
Executable search path is: 
Windows 7 Version 7601 (Service Pack 1) MP (4 procs) Free x64
Product: Server, suite: TerminalServer DataCenter SingleUserTS
Machine Name:
Debug session time: Sat Jan 11 12:58:01.000 2020 (UTC + 8:00)
System Uptime: not available
Process Uptime: 0 days 0:50:59.000
................................................................
................................................
Loading unloaded module list
............................................................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(1b8c.10a0): Access violation - code c0000005 (first/second chance not available)
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for ntdll.dll - 
ntdll!RtlAnsiStringToUnicodeString+0x12c:
00000000`777cf23c 488b7b08        mov     rdi,qword ptr [rbx+8] ds:000005d2`ac238618=????????????????

After running !analyze -v:

Failed calling InternetOpenUrl, GLE=12029

FAULTING_IP: 
ntdll!RtlAnsiStringToUnicodeString+12c
00000000`777cf23c 488b7b08        mov     rdi,qword ptr [rbx+8]

EXCEPTION_RECORD:  ffffffffffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 00000000777cf23c (ntdll!RtlAnsiStringToUnicodeString+0x000000000000012c)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 0000000000000000
   Parameter[1]: 000005d2ac238618
Attempt to read from address 000005d2ac238618

PROCESS_NAME:  httpd.exe

FAULTING_MODULE: 0000000077780000 ntdll

DEBUG_FLR_IMAGE_TIMESTAMP:  0

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_PARAMETER1:  0000000000000000

EXCEPTION_PARAMETER2:  000005d2ac238618

READ_ADDRESS:  000005d2ac238618 

FOLLOWUP_IP: 
ntdll!RtlAnsiStringToUnicodeString+12c
00000000`777cf23c 488b7b08        mov     rdi,qword ptr [rbx+8]

MOD_LIST: <ANALYSIS/>

ADDITIONAL_DEBUG_TEXT:  
Use '!findthebuild' command to search for the target build information.
If the build information is available, run '!findthebuild -s ; .reload' to set symbol path and load symbols. ; Enable Pageheap/AutoVerifer

FAULTING_THREAD:  00000000000010a0

DEFAULT_BUCKET_ID:  HEAP_CORRUPTION

PRIMARY_PROBLEM_CLASS:  HEAP_CORRUPTION

BUGCHECK_STR:  APPLICATION_FAULT_HEAP_CORRUPTION_INVALID_POINTER_READ_WRONG_SYMBOLS_FILL_PATTERN_ffffffff

LAST_CONTROL_TRANSFER:  from 00000000773d1a0a to 00000000777cf23c

STACK_TEXT:  
00000000`5904eb20 00000000`773d1a0a : 00000003`67cb5f10 00000001`0e2bc328 00000000`01fd89e0 00000003`42541b01 : ntdll!RtlAnsiStringToUnicodeString+0x12c
00000000`5904eba0 000007fe`ee296338 : 00000003`4b1038f0 00000000`00000002 00000003`00000000 00000000`00000002 : kernel32!HeapFree+0xa
00000000`5904ebd0 00000003`4b1038f0 : 00000000`00000002 00000003`00000000 00000000`00000002 00000003`4281e670 : msvcr110+0x66338
00000000`5904ebd8 00000000`00000002 : 00000003`00000000 00000000`00000002 00000003`4281e670 000007fe`daf5ea98 : 0x3`4b1038f0
00000000`5904ebe0 00000003`00000000 : 00000000`00000002 00000003`4281e670 000007fe`daf5ea98 00000003`3bb260c0 : 0x2
00000000`5904ebe8 00000000`00000002 : 00000003`4281e670 000007fe`daf5ea98 00000003`3bb260c0 00000001`0e2a8d50 : 0x3`00000000
00000000`5904ebf0 00000003`4281e670 : 000007fe`daf5ea98 00000003`3bb260c0 00000001`0e2a8d50 00000001`0e2bc328 : 0x2
00000000`5904ebf8 000007fe`daf5ea98 : 00000003`3bb260c0 00000001`0e2a8d50 00000001`0e2bc328 000007fe`db4d1370 : 0x3`4281e670
00000000`5904ec00 00000003`3bb260c0 : 00000001`0e2a8d50 00000001`0e2bc328 000007fe`db4d1370 00000001`0e2fe6b8 : php5ts+0xbea98
00000000`5904ec08 00000001`0e2a8d50 : 00000001`0e2bc328 000007fe`db4d1370 00000001`0e2fe6b8 000007fe`daf3e4cc : 0x3`3bb260c0
00000000`5904ec10 00000001`0e2bc328 : 000007fe`db4d1370 00000001`0e2fe6b8 000007fe`daf3e4cc 00000000`5e1955d9 : 0x1`0e2a8d50
00000000`5904ec18 000007fe`db4d1370 : 00000001`0e2fe6b8 000007fe`daf3e4cc 00000000`5e1955d9 00000000`00000001 : 0x1`0e2bc328
00000000`5904ec20 00000001`0e2fe6b8 : 000007fe`daf3e4cc 00000000`5e1955d9 00000000`00000001 00000000`00000001 : php5ts+0x631370
00000000`5904ec28 000007fe`daf3e4cc : 00000000`5e1955d9 00000000`00000001 00000000`00000001 00000003`4281e670 : 0x1`0e2fe6b8
00000000`5904ec30 00000000`5e1955d9 : 00000000`00000001 00000000`00000001 00000003`4281e670 00000003`3bb260f8 : php5ts+0x9e4cc
00000000`5904ec38 00000000`00000001 : 00000000`00000001 00000003`4281e670 00000003`3bb260f8 000007fe`daf42a71 : 0x5e1955d9
00000000`5904ec40 00000000`00000001 : 00000003`4281e670 00000003`3bb260f8 000007fe`daf42a71 00000001`0e323030 : 0x1
00000000`5904ec48 00000003`4281e670 : 00000003`3bb260f8 000007fe`daf42a71 00000001`0e323030 00000000`00000000 : 0x1
00000000`5904ec50 00000003`3bb260f8 : 000007fe`daf42a71 00000001`0e323030 00000000`00000000 00000003`42541b10 : 0x3`4281e670
00000000`5904ec58 000007fe`daf42a71 : 00000001`0e323030 00000000`00000000 00000003`42541b10 00000000`00000001 : 0x3`3bb260f8
00000000`5904ec60 00000001`0e323030 : 00000000`00000000 00000003`42541b10 00000000`00000001 00000003`42541b10 : php5ts+0xa2a71
00000000`5904ec68 00000000`00000000 : 00000003`42541b10 00000000`00000001 00000003`42541b10 00000000`5904ec90 : 0x1`0e323030


SYMBOL_NAME:  heap_corruption!heap_corruption

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: heap_corruption

IMAGE_NAME:  heap_corruption

STACK_COMMAND:  ~110s; .ecxr ; kb

FAILURE_BUCKET_ID:  HEAP_CORRUPTION_c0000005_heap_corruption!heap_corruption

BUCKET_ID:  X64_APPLICATION_FAULT_HEAP_CORRUPTION_INVALID_POINTER_READ_WRONG_SYMBOLS_FILL_PATTERN_ffffffff_heap_corruption!heap_corruption

WATSON_STAGEONE_URL:  http://watson.microsoft.com/StageOne/httpd_exe/2_4_38_0/5c45ba66/ntdll_dll/6_1_7601_23677/589c99e1/c0000005/0004f23c.htm?Retriage=1

Followup: MachineOwner
---------

EDIT:

I monitored another occasion of crash and this is the outcome after running !analyze -v:

Failed calling InternetOpenUrl, GLE=12029

FAULTING_IP: 
ntdll!RtlAnsiStringToUnicodeString+12c
00000000`777cf23c 488b7b08        mov     rdi,qword ptr [rbx+8]

EXCEPTION_RECORD:  ffffffffffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 00000000777cf23c (ntdll!RtlAnsiStringToUnicodeString+0x000000000000012c)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 0000000000000000
   Parameter[1]: 0000072502404c18
Attempt to read from address 0000072502404c18

PROCESS_NAME:  httpd.exe

FAULTING_MODULE: 0000000077780000 ntdll

DEBUG_FLR_IMAGE_TIMESTAMP:  5098826e

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

EXCEPTION_PARAMETER1:  0000000000000000

EXCEPTION_PARAMETER2:  0000072502404c18

READ_ADDRESS:  0000072502404c18 

FOLLOWUP_IP: 
msvcr110+66338
000007fe`ee296338 ??              ???

MOD_LIST: <ANALYSIS/>

LAST_CONTROL_TRANSFER:  from 00000000773d1a0a to 00000000777cf23c

FAULTING_THREAD:  ffffffffffffffff

ADDITIONAL_DEBUG_TEXT:  
Use '!findthebuild' command to search for the target build information.
If the build information is available, run '!findthebuild -s ; .reload' to set symbol path and load symbols. ; Followup set based on attribute [Is_ChosenCrashFollowupThread] from Frame:[0] on thread:[PSEUDO_THREAD] ; Enable Pageheap/AutoVerifer

DEFAULT_BUCKET_ID:  HEAP_CORRUPTION

PRIMARY_PROBLEM_CLASS:  HEAP_CORRUPTION

BUGCHECK_STR:  APPLICATION_FAULT_HEAP_CORRUPTION_INVALID_POINTER_READ_WRONG_SYMBOLS_FILL_PATTERN_ffffffff

STACK_TEXT:  
00000000`00000000 00000000`00000000 msvcr110+0x0


SYMBOL_STACK_INDEX:  2

SYMBOL_NAME:  msvcr110+66338

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: msvcr110

IMAGE_NAME:  msvcr110.dll

STACK_COMMAND:  ** Pseudo Context ** ; kb

BUCKET_ID:  WRONG_SYMBOLS

FAILURE_BUCKET_ID:  HEAP_CORRUPTION_c0000005_msvcr110.dll!Unknown

WATSON_STAGEONE_URL:  http://watson.microsoft.com/StageOne/httpd_exe/2_4_38_0/5c45ba66/ntdll_dll/6_1_7601_23677/589c99e1/c0000005/0004f23c.htm?Retriage=1

Followup: MachineOwner
---------
windows
apache
procdump
asked on Stack Overflow Jan 12, 2020 by user3162662 • edited Jan 15, 2020 by user3162662

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0