Unhandled Exception in Rad Studio Debugger Thread

14

I have a large application that recently started exhibiting rather strange behavior when running in a debugger. First, the basics:

OS: Windows 7 64-bit.
Application: Multithreaded VCL app with many dlls, bpls, and other components.
Compiler/IDE: Embarcadero RAD Studio 2010.

The observed symptom is this: While the debugger is attached to my application, certain tasks cause the application to crash. The particulars are furthermore perplexing: My application stops with a Windows message saying, "YourApplication has stopped working." And it helpfully offers to send a minidump to Microsoft.

It should be noted: the application doesn't crash when the debugger is not attached. Also, the debugger doesn't indicate any exceptions or other issues while the application is running.

Setting and stepping through breakpoints seems to affect the point at which the application crashes, but I suspect that is a symptom of debugging a thread other than the problematic one.

These crashes also occur on the computers on my colleagues, with the same behavior I observe. This leads me to not suspect a failed installation of something on my computer particularly. My colleagues experiencing the issue are also running Windows 7 64-bit. I have no colleagues not experiencing the issue.

I've collected an analyzed a number of full dumps from the crashes. I discovered that the failure was actually happening in the same place each time. Here is the exception data from the dumps (it is always the same, except of course the ThreadId):

Exception Information

ThreadId:         0x000014C0
Code:             0x4000001F Unknown (4000001F)
Address:          0x773F2507
Flags:            0x00000000
NumberParameters: 0x00000001
    0x00000000

Google reveals that Code 0x4000001F is actually STATUS_WX86_BREAKPOINT. Microsoft unhelpfully describes it as "An exception status code that is used by the Win32 x86 emulation subsystem."

Here are the stack details (which don't seem to vary):

0x773F2507: ntdll.dll+0x000A2507: RtlQueryCriticalSectionOwner + 0x000000E8
0x773F3DAB: ntdll.dll+0x000A3DAB: RtlQueryProcessLockInformation + 0x0000020D
0x773D2ED9: ntdll.dll+0x00082ED9: RtlUlonglongByteSwap + 0x00005C69
0x773F3553: ntdll.dll+0x000A3553: RtlpQueryProcessDebugInformationRemote + 0x00000044
0x74F73677: kernel32.dll+0x00013677: BaseThreadInitThunk + 0x00000012
0x77389F02: ntdll.dll+0x00039F02: RtlInitializeExceptionChain + 0x00000063
0x77389ED5: ntdll.dll+0x00039ED5: RtlInitializeExceptionChain + 0x00000036

It is worth noting that there appears to be a function epilog at 0x773F24ED, which rather suggests that the RtlQueryCriticalSectionOwner is a red herring. Likewise, a function epilog casts doubt on RtlQueryProcessLockInformation. The 0x5C69 offset casts doubt on the RtlUlonglongByteSwap. The other symbols look legit, though.

Specifically, RtlpQueryProcessDebugInformationRemote looks legitimate. Some people on the internet (http://www.cygwin.com/ml/cygwin-talk/2006-q2/msg00050.html) seem to think that it is created by the debugger to collect debug information. That theory seems sound to me, since it only seems to appear when the debugger is attached.

As always, when something breaks, something changed that broke it. In this case, that something is dynamically loading a new dll. I can cause the crash to stop happening by not dynamically loading the particular dll. I'm not convinced that the dll loading is related, but here are the details, just in case:

The dll source is C. Here are the compile options that are not set to the default:

Language Compliance: ANSI
Merge duplicate strings: True
Read-only strings: True
PCH usage: Do not use
Dynamic RTL: False

(The Project Options say False is default for Dynamic RTL, though it was set to True when I created the dll project.)

The dll is loaded with LoadLibrary and freed with FreeLibrary. All seems to be fine with the loading and unloading of the module. However, shortly after the library is unloaded (with FreeLibrary), the aforementioned thread crashes the program. For debugging, I removed all actual calls to the library (including, for more testing, DllMain). No combination of calls or not calls, DllMain or no DllMain, or anything else seemed to alter the behavior of the crash in any way. Simply loading and unloading the dll invokes the crash later on.

Furthermore, changing the dll to use the Dynamic RTL also causes the debugger thread crash to cease. This is undesirable because the compiled dll really should be usable without CodeGear Runtime available. Also, dll size is important. The C code contained in the dll does not make use of any libraries. (It includes no headers, even standard library headers. No malloc/free, no printf, no nothin'. It contains only functions that depend exclusively on their inputs and do not require dynamic allocation.) It is also undesirable because "fixing" a bug by changing stuff until it works without understanding why it works is really never a good plan. (It tends to lead to bug recurrences and strange coding practices. But really, at this point, if I can't find anything else, I may admit defeat on this count.)

And finally, my issue may be related to one of these issues:

Any ideas or suggestions would be appreciated.

c++
exception
dll
c++builder
c++builder-2010
asked on Stack Overflow Jun 20, 2011 by alficles • edited May 23, 2017 by Community

4 Answers

9

I solved the above mentioned problem by using a modified version of the PatchINT3 workaround, which was published in 2007 for BDS 2006:

procedure PatchINT3;
const
  INT3: Byte = $CC;
  NOP: Byte = $90;
var
  NTDLL: THandle;
  BytesWritten: DWORD;
  Address: PByte;
begin
  if Win32Platform <> VER_PLATFORM_WIN32_NT then
    Exit;
  NTDLL := GetModuleHandle('NTDLL.DLL');
  if NTDLL = 0 then
    Exit;
  Address := GetProcAddress(NTDLL, 'RtlQueryCriticalSectionOwner');
  if Address = nil then
    Exit;
  Inc(Address, $E8);
  try
    if Address^ <> INT3 then
      Exit;

    if WriteProcessMemory(GetCurrentProcess, Address, @NOP, 1, BytesWritten)
      and (BytesWritten = 1) then
      FlushInstructionCache(GetCurrentProcess, Address, 1);
  except
    //Do not panic if you see an EAccessViolation here, it is perfectly harmless!
    on EAccessViolation do
      ;
  else
    raise;
  end;
end;

Call this routine once after you have loaded the DLL in your thread. The patch fixes a user breakpoint in ntdll.dll version 6.1.7601.17725 and changes it to a NOP.

If there is no user breakpoint (INT3 (=$CC) opcode) at the expected address, the patch routine does nothing and exits.

Hope that helps,
Andreas

Footnote
The original source of PatchINT3 can be found here:
http://coding.derkeiler.com/Archive/Delphi/borland.public.delphi.non-technical/2007-01/msg04431.html

Footnote2
The same function in C++:

void PatchINT3()
{
   unsigned char INT3   = 0xCC;
   unsigned char NOP    = 0x90;

   if (Win32Platform != VER_PLATFORM_WIN32_NT)
   {
      return;
   }

   HMODULE ntdll = GetModuleHandle(L"NTDLL.DLL");
   if (ntdll == NULL)
   {
      return;
   }

   unsigned char *address = (unsigned char*)GetProcAddress(ntdll,
      "RtlQueryCriticalSectionOwner");
   if (address == NULL)
   {
      return;
   }

   address += 0xE8;

   try
   {
      if (*address != INT3)
      {
         return;
      }

      unsigned long bytes_written = 0;
      if (WriteProcessMemory(GetCurrentProcess(), address, &NOP, 1,
         &bytes_written) && (bytes_written == 1))
      {
         FlushInstructionCache(GetCurrentProcess, address, 1);
      }
   }
   catch (EAccessViolation &e)
   {
      //Do not panic if you see an EAccessViolation
      //here, it is perfectly harmless!
   }
   catch(...)
   {
      throw;
   }
}
answered on Stack Overflow Sep 17, 2012 by Andreas Bormann • edited Jun 22, 2013 by truthseeker
0

Just an idea...

Perhaps you need to close in on the crashing thread. The state that you are observing seems to be a bit after the actual error.

First, your stack traces seems incomplete to me. What is the base root of the stack of that thread? What was the origin of that thread?

And, in the VS debugger there is the possibility to break on exceptions, (Debug->Exceptions...->[Add]). Then all threads will freeze at the moment the exception occurs. I dunno about RAD, but the trick to do it programatically seems to be WaitForDebugEvent().

I could be wrong but I think there is a fair chance that the bug is in the debugger not your code. In that case an uggly workaround is IMHO fully forgivable. Good luck!

answered on Stack Overflow Aug 4, 2011 by Adam
0

I cannot answer this because I cannot see the code...

But...

1) In Borland C++ at least with C++ from BDS there can be a provable problem with the realloc function in multithreaded library. Is your C++ code using realloc?

2) The stack you are showing is more than likely being called as a result of your code actually hitting a "CALL BAD_ADRESS" and that can happen as a result of a bug in your own code. In other words, in the DLL you load, there is likely a function that is doing something that is overwriting the executable code in your program with junk, and then when the now junk section runs, it crashes.

Another way is if something in the C++ dll is modifying the stack below where it runs, and then your code is hitting that later.

3) Check out the CPU flags settings for your DLL. Borland libraries sometimes use conflicting CPU flags upon entry and you might need to save and restore before calling into the DLL. For example if you call a VST plugin made with C++ from Delphi and do not set the flags properly, you can get subsequent divide by zero errors from the VST plugin which was compiled with that exception turned off.

answered on Stack Overflow Sep 12, 2012 by (unknown user)
0

We had the same issue today. In our case the crash happens if having a breakpoint after call to TOpenDialog->Execute() (which is using dialog from shell32.dll I think) (Windows 7 x64, C++ Builder XE2)

After uninstalling iCloud (v2.1.0.39), the issue was resoved.

Unfortunately we are still looking into similar issue our customers are having some times with our release product under Windows Vista. After selecting the file using TOpenDialog, the application crashes in gdiplus.dll with Access violation, removing the iCloud seems to also resolve the issue.

answered on Stack Overflow Jan 25, 2013 by Niki

User contributions licensed under CC BY-SA 3.0