Currently I am analyzing a dump with WinDbg.
I ran following commands (following Tess' incredible walkthrough):
~* e !clrstack
Which listed me all stacks of all threads. There are 300 running threads with more or less the same stack, so I am just printing the stack of one here ...
OS Thread Id: 0x107c (166)
Child SP IP Call Site
2bc1e654 77c1015d [HelperMethodFrame_1OBJ: 2bc1e654] System.Threading.WaitHandle.WaitOneNative(System.Runtime.InteropServices.SafeHandle, UInt32, Boolean, Boolean)
2bc1e720 6b2e6dd2 System.Threading.WaitHandle.InternalWaitOne(System.Runtime.InteropServices.SafeHandle, Int64, Boolean, Boolean)
2bc1e73c 6b2e6d9c System.Threading.WaitHandle.WaitOne(Int32, Boolean)
2bc1e750 727f4baa System.Threading.ReaderWriterLockSlim.WaitOnEvent(System.Threading.EventWaitHandle, UInt32 ByRef, TimeoutTracker)
2bc1e78c 729bc154 System.Threading.ReaderWriterLockSlim.TryEnterUpgradeableReadLockCore(TimeoutTracker)
2bc1e7b8 725d250c System.Threading.ReaderWriterLockSlim.TryEnterUpgradeableReadLock(TimeoutTracker)
** MORE LINES **
Then I did the following:
> ~166s
eax=00000000 ebx=2bc1e444 ecx=00000000 edx=00000000 esi=00000001 edi=00000000
eip=77c1015d esp=2bc1e3f4 ebp=2bc1e490 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
ntdll!NtWaitForMultipleObjects+0x15:
77c1015d 83c404 add esp,4
Then I was looking for all the instances in this thread:
> !dso
OS Thread Id: 0x107c (166)
ESP/REG Object Name
** MORE LINES **
2BC1E7D0 039ec48c System.Threading.ReaderWriterLockSlim
** MORE LINES **
Then I tried to get more information on the ReaderWriterLockSlim
-instance:
> !do 039ec48c
Name: System.Threading.ReaderWriterLockSlim
MethodTable: 725ebda4
EEClass: 724543bc
Size: 68(0x44) bytes
File: C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll
Fields:
MT Field Offset Type VT Attr Value Name
6b3b8138 4000755 3c System.Boolean 1 instance 0 fIsReentrant
6b3c3aa4 4000756 1c System.Int32 1 instance 0 myLock
6b3c7ae4 4000757 20 System.UInt32 1 instance 0 numWriteWaiters
6b3c7ae4 4000758 24 System.UInt32 1 instance 0 numReadWaiters
6b3c7ae4 4000759 28 System.UInt32 1 instance 0 numWriteUpgradeWaiters
6b3c7ae4 400075a 2c System.UInt32 1 instance 348 numUpgradeWaiters
6b3b8138 400075b 3d System.Boolean 1 instance 0 fNoWaiters
6b3c3aa4 400075c 30 System.Int32 1 instance 366 upgradeLockOwnerId
6b3c3aa4 400075d 34 System.Int32 1 instance 366 writeLockOwnerId
6b3c0ac0 400075e c ...g.EventWaitHandle 0 instance 00000000 writeEvent
6b3c0ac0 400075f 10 ...g.EventWaitHandle 0 instance 00000000 readEvent
6b3c0ac0 4000760 14 ...g.EventWaitHandle 0 instance 08188858 upgradeEvent
6b3c0ac0 4000761 18 ...g.EventWaitHandle 0 instance 00000000 waitUpgradeEvent
6b3b821c 4000763 4 System.Int64 1 instance 231 lockID
6b3b8138 4000765 3e System.Boolean 1 instance 0 fUpgradeThreadHoldingRead
6b3c7ae4 4000766 38 System.UInt32 1 instance 2147483649 owners
6b3b8138 4000767 3f System.Boolean 1 instance 0 fDisposed
6b3b821c 4000762 3e0 System.Int64 1 shared static s_nextLockID
>> Domain:Value 01742528:NotInit 01783fb8:NotInit 1268c9d8:NotInit <<
725fd46c 4000764 4 ...ReaderWriterCount 0 shared TLstatic t_rwc
>> Thread:Value <<
I've matched the value of owners
(2147483649) against the the information on this page = 0x80000001
private const uint WRITER_HELD = 0x80000000;
private const uint WAITING_WRITERS = 0x40000000;
private const uint WAITING_UPGRADER = 0x20000000;
But how can I identify the thread which holds the lock?
If you have a deadlock, you can use SOSEx's !dlk
command which does all the work for you.
*DEADLOCK DETECTED*
CLR thread 0x4 holds the Writer lock on ReaderWriterLockSlim 02712580
...and is waiting for a Reader lock on ReaderWriterLockSlim 0271253c
CLR thread 0x3 holds the Writer lock on ReaderWriterLockSlim 0271253c
...and is waiting for a Reader lock on ReaderWriterLockSlim 02712580
CLR Thread 0x4 is waiting at System.Threading.WaitHandle.InternalWaitOne(System.Runtime.InteropServices.SafeHandle, Int64, Boolean, Boolean)(+0x1f IL,+0x22 Native)
CLR Thread 0x3 is waiting at System.Threading.WaitHandle.InternalWaitOne(System.Runtime.InteropServices.SafeHandle, Int64, Boolean, Boolean)(+0x1f IL,+0x22 Native)
If you don't have a deadlock, you can still use SOSEx, but type !mlocks
. The output looks like this, so you get all the different thread numbers and the type of the lock.
0:012> !mlocks
Examining SyncBlocks...
Scanning for ReaderWriterLock instances...
Scanning for holders of ReaderWriterLock locks...
Scanning for ReaderWriterLockSlim instances...
Scanning for holders of ReaderWriterLockSlim locks...
Examining CriticalSections...
ClrThread DbgThread OsThread LockType Lock LockLevel
----------------------------------------------------------------------
0x1 0 0x1460 thinlock 02718bcc (recursion:0)
0x5 6 0x1e80 RWLock 027125f0 Writer
0x8 9 0x22ac CritSect 027124c0
0x9 10 0x27b8 SyncBlock 0045f4e8
0xa 11 0x33f8 SyncBlock 0045f4b4
0x7 8 0x388c CritSect 027124a0
0x4 5 0x3d20 RWLockSlim 02712580 Writer
0x3 4 0x3e44 RWLockSlim 0271253c Writer
0x6 7 0x4704 RWLock 027125c4 Writer
If you have DML enabled, you can even click the links in the Lock column which gives more information. You can also type !rwlock <lock>
if you don't like DML.
I did some more research, and decompiled ReaderWriterLockSlim
and determined the meaning of writeLockOwnerId
and upgradeLockOwnerId
: They are actually the managed thread ids of the threads within the corresponding method.
Unfortunately you have to determine the ordinal of this managed thread id, by doing:
> !threads
ThreadCount: 371
UnstartedThread: 0
BackgroundThread: 371
PendingThread: 0
DeadThread: 0
Hosted Runtime: no
Lock
ID OSID ThreadOBJ State GC Mode GC Alloc Context Domain Count Apt Exception
** MORE LINES **
381 366 1c0c 1e1fca78 1029220 Preemptive 3E46102C:00000000 01783fb8 2 MTA (Threadpool Worker)
** MORE LINES **
After this, you can simply do:
> ~381s
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=0000432d edi=4519d490
eip=77c0f8d1 esp=4519d448 ebp=4519d4b4 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246
ntdll!ZwWaitForSingleObject+0x15:
77c0f8d1 83c404 add esp,4
> !clrstack
OS Thread Id: 0x1c0c (381)
Child SP IP Call Site
** MORE LINES **
4519da08 69f35592 System.Data.SqlClient.SqlCommand.ExecuteReader(System.Data.CommandBehavior, System.String)
** MORE LINES **
Et voilĂ : the thread, which holds the upgradable lock, is currently executing some sql.
User contributions licensed under CC BY-SA 3.0