.NET OutOfMemory exception - Reason: Low on memory during GC

I am trying to debug a .NET application that is crashing with strange symptoms.

The stack trace reported alongside the same OutOfMemory exception in Windows Event Log is always different:

Exception Info: System.OutOfMemoryException
   at System.Exception.get_Message()
   at System.Exception.ToString(Boolean, Boolean)
   at System.Exception.ToString()
   at App.Log.AddException(System.String, System.Exception)
   at App.SoftwarePlayerMonitoring.Monitor()
   at System.Threading.ThreadHelper.ThreadStart_Context(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.ThreadHelper.ThreadStart()

---

Exception Info: System.OutOfMemoryException
   at System.Windows.Forms.Integration.ApplicationInterop.ThreadMessageFilter(System.Windows.Interop.MSG ByRef, Boolean ByRef)
   at System.Windows.Interop.ComponentDispatcherThread.RaiseThreadMessage(System.Windows.Interop.MSG ByRef)
   at System.Windows.Threading.Dispatcher.PushFrameImpl(System.Windows.Threading.DispatcherFrame)
   at System.Windows.Threading.Dispatcher.PushFrame(System.Windows.Threading.DispatcherFrame)
   at System.Windows.Application.RunDispatcher(System.Object)
   at System.Windows.Application.RunInternal(System.Windows.Window)
   at System.Windows.Application.Run(System.Windows.Window)
   at App.Main()

---

Exception Info: System.OutOfMemoryException
   at System.Threading.ExecutionContext.CreateCopy()
   at System.Threading.TimerQueueTimer.CallCallback()
   at System.Threading.TimerQueueTimer.Fire()
   at System.Threading.TimerQueue.FireQueuedTimerCompletion(System.Object)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

---

In all of the aforementioned cases the crash is reported as being related to:

Faulting module name: KERNELBASE.dll, version: 6.1.7601.18015, time stamp: 0x50b83c8a
Exception code: 0xe0434352
Fault offset: 0x0000c41f
Faulting process id: 0xf50
Faulting application start time: 0x01d52412866cc441
Faulting module path: C:\Windows\syswow64\KERNELBASE.dll

I have managed to capture a procdump dump of the application in frozen state and here is the analysis of it:

0:000> !sos.AnalyzeOOM
Managed OOM occured after GC #29843 (Requested to allocate 0 bytes)
Reason: Low on memory during GC
Detail: SOH: Failed to reserve memory (16777216 bytes)

0:000> !dumpheap -stat
The garbage collector data structures are not in a valid state for traversal.
It is either in the "plan phase," where objects are being moved around, or
we are at the initialization or shutdown of the gc heap. Commands related to 
displaying, finding or traversing objects as well as gc heap segments may not 
work properly. !dumpheap and !verifyheap may incorrectly complain of heap 
consistency errors.
Object <exec cmd="!ListNearObj /d 024b1000">024b1000</exec> has an invalid method table.

0:000> !vmstat
TYPE                      MINIMUM                      MAXIMUM                      AVERAGE  BLK COUNT                        TOTAL
~~~~                      ~~~~~~~                      ~~~~~~~                      ~~~~~~~  ~~~~~~~~~                        ~~~~~
Free:
Small                          4K                          64K                          36K        477                      17,187K
Medium                        68K                       1,024K                         350K        133                      46,571K
Large                      1,040K                      14,976K                       5,233K         49                     256,447K
Summary                        4K                      14,976K                         485K        659                     320,207K

Reserve:
Small                          4K                          64K                           6K      1,107                       7,619K
Medium                       112K                       1,012K                         459K        260                     119,355K
Large                      1,148K                      20,204K                       6,568K         14                      91,955K
Summary                        4K                      20,204K                         158K      1,381                     218,931K

Commit:
Small                          4K                          64K                          34K      5,826                     198,431K
Medium                        68K                       1,024K                         284K        472                     134,315K
Large                      1,028K                      26,316K                       8,689K        141                   1,225,199K
Summary                        4K                      26,316K                         241K      6,439                   1,557,942K

Private:
Small                          4K                          64K                          29K      1,442                      42,219K
Medium                        68K                       1,024K                         406K        394                     160,275K
Large                      1,028K                      26,316K                       9,867K        105                   1,036,119K
Summary                        4K                      26,316K                         638K      1,941                   1,238,615K

Mapped:
Small                          4K                          64K                          62K      2,157                     135,810K
Medium                        68K                       1,004K                         304K         18                       5,487K
Large                      1,512K                      20,204K                       6,707K         11                      73,783K
Summary                        4K                      20,204K                          98K      2,186                     215,081K

Image:
Small                          4K                          64K                           8K      3,334                      28,019K
Medium                        68K                         984K                         274K        320                      87,907K
Large                      1,032K                      26,112K                       5,314K         39                     207,251K
Summary                        4K                      26,112K                          87K      3,693                     323,177K


0:000>  lmv m clr
Browse full module list
start    end        module name
732e0000 739c1000   clr        (pdb symbols)          C:\Program Files (x86)\Windows Kits\10\Debuggers\x86\sym\clr.pdb\76BC9897F88B4D919C79C4FD96BFC89D2\clr.pdb
    Loaded symbol image file: clr.dll
    Image path: C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
    Image name: clr.dll
    Browse all global symbols  functions  data
    Timestamp:        Thu Jul 14 20:55:59 2016 (5787EE4F)
    CheckSum:         006E02E6
    ImageSize:        006E1000
    File version:     4.6.1590.0
    Product version:  4.0.30319.0
    File flags:       8 (Mask 3F) Private
    File OS:          4 Unknown Win32
    File type:        2.0 Dll
    File date:        00000000.00000000
    Translations:     0409.04b0
    Information from resource tables:
        CompanyName:      Microsoft Corporation
        ProductName:      Microsoft® .NET Framework
        InternalName:     clr.dll
        OriginalFilename: clr.dll
        ProductVersion:   4.6.1590.0
        FileVersion:      4.6.1590.0 built by: NETFXREL2
        PrivateBuild:     DDBLD400
        FileDescription:  Microsoft .NET Runtime Common Language Runtime - WorkStation
        LegalCopyright:   © Microsoft Corporation.  All rights reserved.
        Comments:         Flavor=Retail

So it looks as if the app is crashing during regular garbage collection. I have also tried running ANTS Memory Profiler, taking several snapshots and comparing them, but could not isolate the problem to any particular mechanism within the software. The memory usage or handles are not growing beyond what I would expect, and goes down, as well as up:

2019-06-16 13:43:51,4080000 -  Physical Memory : 982784 ko -  Virtual Memory : 977276 ko -  Handle Count : 3089 -  Thread Count : 150 -  GDI Count : 342 -  Total Managed Memory : 526796 ko
2019-06-16 13:44:01,4070000 -  Physical Memory : 982732 ko -  Virtual Memory : 977176 ko -  Handle Count : 3089 -  Thread Count : 148 -  GDI Count : 342 -  Total Managed Memory : 528396 ko
2019-06-16 13:44:11,4260000 -  Physical Memory : 982760 ko -  Virtual Memory : 977228 ko -  Handle Count : 3118 -  Thread Count : 149 -  GDI Count : 342 -  Total Managed Memory : 526336 ko
2019-06-16 13:44:21,4230000 -  Physical Memory : 982736 ko -  Virtual Memory : 977176 ko -  Handle Count : 3091 -  Thread Count : 148 -  GDI Count : 342 -  Total Managed Memory : 528002 ko
2019-06-16 13:44:31,4380000 -  Physical Memory : 985688 ko -  Virtual Memory : 979912 ko -  Handle Count : 3099 -  Thread Count : 149 -  GDI Count : 342 -  Total Managed Memory : 532605 ko
2019-06-16 13:44:41,4370000 -  Physical Memory : 1001316 ko -  Virtual Memory : 994700 ko -  Handle Count : 3090 -  Thread Count : 149 -  GDI Count : 342 -  Total Managed Memory : 534268 ko
2019-06-16 13:44:51,4740000 -  Physical Memory : 1103400 ko -  Virtual Memory : 1082400 ko -  Handle Count : 3103 -  Thread Count : 150 -  GDI Count : 342 -  Total Managed Memory : 673765 ko
2019-06-16 13:45:01,4850000 -  Physical Memory : 1103172 ko -  Virtual Memory : 1081716 ko -  Handle Count : 3091 -  Thread Count : 149 -  GDI Count : 342 -  Total Managed Memory : 678493 ko
2019-06-16 13:45:11,5000000 -  Physical Memory : 1130252 ko -  Virtual Memory : 1108760 ko -  Handle Count : 3097 -  Thread Count : 150 -  GDI Count : 342 -  Total Managed Memory : 697835 ko
2019-06-16 13:45:21,5010000 -  Physical Memory : 1003092 ko -  Virtual Memory : 1089056 ko -  Handle Count : 3109 -  Thread Count : 149 -  GDI Count : 342 -  Total Managed Memory : 530151 ko
2019-06-16 13:45:31,5130000 -  Physical Memory : 1008980 ko -  Virtual Memory : 1096440 ko -  Handle Count : 3096 -  Thread Count : 150 -  GDI Count : 342 -  Total Managed Memory : 527599 ko
2019-06-16 13:45:41,5270000 -  Physical Memory : 1039512 ko -  Virtual Memory : 1018116 ko -  Handle Count : 3090 -  Thread Count : 149 -  GDI Count : 342 -  Total Managed Memory : 608565 ko
2019-06-16 13:45:51,5310000 -  Physical Memory : 1069892 ko -  Virtual Memory : 1048488 ko -  Handle Count : 3122 -  Thread Count : 149 -  GDI Count : 342 -  Total Managed Memory : 637848 ko
2019-06-16 13:46:01,5430000 -  Physical Memory : 1065032 ko -  Virtual Memory : 1043860 ko -  Handle Count : 3094 -  Thread Count : 149 -  GDI Count : 342 -  Total Managed Memory : 644439 ko
2019-06-16 13:46:11,5420000 -  Physical Memory : 991880 ko -  Virtual Memory : 994636 ko -  Handle Count : 3108 -  Thread Count : 149 -  GDI Count : 342 -  Total Managed Memory : 529213 ko
2019-06-16 13:46:21,5580000 -  Physical Memory : 987684 ko -  Virtual Memory : 990396 ko -  Handle Count : 3124 -  Thread Count : 150 -  GDI Count : 342 -  Total Managed Memory : 529234 ko
2019-06-16 13:46:31,5610000 -  Physical Memory : 991412 ko -  Virtual Memory : 994088 ko -  Handle Count : 3116 -  Thread Count : 149 -  GDI Count : 342 -  Total Managed Memory : 530042 ko
2019-06-16 13:46:41,5730000 -  Physical Memory : 987480 ko -  Virtual Memory : 990488 ko -  Handle Count : 3112 -  Thread Count : 151 -  GDI Count : 342 -  Total Managed Memory : 528080 ko
2019-06-16 13:46:51,5730000 -  Physical Memory : 987592 ko -  Virtual Memory : 990612 ko -  Handle Count : 3121 -  Thread Count : 150 -  GDI Count : 342 -  Total Managed Memory : 530177 ko

What troubleshooting technique/method can I use to investigate the problem further? What could be causing such behaviour?