Occasional 500s using Faces plugin - Delegates cannot be marshalled

We're using 3.4.3 of ImageResizer, and the elite plugins. It seems any trouble we’ve had have been related to the Faces plugin. Images will not show up, and upon looking at the network response, the stacktrace shows "Delegates cannot be marshaled from native code into a domain other than their home domain". Full trace is here

[NotSupportedException: Delegates cannot be marshaled from native code into a domain other than their home domain.]
   System.Runtime.InteropServices.Marshal.GetDelegateForFunctionPointerInternal(IntPtr ptr, Type t) +0
   System.Runtime.InteropServices.Marshal.GetDelegateForFunctionPointer(IntPtr ptr, Type t) +267
   OpenCvSharp.CvInvoke..cctor() +328

[TypeInitializationException: The type initializer for 'OpenCvSharp.CvInvoke' threw an exception.]
   OpenCvSharp.Cv.Load(String filename, CvMemStorage memstorage, String name, String& real_name) +222
   ImageResizer.Plugins.Faces.FeatureDetectionBase`1.LoadFiles() +621
   ImageResizer.Plugins.Faces.FeatureDetectionBase`1.DetectFeatures(Bitmap b) +82
   ImageResizer.Plugins.Faces.FacesPlugin.PostPrepareSourceBitmap(ImageState s) +341
   ImageResizer.Resizing.AbstractImageProcessor.PostPrepareSourceBitmap(ImageState s) +99
   ImageResizer.ImageBuilder.Process(ImageState s) +60
   ImageResizer.ImageBuilder.buildToBitmap(Bitmap source, ResizeSettings settings, Boolean transparencySupported) +276
   ImageResizer.ImageBuilder.buildToStream(Bitmap source, Stream dest, ResizeSettings settings) +149
   ImageResizer.ImageBuilder.BuildJob(ImageJob job) +1353
   ImageResizer.ImageBuilder.Build(ImageJob job) +268
   ImageResizer.ImageBuilder.Build(Object source, Object dest, ResizeSettings settings, Boolean disposeSource, Boolean addFileExtension) +124
   ImageResizer.ImageBuilder.Build(Object source, Object dest, ResizeSettings settings, Boolean disposeSource) +36
   ImageResizer.ImageBuilder.Build(Object source, Object dest, ResizeSettings settings) +23
   ImageResizer.<>c__DisplayClass3.<HandleRequest>b__2(Stream stream) +287
   ImageResizer.Plugins.DiskCache.<>c__DisplayClasse.<TryWriteFile>b__d() +593
   ImageResizer.Plugins.DiskCache.LockProvider.TryExecute(String key, Int32 timeoutMs, LockCallback success) +424
   ImageResizer.Plugins.DiskCache.CustomDiskCache.TryWriteFile(CacheResult result, String physicalPath, String relativePath, ResizeImageDelegate writeCallback, DateTime sourceModifiedUtc, Int32 timeoutMs, Boolean recheckFS) +517
   ImageResizer.Plugins.DiskCache.CustomDiskCache.GetCachedFile(String keyBasis, String extension, ResizeImageDelegate writeCallback, DateTime sourceModifiedUtc, Int32 timeoutMs, Boolean asynchronous) +819
   ImageResizer.Plugins.DiskCache.DiskCache.Process(IResponseArgs e) +209
   ImageResizer.Plugins.DiskCache.DiskCache.Process(HttpContext context, IResponseArgs e) +47
   ImageResizer.InterceptModule.HandleRequest(HttpContext context, String virtualPath, NameValueCollection queryString, IVirtualFile vf) +1687
   ImageResizer.InterceptModule.CheckRequest_PostAuthorizeRequest(Object sender, EventArgs e) +1163
   System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +92
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +165

I found this page and ensured we weren't using overlapped recycling. I tried upgrading to 4.1.7, but it was a bit of a nightmare trying to get all the opencv versions/bitness to line up. The app would crash often and hold the rest of the sites depending on it hostage, so we rolled back to 3.4.3 which at least works, then doesn't. Our regular issues go away temporarily when we restart the IR app pool.

We have our web apps hosted on two VMS load-balanced by BigIP. Each VM has two web apps, and they share a single image resizer hosted on the same server as them which and utilizes the URL API to make resize requests, face detection + cropping, etc.

One other bit of info I was able to obtain was from DebugDiag. I grabbed a crash of the app with the following message:

In w3wp__ImageResizer__PID__5284__Date__06_14_2017__Time_08_23_10AM__122__First chance exception 0XE0434352.dmp the assembly instruction at KERNELBASE!RaiseException in C:\Windows\System32\KERNELBASE.dll from Microsoft Corporation has caused a CLR Exception on thread 28 with the following error information:
    Type:   ImageResizer.Plugins.Faces.AlternateResponseException 
    Message:   Resizing was canceled as JSON data was requested instead 
This exception originated from ImageResizer.Plugins.Faces.FacesPlugin.Render(ImageResizer.Resizing.ImageState).

Debug page can be found here