D3DImage OutOfMemoryException when calling SetBackBuffer

I'm using D3DImage to show images rendered using Direct3D. The Direct3D rendering needs to be happening in its own thread, while the GUI thread takes a surface when it wants and puts it on the screen using D3DImage.

At first I tried to do this using a single D3D render target, however even with locks in place, I had serious tearing, i.e. the rendering thread was overwriting the surface as WPF was copying it on its frontbuffer. It seems like WPF is very unpredictable as to when it copies the data (i.e. it's not on D3DImage.Unlock() or even on the next D3DImage.Lock(), as the documentation suggests).

So now what I'm doing is I have two render targets, and every time WPF displays a frame, it asks the rendering thread to swap its targets. So I'm always rendering into the target WPF isn't using.

This means that on each graphical update of the window, I do something like

m_d3dImage.Lock();
m_d3dImage.SetBackBuffer(D3DResourceType.IDirect3DSurface9, m_d3dRenderer.OutputSurface);
m_d3dImage.Unlock();
m_d3dRenderer.SwapSurfaces();

where OutputSurface is an IntPtr that points to the D3D render target we're not currently rendering to, and SwapSurfaces just swaps the two surface pointers and calls IDirect3DDevice9::SetRenderTarget with the one we'll use to render next.

EDIT: as requested, here is the code of SwapSurfaces():

var temp = m_renderingSurface;
m_renderingSurface = m_outputSurface;
m_outputSurface = temp;
m_d3dDevice.SetRenderTarget(0, m_renderingSurface);

Where m_renderingSurface and m_outputSurface are the two render targets (SharpDX.Direct3D9.Surface), and m_d3dDevice is the DeviceEx object.

This works beautifully, i.e. no tearing, however after a few seconds I get an OutOfMemoryException, and Direct3D has the following debug output:

Direct3D9: (ERROR) :Invalid iBackBuffer parameter passed to GetBackBuffer
Direct3D9: (ERROR) :Error during initialization of texture. CreateTexture failed.
Direct3D9: (ERROR) :Failure trying to create a texture
Direct3D9: (ERROR) :Error during initialization of texture. CreateTexture failed.
Direct3D9: (ERROR) :Failure trying to create a texture
MIL FAILURE: Unexpected HRESULT 0x8876017c in caller: CInteropDeviceBitmap::Present D3D failure
Direct3D9: (WARN) :Alloc of size 1577660 FAILED!
Direct3D9: (ERROR) :Out of memory allocating memory for surfaces.
Direct3D9: (ERROR) :Failure trying to create offscreen plain surface

I've found a related topic here where the proposed solution was to call D3DImage.SetBackBuffer() and pass IntPtr.Zero, however I've added this just before the existing call and it didn't solve the issue. I also tried calling Lock() and Unlock() around the SetBackBuffer(... IntPtr.Zero), and that didn't solve the issue either.

At this point I'm wondering if there's a bug in D3DImage or if I should use a different approach altogether. Could I replace my 2 render targets with a D3D swap chain instead, would that allow me to stop having to call SetBackBuffer with a different pointer all the time? I'm a newbie with Direct3D.

EDIT: I looked in the code of D3DImage.SetBackBuffer() using .NET Reflector, and it's creating an InteropBitmap every time. It doesn't do anything in particular for IntPtr.Zero. Since I'm calling this many times per second, perhaps the resources don't have the time to be freed. At this point I'm thinking of using 2 different D3DImages and alternating their visibility to avoid having to call their SetBackBuffer() all the time.

Thanks.

It appears that D3DImage creates a new Direct3D texture every time you set its backbuffer pointer to something different. This is eventually cleaned, but setting it 30 times per second like what I was doing doesn't leave it enough time, and is anyway a big performance killer. The approach I went for was to create several D3DImages, each with its own surface, put them all on top of each other and toggle their Visibility property so only one shows at a time. This seems to work very well and doesn't leak any memory.