C# (MediaFrameReader API) SoftwareBitmap doesn't show

Question

C# (MediaFrameReader API) SoftwareBitmap doesn't show

I followed Microsoft's instructions to implement MediaFrameReader API in order to process video captured by camera. The problem is I don't know why the bitmap doesn't show up in XAML, I can't save it to jpeg either (bitmap is not empty). I did convert the bitmap to proper format for XAML:

Here's my code to show the frame in XAML control (no error):

public void ShowSoftwareBitmap(SoftwareBitmap softwareBitmap, SoftwareBitmap backBuffer, Image imageElement, bool taskRunning)
    {
        try
        {
            //check if softwareBitmap is in proper format (Bgra8 and premultiplied alpha) to display to XAML Image control 
            if (softwareBitmap.BitmapPixelFormat != BitmapPixelFormat.Bgra8 ||
                softwareBitmap.BitmapAlphaMode != BitmapAlphaMode.Premultiplied)
            {
                softwareBitmap = SoftwareBitmap.Convert(softwareBitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied);
            }

            if(softwareBitmap != null)
            {
                softwareBitmap = Interlocked.Exchange(ref backBuffer, softwareBitmap);
                softwareBitmap?.Dispose();
                var task = imageElement.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal,
                async () =>
                {
                    if (taskRunning)
                    {
                        return;
                    }
                    taskRunning = true;

                    SoftwareBitmap latestBitmap;
                    while ((latestBitmap = Interlocked.Exchange(ref backBuffer, null)) != null)
                    {
                        SaveSoftwareBitmapToJpg(latestBitmap, 2);
                        var imageSource = (SoftwareBitmapSource)imageElement.Source;
                        await imageSource.SetBitmapAsync(latestBitmap);
                        latestBitmap.Dispose();
                    }
                    taskRunning = false;
                }
                );
            }

        }
        catch (NullReferenceException e)
        {
            Debug.WriteLine("Backbuffer is empty." + e.Message);
        }

    }

Here's my code to save it to jpeg (error unsupported bitmap format):

public async void SaveSoftwareBitmapToJpg(SoftwareBitmap softwareBitmap, int _frameIndex)
    {
        StorageFolder captureFolder = await FileAccess();
        StorageFile outputFile = await captureFolder.CreateFileAsync($"capture{_frameIndex}.jpg", CreationCollisionOption.FailIfExists);
        using (IRandomAccessStream stream = await outputFile.OpenAsync(FileAccessMode.ReadWrite)) 
        {
            BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, stream);
            encoder.SetSoftwareBitmap(softwareBitmap);
            encoder.BitmapTransform.ScaledHeight = 300;
            encoder.BitmapTransform.ScaledWidth = 300;
            encoder.IsThumbnailGenerated = true;
            try
            {
                await encoder.FlushAsync();
            }
            catch (Exception e)
            {
                switch (e.HResult)
                {
                    case unchecked((int)0x88982F81): //WINCODEC_ERR_UNSUPPORTEDOPERATION
                                                     // If the encoder does not support writing a thumbnail, then try again
                                                     // but disable thumbnail generation.
                        encoder.IsThumbnailGenerated = false;
                        break;
                    default:
                        throw e;
                }
            }
            //catch error generating thumbnail
            if (encoder.IsThumbnailGenerated == false)
            {
                await encoder.FlushAsync();
            }
        }
    }

c#

bitmap

uwp

asked on Stack Overflow Mar 20, 2018 by

AnneP

2 Answers

I tested with your code snippet, if you invoking the above methods correctly with a correct SoftwareBitmap object, the above methods could work well. Simply for example:

 <Image x:Name="imgshow" ></Image>
 imgshow.Source = new SoftwareBitmapSource();
 SoftwareBitmap _backBuffer = new SoftwareBitmap(BitmapPixelFormat.Bgra8, 300, 400);
 ShowSoftwareBitmap(softwareBitmap, _backBuffer, imgshow, false);

So that issue may be not happened with the methods above. This could help you narrow the issue, you could check if something wrong when getting the SoftwareBitmap object. I saw you may copy code from ProcessFrame method which is in FrameRenderer class of the official sample. The official sample get the SoftwareBitmap by converting VideoMediaFrame, check if anything wrong when you converting.

Also please debug your code to see if taskRunning is false that your code do run into setting image source steps.

answered on Stack Overflow Mar 21, 2018 by

Sunteen Wu

So here's how I changed my code:

In ShowSoftwareBitmap(), I used the same boolean taskRunning in other methods so it was changed crazily in run time. Just create another boolean to solve the problem.

In save SaveSoftwareBitmapToJpg(), the bitmap wasn't properly converted, it has to be Rgba8 or Rgba16 to print to JPEG, so the same conversion used for ShowSoftwareBitmap() wouldn't work. Use this instead:

try
{
    if (softwareBitmap.BitmapPixelFormat != BitmapPixelFormat.Rgba16)
       {
          result = SoftwareBitmap.Convert(softwareBitmap, BitmapPixelFormat.Rgba16);
       }
    else
       {
          result = SoftwareBitmap.Copy(softwareBitmap);
       }
}

answered on Stack Overflow Mar 23, 2018 by

AnneP

User contributions licensed under CC BY-SA 3.0