Getting Hololens RGB Camera frames from Unity

I am building an OpenCV Project in Unity for Hololens for detecting aruco Tags. My project includes .dlls generated from OpenCV as Plugins and in addition, I have MRTK installed in my project.

It works well within Unity but when I deploy on Hololens, tags do not get detected. There are few exceptions thrown when I see Visual Studio output.

The exceptions start from this point :

'MachineViewere.exe' (Win32): Loaded 'C:\Windows\System32\CoreMmRes.dll'. Module was built without symbols.
'MachineViewere.exe' (Win32): Unloaded 'C:\Windows\System32\CoreMmRes.dll'
'MachineViewere.exe' (Win32): Loaded 'C:\Windows\System32\CoreMmRes.dll'. Module was built without symbols.
'MachineViewere.exe' (Win32): Loaded 'C:\Windows\System32\mfsensorgroup.dll'.
Exception thrown at 0x76F83C22 (KernelBase.dll) in MachineViewere.exe: WinRT originate error - 0xC00DABE0 : 'No capture devices are available.'.
Exception thrown at 0x76F83C22 (KernelBase.dll) in MachineViewere.exe: WinRT originate error - 0xC00D36B3 : 'The stream number provided was invalid.'.

After this a bunch of random exceptions continues:

Exception thrown at 0x76F83C22 in MachineViewere.exe: Microsoft C++ exception: Il2CppExceptionWrapper at memory location 0x0223F4C0 ...

I can understand that the camera is not properly accessed as it shows 'No capture device available

But I do not understand if I need to change something in the MRTK Camera profile or even how to understand what is going wrong. I saw a similar project which works when I use Webcam using Webcamtexture class of Unity - https://github.com/KeyMaster-/HoloLensArucoTracking but the difference is they do not use MRTK.

Does anyone have some suggestions for me on where to look or what to look at?I have tried changing the camera profile in MRTK profiles a bit does not change much.