Call was rejected by callee in the Windows task scheduler

I have done an application to create outlook meeting requests automatically. The approach was to create a console application (C#) and schedule it in the windows task scheduler. The application is working fine in the Visual Studio and when i double click the exe also it is working fine.

but When i schedule it in the scheduler, im getting the below error.

Retrieving the COM class factory for component with CLSID {0006F03A-0000-0000-C000-000000000046} failed due to the following error: 80080005 Server execution failed (Exception from HRESULT: 0x80080005 (CO_E_SERVER_EXEC_FAILURE)).

I was trying to figure this out and did some changes to the DCOM objects. I have given permissions to NETWORKSERVICE user and tried to run the scheduler using NETWORKSERVICE user. But no luck so far.

Please help.