Using Outlooks COM class component fails only with admin privileges

I have a PowerShell script which queries the current Outlook session.

Running it just in an unelevated PowerShell window works like expected, but when I'm in an elevated prompt it fails like demonstrated below:

"normal" session:

PS> New-Object -Com Outlook.Application


Application        : System.__ComObject
Class              : 0
Session            : System.__ComObject
Parent             :
Assistant          :
Name               : Outlook
Version            : 15.0.0.4903
COMAddIns          : System.__ComObject
Explorers          : System.__ComObject
Inspectors         : System.__ComObject
LanguageSettings   : System.__ComObject
ProductCode        : {90150000-000F-0000-0000-0000000FF1CE}
AnswerWizard       :
FeatureInstall     : 1
Reminders          : System.__ComObject
DefaultProfileName : Outlook
IsTrusted          : False
Assistance         : System.__ComObject
TimeZones          : System.__ComObject
PickerDialog       : System.__ComObject

Elevated one:

PS> New-Object -Com Outlook.Application
New-Object : Retrieving the COM class factory for component with CLSID {0006F03A-0000-0000-C000-000000000046} failed
due to the following error: 80080005 Server execution failed (Exception from HRESULT: 0x80080005
(CO_E_SERVER_EXEC_FAILURE)).
At line:1 char:1
+ New-Object -Com Outlook.Application
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [New-Object], COMException
    + FullyQualifiedErrorId : NoCOMClassIdentified,Microsoft.PowerShell.Commands.NewObjectCommand

Elevation uses the same user account, which is in the Administrator group. Why does this happen? And how to fix it? As I know un-elevated applications are not allowed to communicate directly with elevated ones, but the other way around should work, shouldn't it? I also tried to start Outlook as Administrator but - like expected - it doesn't make any difference.

EDIT:

C:/WINDOWS/system32> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.14393.693
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.693
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

It's PoSh 5 on Win 10 with Office 2013 Home & Business

Thanks to @Lieven on helping me researching this issue. I wanted to leave this open if someone comes around and finds a solution on this. As stated out by @Lieven and some ongoing research on myself, there is no "solution" for this:

Outlook and PowerShell can concurrently use the same Outlook session using shared memory. As processes with different elevation levels can't share memory (reference needed), the 2nd process (elevated PowerShell in my case) has to open the PST (a new outlook session) by itself, which fails because it's exclusively opened by the first one (unelevated Outlook in my case).

My workaround is to create a low-level process holding the Outlook session and providing a pipeline for higher level processes (and same-level as well) to connect with. Vice versa won't work as unelevated processes are not allowed to connect to elevated pipelines.

This works for me as the tasks doing with the Outlook session from PowerShell are very basic. However it is still a workaround.