System.Diagnostics.Process.Start consistently fails the second time

I have the following code that opens documents from my application:

System.Diagnostics.Process.Start(fileName);

This has been working fine for years, and it works fine on other people's computers as well.

However, I recently got a new computer. On my computer, it works only once: I click the button, and the document opens. When I try to do the exact same thing a second time, I get this error:

Message: System.ComponentModel.Win32Exception (0x80004005): The specified procedure could not be found
   at System.Diagnostics.Process.StartWithShellExecuteEx(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start(String fileName)
   at Citadel.Products.Entity.ProductFactSheetDetail.DownLoad(String path, Boolean includeDateInFileName, Boolean addGuidToFileName, Boolean fileNameKeepSpaces, Boolean viewDocument)
   at Citadel.Products.Entity.ProductFactSheetDetail.DownLoad(Boolean includeDateInFileName)
   at Citadel.Products.Interface.Controls.FactSheetVersionSelectorControl.btnView_Click(Object sender, EventArgs e) in C:\Dev\Src\Tyrus\CodeBase\Development\Citadel.Products\Citadel.Products.Interface\Controls\FactSheetVersionSelectorControl.cs:line 81 - The specified procedure could not be found
Stack Trace:    at System.Diagnostics.Process.StartWithShellExecuteEx(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start(String fileName)
   at Citadel.Products.Entity.ProductFactSheetDetail.DownLoad(String path, Boolean includeDateInFileName, Boolean addGuidToFileName, Boolean fileNameKeepSpaces, Boolean viewDocument)
   at Citadel.Products.Entity.ProductFactSheetDetail.DownLoad(Boolean includeDateInFileName)
   at Citadel.Products.Interface.Controls.FactSheetVersionSelectorControl.btnView_Click(Object sender, EventArgs e) in C:\Dev\Src\Tyrus\CodeBase\Development\Citadel.Products\Citadel.Products.Interface\Controls\FactSheetVersionSelectorControl.cs:line 81
Application: Citadel.Tyrus
Version: 1.0.0.0
OSVersion: Microsoft Windows NT 6.2.9200.0
Description: 

The only way to get rid of this error, is to restart my application.

Clearly this is not a coding issue, something is not right on my computer, but I have no idea where to start looking. Any help will be greatly appreciated.

EDIT: The error cannot be related to the file being opened. Whenever we download a file from the database, we always add a random Guid to the file name to ensure it is unique.

EDIT: The problem seems to only occur with MS Office documents. When I open a PDF document or a bmp, the error does not occur.

Win32Exception There was an error in opening the associated file.

I think your file (like Excel) can't be opened twice.

Check if the file is already opened before opening it