NET Core 3.1 - HTTP Error 500.30 - ANCM In-Process Start Failure

I upgraded my project from .net core 2.2 to 3.1, the project is running fine in my visual studio 2019, the problem occurred in staging server, it prompted the error "HTTP Error 500.30 - ANCM In-Process Start Failure". If I change the hosting model to outofprocess, it works fine.

so I tried with a default .net core project (API - .net core 3.1) and got the same problem -> the issue is not my code. I did install both sdk and hosting bundle 3.1, below is the ancm log and the dotnet info, I noticed that the "Host (useful for support)" is showing "Version: 2.1.4" and the ancm is showing "hostfxr.dll located at 'E:\Program Files\dotnet\host\fxr\2.1.4\hostfxr.dll'"

why it could not load the latest version following Microsoft document: https://docs.microsoft.com/en-us/dotnet/core/distribution-packaging

Any help or suggestion would be appreciated, thanks.

.NET Core SDK (reflecting any global.json):
 Version:   3.1.200
 Commit:    c5123d973b

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.200\

Host (useful for support):
  Version: 2.1.4
  Commit:  85255dde3e

.NET Core SDKs installed:
  2.1.402 [E:\Program Files\dotnet\sdk]
  2.2.207 [C:\Program Files\dotnet\sdk]
  3.1.101 [C:\Program Files\dotnet\sdk]
  3.1.102 [C:\Program Files\dotnet\sdk]
  3.1.200 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.4 [E:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.4 [E:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.4 [E:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

[aspnetcorev2.dll] Initializing logs for 'C:\Program Files\IIS\Asp.Net Core Module\V2\aspnetcorev2.dll'. Process Id: 4200.. File Version: 13.1.20018.2. Description: IIS ASP.NET Core Module V2. Commit: c3acdcac86dad91c3d3fbc3b93ecc6b7ba494bdc.
[aspnetcorev2.dll] Resolving hostfxr parameters for application: '.\WebApplication1.exe' arguments: '' path: 'E:\TEST1\'
[aspnetcorev2.dll] Known dotnet.exe location: ''
[aspnetcorev2.dll] Process path '.\WebApplication1.exe' is not dotnet, treating application as standalone or portable with bootstrapper
[aspnetcorev2.dll] Checking application.dll at 'E:\TEST1\.\WebApplication1.dll'
[aspnetcorev2.dll] Checking hostfxr.dll at 'E:\TEST1\.\hostfxr.dll'
[aspnetcorev2.dll] hostfxr.dll found app local at 'E:\TEST1\.\hostfxr.dll', treating application as portable with launcher
[aspnetcorev2.dll] Resolving absolute path to dotnet.exe from 'dotnet'
[aspnetcorev2.dll] Invoking where.exe to find dotnet.exe
[aspnetcorev2.dll] where.exe invocation returned: 'E:\Program Files\dotnet\dotnet.exe
C:\Program Files\dotnet\dotnet.exe
C:\Program Files (x86)\dotnet\dotnet.exe
'
[aspnetcorev2.dll] Current process bitness type detected as isX64=1
[aspnetcorev2.dll] Processing entry 'E:\Program Files\dotnet\dotnet.exe'
[aspnetcorev2.dll] Binary type 6
[aspnetcorev2.dll] Found dotnet.exe via where.exe invocation at 'E:\Program Files\dotnet\dotnet.exe'
[aspnetcorev2.dll] Resolving absolute path to hostfxr.dll from 'E:\Program Files\dotnet\dotnet.exe'
[aspnetcorev2.dll] hostfxr.dll located at 'E:\Program Files\dotnet\host\fxr\2.1.4\hostfxr.dll'
[aspnetcorev2.dll] Parsed hostfxr options: dotnet location: 'E:\Program Files\dotnet\dotnet.exe' hostfxr path: 'E:\Program Files\dotnet\host\fxr\2.1.4\hostfxr.dll' arguments:
[aspnetcorev2.dll] Argument[0] = 'E:\Program Files\dotnet\dotnet.exe'
[aspnetcorev2.dll] Argument[1] = 'E:\TEST1\.\WebApplication1.dll'
[aspnetcorev2.dll] Loading hostfxr from location E:\Program Files\dotnet\host\fxr\2.1.4\hostfxr.dll
[aspnetcorev2.dll] Canceling standard stream pipe reader
[aspnetcorev2.dll] Loading request handler:  'C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\3.1.2\aspnetcorev2_inprocess.dll'
[aspnetcorev2.dll] Creating handler application
[aspnetcorev2_inprocess.dll] Initializing logs for 'C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\3.1.2\aspnetcorev2_inprocess.dll'. Process Id: 4200.. File Version: 13.1.20018.2. Description: IIS ASP.NET Core Module V2 Request Handler. Commit: c3acdcac86dad91c3d3fbc3b93ecc6b7ba494bdc.
[aspnetcorev2_inprocess.dll] Waiting for initialization
[aspnetcorev2_inprocess.dll] Starting in-process worker thread
[aspnetcorev2_inprocess.dll] Resolving hostfxr parameters for application: '.\WebApplication1.exe' arguments: '' path: 'E:\TEST1\'
[aspnetcorev2_inprocess.dll] Known dotnet.exe location: 'E:\Program Files\dotnet\dotnet.exe'
[aspnetcorev2_inprocess.dll] Process path '.\WebApplication1.exe' is not dotnet, treating application as standalone or portable with bootstrapper
[aspnetcorev2_inprocess.dll] Checking application.dll at 'E:\TEST1\.\WebApplication1.dll'
[aspnetcorev2_inprocess.dll] Checking hostfxr.dll at 'E:\TEST1\.\hostfxr.dll'
[aspnetcorev2_inprocess.dll] hostfxr.dll found app local at 'E:\TEST1\.\hostfxr.dll', treating application as portable with launcher
[aspnetcorev2_inprocess.dll] Resolving absolute path to hostfxr.dll from 'E:\Program Files\dotnet\dotnet.exe'
[aspnetcorev2_inprocess.dll] hostfxr.dll located at 'E:\Program Files\dotnet\host\fxr\2.1.4\hostfxr.dll'
[aspnetcorev2_inprocess.dll] Parsed hostfxr options: dotnet location: 'E:\Program Files\dotnet\dotnet.exe' hostfxr path: 'E:\Program Files\dotnet\host\fxr\2.1.4\hostfxr.dll' arguments:
[aspnetcorev2_inprocess.dll] Argument[0] = 'E:\Program Files\dotnet\dotnet.exe'
[aspnetcorev2_inprocess.dll] Argument[1] = 'E:\TEST1\.\WebApplication1.dll'
[aspnetcorev2_inprocess.dll] Setting environment variable ASPNETCORE_IIS_HTTPAUTH=anonymous;
[aspnetcorev2_inprocess.dll] Setting environment variable ASPNETCORE_IIS_PHYSICAL_PATH=E:\TEST1\
[aspnetcorev2_inprocess.dll] Loading hostfxr from location E:\Program Files\dotnet\host\fxr\2.1.4\hostfxr.dll
[aspnetcorev2_inprocess.dll] Initial Dll directory: '', current directory: 'c:\windows\system32\inetsrv'
[aspnetcorev2_inprocess.dll] Setting dll directory to c:\windows\system32\inetsrv
[aspnetcorev2_inprocess.dll] Setting current directory to E:\TEST1\
[aspnetcorev2_inprocess.dll] Managed threw an exception -1073741819
[aspnetcorev2_inprocess.dll] Starting shutdown sequence 1
[aspnetcorev2_inprocess.dll] Clr thread wait ended: clrThreadExited: 1
[aspnetcorev2_inprocess.dll] Canceling standard stream pipe reader
[aspnetcorev2_inprocess.dll] Event Log: 'Application '/LM/W3SVC/7/ROOT' with physical root 'E:\TEST1\' hit unexpected managed exception, exception code = '0xc0000005'. Please check the stderr logs for more information.' 
End Event Log Message.
[aspnetcorev2_inprocess.dll] Stopping in-process worker thread
[aspnetcorev2_inprocess.dll] Stopping CLR
[aspnetcorev2_inprocess.dll] Event Log: 'Application '/LM/W3SVC/7/ROOT' with physical root 'E:\TEST1\' failed to load coreclr. Exception message:
CLR worker thread exited prematurely' 
End Event Log Message.
[aspnetcorev2_inprocess.dll] InvalidOperationException 'CLR worker thread exited prematurely' caught at F:\workspace\_work\1\s\src\Servers\IIS\AspNetCoreModuleV2\InProcessRequestHandler\inprocessapplication.cpp:420 
[aspnetcorev2_inprocess.dll] Failed HRESULT returned: 0x8007023e at F:\workspace\_work\1\s\src\Servers\IIS\AspNetCoreModuleV2\InProcessRequestHandler\dllmain.cpp:131 
[aspnetcorev2_inprocess.dll] Starting app_offline monitoring in application 'E:\TEST1\'
[aspnetcorev2_inprocess.dll] Starting file watcher thread

Update: I have found the root cause, the hosting bundle 3.1 was installed on C drive while the 2.1.4 was installed on E drive. the system keeps pointing to E drive, so I have to correct the installation on E drive, the problem was solved.

In my case, was because my appsetting.json has a json invalid for configuration and i get that error. Check the jsons you have using in your application.

With a invalid \ on the connection string.